Uploading 300MB file slow; often fails


#1

I’m trying to upload a bunch of 300-400 MB files using SFTP.

(I’m a sound tech for my church. I record the services, then edit them for CD mastering. I upload the files so the guy with the CD duplicator can download them and make duplicate CDs before the next Sunday. Otherwise, I’d either have to drive a CD over to his place during the week, or we’d have to delay delivering CDs by a week.)

I get very slow throughput, averaging about 30KB/s. The instantaneous numbers are often much higher, but then it shows “stalled” for long periods. About half the transfers fail with a “broken pipe” error after doing a hundred MB or so.

  1. How can I transfer the files in a way that I can resume a broken transfer instead of starting it over?

  2. How can I figure out where the slowdown is between my computer and the host?

Thanks,

Pat


Save the Earth. It’s the only planet with chocolate!


#2

I’m having the same issues right now, possibly because I’m doing a handful of other Internet connections at the same time.

  1. I use Transmit http://www.panic.com for the Mac. I use SFTP transfers, and it allows me to resume an upload.

  2. I get about 30KB/s, which is the upstream speed of my broadband connection (~256k). I periodically test this with http://lax.speakeasy.net, or some other speed test.

-Scott


#3

Hi Scott,
Thanks for the suggestion about Transmit. I’ll check it out.

When I ran the Speakeasy test, it reported I had 691kpbs (86.4 KB/s) upload rate.

This upload rate test still leaves me thinking there’s some intermittent problem between me and dreamhost. My upload shows ‘stalled’ a lot of the time, then it races ahead, as if it were uploading at around 100 KB/s for a few seconds. Then, it stalls again.

This wouldn’t be so bad, except that sometimes the stall lasts long enough to break my pipe.

Pat


Save the Earth. It’s the only planet with chocolate!


#4

Are you using an ftp program that supports resuming, or are you trying to upload through a web browser? Browsers are notorious for failing with large file transfers, because they assume there is just no response from the server. They also don’t support transfer resuming. You should be using a halfway decent ftp client for that.

FileZilla Open Source FTP / SFTP Client
FileZilla is a free, open source FTP client for Windows distributed under the GNU General Public License. You can do the usual stuff, upload and download files, as well as resume uploads or downloads, works with firewalls, supports SFTP (Secure FTP) and SSL secured connections, handle queues of files to be uploaded/downloaded, etc.

Get it here: http://sourceforge.net/projects/filezilla/

http://benconley.net
http://teamshocker.com


#5

I am experiencing the same problem with ftp uploads to DH slowing to 0.0KB/s and stopping. I have to restart the upload after several retries. My Internet connect is fine… I am able to browse sites and ftp other sites when this problem with DH occurs.

Is this a typical problem with DH? Is this a symptom of overloading/oversubscribing servers?

Thanks for any info.