I get an error message when uploading large files


#1

I use leech ftp, and when I upload my 55mb file, it says ! Socket Error: connection reset by peer
~ Disconnected
like half way through, can somebody explain the cause to this error?

thanks,
Matt Crissinger


#2

thanks for the reply, it seems it stops at random times, but then a couple times it stopped at 45mb each time which was strange, I’ll check the quota, thanks.


#3

Hi, I am having the same problem and I just sent a message to dreamhost support which I am pasting below.

Hi,

I have had this problem for several months now and I am at my wits end, please help me.

I can’t upload my large files. The connection resets just before completion. My main ftp program is smartftp, and this program is supposed to be able to resume uploads when disconnected - but when it reconnects it starts back from the beginning. This happens if I use
direct upload or the global queue. I have also tried increasing the time out time to 3 minutes.

I have also tried using cuteftp, coffecup, and fireftp.

Please note that sometimes even my small files disconnect and restart.

I am on a home network and I have tried uploading from other family computers, each of which have different software and software configurations - but no luck.

I have another site hosted by another company and I have no troubles uploading to it.

Please advise.

Thank you.

Note: I am a dreamhost customer - not an employee. I’ll do my best to give you an accurate answer, however, you must use my information and advice at your own risk.


#4

Have you checked to make sure that your various FTP client programs are using “passive” (PASV) mode ?

–rlparker


#5

Yes, it was set to pasv.

I just got a response from dreamhost support, but somehow it doesn’t seem like it would help me and it also feels like it might be missing an important step or two. I guess this is because I can and do connect to the sever. The problem is that I don’t stay connected and when I do reconnect the resume feature doesn’t work. anyhow, below is their response:

It sounds like you may have a problem with
your path to the server. If you are still unable to connect to the server
properly, please send in the results of a traceroute.

To run a traceroute in Windows:

  • Open a command prompt (Start -> Run -> type ‘cmd’ and click ok)
  • Type ‘tracert doamin.com’ and press enter

To run a traceroute in OS X:

  • Open a terminal
  • Type ‘traceroute domain.com’ and press enter

Note: I am a dreamhost customer - not an employee. I’ll do my best to give you an accurate answer, however, you must use my information and advice at your own risk.


#6

I have no other idea what might be happening, but I would definitely respond to support’s request with a traceroute; maybe that will help them help you.

–rlparker


#7

I really don’t see how that would help them. I don’t need to do a trace route to determine if I am connecting; I know positively that I am connecting - as I am uploading lots of files, generally only the very large files disconnect and reset just before completion.

anyhow, I don’t know everything, so I sent them a trace route, but as I suspected… it did not help.

And I apologize for being blunt here, but I am now wishing that I had not contacted them at all. Don’t get me wrong, I am not trashing dreamhost customer service, they are trying, they really are, bless their hearts - but their suggestions don’t seem to make much sense in light of what I had told them and I especially didn’t like being written back and told that I may have copy written files on my site that need to be removed. Right now I am very concerned that some overzealous employee might take it upon himself to delete MY files…

Whew…

…below are some of the answers that I just sent dreamhost support. If anyone wants to pick apart my thinking here, be my guest; I want you to find something I missed, something I misunderstood, or even a flaw in my logic - because if you do, that may be the information I need to solve my problem.

The “copy written files” are mine; I spent years making them and I own the copyrights. You are scaring me here. Please do not delete MY videos.

Successfully uploading a 113mb file is not surprising. I (generally) have no problem uploading files of around that size. Allow me to clarify: My large 650-780mb files (and one 2.65gb file) are quitting just before completion. Can you imagine the feeling of uploading a file for 3-4 hours and sometimes 24 hours or more (tying up your internet the whole time) and then it quits just before completing? There have also been times when I slowed my upload speed to *12kbs so that my kids could use the net while I was uploading and the file uploaded for days and quit just before completion.

-as for my firewall blocking me. Consider this: I am connecting and I am uploading files. And why would my firewall decide to block me at such random times.? This same behavior also happens whether I use my personal or work computer both of which are on my home network but have different firewalls. It’s possible, but extremely doubtful.

I know what you are going to say…perhaps it’s the router? And again. Consider this: I am connecting and I am uploading files. And why would my router decide to block me at such random times? It’s possible, but doubtful.

So what to do now???

I appreciate your efforts, but I seem no closer to determining the problem. Please keep me informed if you come up with new information.

Best Regards.

Sincerely,

-frank

*Note: I have high speed cable but Charter limits upload speed to a measly 32kbs.

Note: I am a dreamhost customer - not an employee. I’ll do my best to give you an accurate answer, however, you must use my information and advice at your own risk.


#8

I understand your thinking there, but it is generally a good idea to give the suppport folks what they ask for even if you don’t see a reason for it; in this case a traceroute is a simple enough request and, if it helps them eliminate a possible source fo the problem, providing it is time well spent.

I realize that you are “connecting” without problem, but with transfers that large, taking that long, and at that speed, lots of things can happen along the route between your box and the ftp server that can, and often do interrupt the transfer. That is one of the reasons many of the linux distros now only distribute their iso’s via bittorrent - it’s more robust for largish transfers. And your transfers are "largish, to say the least. I’d also be suspicious of your cable provider jumping into what it sees as a very long continuous stream of outgoing data from your box, and if you haven’t contacted them about the problem you are having, you should. Unless you are monitoring the route carefully, and checking for packet loss during the extended upload period, you can’t really assume it’s a DH problem (though it very well may be).

It’s pretty easy to tell that you are frustrated, but you might do well to realize that moving multi-gigabyte files via an 32kbs upload link is not a trivial task across a distributed network; any router along the way that “craps”, “burps” or takes a few minutes to respond or reset could be contributing to the probelm…and during the extended transfers, there is a lot of opportunity for that to happen. :frowning:

As for what to do, as I said before, I do not know the answer to your problem. I suspect that it will take some significant investigation involving the whole transfer process before anyone is likely to determine authoritatively what is causing the behavior. That said, and since you immediate goal is to successfully complete the transfers, I suggest you do what the military does when having transmission problems…break the transmission into smaller packets.

You can use tools on your computer to “split” those monster files into smaller chunks for transfer (only small enough to transfer successfully) and then use the linux shell to “reassemble them” into their original “monster sized” state once all the "chunks have been received.

I don’t know what system you are using at your end, but there are loads of such tools that have *nix counterparts…you just need to locate a tool that will “split” files in a way that an appropriate *nix tool (DH uses Debian linux) can reassemble, and you have a workaround until the problem is identified and resolved or for another time when network condition, sunspots, the position of the stars, or whatever other “voodoo” interferes with one of your massive transfers.

I’m not saying it’s ideal, but it is a “tried and true” way of dealing with such gremlins when it is more important to get the files transferred than it is to track down the source of the problem…

This is the first time I noticed you mentioning the size of the files you are having trouble with; moving that size files over a consumer-grade ISP account is likely to continue to give you periodic problems. That , unfortunately, is the state of consumer-level connectivity in the United States at present. And I know that sucks, but don’t shoot me for pointing it out - I’m just the messenger. :wink:

–rlparker


#9

This may or may not be your issue, but it sounds like it.
Try the program ‘Winsock Fix’, to see if it will resolve your incomplete uploading issues.
http://www.iup.edu/house/resnet/winfix.shtm

I long long time ago I had the same problem and used that program to resolve it. It seems some program I’d installed had modified Window’s networking behavior and was causing many of my downloaded/uploaded files to become corrupt.

Again, this may not fix your problem, but I figure it’s worth a shot at least. Though I am rather surprised you don’t just resume the failed upload, rather than try to re-upload it again.


#10

@Mousee, thanks. I will try that.

@rlparker, “Pow, Pow, Pow, Pow” goes my machine gun. Better duck, don’t worry, just kidding…I never shot an honest messenger. LOL.

Everything you said makes sense. I do want to mention a couple things: smartftp is supposed to be able to resume transfers but it doesn’t when connecting to my dreamhost (DH) account, however, it does when connecting to another account that I have. I know this because I tested it, purposely breaking the connection and then reconnecting. There is something particular about DH or the way my comp. interacts with DH, or the way my isp and my comp and my router and Dh interact, etc. etcetera…Like you said, there are many opportunities for those pesky gremlins…thus I can’t single out DH for blame.

The file splitting solution could resolve this frustrating issue for me. And I had thought of that, however, I doubted that a shared server would allow such necessary permissions. I am also not familiar with the OS that dreamhost is using on my shared server. If possible, I’d like to use RAR, if they have a version available that will work on the DH server. I will contact DH and let you know of their response.

BTW, I have decent download speed of 3mbps. It really sucks that charter is so “chincy” (sp?) on the upload speed.

And before I go, I want and need/should say, thank you, Thank You, THANK YOOOOOOOOOOOOOOOOOU!!!

“I really mean it” (Note: you are supposed to sing that last line.) :wink:

Note: I am a dreamhost customer - not an employee. I’ll do my best to give you an accurate answer, however, you must use my information and advice at your own risk.


#11

Dreamhost has both rar and zip available for user use from the command line. :wink:

Of course, as a courtesy to others (and to avoid attracting the attention and subsequent wrath of the Dreamhost omnipresent “prockiller”), you should “nice” your commands whenever maipulating really large files with these tools.

From the shell command line, you can learn about these tools use on DH by just typing “man commandname” (no quotes - replace ‘commandname’ with the program/command you want to learn about). “man nice” explains how the “nice” program works, “man rar” shows the manual for “rar”, etc. Good Luck!

–rlparker


#12

LOL, that’s funny, I just got a response from DH support that said…

“we don’t have the program that
lets you rar the files together on the server as its too resource intensive and it would effect other users.”

Perhaps you are working on a dedicated server? Or perhaps the technician is mistaken? In any event, how do I execute command line commands on a shared server? I imagine there is a program that I’ll need to do something like that - to my knowledge it can’t be done via standard ftp programs.

And hey, thanks for responding. I am starting to have the feeling that somehow this can be worked out.

Note: I am a dreamhost customer - not an employee. I’ll do my best to give you an accurate answer, however, you must use my information and advice at your own risk.


#13

[quote]“we don’t have the program that
lets you rar the files together on the server as its too resource intensive and it would effect other users.”[/quote]
Nope, I’m not on a dedicated server, just a shared server like most everyone else. As for whether or not the technician is mistaken, I’d prefer not to characterize his statements at all, but rather encourage you to check it out for yourself. :wink:

I see his point about resource usage, and I cautioned you about that, but check it out for yourself. Log into the shell, and type “which rar”, or “which zip” and notice the locations of the executables. They are there, and they work, irrespective of what the tech told you.

You access the shell command line via the SSH protocol; the program you need on your computer varies depending upon your host system (you have never said what you are using). There are plenty of “free” SSH client programs available for whatever system you are running at home.

The Dreamhost Wiki article on the shell, and related articles on SSH and Unix commands will get you started. Make sure and read the “see also” links you encounter, particularly the one about “enabling shell access” (in the event your user is not, at present, a “shell enabled user”).

The appropriate use of “nice” is the real key here. :wink:

–rlparker


#14

Hey, thanks for all your help. I just wanted to let you know that for some reason the resume feature on smartftp is now working for me. I haven’t tried to upload that huge file yet, but I stopped and resumed a 700mb file with no problem. If it doesn’t work for that big file, I will return to let everyone know about it.

Thanks again.

Note: I am a dreamhost customer - not an employee. I’ll do my best to give you an accurate answer, however, you must use my information and advice at your own risk.