problem LQATUZ post LQDD1Q
I’m having the problem described in the subject (“My DreamHost Subversion now can’t commit/add any new file over ~.5GiB or .5GB”). Anyone else know-of or experiencing this? As it’s certainly got me stuck; and it looks like it could be affecting others, potentially everyone on DreamHost.
And based on (the cryptic error message which occurs and my frustrating experiences) which I present below, possible causes I can see include, from most to least likely, are: [list=1]
Possibly DreamHost quietly added (else notably reduced) a limit on Subversion checkin size (for the whole checkin else for a file), seemingly .5GiB (a typical value) else .5GB (a somewhat typical value) when before I was having no trouble at over 1GB, and a limit which manifests itself as a cryptic error (described below) which also oddly doesn’t appear until after 24 to 87MiB are transferred. If DreamHost did this, based on my experiences, they must have done this between 2011.06.04 (when I added-via-commit a 1.03GiB file with no problem) to 2011.08.21 (when now I can’t commit any file .51GiB or larger). And if DreamHost did this, they would have done this either directly (by somehow setting some explicit limit) else indirectly (by say changing one of the involved server programs, which somehow then imposed (as added) this limit as say a new “feature”).
 DreamHost’s software is has become out-of-date else inconsistent. While indeed DreamHost software is mostly out-of-date (as most notably, my account (somewhat disappointingly) has “svn, version 1.5.1 (r32289)” even though (the current version 1.6 has been out for 2.4years says http://subversion.apache.org/docs/release-notes ), and the OS (Debian) behind at least 6mo, so I wouldn’t be surprised then if the web server (Apache) was as well), DreamHost’s software being out-of-date seems unlikely to be the problem as DreamHost’s supplied Subversion and some/all this related software always been out-of-date apparently due to the problem detailed here on the Dreamhost Debian wiki page yet still I was able to easily add-via-commit a 1.03GiB file 2mo ago as mentioned; so while still I would very much like DreamHost to update the software of me (and of everyone else who might need it, as being 2.4years behind is kind of scary, especially in IT, and especially when it comes to storing precious sources, what Subversion does), I still don’t strongly suspect DreamHost’s out-of-date software being the present cause of this problem, unless say it is from some (older?) Subversion bug (if there is one) where the (supposedly unlimited) file/checkin size gets limited as the repository grows, or from some fringe-combo-version-incompatibility-bug (if there is one) from say DreamHost updating some other software on my server (if they did) while still letting others (as Subversion) lag 2.4years behind.
 Possibly something corrupted in my SVN repository (note I have only tried files over .5GiB in this one repository, since these files are large and Subversion keeps files forever, so this is my Subversion repository for oversized items & collections), but this repository has always worked before, including always working fine for at least for all files under ~.5GiB or GB, and before also working fine for the 1.03GiB file I mentioned 2mo ago , and I’m experiencing this failure identically on 2 of 2 Subversion clients, so I kind of doubt it is my repository.
 It’s not seemingly my client computer & it’s SVN checkout, as the problem occurs identically on 2 of 2 different ones as mentioned.
 It’s not seemingly my Internet connection as the problem occurs identically on 2 of 2 totally separate Internet connections.[/list]
So in response to my #1 suspect, Did DreamHost (or the Apache web server or involved server software) suddenly add-else-notably-lower a file-size or checkin-size limit? While I certainly hope not, it unfortunately looks like it, and if so, then
not only was that done without a meaningful error message when the limit is reached (so costing me 2 days of time making & analyzing these tests),
it also appears to have been done without any public warning & announcement & documentation, as:[list=1]
 Google Search(DreamHost +Subversion OR +SVN size OR GiB OR MiB limit) currently finds for me:[list=a]  Nothing specifically mentioning any limits. Notably it does NOT find any size limits mentioned where one would expect them to be mentioned: on the official page http://wiki.dreamhost.com/Subversion .
 http://discussion.dreamhost.com/thread-67887-post-67909.html#pid67909 saying in 2007 that the servers (including those hosting Subversion) had a 2GiB file Apache web server limit, which 4x higher than what I’m experiencing here.[/list]
 Google Search(+Subversion OR +SVN size OR GiB OR MiB limit) currently finds for me:[list=a]  1st the official document saying “A nice feature of Subversion is that by design, there is no limit to the size of files it can handle. …”.
 2nd this useful thread http://www.svnforum.org/threads/39795-Is-there-any-inherent-Subversion-repository-size-limit also saying Subversion itself imposes no practical limits but can be limited to generally 2GiB but only if the filesystem isn’t properly upgraded, and while that is also not good, it is still 4x larger than the ~.5GiB limit I am experiencing.[/list][/list]
So what’s the problem I’m experiencing exactly? Subversion commits now get their connection mysteriously aborted when the commit contains (adds) any new file over ~.5GiB, including:[list=1]
 This didn’t happen 2mo ago (when I committed a new 1.03GiB file without a hitch), though (as noted below) I did get not-the-same-but similar failure 5mo ago when trying to add-via-commit a 2.32GiB file.
 This failure doesn’t occur at the start nor end of commit, but oddly after transmitting the first ~25 to 87MiB
 They error message is upsettingly not clear being vague, phrased either:[list=a]
 From TortoiseSVN 1.6.16 (latest) “
Adding: …\L9LIBO_LIP2I8_oversized_L9QDKC\Asus_Eee_PC_Vmx__LQ7F9N\LQ7KPS\LQ7KPS-s002.vmdk application/octet-stream
Sending content: …\L9LIBO_LIP2I8_oversized_L9QDKC\Asus_Eee_PC_Vmx__LQ7F9N\LQ7KPS\LQ7KPS-s002.vmdk
Error: Commit failed (details follow):
Error: PUT of
Error: Could not send request body: An existing connection was forcibly closed by the
Error: remote host.
” after transmitting just 87.38MiB of this 2,143,617,024 byte file. OR
 from Cygwin svn 1.6.17 (latest) “
Transmitting file data .svn: Commit failed (details follow):
svn: PUT of ‘/L9LIBO/!svn/wrk/258318e6-cc9a-11e0-aa24-2bb4316230e0/_oversized_L9QDKC/Asus_Eee_PC_Vmx__LQ7F9N/LQ7KPS/LQ7KPS-s011.vmdk’: Could not send request body: Software caused connection abort (http://subdomain1_alias.dreamhosters.com)
” after transmitting ~2min worth (~25MiB) of this 547,815,424 byte file.[/list][/list]
As suggested, this failure doesn’t occur everywhere for me, but so far only with files over ~.5GiB, and specifically:[list=1]
 This problem has NOT occurred (instead did a checkin with no problem):[list=a]
 On all other checkins within at least that last 2 months, which have been as big as ~126.63MiB (a recent diff on this Outlook folder LM3NRK/).
 On 2011.06.04 (just 2mo ago), a new file of 1.03GiB to this same repository (file LM3NRK/outlook_LCC5FC.ost, so also to another folder).
 Within this particular source folder where I’m now seeing problems, folder LQ7KPS/ : all files from 0 bytes to 461,832,192 bytes(=.43GiB=.46GB) checkin no-problem.[/list]
 Indeed, so far all files having this failure-to-checkin problem have been:[list=a]
 On committing every file 547,815,424 bytes(=.51GiB=.54GB) or larger, where:
 On commits of new files (so on adding files via commit) but not on file diffs (probably since I haven’t had any big diffs over ~.5GiB), though even if the problem is just for adds, it is still a serious problem (as hopefully one wouldn’t have to do a workaround of explicit file splitting & reassembly).
 Within one particular source folder, folder LQ7KPS/ – but I don’t think this folder is the problem for, as mentioned, all is files from 0 to .43GiB HAVE added-via-commit without a problem.[/list]
 As mentioned, the failure occurs not at the start nor end of the transfer, but right after the first 24.13MiB to 87.38MiB.
 When a commit is reattempted, the error occurs not at exactly the same point in the transfer but very close, within +/- .3MiB. This repeatability suggests to me that this problem is not a flaky hardware (or software), but a situation where the (server=recipient) reaches some (likely software) limit (seemingly .5GiB) so reports back to the client to abort, which then immediately aborts within +/- a few packets transmitted.
 The client computer probably doesn’t matter (as happened identically on both a Win7 (Starter & Home Premium)).
 Internet connection doesn’t seem to matter (as happened on office Wi-Fi, office Ethernet-to-router, and Starbucks Wi-Fi)
 Subversion client doesn’t seem to matter (as happened via latest TortoiseSVN and Cygwin svn, using each to do both the add-op and the checkin-op)
 About 5mo ago, I got not the same but similar TortoiseSVN error (only different in the last 1/2 of error message, and only somewhat: “Error: Could not send request: An established connection was aborted by the software \ Error: in your host machine.”) trying to add-via-commit a 2.32GiB file (file X15-65804.iso commit LI983N on 2011.03.18).[/list]
Google Search(SVN Commit failed SVN wrk send request body connection forcibly closed) led me to number of possible fixes for this, such as http://code.google.com/p/support/issues/detail?id=2726 suggesting doing a Subversion Cleanup first, updating the router’s firmware, and trying with a command-line tool, but I’ve done all 3 things (including not only updating the router’s firmware but connecting to the router via Ethernet instead of 802.11g) and none of them seem to improve nor change anything.
To my notable frustration, Google Searching for the error message as I explain above, plus even for its likely cause as I explain above, I have not been able to find any explanations, why then I make this post.
Today or tomorrow I will be asking DreamHost support to reply to this.[list=1]
And if DreamHost indeed does have any Subversion file size limit (such as mere .5GiB), for me at least, that would be[list=a]
 a serious drawback in choosing & using & recommending DreamHost (especially when DreamHost’s inexpensive Subversion hosting was the #1 reason I picked them), and
 seemingly false-advertising as (DreamHost clearly advertises “unlimited” storage including definitely up to 50 GB or GiB, notably here http://dreamhost.com/unlimited-policy ).[/list]
The only size limit which would seem acceptable would be one truly imposed by the software if this-couldn’t be fixed/enlarged by an upgrade, or a limit at the highest practical limit, notably at least enough to distribute (as a single file) the largest common removable media, which currently is (46.6GiB per http://en.wikipedia.org/wiki/Blu-ray_Disc#Physical_media ).
So if DreamHost indeed has such limits, then hopefully they will now:[list=a]  clearly & publically announce any limits, and any limits they did have, including where possible in the error messages when the limit is reached
[*] most especially, remove any practical limits as I describe here.[/list][/list]
But, especially as this would seem to affect anyone and the error message is not clear (and maybe someone will know about it), I also post this here report on the public DreamHost forums, so your thoughts & suggestions appreciated, too.
[hr]Aside: Here (& officially), GiB is 2^30=1024^3 bytes, as defined by http://en.wikipedia.org/wiki/Gibibyte ; so then “GB” is really 2^9=1000^3 bytes, as defined by http://en.wikipedia.org/wiki/Gigabyte . Not knowing of the “Gi” symbol (and in general the “i” insert) until now, until now I (as many people, including MS Windows 7) have miscalled “GiB” (and “MiB, KiB”, etc) as “GB” (and “MB”, “KB” respectively); but since 1024^3 (“Gi”) is not quite the same (slightly larger) than 1000^3 (classic “G”=“Giga”), it makes sense that it has its own symbol “Gi”, and indeed http://en.wikipedia.org/wiki/Blu-ray_Disc#Physical_media alerts me of this, so I will be more careful from now on and not use “G” when I mean “Gi”. Indeed per http://en.wikipedia.org/wiki/Binary_prefix , I see one should insert the “i” (as “Ki, Mi, Gi, Ti” instead of “K, M, G, T” respectively) whenever one really means the binary multiplier 1024^n instead of the decimal multiplier 1000^n.