I have a website which I host on a dedicated server at a company other than DreamHost. I would like to use shared hosting on DreamHost to serve the static media for the site because of DreamHost’s significantly larger bandwidth and storage offering.
My cumulative file size and bandwidth usage would be well within the monthly DreamHost allowances, but their language in the End User License Agreement in regards to dissallowing “severely CPU intensive CGI scripts” gives me pause. As I say, I will only be serving static files, but I’ll be using rsync to syncronize the files from another server perhaps once a day. Since there are about 500,000 files, the rsync calculation could take a few minutes, which I’m sure would be a few minutes of a pegged CPU.
Does this sound like it could be a problem, or do you think this would be acceptable usage?
I’ll suggest you to contact DH support for best answers.
As a customer, I’ll suggest you to consider at least a Dreamhost PS. If you are on a shared server, a persistent process which may affect the server will be killed by DH. And this does not happen on Dreamhost PS.
There’s still the 97 Day guarantee, so if it’s too CPU intensive, you can probably get a refund, but you’ll have to ask Sales for sure.
Many of us use rsync to back up our data on a regular basis. The total number of files I back up via rsync is about 350,000, and I’ve never heard any complaints from DreamHost. A few minutes of heavy lifting shouldn’t raise too many eyebrows. The servers have multiple processors (four, I think), so if you’re running just one Rsync process, I’m thinking it’ll only hog one CPU.
You may be able to reach an agreement on a more personal level (email support), but do take note that the account could be closed at any given moment if you haven’t clarified with DH your intended purposes of the account “up front”.
You also need to be careful with the “bandwidth allowance”. Even if you only serve static files and are within the limits, your domain may still be disabled for impacting other users on the server.
As for rsyncing 500k files : I’d look for a better alternative. It should not be necessary to run rsync over 500k files just to update a few dozen (or even hundred). You could, for instance, do a find on the files on the originating server, only selecting files that are less than X hours old (modification or change time) and feed those to rsync in a list (or even just scp) ; this would still incurr the calculation on your server, but not dreamhosts. You could do an rsync monthly just to make sure the two directories stay in sync (the method using find would not DELETE files on the target side if they were deleted on the source side – you’d have to account for that by other means).
I wouldn’t worry about bandwidth allowance. They give gobs of bandwidth, and it still sounds like your only concern is CPU usage for the rsync, rather than users hammering the server for data.
I just ran rsync on a site with 50,000 files. CPU usage on the compare portion of the sync hovered around 2-3 percent with occasional spikes to 5%. During download, CPU went high during compression on some files (I use the -z flag), but hovered between 5 and 10 percent.
If you want to be extra nice, you might ask support when the low-use hours typically are, then schedule your sync for then.
TOS says: “primarily for the purpose of hosting a website…”
The OP is serving the actual website from a dedicated server at another company and wants an account purely for data storage and bandwidth purposes. It’s widely known that bulk hosts such as DH frown upon resources being used for this purpose. They’re webhosting companies, not file servers. I’ll suggest again it would be in his best interests to contact support first - lest he finds his 500k files >/dev/null one morning due to an industrywide default TOS clause.
Assuming the files are otherwise legitimate in terms of licensing, does this mean that files that exist purely for the purposes of hotlinking are against the ToS? If so, would it legitimize the files by having the OP host a subdomain for access to them?
BTW, as a question to the OP, if you like Dreamhost’s space and bandwidth limitations and need a dedicated server, why not just move your whole dedicated server over?
Like the OP’s original question, those would be questions only a Dreamhost official could answer with any authority on the matter. I’m just hinting at the likely reason that particular phrase is found in the TOS here (as it does with every other bulkhost on the Internet) and suggesting he go straight to the top in order to clarify whether he can do as he plans with an “official okay” before proceeding.
I agree that is a reasonable price. I also agree that the OP is better off talking to DreamHost directly over this, as there is considerable ambiguity in that TOS clause.
I mean, if the files are being stored to be served by a website (even on another host) that part of the TOS seems to be met.
That said, the heavy rsyncing may very well be problematic form a CPU usage standpoint; then again - maybe not.
I suspect the “bottom line” will end up being the effect on the server’s performance, not the actual storage or the bandwidth and I wouldn’t be surprised at all if DreamHost were to look at it that way.
I think this in one where it is easier to ask for (get confirmation of) permission rather than relying on forgiveness.
Thanks for all your feedback. I contacted support directly just a few minutes ago and Mike S responded (very quickly) that it may cause a problem, but that I should make use of their 97 day moneyback guarantee, and sent me some additional pointers, including an offer to check the process’s resource usage after I run it. Great idea!
To address specific questions:
You are correct. I am not putting these files up to be archived, which I see is specifically disallowed. They are being served, and not just served to a single company, but to a large group of disparate people. They are accessible to anyone with an internet connection.
I don’t imagine this would be a problem because I’ll be well within the limits. Your other comment about using “find” on the local files could potentially be very useful if rsync is killed for being too resource intensive.
Lots of websites pull from multiple sources these days. I’m not sure that having the html come from a different place than the media makes the place with the html the “website” and the place with the media a “file server”. Html files are also files which are served. Seems like an even split to me. I was also thinking of hosting the css file on DreamHost, too, which makes the distinction even more blurry. For a technical service to say “I’m not going to define what a website is, but I know one when I see one” is, in the long run, untenable. Not that it should matter, but these files are not what some people in the hosting world might think of as “files” as opposed to website media. They are .jpg images. So, it’s not like I want to use DreamHost to host my company’s Microsoft Office documents, which I think might be the concern.
The files in question are legit, and are being served to the public, not being used as an archive.
We already have a dedicated server which we get for free (because this is a volunteer gig for a local charity). Unfortunately the server is on a really slow connection, so although it can calculate and serve small amounts of html relatively quickly, it doesn’t have the bandwidth to serve thumbnails or full size photos without the internet connection getting bogged down.
Well that was a fun experiment while it lasted. My account was disabled when I woke up this morning. Can’t ssh into the server, and the files are not accessible from http.
I hadn’t actually run the rsync script in the past day or so, so my guess is that it was the actual serving of the photos which made them decide to shut the site down.
I think the media may have been just a bit too much for the shared hosting plan. I’m dissapointed, but not too surprised, given the tentative responses from people in this thread and from support.
Thank you all for your input, and thank you Lensman for the link to the non-profit information. I think I’ll be hosting the media elsewhere, though. I can’t afford the time to set up another account only to have it potentially fail again in the same way.
Turns out they hadn’t disabled the account… the server was just offline for a little while. I didn’t discover that I had jumped to conclusions until I had already started making the move to another service.
Still, the fact that the server was offline on the third day I was using it didn’t really boost my confidence with DreamHost. (I’m sure it wasn’t a connectivity issue on my end because I heard about it from numerous people who weren’t able to see the photos)
So in the end, I’ll part ways with DreamHost, but do it California style. A “no fault” divorce… things just didn’t work out between us.