Preparing for high bandwidth


#1

My sites are basically PHP code serving data from MySQL, and have been relatively low traffic. I now have a project that anticipates up to 100K hits per day, and perhaps 500K records in the database. My account says I have unlimited storage and unlimited bandwidth; do I need to do anything to prepare for this new project? My boss is suggesting we use Amazon. Thanx in advance for any suggestions.


#2

500k records in the database is not an issue.

100k hits per day may be an issue on a shared server. Are you on VPS? You will need dedicated CPU and memory resources to support it.


#3

Thanks for the reply! I am not on VPS. I see the cheapest dedicated server is $1089/yr, which is cheaper than Amazon. It’s not clear to me whether I can upgrade to a better server if needed during the year; something Amazon touts with their virtual machines. It also looks like I would need to write my own script to perform regular backups, although that should be simple to do.


#4

Do you mean hits as in actual requests or hits as in the more colloquial pages or visitors? If the former, then you might be able to do that on a shared server if your code is efficient enough. I occasionally get 100k hits (requests) and I haven’t had any problems. Granted, an average day is 5-10k.

The biggest issue for me has been the unrelability of the shared database. Sometimes connections and requests take a loooong time. On my laptop, I can get my most DB-intensive pages to load in <500ms, but that jumps up to 2,000 or more occasionally because the DB becomes slow. It’s even gone down for short periods of time which doesn’t work for me because most of the visitors are going to click through a series of pages and can’t stop in the middle. I’ve recently set it up so that everything gets cached from the DB from the first load so that the next 100+ clicks don’t touch it at all.


#5

Hits meaning just requests. In this project I have no public web pages; the requests are coming from an app. The app caches all data when it loads, and then occasionally writes. It only tracks a single record; it’s not like a leaderboard, so it’s only a single record SELECT (or INSERT if not found), or a simple UPDATE. The initial load of the app is already 10 seconds, and I do the request async, so a 2-second delay is no problem at all. The write is also async, so it’s not really a problem (for us) if the response is slow, as long as it’s less than the 10-second timeout.

Still, we really don’t know what the traffic will be like. We did a similar project last year, which the publisher put on Amazon (they already had leased servers), but I don’t have access to the stats.

Unfortunately, I don’t know how to compare a shared server with the dedicated ones. The Wiki mentions limits placed on shared servers (http://wiki.dreamhost.com/Shared_server), but not what they are. Complicating matters is that we expect the most traffic to occur upon release. If we go with Dreamhost, I’m sure we’ll go for a dedicated server, but I don’t know which one.

I suppose no one really knows how much traffic they will get, and always hopes for a lot. I need to ask Dreamhost how fast we could switch to a better server if necessary.


#6

i think switching to VPS from shared is almost instantaneous. at least they sort of promote it that way. i don’t have the funds to know.

i do wonder what DH will say about using a lot of shared resources for something that’s not primarily a website though.


#7

Also I’ll add since in post #3 above you jump right over VPS and into the price of a dedicated server that those are two different products.

In the panel, near the button where you active a VPS appear the words that you will be notified “usually within a day” when the server is provisioned.

In regards to dedicated servers the wiki says "Sometimes, there is a slight delay from the time you order your Dedicated Server account, to the time the server is provisioned, until then, neither Users nor Domains can be created. "


#8

I jumped over VPS because I don’t know much about it.

“Slight delay” or even “within a day” is not too bad, assuming the account is still active during the transition.

I shouldn’t think any provider would care where the requests are coming from, as long as it’s a reasonable bandwidth and they are getting paid. I know lots of developers who use DH for leaderboards and such, although Amazon now offers a turnkey solution for them.

I appreciate all the advice and suggestions!

—MikeB


#9

I recommend that you study up on the VPS, it’s the step between Shared hosting and dedicated server and can be re-sized at will.


#10

You are correct; it looks like VPS is the way to go. I’m going to take advantage of the free week and run some load tests based on our expectations, and see what happens. Thanx!

—MikeB


#11

VPS is looking good. Thanx all!

I now have a server for my domain, and a server for MySQL; is this normal and/or required? Basically it’s twice the cost I thought it’d be.

—MikeB


#12

I believe you can use the shared databases since (I think) the VPS is an add-on to shared hosting. I’ve heard that the opposite is true as well. You can use shared hosting but rent your own personal MySQL server.


#13

Just a quick update: After almost a month, my VPS site has been down 3 times. The latest, today, was for over 2 hours, and still no reply from support. My personal shared account, which I’ve had for years, has never (AFAIK) been down, so in my limited experience, VPS is not the way to go, at least not on DH. I should have checked their status blog first; they are always having issues with VPSs. YMMV, and I hope it does. P.S. My project still has not gone live, so it’s not bandwidth or resource related.