The Slashdot Effect


#1

I’m just curious. No plans at present or anything.

Can Dreamhost handle a “slashdotting” on a shared server website? As customers, we certainly have the storage space and bandwidth (I think I have 11 TB a month, currently… been a customer for a while) to survive one, but what about CPU utilization and peak bandwidth?


#2

There are a lot of thread, and discussion, about this on these forums and users’ experiences vary greatly. Some have come through just fine while others have been brought down or had to be throttled.

Bottom line: It depends entirely upon your site. A static html site should survive pretty well, and is generally only going to risk running up against too many concurrent connections, though that is less than probable.

A dynamic site, using PHP-CGI and/or MySQL will not fare as well.

It’s all about managing CPU usage and RAM - bandwidth is not lilkely to be an issue. :wink:

–rlparker


#3

Getting slashdotted (or “Dugg”, for those n00bz out there) shouldn’t be a problem for a DreamHoster. If you have a dynamic site and a particular document is getting hammered, it makes sense to create a temporary static version of that document. This will effect a dramatic reduction on resources, particularly with respect to database access. In the case of a “blog page” that includes user comments, it might be prudent to temporarily close the comments while riding out the storm.

si-blog
Max discount on any plan with promocode SCJESSEYTOTAL


#4

This is the key right there. Greater than 90% of your visitors will only read the landing page and if you can make that page as fast a static page as possible, you should be fine.

Even dynamic sites that are on dedicated hosts can have trouble with the extra traffic generated by the slashdot effect.

What are [color=#CC0000]50DISK50[/color], [color=#CC0000]3DOM50[/color], and [color=#CC0000]1IP1DOM50[/color]?
They’re Dreamhost coupons!


#5
This is the key right there. Greater than 90% of your visitors will only read the landing page and if you can make that page as fast a static page as possible, you should be fine.

Even dynamic sites that are on dedicated hosts can have trouble with the extra traffic generated by the slashdot effect.

Indeed; so long as you don’t expect DreamHost to have unlimited CPU and bandwidth-resources and plan ahead accordingly, you should be fine. For instance, it is a very, very bad idea to start thinking about how to do static pages when the slashdotting of a heavily plugged WordPress is already in progress.
If you expect regular traffic surges, it might be a good idea to start thinking about how to automatically generate completely static pages off of your dynamic ones, and how to collect them/update them later on (or when comments are posted or such). You do, after all, have loads of storage.


#6

Agreed. And even if you don’t expect heavy traffic, it’s good practice for all dynamic sites to employ a static caching system.

This is especially so in a shared environment :wink:

Maximum Cash Discount on any plan with MAXCASH


#7

Would WP Cache do the trick for wordpress?

Now taking WP out of the equation, on a completely seperate website which is completely database-less. It uses php with some homemmade SEF url code, and pulls the page content from static pages… the site therefore does use cpu time on the php side of things, but no database calls at all.

On the aforementioned site, how would embedded videos cope if the site is “dotted”? I’m currently creating a series of Joomla tutorial vids which will be hosted on my company website, which uses the homebrew system i mentioned above. If the site received a big traffic surge for the vids, would the server be able to cope?

Cheers,
Karl

web design, development, pay per click marketing (PPC) & Search Engine Optimisation (SEO) by DigitalVibe


#8

I believe rlparker stands firmly behind wp-super-cache.

I’d suggest database calls are the bottleneck in any environment, shared or otherwise. Considering you’ve taken them completely out of the equation with static pages is the best step toward stability. Having said that, there is still the quantity of hits generated by a slahdot to consider as any server getting 10k hits a second is going to meltdown fairly quickly and require load balancing.

With static content it depends on the ammount of hits rather than excessive script processing overhead and queued database calls. I haven’t tested DH at all for concurrent connections piping large files but from what I’ve read in this forum you can have a couple hundred concurrents and remain viable. There was also a note somewhere that DH users have survived an occassional slashdot.

I’d reckon your video site would run okay.

Post a link for us when it’s complete! :wink:

Maximum Cash Discount on any plan with MAXCASH


#9

I recommend reading about other people’s experiences with traffic spikes.

I think the fact that the server can handle a couple hundred concurrent connections doesn’t mean that it can handle a couple hundred new connections every minute. If old requests aren’t being completed faster than new ones are being requested, no server is going to be able to handle the traffic.

For video streaming, I’d recommend making sure that the video traffic got handled by a specialized video streaming server like the Darwin Streaming Server (Quicktime) offered here. Now I don’t know whether the setup of Darwin will actually help in traffic spikes because I don’t know that much about whether it will “hook into” the multicast capabilities of the server when it’s not a live broadcast. Perhaps some real expert can opine about this?

What are [color=#CC0000]50DISK50[/color], [color=#CC0000]3DOM50[/color], and [color=#CC0000]1IP1DOM50[/color]?
They’re Dreamhost coupons!


#10

It would certainly help, but I think wp-super-cache would do a lot better! :wink:

I’m not at all sure what the correct answer is for that question, though I suspect that it could be a challenging situation.

A possible strategy for dealing with a video that gets “slashdotted” might be to stick that puppy on Youtube for the duration of the traffic spike and just embedd it from there for a while? You can always delete it later from YouTube to keep it “exclusive” to your site (after you have used YouTube’s infrastructure to get you over the traffic spike!) :slight_smile:

–rlparker


#11

Oops! That’s what I get for responding when reading “last 24 hours” instead of “threaded” or “flat” style, and seeing all repliles before I respond.

“What you said!” - I’m sorry for just repeating your answer.

–rlparker


#12

Re-affirmation is good. I’m just glad I didn’t namedrop you and get it wrong!

I don’t use WP myself, but I read and (try to) remember the Guru-standard advice I come across in this forum and others. After a recent thread on this concerning a DH user who was slashdotted I now suggest to everyone that they consider implementing wp-super-cache right from the get-go if questioned about WP installation and stability :wink:

Maximum Cash Discount on any plan with MAXCASH


#13

sXi, rl, thanks for the wp-super-cache suggestion, i’ll get that installed now :slight_smile:

We were are all helping each other out a lot these last couple of days! Hehe.

Cheers,
Karl

web design, development, pay per click marketing (PPC) & Search Engine Optimisation (SEO) by DigitalVibe