Upgrade to VPS needed or is a problem I'm noto understanding?

Note: total rookie here so please use small words accompanied by hand gestures.

So I have a little site and traffic has been growing steadily (15-20%/month) to 70+K unique visits and about 250K page views a day on shared hosting running on WP (min plugins). So when the tech guys explained that my site was crashing constantly because I had outgrown shared hosting I figured - OK that must be true. So I’m on my 1 week upgrade to VPS which is confusing to say the least.

According to my VPS usage graph I would need min=600MB of allotment now (if traffic continues to grow I figure I would need to go higher or I would be constantly dropping visits that went OVER 600 MB right)?

But I’m still confused about how/why I outgrew shared hosting. I frankly don’t understand the server logs or what they’re supposed to tell me. But when I look I see this (screen capture from daily log) with these big RED BARS which look alarming. Unsurprisingly these coincide with when the site was crashing constantly.

So my very long question is - did I literally outgrow my hosting package? Or did something strange happen those days that made it LOOK like I outgrew my package? Yes I’ve read all the troubleshooting wikis which are not helpful/way beyond my ability to comprehend.

Anybody have a similar experience? All input/advice is warmly welcome. Thanks a bunch.


Well first up, if you’re a rookie you need to be teaching the pros how to get hits!

600MB doesn’t seem too high, but I have to ask: Are you using any caching at all ?

Again, congrats on your site’s success! :slight_smile:

You have (inactive) WP Super Cache. I’d use Total Cache (though that will bump your memory a little).

I’d switch to PHP 5.3.x FastCGI as well, which will lower your memory.

WOOPS! Typo - So I have a little site and traffic has been growing steadily (15-20%/month) to 70+K unique visits and about 250K page views a day

That’s a MONTH (see not so impressive - 5-6K a day).

Will try the caching plugin you suggest (I disabled Total Cache it as I thought it was interfering with something else). Not entirely sure how to upgrade the PHP I’m running but will google it. (Sorry, I’m about 10 steps below the types who normally post in this forum).

Thanks for the suggestion!

To upgrade the PHP environment:

Panel > Domains > Manage Domains

Press the Edit button next to troublesometots.com

On the next page you’ll see a blue heading “Web Options” with “PHP Mode” directly beneath it.
Just select PHP 5.3.x FastCGI using the dropdown list.

Once selected, scroll down a bit further (just past the HUGE Google Apps button) and click Change Settings.

It’ll automatically update the environment to PHP 5.3 in 5-10 minutes.

I have a site on shared hosting that gets 250k page views per month if you count ajax requests. I count them as page views because it’s not a request for static content; fulfilling the request requires the same framework as rendering a normal page.

Anyway, the point is, you can easily accommodate > 250k page views per month if you have optimised your site. Use caching, move your static assets to a cookie-less subdomain (or even better, another domain altogether and put it behind cloudflare or similar), and benchmark your setup with FCGI vs regular PHP (in my case, FCGI is actually worse). It takes time and knowledge to make these adjustments.

You may want to hire a freelancer to help you if you’re not sure how or don’t have the time to learn how to do it. It’s beyond the scope of the replies you’ll get on a forum.

Enable it and under Advanced choose the Use mod_rewrite to serve cache files. option, then scroll down and hit the Update Status button.
You could toy with the other options if you like, but just doing the above should give you good results for your site in it’s present state and load.

I’m unsure if wp-cron is still an impact these days, and at this stage it might be negligible burden anyway. Ipstenu would be able to give you the good oil regarding that. (pssst… She’s actually a Guru over at WordPress.org too!)

Cron. I want to say “It’s really not an issue” but the truth is that it can be an issue. If you get a huge amount of traffic (and I’m talking like as much as dreamhost.com would get in a day), then cron may get hammered like mad. For most of us, 250k page views per month will run with minimal effort on our part. Just the basic cache and we’re done. 250K page views a day though is getting up into awesomeland :wink: Glad it was a typo, cause that totally needs a VPS!

bobocat is right. 250 a month should be totally doable on shared, if you optimize things. Now, in some cases, it’s going to be better on VPS simply because throwing more memory at things will help, but that really depends on the nature of the site. How dynamic is it? I have a heavily dynamic site that no amount of CDN/caching would help as much as 1G of memory. So my wee 100k/month site is on a VPS, along with two other 50k/month sites, and the 25k ones. It does all add up, and while four of the six would still be fine on shared, that one justified the rest. Juggling the PHP versions is also a good idea. FastCGI uses more memory than, say DSO (which you can install yourself, it’s not standard here), but DSO has file writing shenanigans with uploads (it saves things as nobody, go figure).

It’s all (as you can see) a lot of ‘If this, then that’ and terribly subjective :confused:

Outgrowing shared on a 250k/month site isn’t out of the ordinary, though. In fact, it’s more the norm that being able to stay on shared with that. Caching will help a ton, as everyone’s said. W3TC is better on a VPS, IMO, since you can have it minify and compress, AND use more dynamic caching like xcache.

(I’m a Half-Elf Support Rogue on WP.org :wink: That’s how I got the gig here being your extra special WP Guru)

So, just to follow up on this idea, it’s possible to calculate approximately how many pageviews per day/week/month a site could receive before outgrowing a shared server. This is rarely discussed because the whole CPU resource concept is difficult to measure and most people don’t think about it when they see that the disk space, bandwidth, etc is all unlimited.

According to the wiki, a customer can use up to around 300 CPU minutes without causing problems. It’s not clear whether it’s per customer or per user, but the first revision of that page states per customer, so we’ll go with that.

If you turn on resource reporting, you can get a feel for how many minutes or seconds your users consume each day and use that with your pageview stats to calculate how many CPU seconds are consumed by an average page request. If your site is totally static, it would be just a few milliseconds per page. In fact, it’s such a small amount that it won’t even show up on the resource usage reports! An efficient dynamic site, however, may take anywhere from 20—200ms per page. WordPress or other behemoths may take much more than that just for a single page.

So let’s say that, on average, your dynamic site consumes 171ms of CPU time per request (personal example). Assuming demand for your site was relatively constant throughout the day and you managed to stay within memory limits, you could get away with about:
[]pageviews per day: 18,000
]pageviews per week: 126,000
[*]pageviews per month: 546,000

As Ipstenu noted, 250k / month is doable on shared hosting, but based on these simple calculations, doubling that might be pushing the limits of shared hosting.

That’s humbling for me. My busiest month was 296k pageviews, so I either need to optimise more if I want to expand or start monetising my site!

One thing that’s confusing is that the CPU resources consumed seems to depend on the type of server that you are on. I was moved to a different server about half way through 2012 and noticed that, for the same traffic, the amount of resources my site was consuming increased. It increased by a lot. ≈­4× to be precise. I assume it’s because my ‘new’ server was actually an older, less capable server with less RAM and fewer CPUs, but it does make one wonder if one can get away with more pageviews by being placed on a newer server.

DH used to talk about shifting popular sites to relatively quiet servers in an effort to keep customers on shared plans from running out of resources. I wonder, out loud, if this strategy is still official policy.

That’s exactly why the metrics are a great start (which they are!) but only that. Caching can lower CPU but increase memory (see w3tc), and more efficient PHP can do the opposite (that’s usually where I start to swear). You can’t have super dynamic, low cpu, and low memory, without some serious tradeoffs (like CDNs and caching). FWIW I can get a WP site down to 0.176s for page generation with caching + server optimization, and keep my dynamism, and that is still slower than how fast I can make a mostly static MediaWiki, which is again slower than the static HTML with no external images/css. And yet I still need a gig of memory to do everything.

Oh that’s always true, and it’s not as confusing as you may think. Take the example of your personal computer. If you have a 2008 MacBook and you speed-test it against a 2012 MacBook, you’ll see the newer one is much faster. This is not all due to hardware (though that helps). Software is always being optimized, and while yes, the newer OSs tend to swallow more memory, they also share a lot better and do a lot more. I like to tell people that every server is different, no matter how hard we try to make them very similar, and upgrading will never be ‘as good’ as moving to a new server.

We do a lot of load balancing and I’ve see us shuttle users around because sometimes you do end up on a noisy neighbor server (we try to get the neighbor somewhere else where they can’t hurt everyone else!) but sometimes it’s just a case of which sites are on the server that’s a perfect storm of ‘Oops, my bad.’

I’ll quietly point out at this stage that the above 2 posters are incessant tweakers Gurus of optimisation and that the average user who installs a CMS and happily slaps in a billion plug-ins should probably not assume they can serve 500k pageviews per month on a shared server without encroaching upon a limit here and there :stuck_out_tongue:

Hmm, I recall spending about 3 days (~18 hours per day) tweaking the framework I use to shave a whopping 20ms off of my pageview load times. With that time savings and around 300k pageviews per month, the time total time saved will be greater than the time invested after a mere 28.5 months!

I would say that puts me in the incessant novice optimiser category.

Fortunately for me, my app uses a mere 12-15MB per request rather than WordPress’s 30+.

Outgrowing shared on a 250k/month site isn’t out of the ordinary, though. In fact, it’s more the norm that being able to stay on shared with that. Caching will help a ton, as everyone’s said. W3TC is better on a VPS, IMO, since you can have it minify and compress, AND use more dynamic caching like xcache.

OK - I’m going with you on this because clearly you know way more than I am, a mere human with no elf-like capabilities.

But I’m beyond confused. Here’s where I’m at:

  1. Serving up ~300 page views/month or relative static content, appropriately sized images, decent WP theme (no unnecessary garbage in there), using wpsmushit and wpsupercache. Have minimal plugins.

[Had technical friend look over things to make sure I wasn’t doing anything stupid and he found nothing amiss in my setup or files.]

  1. Mystery server crash seemed to come out of nowhere and coincided to the DAY when Dreamhost was moving my servers from CA to NY. Although support swears this was coincidence. Of course nothing has worked well since so my non-technical non-elf-like-brain is still hung up on this fact.

  2. Upgraded to VPS with 818MB/month (for $40/month - OUCH!) but figured that there was NO way I could be using more memory than that and I could gently downsize over time once I got things sorted out.

  3. Um…apparently no. Because I get this automatic error/restart message about 1X a week.
    Nobody likes to hear that they’ve packed on a few pounds, but we wanted you to hear it from a friend.

Your DreamHost VPS, ps150501, has just exceeded the memory allocation that you’ve established. If left unchecked that behavior could begin to negatively impact the VPS services of every other customer on your server.

As a result we’ve had to reboot your VPS, effectively restarting your virtual machine. You may experience several minutes of downtime while your services come back online.

  1. Also when I look at my VPS usage graph I never see anything above spikes in the 500 MB range (well below my 818 limit) so never see why I’m crashing/force rebooting my VPS.

So I can’t see anything that would suggest why I CAN’T run on shared hosting or worse, why I seem to be blowing out my very generous 818 MB VPS hosting. bangs head on table

Lemme guess: it’s a WordPress site, right?

And when you say 300 page views / month, you really mean 300k, right?

I don’t know WordPress well, but from the little experience I’ve had with it, it’s memory and resource intensive. If you get 5-10 simultaneous requests, then you’ll easily beyond the limits of shared.

There’s no simple answer for all cases. It depends on the site, how users are interacting with it, traffic patterns, etc.

I know it’s possible to deal with 100-200 requests per minute on shared (equivalent to 4m - 8m requests / month) on shared hosting for short periods of time with an extremely fast and light footprint framework (10MB memory per request @ 50ms per request), but that’s not going to happen on a highly interactive, dynamic WordPress site.

It’s all about memory and CPU limits, the parts that hosts don’t like to talk about in their promotional offers and hide away in their ToS, if they even mention them at all (DreamHost refuses to specify exactly how much memory can be used on shared although it’s widely known to be about 100 MB).

And here I am with a site that gets on WP less than 1,000 hits a day and I moved to VPS because of 500 errors and now even with cache, less than 10 plugs ins and reducing load as much as I can, I’m looking at needing 1.5 GB or more to support my site.

sighs and continues to try and fix things without getting anywhere really

I’d love it if someone from DH like the resident WP guru here could help

[quote=“TeraS, post:15, topic:58990”]
And here I am with a site that gets on WP less than 1,000 hits a day and I moved to VPS because of 500 errors and now even with cache, less than 10 plugs ins and reducing load as much as I can, I’m looking at needing 1.5 GB or more to support my site.[/quote]

1,000 hits = 1,000 unique visitors? 1,000 unique page views? 1,000 http requests including static assets?

I have a hard time believing you’d need a VPS for almost any of those definitions. 1,000 unique pages views is < 1 / minute on average. But it depends on your traffic profile. Are they all coming at the same time? Is it highly interactive (where caching is less effective). Are you static assets on another (sub)domain / user? How many actual requests to the server are made to render a single page?

1,000 hits = 1,000 unique visitors? = YES
1,000 unique page views? = YES
1,000 http requests including static assets? = YES

Are they all coming at the same time? = NO
Is it highly interactive (where caching is less effective) = NO
Are you static assets on another (sub)domain / user? = NO
How many actual requests to the server are made to render a single page? = I do not know that answer at the moment.

Here’s the other thing, when posting a new post, server load jumps to 15 every time.

I am running jetpack for stats only, I turned everything else off.

I even turned off my antispam measures because it would shove load through the ceiling as well.

I <3 DreamHost, but it sounds like you guys are paying more than you should be.

I don’t mind paying. We paid for five years of hosting on our last renewal last year, and we’ve been with DH for seven years now. We’ve always have had wonderful support from DH and it’s the main reason we stay here.

I know, somehow, this can be resolved… just don’t know how.

Yeah I think so too. Is there a hosting company that is for people who need technical handholding? I need some technical handholding. Or that guy who is a master of optimization. Where do I find that guy?