Wordpress 404 errors: Dreamhost killing processes?

When I migrated my blog over to Dreamhost about a month ago, I began to get random 404 errors while doing functions in the Wordpress Admin area. Sometimes, even saving a post will being me to a 404 page. After searching around the Wordpress forum for answers, I found that a number of people hosting WP on Dreamhost have had this problem, and that it relates somehow to Dreamhost killing scripts automatically due to memory issues. Apparently this happens even to fresh WP installs with no plugins.

Anyone know anything about this and what can fix it?

When I was having problems with scripts being killed, I wasn’t getting 404ed; I was getting 500 server errors.

I have seen an occasional instance where invoking a WP function will bring up a 404, which I believe to be simply the result of a timeout. To my knowledge, I haven’t had to have a script put to death in about a year.

I got in touch with DH support about this and they said that something was pushing their memory limits and so they were killing processes. This is a weird Dreamhost policy thing, because I’ve never experienced this before on other hosts for my blog. Anyway, I’ve been trying to reduce the load by paring down the number of plugins my wordpress install uses. All in all, an annoying situation. The tech support rep did note that in my situation it was “admin-ajax.php” that was pushing my memory limits. I have some admin-related plugins, so I’ve started getting rid of superfluous ones.

The more time passes, the more serious this problem is. I believe that somehow DreamHost has been lowering the memory thresholds more and more, until at this stage even a fresh, clean install of WordPress will almost always give 404 or 500 errors, even with zero plugins installed. From “mildly annoying” (last year) it has begun to become “a constant nightmare”. Well, I use CloudFlare on top of everything, which will minimise the 404/500 errors somewhat (since 40-60% of all traffic will come from CloudFlare and not hit DH’s Apache webservers), but during PST “prime time”, it’s almost impossible to completely avoid all those errors.

I wonder if this gets worse with PHP 5.3 FastCGI. Some tests I’ve been doing show that WordPress with PHP 5.3 FastCGI and Page Optimisation almost always use close to 256 MB of RAM, while other software — like, say, phpBB or even Joomla — under PHP 5.2 (not fast) consume far less, e.g. 100 MB or so. Comparing different software is hardly fair, but I wonder if there are some recommended settings to keep WP memory consumption low. And please, don’t simply advise “delete plugins” or “use simpler themes”. As said, I have been getting 404/500 errors with a fresh WP install with just the TwentyEleven theme.

I wonder if turning Page Optimisation or Extra Web Security off will consume less memory? Also, caches reduce bandwidth and CPU, but sometimes at the expense of using more RAM. Has anyone made a few tests to see what combination works best?

Well, now that I’m seeing this more and more, I’m way more convinced that they’ve ratcheted this down in the last few months.

It’s time to move WordPress off Dreamhost. Everything else seems to work, but WordPress and Gallery2 just are not compatible here. They both fail miserably.

I’ve tried killing FastCGI, as some have suggested that it spawns lots of processes that don’t terminate themselves as fast, thus driving up the memory loads. I know someone will probably say that the opposite is happening, and it’s worse now. Point is, it’s not any better either way, so whats the use?

Sad, that nearly every other host provider uses cPanel, the DH panel is much nicer.

Just for the record, this is what I used to fix this problem forever: http://discussion.dreamhost.com/thread-131416-post-142854.html#pid142854

The shortlist:

  • Move all your WP sites to PHP 5.3 (NOT FastCGI!!) — this is less efficient than FastCGI, but consumes half the memory
  • Uncheck Web optimization, secure Web, etc. which consume a lot of memory and some CPU. This will make the site more open to attacks, but see below.
  • Install the W3 Total Cache plugin (this will increase performance due to very good caching and lots of aggressive optimisation of CSS, JS, etc.) or at least the more popular and less feature-rich WP SuperCache (easier to configure than W3TC)
  • Sign up with CloudFlare.com as a free, simple-to-use CDN. This will distribute your static content across 12 data servers spread around the world and eliminate some W3 Total Cache will communicate with CloudFlare as well. WP SuperCache won’t, but you can install the CloudFlare plugin which will do pretty much the same. And it provides far more security than DH’s solution and another level of optimisation on top of what the cache plugins can offer — all running on their servers, of course.