This is a long one, so skip to the bottom if you just want to see the question and not the sad story that precedes it.
One week ago, we switched our Wordpress site to Dreampress, and we’ve had nothing but headaches since. While the site initially ran nice and speedy, on the first day it was live, Apache crashed. We asked the support web chat folks to restart it, which they did. A few hours later, it crashed again. Then again the next morning. Then again the next afternoon. Every 6-8 hours, it seemed, Apache would crash, unrelated to traffic or time of day.
With the help of both web chat support and ticket responses, we did a ton of things to optimize the site: tinkered with .htaccess and robots.txt to slow or block bad bots, deleted unneeded plugins (and even some needed but not vital ones), updated everything to the latest version, etc. Still the Apache crashes continued. A ticket reply said that Dreamhost’s “WordPress ninja” had determined that though traffic was high, it wasn’t causing the problems.
Finally, a support chat tech suggested that the problem was something with our site conflicting with Varnish, and said maybe we should switch to VPS+MySQL VPS instead, since that doesn’t run Varnish. He said our CPU and RAM loads looked low enough that we would run fine on VPS. Sure, worth a try!
I switched the site over last night. Since then, the site has been virtually inaccessible: Mostly 503 errors, though occasionally I can get through after long, long lag. A different chat support tech suggested that the problem is that VPS doesn’t scale like Dreampress, so it can’t handle the traffic; yet another tech said that traffic doesn’t seem to be the problem.
I have my umpteenth ticket filed now, but I’m wondering from both any Dreamhost folks reading this or Dreamhost users: Any idea what we should be pursuing here? This is a site that gets about 10k non-bot pageviews a day, so we shouldn’t be crashing like this. Our WP install doesn’t use any crazy plugins — the theme is Genesis (needlessly complex, but it predates me and we don’t want to do a full site redesign), so that requires a few plugins of its own, but they all seem okay, and ran fine on our old host. Is Varnish that big a conflict problem that we should avoid it? Is VPS such a big non-scalability problem that we should avoid it? Is Dreamhost just a bad solution for us? We’ve been very happy with the control panel usability and the support staff, notwithstanding the fact that the site keeps going down…