Zorg/randy/aiai SLOW

Anyone else on this server/cluster/MySQL combination having problems? I host a couple of phpBB forums, and they have been horribly slow for the past few days.

The problem actually started on May 26th when Apache on zorg “became unresponsive” (according to DreamHost support). I assume they restarted the server to fix that. Then it was good for about 5 days, until the randy cluster issue on June 2nd that affected a lot of people. Ever since then my sites have been intermittently (but consistently) slow. Even simple Linux commands like uptime, ls, mkdir, etc. sometimes respond super-slowly when I’m ssh’d to zorg.

In the past I have been suspicious of MySQL when my forums are slow… but this time evidence suggests a file system / IO issue. I have seen load averages (as reported by uptime and top) of around 8 or higher (zorg is a quad-core machine, I believe), yet CPU usage is not maxed… From my research, this could indicate an NFS problem, but I am just guessing.

DreamHost support has been unable to fix (or even reproduce, apparently) this problem in over 48 hours, despite much effort on my part to assist them. I’ve been a DreamHost customer now for almost 4 years, and this is not the first time I’ve had such frustration with performance. I would really love to be moved off of the zorg server, as I’ve read a lot of complaints about it. If these issues aren’t resolved, I’m sad to say I will be looking for a new host.

Well, they moved me from zorg to ultima, but I’m experiencing the same intermittent slowness still. zorg and ultima are both on the randy cluster, and I think this is an important clue. It seems that the network file server being used by this cluster is severely overloaded, or is having some difficulty.

I had a friend write a simple Ruby script to test with, which creates 100 empty directories and then reports how much time it took. On a local drive (e.g., /tmp) it takes a second or less. On a network drive it can take multiple minutes!

Just using the shell can be a painful experience it is so unresponsive sometimes.

Is anyone else experiencing this?