Defunct php5.cgi processes / 503 Service Temporarily Unavailable


#1

I just started getting a 503 Service Temporarily Unavailable page after a long delay, which from a bit of searching seems to indicate I’m hitting the connections limit. Since my site doesn’t get that many hits, that has to be from hanging processes, and sure enough here’s the output of top:

top - 21:52:37 up 152 days,  6:33,  5 users,  load average: 5.93, 5.31, 5.05
Tasks:  21 total,   1 running,  12 sleeping,   0 stopped,   8 zombie
Cpu(s): 24.8%us,  2.2%sy, 23.2%ni, 49.6%id,  0.0%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  32953548k total, 31792792k used,  1160756k free,   625736k buffers
Swap:  3998716k total,   894276k used,  3104440k free, 17336228k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
27451 mdp       22   2 18992 1304  996 R    0  0.0   0:00.05 top
 4435 mdp       20   0     0    0    0 Z    0  0.0   0:14.75 php5.cgi <defunct>
 5447 mdp       20   0     0    0    0 Z    0  0.0   0:09.67 php5.cgi <defunct>
 5942 mdp       22   2  129m 3916 1028 S    0  0.0   0:02.96 screen
 5943 mdp       22   2  122m 3028 1528 S    0  0.0   0:00.08 bash
 6143 mdp       22   2  134m 4652 2376 S    0  0.0   0:04.62 vim
15050 mdp       22   2  122m 3060 1556 S    0  0.0   0:00.08 bash
15241 mdp       22   2  134m 4428 2128 S    0  0.0   0:07.82 vim
15614 mdp       22   2 74772 1684  900 S    0  0.0   0:01.09 sshd
15615 mdp       22   2  122m 2804 1444 S    0  0.0   0:00.08 bash
15787 mdp       22   2  126m 1108  884 S    0  0.0   0:00.12 screen
18255 mdp       22   2  122m 3044 1536 S    0  0.0   0:00.08 bash
18684 mdp       22   2  134m 4820 2248 S    0  0.0   0:13.77 vim
19954 mdp       20   0     0    0    0 Z    0  0.0   0:12.66 php5.cgi <defunct>
20845 mdp       20   0     0    0    0 Z    0  0.0   0:06.25 php5.cgi <defunct>
21820 mdp       20   0     0    0    0 Z    0  0.0   0:32.84 php5.cgi <defunct>
21870 mdp       20   0     0    0    0 Z    0  0.0   0:09.90 php5.cgi <defunct>
25456 mdp       20   0     0    0    0 Z    0  0.0   0:34.81 php5.cgi <defunct>
26806 mdp       20   0  265m  39m 9688 S    0  0.1   0:02.20 php53.cgi
27292 mdp       22   2  122m 3032 1520 S    0  0.0   0:00.08 bash
30581 mdp       20   0     0    0    0 Z    0  0.0   0:05.89 php5.cgi <defunct>

Thing is, I can’t kill the processes, seems I don’t have the permissions. (The kill command, even with -9, just silently fails.) Anything I can do to fix this? And to keep Wordpress from misbehaving like this in the future?
[hr]
Following up, a couple of other unexpected things: I have two other wordpress sites hosted on the same server (one on a subdomain of the one that’s having problems) and both are working fine. Also, plain html pages load instantly with no problems, but a php file with just phpinfo() gives me a 503.


#2

So another update: upgrading to PHP 5.3 seems to have fixed the problem. Oddly enough my other domains were already configured to run 5.3.
[hr]
Or perhaps I spoke too soon. No 503s (yet), but from top:

 3629 mdp       20   0     0    0    0 Z    0  0.0   0:47.96 php53.cgi <defunct>
 6252 mdp       20   0     0    0    0 Z    0  0.0   0:14.08 php53.cgi <defunct>
 7868 mdp       20   0  284m  57m  10m S    0  0.2   0:24.58 php53.cgi

Any suggestions on how I might diagnose what’s going on?