503 Error / php5.cgi processes?


#1

Hello,

So I’ve had issues with my entire website being down constantly due to 503 error. I got moved to a different server because the one I was on was busy, but still getting the error. I was told by support that one of my sites looks like it is having processes that are being killed.

Well, the site in question is just a simple site, a couple php pages (to pull header/footer) and an email contact form, just a small business site for my mom.

In shell, the ‘top’ command shows that I’ve got 14 processes running, most of them seem to be php5.cgi, so my only guess is that is what is causing the problem?

[php]top - 10:06:17 up 186 days, 21:35, 1 user, load average: 27.31, 29.46, 28.82
Mem: 32959040k total, 32450708k used, 508332k free, 339888k buffers
Swap: 3998712k total, 113572k used, 3885140k free, 17476668k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6244 koneko 30 10 175m 19m 9868 R 0 0.1 2:29.45 php5.cgi
6925 koneko 30 10 182m 26m 9.9m R 0 0.1 2:16.12 php5.cgi
7129 koneko 30 10 180m 24m 9.9m R 0 0.1 2:10.89 php5.cgi
7285 koneko 30 10 181m 25m 9872 R 187 0.1 2:10.40 php5.cgi
7355 koneko 30 10 179m 23m 9.9m S 187 0.1 2:04.86 php5.cgi
7708 koneko 30 10 181m 26m 9.9m R 0 0.1 2:00.78 php5.cgi
9741 koneko 30 10 181m 25m 9.9m S 187 0.1 1:39.71 php5.cgi
9851 koneko 20 0 194m 40m 9196 S 0 0.1 0:02.53 php5.cgi
11492 koneko 20 0 178m 23m 9492 R 187 0.1 0:01.59 php5.cgi
7684 koneko 22 2 18948 1272 984 R 187 0.0 0:01.57 top
31476 koneko 22 2 68448 1580 868 S 0 0.0 0:00.44 sshd
31477 koneko 22 2 121m 2464 1508 S 0 0.0 0:00.23 bash
1358 koneko 22 2 68432 1708 980 S 0 0.0 0:00.03 sshd
1359 koneko 22 2 12384 940 696 S 0 0.0 0:00.03 sftp-server
[/php]

As I watch it, php5.cgi will show then disappear. Any idea how to fix this?

Edit: I checked out my other website and and within about 30seconds I’ll get a ‘Yikes!’ warning telling me a process was killed. So I guess the issue is across all my domains.


#2

There’s something bad happening at DreamHost. Two shared severs in one month have RAID meltdowns, and it’s taken over 2 weeks to partially restore one of them, barker, with no resolution date possible. Both my sites have been mostly down, with occasional periods where they are partially available, but I have major PHP header problems even when they’re “up”. I tried to update some plugins, especially the security plugin I use, and the updates fail with no error being logged. Now, I tried to SSH in to check some files in my home directory and see what the load is, and the SSH session gets the password prompt and stops.

Something bad is happening.


#3

Dreanhost has actually had the RAID meltdown problem for a really long time, but it has seemed to be better this year, but maybe that’s just because they don’t post them on dreamhoststatus.com anymore. One would assume that’s as long as the problems existed (search this forum and the old old posts on dreamhoststatus.com) that they would have come up with a better way to deal with the meltdown but alas it does seem to interrupt everything in a customers life for around 2 weeks (again look at old posts in this forum).


#4

Hm, this has gotten worse for me.

One of my other domains, my most active one, was also having a ton of phpx.cgi processes, so I updated it from php 5.2 to php 5.3 to fix it. When it kicked in I got a TON of php53.cgi processes. I was watching it and I’d have like 120+ processes coming and going.

I just got a notice that my site has been disabled due to the high load it is causing on the server, but I have no idea what is causing all these processes. This all started about 3-4 weeks ago, but I hadn’t changed anything before hand so I’m not really sure how to fix it. This is driving me mad.


#5

I was going through my errors logs and did find a bunch of this:

Invalid argument: mod_fcgid: can’t lock process table in pid 9330 (and other numbers/listings)

Google search says “Any fcgid processes that were active/sleeping before executing graceful apache httpd restart, never gets killed after such restart. They get killed only after hard apache restart. This problem causes fcgid processes to „pile up“after each graceful restart, therefore consuming memory and eventually making the whole system unusable.”

So could my processes buildup crazyness be due to some type of soft restarting that has happened? Though, I can’t restart anything myself.


#6

There are issues at DreamHost with concern to Apache and their response has been to restart processes, but it also looks like your site is being DOS’d to some degree. I’d suggest opening your php files and visually inspecting the code to ensure everything is in order.