I sometimes have the 503 errors with my Gallery2 installation and my logs indicate memory problems in those cases.
Do you host your domains with different users or do you use one user for all your domains? I recently started moving the more resource intensive domains to different users because AFAIK the limit should only be applied to the current (shell) users which is the owner of the PHP process. So splitting up your domains with different users should spread the load.
However since you are mentioning security problems I would think that someone might have placed a script on your server (Perl, PHP, Python, Shell...) which might be running a lot. Are you sure that you deleted all files and replaced them with backups/original files? Maybe you still have a harmful script somewhere on your account which is hogging all the resources? Did you check all the directories, logs? Check the apache logs for any calls to PHP scripts which look suspicious.
Promo-Code: [color=#CC0000]SAVEMONEY97[/color] - Save [color=#CC0000]$50[/color] on your first year of hosting.
Get more promo codes here