Your web server is just that - a web server. DreamHost expects it to be doing work related to making a web site work. A web server gets a request from a browser and sends a response back as quick as possible. As a result that kind of work shouldn’t take a long time. Now the problem is that if it does take a long time, and the server gets more requests, the machine reaches capacity and no more requests can be handled. And then the 20 - 30 customers DreamHost has using that machine have their web sites go down. Bummer!
Now I’m sure you don’t want to make 30 customers unhappy anymore than DreamHost does, so they have a process killer that checks how long a web server is handling a request, and if it takes to long, poof. Besides that, theres only so much memory to go around as well, and you can’t do work with out memory. PHP alone takes up 100 megabytes of RAM.
Anyways, you’re utility in this regard is not doing web site work. It is reading dozens of text files and shoveling that data off to a database server. And thats a lot of data and thus a lot of time. But wait, its being run by the web server, so its subject to the process killer. Ooops.
So the obvious solution is to avoid running it under the web server and its process killer. So you can login to a shell account or use your own computer. Splitting the workload into small batches might also work if you absolutely must run it under the web server, but thats complicated to setup.
Customer since 2000 openvein.org