Anyone have any suggestions on how I can run my Java process without it being killed off after < 10 seconds?
The process basically downloads ~100 rss feeds (just the xml files), analyzes them, then puts the results into a mysql database. The process needs to run every 6 hours or so. It takes ~10 minutes to complete (on a non-dreamhost server where it doesn't get killed) but most of that time is spent waiting for the HTTP responses.
Here are the things I have already done to try to make my process less offensive. But it still gets killed:
Run with "nice -n+20 java ......"
Run java w/ low memory options (e.g. -Xmx8m).
Make the process single threaded (i.e. it checks the rss feeds serially so as not to use too much socket or cpu resources at one time. This obviously makes the process take a lot longer to complete but that doesn't matter to me).
Optimize my calls to MySQL (e.g. re-using connection objects, minimizing the number of insert statements, locking tables and disabling keys when doing many inserts).
I even tried to put a few calls to Thread.sleep(10) around within a few loops to further reduces CPU usage.
Nothing has helped. The process still gets killed almost immediately.
Anybody have other ideas to try? All guidance and suggestions greatly appreciated!