I'm developing with Ruby on Rails and dreamhost. One of the things my app does is parse an uploaded XML file into a database. It works fine for small amounts of XML data. When I send it a 14mb XML file it starts processing but eventually dumps out "rails application failed to start properly". I watched my processes while it is working and dispatch.cgi is chugging through the XML, apparently. Fairly CPU intensive. The dispatch.cgi process gets killed (probably by a dreamhost watcher script) and that's when Rails spits out the error.
Any ideas how I can do this without my dispatch.cgi getting killed? Even if I saved the file and ran an XML processor from the shell I would imagine it would get killed by dreamhost before it could finish.