[quote]if you do exceed this quota the process will be killed anyway, so I
don’t think you run the risk of bringing the server down to it’s
This is true to some degree, but our system is fairly lenient when it can be. A process can severely overload a server before we ‘catch’ it, so we still depend to some degree on the kindness of our customers. As long as everyone realizes that our hosting servers are a shared resource, it’s usually okay.
[quote]A few suggestions if I may: download the files and process them
offline on your servers. After all, you shouldn’t really be using a
production server for this kind of work.
This is the best option, but…
[quote]Or chop up the files into multiple batches and work on smaller
batch files to get your results without consuming too much
…this is probably okay too. Break the file in XX MB chunks, and do the processing in ‘spurts’.
I’m not sure what kind of logs are being processed here, or how large the files will be. If they’re pretty small, this is all really sort of academic.
- Jeff @ DreamHost
- DH Discussion Forum Admin