How do I split big files (100, 200mb plus) into smaller parts (50 mb, or 10 mb) on the server?
[quote]12.3 Splitting files into smaller pieces
Sometimes a file is too big to fit on a disk, or you don’t want to send a huge file over the net in a single chunk. You can split the file using the split utility, and reassemble it using the cat (concatenate) utility.
- cd; cp /bin/bash myfile; ls -l myfile
Copy the bash executable to a file in your home directory called myfile. Observe that myfile occupies a little over 400,000 bytes, or around 400 kilobytes.
[b]2. split -b100k myfile myprefix
Splits the file into sections of 100 kilobytes, naming the sections myprefixaa, myprefixab, etc. Type ls -l so see the results.
You can specify any number after the -b: choose one that makes sense. If you leave off the k, it will be understood as bytes instead of kilobytes. If you use m instead of k, it will be understood as megabytes. [/b]
- cat myprefix* > mynewfile
Concatenate all the files and write them to mynewfile. (The * and > are tricks you’ll learn in another chapter .)
- rm myfile mynewfile myprefix*
Remove everything. [/quote]
The short answer:
Use “split” from the command line in the shell (see “man split”) making sure to use “nice” (see “man nice”) to keep the process from getting killed by the DreamHost procwatch program when processing really big files. .