Does anyone use DH as a backup device?

Does anyone know if there are file size limits on DH? In other words, if I’m patient enough, can I upload a 40G file, or would it have to be broken up into smaller chunks (like 4G)?

Bob

I always get something close to 150KB/s when downloading from Dreamhost, but that is the limit of my internet connection.

Pay just [color=#CC0000]$22.40[/color] for 1 year Dreamhost hosting, use promo-code [color=#CC0000]MOJO97[/color]
Save [color=#CC0000]$97[/color] on all 1 and 2 year plans!! (Max saving on monthly plans too!!)

In case anyone else wants to know, here’s the response from DH support:

“While we don’t have any inherent filesize limits, I think it would be
practically better to break your files into smaller, less than 2GB
pieces. This is because our FTP servers do not support upload resuming,
so you would best be off uploading your data in small pieces.”

Bob

I am now using DH as my offsite backup, including encryption.

This is a bit long, but here is what I am doing.

Our home LAN consists of 4 PCs (my Laptop, my wife’s laptop, my daughters’ computer and my MAME machine). I also have 3 special purpose servers: (1 IPCop firewall, 1 Linux box running Firefly media server and 1 XP Pro box that has been hacked to support RAID 5 and is the main file server for the house).

The only place that there is a need for automated backups is of the 2 laptops and here is what I have done:

The XP Pro “server” is running TrueCrypt and is sharing out a 50GB volume to the LAN.

Also running on the server is DeltaCopy Server (a freeware rSync implementation for backup on Windows). Each night (in the middle of the night) the DeltaCopy client on each laptop syncs their data to a folder on the encrypted volume. This takes only a couple of minutes over the LAN since most of the data has not changed. Currently we are using about 19GB of the 50GB TrueCrypt volume.

Once a week I have a script that does the following.

  1. Dismount the TrueCrypt volume
  2. Use a windows version of the unix Split command to chunk the TrueCrypt volume file into 250MB chunks. (200 chunks total).
  3. Re-mount the TrueCrypt volume… putting this step in here minimizes the amount of time that the LAN backup target is unavailable
  4. runs rsync on the PC (cwRsync) to rsync the chunked up TrueCrypt volume to my space at DreamHost.
  5. Deletes the local copy of the chunks.

The whole script takes about 4-5 hours to run, though the impact of having the local data backup unavailable to the LAN is only about 30-40 minutes.

Much of the time is just spent in calculating the block hashes by rsync and not in actual data transmission, but when you are dealing with 50GB of data it still takes a long time.

The inital upload of the 50GB of data took about 4 days, but the rsync of changes taking 3-4 hours is perfectly acceptable to me. If I can get it running smoothly enough, I may schedule it to automatically run each day while we are at work.

I have not tested the restore of the data - I have to download all 50GB in order to re-create the viable TrueCrypt volume, but I plan on doing that soon.

Some additional info:

  1. I have Verizon FiOS service 15Mbps download/2Mbps upload
  2. I am not terribly concerned with the security of the truecrypt volume on the local server so I leave it mounted all the time. The security of the encryption here is just so that someone at dreamhost (or a hacker who might break into dreamhost’s servers) cannot peruse my data.
  3. It is a potential security threat to have 2 copies of the same TrueCrypt volume with different states of data in them - potentially there can be vectors of attack that depend on differential analysis between the two volumes. That is why I delete the local copy of the rsynced chunks as soon as I am done with them.

Again, I am not overly concerned with high end secutiry from TrueCrypt, I’m mostly looking to keep prying eyes out of my data when I back it up offsite.

Ted

Ted,

I am trying to use DeltaCopy to sync data directly from a Windows XP (Home) machine to the backup account, but my inexperience with a Linux environment is hampering my ability to troubleshoot…

EDIT: I have merged several cross-posts into a new post:

http://discussion.dreamhost.com/showflat.pl?Cat=0&Board=forum_troubleshooting&Number=112802&page=0&view=collapsed&sb=5&o=14&part=