What is an efficient way to transfer a large number of files from my backup user account to a DreamObjects bucket? All I have come across so far is to first move them to my web space, then to an object. The problem is that we are not given shell access to the backup account.
maybe something like (with your backup account as BU_USER):
rsync -e ssh -a -v --delete --protocol=29 BU_USER@hanjin.dreamhost.com:path/to/backedup/folder/ path/to/local/folder/
test on empty folders first…
just tested on my account and it worked, but I have ssh keys set up between the two …might ask for a password if you dont
what is the path/to/local/folder/ for a dreamObjects bucket?
My first thoughts were s3cmd or boto-rsync but it looks like both of those need to sync from a local directory instead of using a remote location.
You could try mounting the backup account as a local filesystem using something like fuse. Then you could use either s3cmd or boto-rsync to send the data to DreamObjects.
I just thought of another you could do this. You could use cyberduck and open 2 connections, one to your backup account and one to DreamObjects. Then just drag the files from the backup account window to the DreamObjects window. The bad part about this would be that all the data would be transferred to your computer first then uploaded to DreamObjects - not ideal.
my apologies, I was going from my Backup account to my web-account…I am not sure what the path to a DreamObjects bucket is. I should have read your last sentence of the original post.
From reading the original post, and the posts that follow…
It seems like the most efficient way is to
Which it seems like that was where this was headed:
Not sure what the resistance to doing it that way is, but it would be much more efficient than downloading to upload. With server to server transfer speeds it should be pretty quick.
That is what i had in mind. I’m going to move 50 Gb of photos and music and wasn’t sure how appropriate staging it on the webspace was though. I’ve dropped a note to support to confirm. I’ll update when they get back to me. thanks!
I’m going to have someone from our tech support team contact you. We should be able to move this for you so it doesn’t have to be downloaded or moved to your web server first.
As a followup for others that stumble on this: justinlund took care of transferring directly from hanjin to the DreamObject for me, removing the need to first transfer to my web space. And, I’ve been using CyberDuck from windows and rsync for additional transfers.