To prevent thread revival...need help regarding site backup

dreamobjects

#1

I’ve searched and read a lot of post on the forum and the mighty Google…although everyone seems to be trying to do something grander than what I want to do. I have found these threads that I think are what I want/need but I’m not sure if they are what I want.

https://discussion.dreamhost.com/thread-142149.html (which lead me to https://www.interworks.com/blog/jkhalaj/2014/07/02/use-dreamobjects-backup-your-mysql-database-php-script for MySQL backup to DreamObjects which I hope is what I want…not sure though…)

https://discussion.dreamhost.com/thread-141371.html (which I think is moving my /home/user/(All Files) over to dreamobjects and is what I want, although not too sure)

So here is what I am trying to do. I want a fully automated way to backup my website and MySQL Databases with the ability to access my backups to download/delete/modify whenever I want. Currently I manually backup the sites that I manage, which is growing and becoming unmanageable, with FTP(WinSCP) and mysqldump utility. This takes a great deal of time. Will DreamObjects allow me to do this and how. Let’s say I’m in need of more detailed guidance than what I have currently found through Google. :frowning: So please, no “Use the rsync command and there you go” type of replies. :slight_smile:

I was going to use a cronjob to put my mysqldump in a folder in my home/user/bkfolder which would delete the file after 3 days, although I don’t want to be in violation of the TOS. I use shared hosting and have VPS accounts that I’m managing.

Any help would be greatly appreciated!


#2

You don’t mention how you are managing those sites.

If you are using WordPress, our own Ipstenu has a wonderful plugin (DreamObjects Backups) that backs up your WordPress site and drops into a DreamObjects bucket, seamlessly. Here’s my review of it.


#3

Sorry good point. No I don’t use any CMS or anything that uses plugins for these domains. Just standard PHP and MySQL. Hence the need outside of plugins and what not.


#4

I think the two threads I linked in the original post will work for my needs of making a copy of everything in my /home/user/ directory and the other post with the link to get a copy of my databases dump files over to DreamObjects. I just want to make sure that this is a good idea with DreamObjects before I waste time and money on it.

My understanding of how DreamObjects works is about nil. I’ve tried to read up on it, although…that is why I’m about nil and not nil of understanding how it all works. :slight_smile:

I’ve talked to Dreamhost Online Chat about a backup solution and they suggested DreamObjects although they said they don’t have any ready-made scripts that will do the job. I can Google all week and find several solutions for backup such as cronjobs and PHP scripts that will do it all, but will it work with Dreamhost and be compliant to the TOS, which I want to be fully.

-EDIT- After reading some more from my Google of Automatic backup of Dreamhost hosted data. It seems that there is not a easy to configure solutions to do backups automatically. The last thing I read, which was promising, was from 2008 when Dreamhost had Backup Accounts. What are people doing to backup their sites? Or am I the only one not using WordPress…lol!


#5

The “backup.php” script you link to is a good option for doing the MySQL dump and shipping it to DreamObjects. You could also modify it to tar your user directory and send that over as well. Otherwise, both s3cmd and boto-rsync are installed on DreamHost machines by default and can be used to upload files to DreamObjects.

I’ve heard a lot of complaints about how painful creating and managing backups can be. That’s why we’re working on an optional backup service that will backup your sites and databases automatically to DreamObjects. I’m hoping to get that released by the end of the year.

That’s not to say we won’t help you with your own backups! Post here which route you decide to go (backup.php, s3cmd, boto-rsync, etc.) and we can help you out.


#6

Thank you very much! I was hoping one of those post would work for me. I’m not too savvy with the CLI yet.

If I read correctly boto-rsync is the route I would want to go to save bandwidth (which I do). Does it only transfer files that have changed from those in DreamObjects? If so, then this would be perfect since I will only make a few changes to config files and of course MySQL databases and wouldn’t benefit from a full backup every time.

To use boto-rsync, I would use the following in a cronjob?
boto-rsync -a ACCESSKEY -s SECRETKEY --endpoint objects.dreamhost.com /SOURCE/PATH/ s3://DESTINATIONBUCKET/PATH/

I read in the Wiki that I can include --delete to remove files no longer needed? How would that work?

If I go the route of the backup.php, this would use the most bandwidth since it would transfer the entire contents over to DreamObjects every time it is run. To me, this would be the easier way…I just want to be server friendly and as I mentioned before, I will only be changing a few files from time to time.

Honestly I would rather wait to see what options ya’ll come up with for backups. Leave it to the experts as they say. The down side is I also would like a secure backup location now without downloading my sites…which I’m doing now and I started around 0900 this morning…it’s 1049 now…falling asleep at the keyboard… :slight_smile:


#7

Regarding boto-rsync, you’re correct that it only transfers files that have changed. It acts differently from regular rsync in that boto-rsync only compares the sizes on the source and destination to determine if a file has changed - it’s likely a file would be missed if it was modified but the size remained the same. This generally isn’t a problem though as most changes usually result in a change in file size.

Your syntax is correct in the example given. Just be sure to keep in mind the trailing slashes - http://defindit.com/readme_files/rsync_backup.html

The --delete option helps ensure the source and destination stay in sync. Without the --delete option, boto-rsync will copy all of the files in the source to the destination. When you remove a file on the source, it will remain on the destination if you do not use the --delete option. If you use the --delete option, that behavior changes. Using the option, if you delete a file on the source, it is also deleted in the destination.

When it comes to database backups, my opinion is that the best option is a full dump copied to an external destination. It’s very difficult to partially restore a database or to back it up from the filesystem. The dump file is just text (assuming no blobs) that is easily compressed. So my suggestion is to use boto-rsync on the site and the backup.php script on the database.

Remember that uploads to DreamObjects are free. You’ll want to keep an eye on your disk usage though. You likely won’t want to keep an endless amount of backups. You could use something like this script to remove older backups - https://discussion.dreamhost.com/thread-141706.html


#8

We use Duplicity to perform Incremental backups to DreamObjects, its been working flawlessly for the past 2 months. It backs up our MySQL and website data without any problems.

This is a good tut :- http://mindfsck.net/incremental-backups-amazon-s3-centos-using-duplicity/


#9

Thanks for the reply! I will work on implementing this today…looks pretty straight forward to me though…here’s hoping. :slight_smile:

EDIT - Well, the steps to create the backup.php is out of date…Step 1…AWS SDK for PHP has since been updated and no longer has aws-autoloader.php or a aws folder. Going to the 2.8 branch doesn’t have this either. Any help would be greatly appreciated…


#10

Will this work with Shared Hosting accounts? I’m pretty sure I won’t be able to install anything to the Shared Hosting sites. Is Duplicity installed on the Shared servers?


#11

I have decided to store the backups on my VPS instead of trying to figure out DreamObjects, which according to http://wiki.dreamhost.com/Backup it is not breaking the TOC. Thanks to Google and the forums I now know how to backup my computer to DreamObjects (still fuzzy though), I just can’t figure out how to change these solutions to work with my Shared or VPS accounts.
[hr]

I guess I could install Wordpress just to manage my backups. :slight_smile:


#12

I went this route instead…not ideal, but I can understand the code better.

I am interested in a solution that uses DreamObjects, I just can’t figure out how to use it with my Shared and VPS sites. I will be watching for the optional backup service you mentioned.