Backup/Data Stability

dreamcompute

#1

What are my options as far as making sure my data is safe on a DreamCompute instance? If the instance was to go down for some reason, would I lose my data/setup? How should I go about backing up my instance? I had read about DreamObjects and storing there, but I’m not 100% sure what I’d need to store/best practices.


#2

That’s a great question!

It would be appropriate to first bring up the difference between ephemeral and volume-based instances. The main difference is that the ephemeral storage is best used for non-persistent data or data that can be easily re-generated, such as operating systems and scratch data, whereas volume can also be used for that data and also persistent data and important data. When an ephemeral instance is destroyed, the data is destroyed with it. Volume-booted instances will retain the data on the volume for re-use or re-purpose when the instance is deleted (except if a “delete on terminate” checkbox is checked during instance creation).

Both ephemeral and volume-based instances can have snapshots taken of them, which is a copy of the entire file system, with ephemeral instance snapshots storing in the OpenStack image service (glance) and volume-based instance snapshots storing their data as a volume snapshot, using volume storage quota. Either instance type can have additional volumes attached, which can be used to store important data as well and also be snapshotted.

When it comes to making backups, there are a couple options.

  1. The recommended way would be to look into backing up just the data that is important, and now the entire file system. Storing this data on object storage like DreamObjects, is also recommended if you need to get to the data should DreamCompute have any issues, because the systems are entirely separate and in different datacenters (currently). To make the backup, you could just do a file copy/sync with tools like s3cmd or boto-rsync:


or you could use a tool like duplicity that supports full/incremental backups, encrypts the data and has some other nifty features:

Should any data get lost, it can simply be restored from these methods. If the entire instance has an issue, you would need to re-create the operating system setup and restore the data.

  1. Ephemeral / Volume snapshots - If the snapshots are of an ephemeral instance, it will not count towards your storage quota, and you can take them as needed. Volume-based instance snapshots do use volume storage quota, and may become locked and un-deleteable if they are in use. When it comes to restoring data, an entire new instance would need to be created to replace the broken one, or a new volume created with the entire file system, and then the data recovered that way. If a broken instance is replaced with a snapshot, IP addresses will change, and all data is reset to the state of when the instance was snapshotted.

I hope that makes sense!