Automated instance management

dreamcompute

#1

OK.

Entirely new to OpenStack, not to virtualisation or networking though.

My current assumption is that I could manually spin up an instance and treat it as a normal VM (install stuff, configure bits…)

However, I could also terminate it when I’m not using it, and then respawn it at a later time, reaping the benefits of the ‘per hour’ charging.

  • I assume that the root FS etc will all be preserved somewhere when that happens (i.e. I don’t need to spin up and reload from a backup, or reconfigure anything)
  • I assume that the IP address will change each time I spin it up in this way

If my assumptions are sound and I go down this route then a couple of questions spring to mind very quickly…
How would a script inside an instance call for that instance to be terminated (i.e. an ‘end of job’ script)?
How would I schedule the instance to start again to run the job automatically?


#2

[quote="[XAP]Bob, post:1, topic:63685"]
My current assumption is that I could manually spin up an instance and treat it as a normal VM (install stuff, configure bits…)
[/quote]

Yes, you can do exactly this: DreamCompute’s machines are just a virtual machine from the user’s perspective.

[quote="[XAP]Bob, post:1, topic:63685"]
However, I could also terminate it when I’m not using it, and then respawn it at a later time, reaping the benefits of the ‘per hour’ charging.
[/quote]

Uhm sort-of. The ‘per hour’ charges may keep on occurring unless you delete/terminate the machine. If you pause or suspend the machine, it’s still billed as if it was running. The reason for the charge is that the suspended machines still hold up resources in the cluster.

[quote="[XAP]Bob, post:1, topic:63685"]

  • I assume that the root FS etc will all be preserved somewhere when that happens (i.e. I don’t need to spin up and reload from a backup, or reconfigure anything)
    [/quote]

If you want to keep the root FS you should take a snapshot of the machine, and later boot from such volume. The process is not 100% reliable though and you should still keep a backup or configure things in another way (see below).

[quote="[XAP]Bob, post:1, topic:63685"]

  • I assume that the IP address will change each time I spin it up in this way
    [/quote]

Indeed, the IP will likely be different every time a machine is booted.

[quote="[XAP]Bob, post:1, topic:63685"]
How would a script inside an instance call for that instance to be terminated (i.e. an ‘end of job’ script)?
How would I schedule the instance to start again to run the job automatically?
[/quote]

I’d do all this from outside. For example, if the reason for the cloud is to run a test or compile a piece of code, I’d suggest you to write a script in your favorite language that:

  • starts a machine
  • configures it to run the test
  • drops the workload that executes the tests and sends a signal when it’s done
  • terminates the machine upon signal

Depending on your exact use case, we can suggest you best practice that don’t involve complicated backups or costly solutions. For example, you could create and upload a virtual image that has exactly the software and configuration you need, and start that one instead of a bare Ubuntu (for example) and configure it all the time. Another approach is to automate installation and configurations with tools like ansible or puppet, chef and the like… There are lots of options. Why exactly do you want to use a VM for little time?


#3

Basically because I’m a cheapskate :wink:
Well, not totally - but I am looking at a number of things…
The opportunity to spin up random machines is, in itself, attractive - there are a couple of different systems I tend to want available. Certainly I have a couple I could happily run for a couple of months at a time, and others which only need to run for a few minutes a day, but want to do that every day…

Do the IP addresses get changed when the server reboots, or only if it is terminated and reinitialised?
(Potentially important for secure access from mobile connections - though I’m sure I can find ways around it)

If the root FS isn’t persistent… how does ‘stuff’ get loaded…
For instance I run tech for a charity radio station - it only runs for four weeks a year, so this would be a ‘two month’ compute task from my point of view. It’s one that sits idle (and firewall blocked) on my ‘normal’ VM at the moment, but having a dedicated machine would offer me more freedom over my normal VM.

The DC filesystem is persistent until I hit delete/terminate yes?

I can take a snapshot (which would be good to allow for the same VM to be spun up next year) - but that’s not always reliable?
Ah - but I could create a perfect ‘custom image’…

I think I need to have a bit of a ponder over what I actually use… It’ll be good for the soul…


#4

[quote="[XAP]Bob, post:3, topic:63685"]
Basically because I’m a cheapskate :wink:
[/quote]

Nothing wrong with that :slight_smile:

[quote="[XAP]Bob, post:3, topic:63685"]
Do the IP addresses get changed when the server reboots, or only if it is terminated and reinitialised?
(Potentially important for secure access from mobile connections - though I’m sure I can find ways around it)
[/quote]

If you suspend the VM, the IP address remains allocated to it. Rebooting the VM doesn’t change its IP address either.

[quote="[XAP]Bob, post:3, topic:63685"]
If the root FS isn’t persistent… how does ‘stuff’ get loaded…
For instance I run tech for a charity radio station - it only runs for four weeks a year, so this would be a ‘two month’ compute task from my point of view. It’s one that sits idle (and firewall blocked) on my ‘normal’ VM at the moment, but having a dedicated machine would offer me more freedom over my normal VM.

The DC filesystem is persistent until I hit delete/terminate yes?
[/quote]

Exactly. So for that use case, I assume for the radio station you need some special software package (say Shoutcast), some configurations and maybe some files… for example, stored on /opt/radio-files. In this case, I’d keep the files in a block volume on DreamCompute, an Ansible role to install and configure shoutcast and a playbook. When that time of the year comes, I’d run the playbook to create a new machine, assign it the shoutcast role, and to attach the volume on /opt/radio-files. When you’re done with the machine, just terminate it and run the playbook again next year. If you write anything on /opt/radio-files, those will keep until next year.

Another possibility is to create a custom VM image with shoutcast, the configuration and the files you need, and upload that in your account. When time comes, boot a new VM from your custom image and shut it down when you’re done.

If you don’t need to write new files each year then you may not need to have a separate volume to keep around.

[quote="[XAP]Bob, post:3, topic:63685"]
I can take a snapshot (which would be good to allow for the same VM to be spun up next year) - but that’s not always reliable?
[/quote]

That’s another approach. The unreliability comes from some bugs that show up at times with some images where they don’t start cleanly… I personally don’t recommend this approach because many people think of snapshots like backups and that’s dangerous. But with in this specific use case, it would probably work: just make sure that the snapshot works correctly before the radio season starts :slight_smile:

HTH


#5

Ok - I really need to read up on some of the technologies available to me then…

playbook, ansible, block volumes… anything else that needs to be on my reading list?

I presume, from what you have just said, that DreamStorage can be carved up into virtual block devices which are then mounted somewhat normally?


#6

[quote="[XAP]Bob, post:5, topic:63685"]
Ok - I really need to read up on some of the technologies available to me then…

playbook, ansible, block volumes… anything else that needs to be on my reading list?
[/quote]

Ansible is a configuration management tool. Start with the basics on https://www.ansible.com/ and once you grasped the basic concepts, move on experimenting with DreamCompute. We have some examples to get started, like:


[quote="[XAP]Bob, post:5, topic:63685"]
I presume, from what you have just said, that DreamStorage can be carved up into virtual block devices which are then mounted somewhat normally?
[/quote]

DreamCompute is basically three things: virtual computers, virtual storage devices and virtual networks. The virtual storage devices are basically disks (or volumes) that you can create on-demand, attach and format as you would with a hard disk (for example) and mount it.

This article can give you an overview of DreamCompute:

and here you’ll find instructions on how to create virtual volumes:
https://help.dreamhost.com/hc/en-us/articles/215912858-How-to-create-and-manage-volumes-with-the-DreamCompute-dashboard

HTH