How to create a new server in DreamCompute and register a FQDN name with Ansible

dreamcompute

#1

I use Ansible to create new DreamCompute machines quite often but until recently I didn’t think of using DreamHost DNS API to assign the new server a full qualified domain name. All you need is an API token that has the permission to add a DNS record.

Once you got your API token, all you need is a playbook that looks like this:

- hosts: localhost
  connection: local
  vars:
    private_key: /path/to/your/ssh/key
    fqdn: subdomain.example.com
    api_key: $YOUR_API_KEY
  tasks:
    - name: create a Ubuntu server
      os_server:
            cloud: iad2
            name: shiny-new-server
            state: present
            image: Ubuntu-16.04
            flavor_ram: 512
            key_name: your_ssh_keyname
            boot_from_volume: True
            volume_size: 5
            network: public
            security_groups: default
            wait: yes
      register: new_server

    - name: get facts about the server (including its public v4 IP address)
      os_server_facts:
        cloud: iad2
        server: shiny-new-server
      until: new_server.server.public_v4 != ""
      retries: 5
      delay: 10

    - set_fact: public_v4="{{ new_server.server.public_v4 }}"

    - name: add the server to our ansible inventory
      add_host: hostname={{ public_v4 }} groups=web ansible_ssh_user=ubuntu ansible_ssh_private_key_file={{ private_key }}

    - name: wait for ssh to be available
      wait_for: host={{ public_v4 }} port=22 state=started

    - name: register DNS name
      shell: 'curl -X GET "https://api.dreamhost.com/?key={{ api_key }}&type=A&unique_id={{ fqdn | to_uuid }}&cmd=dns-add_record&record={{ fqdn }}&value={{ public_v4 }}"'

Save that file as newserver.yml and run it with ansible-playbook -vvv newserver.yml. This playbook will create a new Ubuntu machine, booting from a 5GB volume, with 512MB RAM. Once the machine is created, the script will take its public IPv4 address and register a subdomain called subdomain.example.com with that IP address.

It’s also possible to create other types of DNS records, which opens a lot of other possibilities to automate things using Ansible.


#2

More is needed than indicated here. I believe this is because your environment is already setup but the rest of us need some setup…

The first thing that I needed was to install pip, and the shade package.

sudo apt-get install python-pip
pip install shade
python -c "import shade"

The last line verifies that the package is available. (no error = no response = OK)
Now the os_server modules work.

But once in there, I kept getting the error:

os_client_config.exceptions.OpenStackConfigException:
Cloud iad2 was not found.

I couldn’t work out a way to otherwise identify the cloud (authorization zone). I tried “iad-2”. I tried leaving the cloud out but got another error:

keystoneauth1.exceptions.auth_plugins.MissingRequiredOptions:
Auth plugin requires parameters which were not given: auth_url

Hmmm, auths… The Ansible page for the os_server module says:

[the cloud] parameter is not needed if auth is provided or if OpenStack OS_* environment variables are present.

I’m using Linux (setting up an instance from within an instance) but this HowTo for Windows provided a tip about how to get the openrc.sh file for Linux to setup those environment variables.

So I go the file, changed perms to 774, then executed:

source ./my_openrc.sh

I entered my DC project password and it returned to the shell prompt. Now I executed the playbook and it went a lot further. After a very long period it terminated with a timeout and:

keystoneauth1.exceptions.connection.ConnectFailure:
Unable to establish connection to 
https://iad2.dream.io:8774/v2/my_project_id/servers/detail:
('Connection aborted.', error(104,Connection reset by peer'))

I tried the URL from my PC browser and that worked, with the expected error that auth was needed, but at least it connected. I tried wget from my instance and that worked the same. So the issue wasn’t with a firewall.

I used su to root, had to re-install shade for root user (I’m sure that should have been a global install, whatever) but the result was the same. Rebooted server, got in as ubuntu, immediate su, re-ran openrc.sh, and re-ran the playbook with four -v options ( -vvvv for connection debugging) … no joy.

All of the env variables for OS_* look correct. It looks like the DC server just isn’t honoring the request to create a new instance. :sob:


#3

You’re correct, I forgot to mention that I use cluods.yaml configuration for to specify OpenStack credentials. That’s where I have specified the iad2 cloud. In any case, you solved that puzzle :slight_smile:

The error you get is strange and should not depend on executing ansible as root. Once you source openrc.sh can you connect using the OpenStack client?


#4

The error you get is strange and should not depend on executing ansible as root. Once you source openrc.sh can you connect using the OpenStack client?

No, installed globally and executed openstack client as root, requests for server list and image list both hang in the same way, I didn’t wait for the timeout. With keywords munged, here are my details.

OS_AUTH_URL=https://iad2.dream.io:5000/v2.0
OS_ENDPOINT_TYPE=publicURL
OS_IDENTITY_API_VERSION=2
OS_PASSWORD=zzzzzzzzzzzzzzzzzzzzzzzzz
OS_REGION_NAME=RegionOne
OS_TENANT_ID=6zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz2
OS_TENANT_NAME=dhc999999
OS_USERNAME=myuser

openstack server list
openstack --os-tenant-name $OS_TENANT_NAME --os-username $OS_USERNAME --os-password $OS_PASSWORD --os-auth-url $OS_AUTH_URL server show myserver01

If I clear the password, the CLI does prompt for it, but doesn’t go anywhere afterward.

That is the password we use to login to the DC dashboard, right?
Do we need to install some Keystone client component?

Thanks!


#5

I just tried downloading the newest version of openstack RC file and the authentication works, openstack client sends the list of servers and all. Maybe you’re using an older version of the file? I don’t know, honestly… try downloading the file again and see if that helps.

Alternatively, try creating the clouds.yaml file as described in openstack client docs https://docs.openstack.org/python-openstackclient/pike/configuration/index.html

It should look like this:

clouds:
   iad2:
      profile: dreamhost
      auth:
          username: $THE_USERNAME
          password: $THE_PASSWORD
          project_name: $THE_PROJECT_NAME
          auth_url: https://iad2.dream.io:5000
      region_name: RegionOne

You’ll find username and project name in the newest RC file, and the password is the one you use to authenticate on https://iad2.dreamcompute.com


#6

Nope. Created clouds.yaml as root in a new directory owned as root:root.
Ensured LF EOLs. Hard-coded user/password rather than using env vars.
CLI in that directory:

openstack --os-cloud iad2 server list

Also added trailing slash to auth_url just in case (per referenced doc).
I’m positive that it’s using the clouds.yaml file because as I was ensuring correct spacing I made an error and it got flagged. On correction the above command still just sits there.

Thanks for the help.


#7

Strange… Local users like root or unprivileged should not be an issue… Are you sure you’re using username/password combination that you use on cloud.dreamhost.com for example? Aside from that, I’ve exhausted my options… I suggest you open a ticket, I’m not sure what is the error here at this point.


#8

Taking this through Support, thanks.

This isn’t even related to Openstack or my creds. Simple HTTPS ccess to the auth server from within an instance is random.

Try this:

curl -g -i -X GET https://iad2.dream.io:5000/v2.0 -H "Accept: application/json" -H "User-Agent: osc-lib/1.9.0 keystoneauth1/3.4.0 python-requests/2.18.4 CPython/2.7.12"

Execute that within an instance and it may or may not hang.
Execute from outside the network (from my Windows PC) and it always works.

Other curl requests from the instance to any domain always seem to work:

curl -g -i -X GET https://www.google.com "Accept: text/html"

So I think there’s something going on with the network in terms of timing, firewall, permissions, etc, either outbound from my instance or (more likely) inbound to your Identity service.


#9

Support resolved a reprocible network issue which caused the described issues.

I completed the task of creating a DC server with FQDN and have adapted the notes into my own Ansible scripts. (@smaffulli you’ve created a monster.)

Since the issue was temporary, would you like me to edit out my notes on that? I feel like I’ve polluted your nice HowTo. I have other notes I’d like to add soon for others who follow.

Thanks.


#10

Glad to hear that the issue is fixed now. There was no pollution, more like added details :slight_smile: