A quick tutorial on Vagrant + Ansible

from: https://adamcod.es/2014/09/23/vagrant-ansible-quickstart-tutorial.html

Ansible and Vagrant

First, make sure you have Vagrant3 and Ansible4 installed. You can find installation documentation on their respective websites, they’re very easy to get installed.

Basics

We’ll start by creating a new directory to hold our project.

mkdir -p ~/Projects/vagrant-ansible
cd ~/Projects/vagrant-ansible

Next, we can use Vagrant to create a new vagrant file based on the latest ubuntu image.

vagrant init ubuntu/trusty64

You should now have a file called Vagrantfile in the root of the directory. This contains some basic information about the box you want to provision, and then a whole bunch of commented out stuff you don’t need to worry about now. Remove all of the commented lines, so you’re left with the bare minimum:

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "ubuntu/trusty64"
end

We’ll need a way to access our webserver once it’s provisioned, so we’ll tell Vagrant to forward port 80 from our box to port 8080 on localhost. To do that, add the following line just before end:

  config.vm.network "forwarded_port", guest: 80, host: 8080

Now, there is one last thing we need to do to configure Vagrant, and then we’re finished with it. We need to tell Vagrant that we want to use Ansible as its provisioner, and where to find the commands to run. To do this, add the following lines to your Vagrantfile, again, just before end:

  config.vm.provision :ansible do |ansible|
    ansible.playbook = "playbook.yml"
  end

Once you’ve done that, the entire contents of your Vagrantfile should look like this:

VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
  config.vm.box = "ubuntu/trusty64"

  config.vm.network "forwarded_port", guest: 80, host: 8080

  config.vm.provision :ansible do |ansible|
    ansible.playbook = "playbook.yml"
  end
end

Basic Terms

Ansible works by running a series of Tasks on your server. Think of a Task as a single Bash command. You then have what is called a Playbook. A Playbook tells Ansible what Tasks you want to run on your server. Each Task is run using an Anisble Module. A Module is a built-in way to do something, like access Yum, create a user and so on. That will become clearer later.

Your First Playbook

Create a new file called playbook.yml, this has to match the value we put in our Vagrantfilefor ansible.playbook.

All Ansible Playbooks should be in YAML format. Tradition dictates that you start your YAML file with three dashes. Whilst Ansible doesn’t technically require this, the community still seems to follow this rule.

Your Playbook should be a YAML list. The list should contain a list of hosts (servers) you want your Playbook to run on, as a single Playbook can control multiple hosts, and then a list of Tasks you want to run on that host.

Start by adding the following to your playbook.yml file.

---
- hosts: all
  sudo: true
  tasks:

We’re using Vagrant and only have one host for now, so we can just set the value to all, which is a magic value that says: “run these tasks on all servers you know about”. Then we tell Ansible that these tasks will require sudo and finally add our tasks: key to begin specifying our Task list.

To install a LAMP stack, there are four basic steps we need to take:

  1. Update Apt Cache
  2. Install Apache
  3. Install MySQL
  4. Install PHP

That’s kind of all we need to do. We’re using an ubuntu box, so all of that can be done via apt. To do that, we need to use Ansible’s apt module5.

To start with, we give each Task a name: key. This can be anything you want, and is used to describe the Task that follows.

- name: this should be some descriptive text

We then specify a key telling Ansible the name of the module we want to use, which in our case is the apt module:

apt:

You follow the name of the module with a series of key=value pairs, separated by spaces, corresponding to the options and values you want to pass to the module (you can look these up, along with their possible values, in the Ansible module documentation).

For installing Apache, our task would look something like this:

- name: install apache
  apt: name=apache2 state=present

And that’s all there is to it. Pretty simple, right? We can then re-produce that for MySQL and PHP, and include them all under our original tasks: key in our playbook.yml, which should look something like the following:

---
- hosts: all
  sudo: true
  tasks:
    - name: update apt cache
      apt: update_cache=yes
    - name: install apache
      apt: name=apache2 state=present
    - name: install mysql
      apt: name=mysql-server state=present
    - name: install php
      apt: name=php5 state=present

Now we’re ready to provision. Drop back to your terminal and run vagrant up, and you should see output similar to the following:

Vagrant Ansible LAMP Output

That’s it. If all you wanted was a working LAMP server, you now have it. SSH into the vagrant box and put an info.php file in /var/www/html with:

<?php phpinfo();

And load it in your browser at http://localhost:8080/info.php and you should see everything working as expected.

Refactoring

The next step after you have any application working is to make it better, and Ansible is no different. The first thing we can do is eliminate the repetition. This isn’t always the goal of course, and only makes sense in certain circumstances. As you will see in a moment, this is not one of those circumstances, but this is a good chance to introduce you to a type of loop in Ansible, called a Standard Loop.

A Standard Loop uses a with_items: key in your YAML list, and allows you to perform the same action on a list of items. In our case, we have a list of packages we want to install. It looks something like this:

---
- hosts: all
  sudo: true
  tasks:
    - name: update apt cache
      apt: update_cache=yes
    - name: install required packages
      apt: name={{ item }} state=present
      with_items:
        - apache2
        - mysql-server
        - php5

We’ve now replaced the three install steps with a single install step that installs all of the packages we require. Ansible will run that same task, substituting the value of item with each item in the with_items: list in turn.

You can test it works by running vagrant destroy followed by another vagrant up in the root of your project.

Extracting Include Files

Usually, when you’re installing packages for a new server you want to do more than just install the packages – you probably want to configure them too. You might want to tell apache to use /vargrant instead of /var/www/html (which is the default location for vagrant to mount the current directory), or install php_mysql and php_pdo so you can access your MySQL Server from PHP.

In Ansible, even though include: is technically a language construct, it behaves exactly like a module would. You include it as a task with some key-value pair parameters.

Before we do that, let’s restore our install tasks to three separate tasks to make them easier to break up:

---
- hosts: all
  sudo: true
  tasks:
    - name: update apt cache
      apt: update_cache=yes
    - name: install apache
      apt: name=apache2 state=present
    - name: install mysql
      apt: name=mysql-server state=present
    - name: install php
      apt: name=php5 state=present

Now, let’s create our individual task files that we’re going to include in our playbook. Start by adding a new directory called tasks and then add the following three files:

In tasks/apache.yml:

---
- name: install apache
  apt: name=apache2 state=present

and tasks/mysql.yml

---
- name: install mysql
  apt: name=mysql-server state=present

and finally tasks/php.yml

---
- name: install php
  apt: name=php5 state=present

The first thing you should notice is that at the highest level, we have a list of Tasks, not a list of Hosts. That’s because these files will be included in the tasks: section of your playbook, so they’re already scoped to that, and can just be a top-level list of Tasks. Other than that, they’re exactly the same as before. We have our traditional three dashes to indicate it’s a YAML file, but then there is no other change.

Now, we need to update our playbook to include these new files. Change your playbook.yml file to match the following:

---
- hosts: all
  sudo: true
  tasks:
    - name: update apt cache
      apt: update_cache=yes
    - include: tasks/apache.yml
    - include: tasks/mysql.yml
    - include: tasks/php.yml

That’s all we need to do. I’ve not included a name for these, as each Task has a name in the included file. You could include a name here if you want, it will still work, but it will not appear in the output.

Test this new configuration by running vagrant destroy followed by vagrant up again. Everything should work exactly as before.

Templates and Files

As we mentioned previously, you will probably want to include related tasks in your new include files. As an example, we’ll update your Apache DocumentRoot setting to point at /vagrant rather than /vat/www/html, so you can serve your local files. There are a few ways to do this, but the simplest is to include your Virtual Host as part of your Ansible Playbook, either in the form of a Template or a File.

The difference between a Template and a File is a small but important one. Ansible will copy a file to your server exactly as it appears in your Playbook. A Template, on the other hand, can contain variables that Ansible will substitute with real values before copying across to the server. This is the path we’ll take for our Virtual Host.

In order to create our Virtual Host, we need to follow these four steps:

  1. Copy the new Virtual Host (with the DocumentRoot set to /vagrant) to /etc/apache2/sites-available
  2. Disable the 000-Default Virtual Host
  3. Enable our new Virtual Host
  4. Reload Apache config.

Templates

We’ll start by copying our Virtual Host to the server, and to do that we’ll use a Template. We could use a File for this, and hard-code the DocumentRoot, but the process is almost the same for Templates, and a Template is more flexible.

As Ansible is written in Python, for templates it uses the Python Jinja26 library. Don’t worry too much, the syntax is very similar to all modern template languages, such as Liquid, Twig, Mustache, etc.

To get started, create a directory in the root of your project called templates, and in that new directory create a file called virtual-hosts.conf.j2. It’s customary to call a template file it’s normal name, including file extention, and then append .j2 to the end.

I ssh’d into our already provisioned vagrant box (using vagrant ssh), and grabbed the contents of the existing Virtual Host (at /etc/apache2/sites-available/000-Default). Now put the contents of that file in templates/virtual-hosts.conf.j2 (I have stripped out the comments for the sake of brevity):

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot /var/www/html

  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Now replace the line:

DocumentRoot /var/www/html

with

DocumentRoot {{ document_root }}

This is how you echo a variable in jinja2. The variable in this case is called document_root.

Because of the way Apache works, you need to add a new config section for the /vagrantdirectory, otherwise you’ll get 403 Forbidden errors, so add the following below your DocumentRoot line:

<Directory {{ document_root }}>
  Require all granted
</Directory>

All together, it should look like this:

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot {{ document_root }}

  <Directory {{ document_root }}>
    Require all granted
  </Directory>

  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Now save the file and open up tasks/apache.yml, we need to add a new Task to copy this Template to the server.

- name: Copy across new virtual host
  template: src=virtual-hosts.conf.j2 dest=/etc/apache2/sites-available/vagrant.conf

You don’t need to include the templates directory in the src value, Ansible always assumes Templates will be in a templates directory, relative to the current context (which is important for when we look at Roles later). If it can’t find the file in the templates directory, it will check again in the root before giving up.

The whole file should now look like this:

---
- name: install apache
  apt: name=apache2 state=present
- name: Copy across new virtual host
  template: src=virtual-hosts.conf.j2 dest=/etc/apache2/sites-available/vagrant.conf

Those lines are starting to look a bit long. Luckily, Ansible lets you break up key-value pairs onto their own lines, so lets do that now:

---
- name: install apache
  apt: name=apache2 state=present
- name: Copy across new virtual host
  template:
    src=virtual-hosts.conf.j2
    dest=/etc/apache2/sites-available/vagrant.conf

That makes it a bit easier for us to read, but doesn’t change any behaviour. The final thing we need to do to copy our template across is to tell Ansible what to use for the value of document_rootwhen it’s copying the Template in to place. Again, there are a few ways to do this, but the simplest is to add a vars: key to your playbook.yml, so that’s what we’ll do. Update your playbook.yml to match the following:

---
- hosts: all
  sudo: true
  vars:
    document_root: /vagrant
  tasks:
    - name: update apt cache
      apt: update_cache=yes
    - include: tasks/apache.yml
    - name: include mysql
      include: tasks/mysql.yml
    - include: tasks/php.yml

The vars: key can live in any position, but convention dictates it comes somewhere before tasks: in your Playbook. Drop back to your terminal and run vagrant provision. This demonstrates the idempotency of Ansible. We can provision the server as many times as we want, and Ansible will only update things that need to change. In this case, it will create our new Virtual Host file for us.

We can test that this has worked by sshing into the server and checking the contents of the new file:

vagrant ssh
cat /etc/apache2/sites-available/vagrant.conf

And you should see something like this:

<VirtualHost *:80>
  ServerAdmin webmaster@localhost
  DocumentRoot /vagrant

  <Directory /vagrant>
    Require all granted
  </Directory>

  ErrorLog ${APACHE_LOG_DIR}/error.log
  CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Excellent. Now we have three steps left to complete:

  1. Disable the default Virtual Host
  2. Enable our new Virutal Host
  3. Reload Apache

The File Module

Ubuntu’s configuration convention for Apache Virtual Hosts is to put the contents of all possible Virtual Hosts in /etc/apache2/sites-available and then use the a2ensite and a2dissite to create and remove symlinks for each active Virutal Host in the /etc/apache2/sites-enableddirectory.

When using Ansible, we bypass these tools and create and remove the symlink ourselves using the File Module7. We need to add two tasks to our tasks/apache.yml file, one to remove the existing symlink, and one to create our new one. They look something like this:

- file: path=/etc/apache2/sites-enabled/000-default.conf state=absent
- file: src=/etc/apache2/sites-available/vagrant.conf dest=/etc/apache2/sites-enabled/vagrant.conf state=link

We can break that long line up again, and add some names, to make the whole file look like this:

---
- name: install apache
  apt: name=apache2 state=present
- name: Copy across new virtual host
  template:
    src=virtual-hosts.conf.j2
    dest=/etc/apache2/sites-available/vagrant.conf
- name: Remove default virtual host
  file:
    path=/etc/apache2/sites-enabled/000-default.conf
    state=absent
- name: Enable new vagrant virtual host
  file:
    src=/etc/apache2/sites-available/vagrant.conf
    dest=/etc/apache2/sites-enabled/vagrant.conf
    state=link

Drop back to your terminal, run vagrant provision and you should see the normal Ansible output displayed, including your new Tasks. When it’s done, ssh into your vagrant box and run ls /etc/apache2/sites-enabled. You should see that 000-default.conf is now missing, and vagrant.conf is in its place.

The Service Module

The last thing we need to do is reload Apache so that our new configuration can take effect. To do that, we use the Service Module8, and add another Task:

- name: reload apache
  service: name=apache2 state=reloaded

Now run vagrant provision again, add a file in the root of your project called info.php with a simple <?php phpinfo(); line, then visit http://localhost:8080/info.php in your browser. You should see the familiar phpinfo() output and everything should be working smoothly. You can quickly search your phpinfo() output for DOCUMENT_ROOT to double check if you want to.

Handlers

Telling a service to restart or reload itself manually after every configuration change can be a real pain. Fortunately, Ansible has a way of automating that process for you, and that’s by using Handlers. Handlers are a list of Tasks with a name assigned that you can call by name after another Task has run. They won’t run unless another Task triggers them.

The quickest and simplest way to define handlers is to put them in your playbook.yml. Let’s add a new handler to reload apache for us. Open up tasks/apache.yml and remove the reload apacheTask, then open up playbook.yml and re-add it under a new handlers: key, like this:

handlers:
  - name: reload apache
    service: name=apache2 state=reloaded

That’s pretty cool. It’s exactly the same as it would be in our tasks: key, except it’s in a handler: key. Your whole playbook.yml should now look something like this:

---
- hosts: all
  sudo: true
  vars:
    document_root: /vagrant
  handlers:
    - name: reload apache
      service: name=apache2 state=reloaded
  tasks:
    - name: update apt cache
      apt: update_cache=yes
    - include: tasks/apache.yml
    - name: include mysql
      include: tasks/mysql.yml
    - include: tasks/php.yml

Now we need to tell Ansible to run this new Handler whenever we change the apache config. To do this we add a list of Handlers to notify when a Task is complete. Open up tasks/apache.yml and add the following on to each task that modifies your apache config:

notify:
  - reload apache

You call a Handler by the name you gave it, and you can call as many Handlers as you like, it’s just another list. Your whole tasks/apache.yml should now look similar to this:

---
- name: install apache
  apt: name=apache2 state=present

- name: Copy across new virtual host
  template:
    src=virtual-hosts.conf.j2
    dest=/etc/apache2/sites-available/vagrant.conf
  notify:
    - reload apache

- name: Remove default virtual host
  file:
    path=/etc/apache2/sites-enabled/000-default.conf
    state=absent
  notify:
    - reload apache

- name: Enable new vagrant virtual host
  file:
    src=/etc/apache2/sites-available/vagrant.conf
    dest=/etc/apache2/sites-enabled/vagrant.conf
    state=link
  notify:
    - reload apache

Let’s test this new setup by running vagrant destroy followed by vagrant up. We’re not just running vagrant provision this time because nothing would change, so our Handlers would never get called!

Now if you visit http://localhost:8080/info.php it should work straight away. You should also notice an extra line in the Ansible output:

NOTIFIED: [reload apache] *****************************************************
changed: [default]

Even though we called notify reload apache in three places, Ansible is smart enough to catch it and only run it once for us.

Roles

We’ve covered most of the basic functionality of Ansible now, but we’re left with a few small problems. Firstly, what happens if we want to write a generic Playbook or set of Tasks and then re-use them, and secondly, what happens when we want configure more than just a couple of modules? Our playbook.yml is going to become full of unrelated Handlers and variables and will get difficult to maintain.

Fortunately, Ansible can solve this problem with Roles. A Role is a group of Tasks, Handlers, Variables, Templates, Files, and so on all related to a single purpose. You could create specific “apache”, “MySQL”, and “PHP” Roles, or a generic high-level “webserver” Role. It doesn’t matter. How you group things in to Roles is completely up to you. For our example, we’ll take our existing structure and create two Roles:

  1. A Webserver Role (apache + PHP)
  2. A Database Role (MySQL)

Let’s start by creating a roles directory in the root of our project, and inside there we need to create two more directories, one to hold each Role.

mkdir -p roles/{webserver,database}

Now we’ve done that, we need to move things to the correct places. The directories inside each Role should be named the same way we’ve been naming them so far, so tasks move to a tasksdirectory within the Role, and templates to templates and so on. First, let’s move our Tasks:

mkdir roles/webserver/tasks
mv tasks/apache.yml roles/webserver/tasks/apache.yml
mv tasks/php.yml roles/webserver/tasks/php.yml

mkdir roles/database/tasks
mv tasks/mysql.yml roles/database/tasks/mysql.yml

We can now remove our top-level tasks directory which should already be empty:

rm -rf tasks

Next, we only have one Template which is for apache, so let’s move the entire templatesdirectory into our webserver Role:

mv templates roles/webserver/templates

You can create more templates directories in your other Roles if you need to, they will work the same way as the webserver one.

Once we have our files in the correct places for our Roles, we need to update our playbook.yml to tell it about the Roles. Open up playbook.yml and remove the following lines from our tasks:key:

- include: tasks/apache.yml
- name: include mysql
  include: tasks/mysql.yml
- include: tasks/php.yml

Now add our new roles: key and tell it we want to use the webserver and database roles we just created:

roles:
  - webserver
  - database

When you’re done, your playbook.yml file should look something like this:

---
- hosts: all
  sudo: true
  vars:
    document_root: /vagrant
  handlers:
    - name: reload apache
      service: name=apache2 state=reloaded
  tasks:
    - name: update apt cache
      apt: update_cache=yes
  roles:
    - webserver
    - database

Main Task File

You’ll notice that with our list of Roles we’re just specifying the name of the Role, which is just the name of the directory that contains the Role. Ansible doesn’t know which Tasks to run inside that Role though, as a Role can contain multiple Task files. The secret here is that Ansible will always look for a Task file called main.yml inside the tasks directory of a Role. Let’s create that for our webserver Role now:

Create a new file at roles/webserver/tasks/main.yml and add the following:

---
- include: apache.yml
- include: php.yml

Here, we’re just using the same include module from earlier. main.yml is just like any other Task list, so we can use all the same stuff. This time because main.yml is in the same directory as the files we want to include we don’t need to specify the directory name, it’s relative to the file you’re including them in to.

As our MySQL Role only has one file, we can just rename that to main.yml and it will work straight away:

mv roles/database/tasks/mysql.yml roles/database/tasks/main.yml

Pre & Post Tasks

Before we try this out, there’s one more thing we need to do. Ansible runs your Roles before your Tasks, so our packages will be installed before we’ve updated the apt cache. To fix this, we need to rename our tasks: key in playbook.yml to pre_tasks: to ensure it’s run before anything else:

---
- hosts: all
  sudo: true
  vars:
    document_root: /vagrant
  handlers:
    - name: reload apache
      service: name=apache2 state=reloaded
  pre_tasks:
    - name: update apt cache
      apt: update_cache=yes
  roles:
    - webserver
    - database

Once you’ve done that, save the file and drop back to your terminal to run vagrant destroy and vagrant up. Visit http://localhost:8080/info.php in your browser and everything should be working just as before.

Handlers

There’s one last thing we can improve here, and that’s to move our Handlers inside our Roles too. We only have one Handler at the moment, to reload our apache config, but the below will work just as well for any Role or package.

Handlers inside Roles work in exactly the same way as Tasks inside Roles, with a handlersdirectory and a main.yml file. Create a handlers directory in your webserver Role:

mkdir roles/webserver/handlers

Then take the handler from playbook.yml and add it to roles/webserver/handlers/main.yml:

---
- name: reload apache
  service: name=apache2 state=reloaded

The handlers: key in playbook.yml should now be empty and you can safely delete it, leaving our playbook looking something like :

---
- hosts: all
  sudo: true
  vars:
    document_root: /vagrant
  pre_tasks:
    - name: update apt cache
      apt: update_cache=yes
  roles:
    - webserver
    - database

One last vagrant destroy and vagrant up (or vagrant destroy -f && vagrant up if you like one-liners), and everything should still be working as expected.

Conculsion

When we started this post, I said that most Ansible Tutorials went in over-the-top and covered things you didn’t need to know for a basic and quick introduction. With this post I have tried to start with the minimum amount of knowledge required to get to something useful, and then built the more in-depth stuff in smaller easy to follow chunks. Hopefully this has given you a solid foundation in Ansible and made you realise you don’t need to jump in at the deep end, and you’ll be able to go off and apply this to your code or environment in a way that’s useful to you.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s