Let's roll


Roles, the building blocks of Ansible. They have the same concept as the modules in Puppet if you are familiar with it. Of course you can do without, but eventually you will stumble upon a set of playbook tasks that you have done before. That's where roles come in. They allow you to create repeatable idempotent tasks that belong together, with their own variables files and templates.

Image of bricks

Let's break down the roles setup in Ansible below.

The roles path


The default path of your roles is in the root of your Ansible directory. You can override this in your ansible.cfg file with the roles_path variable and even use multiple lookup locations. If you have multiple roles directory's and use some sort of version control this can come in very handy.

ansible.cfg

roles_path = /etc/ansible/global_roles/v1/:/etc/ansible/global_roles/v2/

Roles will first be searched for in the playbook directory, so you can overwrite a role in the roles_path with your own version. Let's say you have a default Apache role included in your global_roles path, but this special customer needs a heavily reworked role for his Apache setup? No worries, just create it in the ansible root of your project and it will take precedence over the Apache role in your roles_path.

Inside a role directory


To create a proper role directory, you can use the ansible-galaxy command to bootstrap this for you. To create a new Apache role in your roles directory, execute the following command in your Ansible root directory: /

ansible-galaxy init roles/cloudrkt.apache

This command will create the entire apache role directory structure for you. It's common to append your custom identifier like id.role if you are sharing them later in the Ansible Galaxy. Let's see whats inside:

roles/cloudrkt.apache/

-- cloudrkt.apache
   |-- README.md
   |-- defaults
   |   `-- main.yml
   |-- files
   |-- handlers
   |   `-- main.yml
   |-- meta
   |   `-- main.yml
   |-- tasks
   |   `-- main.yml
   |-- templates
   |-- tests
   |   |-- inventory
   |   `-- test.yml
   `-- vars
       `-- main.yml

Let's go over all the directory's and files found in the newly created /roles/cloudrkt.apache directory.

README.md


The README.md file is pre-filled with some useful information explaining how to enable and control your role in your playbook and configure the default variables. To manage your roles properly, you want them in a version control system like Git and Markdown is the default way off displaying information nowadays. Please fill this in properly, you'll thank yourself later when you need to point your new colleague to your roles repository.

You should include all the different variables you use in your defaults directory as described below to configure your role in the README.

roles/cloudrkt.apache/README.md

Welcome to the CloudRKT apache role.

Install info:

- hosts: webservers
  roles:
    - cloudrkt.apache

Defaults


The defaults directory should contain all the default variables you are using in this role. For the Apache role it could be the version of the Apache installation package, your default port or version settings. Don't worry about flexibility, the reason we put this data in a variable is that you can change or overwrite it later when including a role in your playbook. The default variables are meant to be overwritten or provide a sane default when not set in your playbooks, host_vars, group_vars or inventory file.

roles/cloudrkt.apache/defaults/main.yml

---
# Default variables for the apache role.

apache_server_version: 2.4
apache_server_state: "running"
apache_server_service_name: "apache2"

apache_server_global_cnf_timeout: 300
apache_server_global_cnf_keepalive: "On"
apache_server_global_cnf_maxkeepaliverequests: 100
apache_server_global_cnf_keepalivetimeout: 5

Why the long variable names you ask? This is to identify the origin role of your variables and make it less likely to get overwritten by a similar role or task. You can also group variables in a yaml dictionary like this:

roles/cloudrkt.apache/defaults/main.yml

---
# Default variables for the cloudrkt.apache role.

apache_server_vars:
    default_ssl_port: 443
    default_nonssl_port: 80

Be careful that you do not overwrite this dictionary with another dictionary. If you set this dictionary again but miss out on some variable,it will not get set.

---
# Default variables for the cloudrkt.apache role.

apache_server_vars:
    default_ssl_port: 443

Will unset: default_nonssl_port: 80. This could lead to strange behavior and a lot of head scratching.

It's mandatory to name your default variable file main.yml. The name main.yml is hard coded into the ansible source code. In practice you will set variables for a role from your playbook, host_vars and group_vars directly as much as possible. The default directory helps out if there must be a default value present.

Vars directory


The variables in the vars/ directory are meant to stay and can be used to form paths or set variables that are used in executing the right tasks in your role.

roles/cloudrkt.apache/vars/main.yml

---
# Main variables for the cloudrkt.apache role.

apache_server_role_os_family_supported:
  - 'Debian'

apache_server_versions_supported:
  - '2.4'

apache_server_dependencies:
  - "apache2={{ apache_server_version }}*"

apache_server_main_cnf: "{{ apache_server_cnf_path }}{{ apache_server_cnf_file }}"

Files


In the files directory you can put files you don't alter. This could be a binary file that you need to transfer from the ansible host, or even configuration files that you don't want to setup as a template. Files in this directory can be called upon like this:

roles/cloudrkt.apache/tasks/main.yml

---
# Tasks for the cloudrkt.apache role.

- name: copy this binary blob to /tmp/
  file:
    src=usr/local/bin/myscript.exe
    dest=/tmp/

Files path


Your role in combination with the file or copy module will use the files folder by default, this way you don't need to refer to the files directory. Always try to use the complete path in your files directory for your source files like "etc/apache/conf.d/security.conf". This will help you set the correct destination path when creating a task.

Large files


In practice it turns out you try to avoid copying big files to the server via the files directory and rather download them from a location you control directly. Who wants a 100Mb binary blob in their roles repository anyways?

Handlers


The handlers directory is a directory where you place your handlers that behave the same way as in your playbooks. Here you can specify all the actions that need to happen when a task in the role makes a change and sends out a notification.

roles/cloudrkt.apache/handlers/main.yml

---
# Handlers for the cloudrkt.apache role.

- name: restart apache
  service:
    name: "{{ apache_server_service_name | default(apache2) }}"
    state: restarted
  when: apache_server_state == "running"

- name: reload apache
  service:
    name: "{{ apache_server_service_name | default(apache2) }}"
    state: reloaded
  when: apache_server_state == "running"

Meta directory


The meta directory is used to describe your role. The main parts are the galaxy info and the dependencies used for the role.

Galaxy Info


In the meta/main file you can specify the OS you support under the platforms. This does not enforce the OS but offers information to the Ansible Galaxy info command and website when searching for roles.

Dependencies


The meta directory is foremost used to describe dependencies on other roles and let them execute automatically. For example; if you need PHP installed and your standard web server is Apache, you can specify a dependency in your roles/php/meta/main.yml like this:

webserver-playbook.yml

---
# Playbook for a webserver 

- hosts: localhost
  connection: local
  gather_facts: no
  roles:
    - { role: php, webserver: cloudrkt.apache }

roles/php/meta/main.yml

---
# Role dependency in meta/main inside php role 

dependencies:
  - { role: 'cloudrkt.apache', phpmode: "{{ role_php_apache_mod_php }}", when: webserver == 'apache_server' }
  - { role: nginx, phpmode: "{{ role_php_php_fpm }}", when: webserver == 'nginx' }

roles/php/default/main.yml

---
# Default variables for the php role.

php_apache_mod_php: apache2-mod-php
php_fpm: php-fpm

The rest of the meta.yml can be used to describe the role for the galaxy repository, ansible version, supported operating system etc. In practice I never had to use dependencies when configuring servers and middleware, but this can come in handy when you use Ansible roles for deploying applications.

Tasks


The tasks directory contains all the tasks that install on the selected hosts in your playbook. You can include your tasks per file for a better representation and separation of concerns. Your main.yml is your entry point for the role, and the tasks defined here get executed first, but only if the role does not have any dependencies in the meta/main.yml file.

roles/cloudrkt.apache/tasks

|-- tasks
|   |-- main.yml
|   |-- version-check.yml
|   |-- install.yml
|   |-- configure.yml
|   |-- vhost.yml
|   `-- state.yml

Let's go a level deeper and checkout all the files in the task directory.

Main.yml


This is an example of the main.yml you could use to create the apache role. The main file includes the rest of your tasks and takes care of the right order. Try to keep this file as clean as possible, and include files as you need them. When you need to extend your role you can use the main.yml to easily separate the tasks.

roles/cloudrkt.apache/tasks/main.yml

---
# main.yml file for roles/cloudrkt.apache/tasks

- include: version-support.yml
  tags:
   - role-apache
   - role-apache-version-support

- include: install.yml
  tags:
   - role-apache
   - role-apache-install

- include: configure.yml
  tags:
   - role-apache
   - role-apache-configure

- include: vhost.yml
  tags:
   - role-apache
   - role-apache-vhost

- include: state.yml
  tags:
   - role-apache
   - role-apache-state

Always make use of tags in your roles. This allows you to execute specific parts of your role like a adding a virtual host without running the entire role from the beginning. Also try to map your tags to the include names for better control.

Version-support.yml


The version-support file can stop a user from shooting himself in the foot. The galaxy info in the meta directory will not prevent installing the role on the wrong OS with the wrong version. This playbook will fail if it gets executed on the unsupported OS or with an unsupported Apache version.

roles/cloudrkt.apache/tasks/version-support.yml

---
# Check version compatibility for roles/cloudrkt.apache

- name: check | apache version support
  fail:
    msg: "Apache Server version {{ apache_server_version }} is not supported"
  when: apache_server_version | string not in apache_server_versions_supported
  tags:
    - apache-server-version-support-check

- name: check | OS version support
  fail:
    msg: "Operating System version {{ ansible_os_family }} is not supported"
  when: ansible_os_family  | string not in apache_server_role_os_family_supported
  tags:
    - apache-server-version-support-check

Install.yml


This file points to the right tasks so you can install the packages en set the configuration per operating system.

roles/cloudrkt.apache/tasks/install.yml

---
# Main install of the cloudrkt.apache role. 

- name: install | dependencies
  apt:
    name: "{{ item }}"
    state: "{{ apache_server_update_packages|default(installed) }}"
  with_items: 
    - "{{ apache_server_dependencies }}"
  tags:
    - apache-webserver-install-dependencies

- name: install | modules
  apt:
    name: "{{ item }}"
    state: "{{ apache_server_update_packages|default(installed) }}"
  with_items:
    - "{{ apache_server_modules }}"
  tags:
    - apache-webserver-install-modules

- name: install | additional
  apt:
    name: "{{ item }}"
    state: "{{ apache_server_update_packages|default(installed) }}"
  with_items:
    - "{{ apache_server_additional_packages }}"
  tags:
    - apache-webserver-install-additional

In this example I point to the install.yml but you can set it up like centos.yml and debian.yml for CentOS and Debian settings if you are installing your role on multiple operating systems. You can make it smarter by using:

roles/cloudrkt.apache/tasks/main.yml

  include: "{{ item }}"
  with_first_found:
    - "{{ ansible_distribution }}.yml"
    - "{{ ansible_os_family }}.yml"
    - "install.yml"
  tags:
    - role-apache
    - role-apache-install

Configure.yml


In the configure.yml you can configure your installation. Take care that your role will install even if nothing is set. This file will contain the general settings needed to configure the installed role.

roles/cloudrkt.apache/tasks/configure.yml

---
# Configure the cloudrkt.apache role.

- name: configure | setup apache server configuration
  template:
    src: "etc/apache2/{{ item }}.j2"
    dest: "{{ apache_server_cnf_path }}{{ item | basename | regex_replace('.j2$', '') }}"
    backup: yes
  with_items:
    - "{{ apache_server_global_cnf_file_includes | default([]) }}"
    - "apache2.conf"
  notify:
    - restart apache

- name: configure | enable installed apache modules
  apache2_module:
    name: "{{ item }}"
    state: present
    force: yes
  with_items:
    - "{{ apache_server_modules_enabled }}"
  notify:
    - restart apache

- name: configure | disable installed apache modules
  apache2_module:
    name: "{{ item }}"
    state: absent
    force: yes
  with_items:
    - "{{ apache_server_modules_disabled }}"
  notify:
    - restart apache

Vhost.yml


In the vhost.yml you can set configuration for vhosts obviously, but the problem is that some configurations are not generic enough to fit in the default configuration.yml. For Apache you will configure a lot of settings per vhost that differ per domain.

To address this we use a separate configuration file just for vhost called vhost.yml. This can be called with tags as a separate set of tasks.

roles/cloudrkt.apache/tasks/vhost.yml

---
# Configure the vhosts in your apache installation

- name: vhost | add apache vhosts configuration.
  template:
    src: "vhost.conf.j2"
    dest: "{{ apache_server_cnf_path }}/sites-available/{{ item.vhost }}"
    owner: root
    group: root
    mode: 0644
  with_items: "{{ apache_server_enabled_vhosts }}"

- name:  vhost | add vhost symlink in sites-enabled.
  file:
    src: "{{ apache_server_cnf_path }}/sites-available/{{ item.vhost }}"
    dest: "{{ apache_server_cnf_path }}/sites-enabled/{{ item.vhost }}"
    state: link
  with_items: "{{ apache_server_enabled_vhosts }}"
  notify: 
    - reload apache

- name: vhost | remove apache vhosts configuration.
  file:
    dest: "{{ apache_server_cnf_path }}/sites-available/{{ item.vhost }}"
    state: absent
  with_items: "{{ apache_server_disabled_vhosts }}"

- name:  vhost | remove vhost symlink in sites-enabled.
  file:
    dest: "{{ apache_server_cnf_path }}/sites-enabled/{{ item.vhost }}"
    state: absent
  with_items: "{{ apache_server_disabled_vhosts }}"
  notify: 
    - reload apache

To set vhosts correctly you'll need templates, variables and a default or standard way to interact with the Apache role.

[Optional] Security.yml


The security.yml is an extension for your configuration.yml focused towards security related settings in your installation. You cannot only set security settings but also assert and validate your settings for example.

[Optional] Backup.yml


The backup.yml file will help you setup the best way to backup your Apache settings or files, in conjunction with your backup software. This could be as simple as setting a scheduled task or cronjob to backup vhosts, or copy entire directory's to another server.

[Optional] Monitor.yml


The monitor.yml will help you setup the hooks for your monitoring system or even setup alert notifications from the server itself depending on your preferences. It can also setup the right configuration for your specific monitoring solution.

State.yml


The state file will run as a final step to ensure the services needed are running and available if defined. This file will help you change the state of your service if needed.

roles/cloudrkt.apache/tasks/state.yml

---
# Configure the default state in your apache installation

- name: start apache
  service:
    name: "{{ apache_server_service_name | default(apache2) }}"
    state: started
  when: apache_server_state is defined and apache_server_state == "running"

- name: stop apache
  service:
    name: "{{ apache_server_service_name | default(apache2) }}"
    state: stopped
  when: apache_server_state is defined and apache_server_state == "stopped"

Templates


The template directory is used for the templates that come with your role. Please try to use the full path if possible as it is easy to setup the src and dest in your tasks later on.

roles/cloudrkt.apache/templates/

|-- etc
|   `-- apache2
|       |-- apache2.conf.j2
|       `-- ports.conf.j2
`-- vhost.conf.j2

Example for the apache2 vhost:

# {{ ansible_managed }}

{% if item.http_listen is defined %}
{% for listen in item.http_listen %}
<VirtualHost {{listen.ip}}:{{listen.port}} >
{% if item.servername is defined %} 
    ServerName {{ item.servername }}
{%- endif %}
{% if item.serveralias is defined %} 
    ServerAlias {{ item.serveralias }}
{%- endif %}
{% if item.serveradmin is defined %} 
    ServerAdmin {{ item.serveradmin }}
{%- endif %}
{% if item.document_root is defined %} 
    DocumentRoot {{ item.document_root }}
{% endif %}
{% if item.setenv is defined %} 
{% for env in item.setenv %}
    SetEnv {{ env.name }} "{{ env.value }}"
{% endfor %}
{% endif %}
{% if item.directory is defined -%} 
{% for directory in item.directory %}

    <Directory "{{ directory.path }}">
      {{ directory.parameters }}
    </Directory>
{% endfor %}
{% endif %}

{% if item.extra_http_vhost_parameters is defined %}
    {{ item.extra_http_vhost_parameters }}
{% endif %}

</VirtualHost>
{% endfor %}
{% endif %}

# - 

{% if item.https_listen is defined %}
{% for listen in item.https_listen %}
<VirtualHost {{listen.ip}}:{{listen.port}} >
{% if item.servername is defined %} 
    ServerName {{ item.servername }}
{%- endif %}
{% if item.serveralias is defined %} 
    ServerAlias {{ item.serveralias }}
{%- endif %}
{% if item.serveradmin is defined %} 
    ServerAdmin {{ item.serveradmin }}
{%- endif %}
{% if item.document_root is defined %} 
    DocumentRoot {{ item.document_root }}
{% endif %}
{% if item.setenv is defined %} 
{% for env in item.setenv %}
    SetEnv {{ env.name }} "{{ env.value }}"
{% endfor %}
{% endif %}
{% if item.directory is defined -%} 
{% for directory in item.directory %}

    <Directory "{{ directory.path }}">
      {{ directory.parameters }}
    </Directory>
{% endfor %}
{% endif %}

    SSLEngine on

{% if item.sslhonorcipherorder is defined %}
    SSLHonorCipherOrder {{ item.sslhonorcipherorder }}
{% endif %}
{% if item.sslciphersuite is defined %}
    SSLCipherSuite {{ item.sslciphersuite }}
{% endif %}
{% if item.sslusestapling is defined %}
    SSLUseStapling {{ item.sslusestapling }}
{% endif %}
{% if item.sslcompression is defined %}
    SSLCompression {{ item.sslcompression }}
{% endif %}

{% if item.certificate_file is defined %}
    SSLCertificateFile {{ item.certificate_file }}
{% endif %}
{% if item.certificate_key_file is defined %}
    SSLCertificateKeyFile {{ item.certificate_key_file }}
{% endif %}
{% if item.certificate_chain_file is defined %}
    SSLCertificateChainFile {{ item.certificate_chain_file }}
{% endif %}

{% if item.extra_https_vhost_parameters is defined %}
    {{ item.extra_https_vhost_parameters }}
{% endif %}

</VirtualHost>
{% endfor %}
{% endif %}

And variables that you could use in the host_vars, group_vars, separate var files or playbooks.

apache_server_enabled_vhosts: 
  - vhost: "000-cloudrkt.com.conf"
    servername: "www.cloudrkt.com"
    serveralias: "cloudrkt.com"
    serveradmin: "systemadministrator@cloudrkt.com"
    document_root: "/var/www/html"
    setenv:
      - name: "APPLICATION_ENV"
        value: "Production"
      - name: "SPECIAL_THANKS_ENV"
        value: "CloudRKT"
    directory:
      - path: "/"
        parameters: >
          AllowOverride All
                Options -Indexes +FollowSymLinks
                Require all granted
      - path: "/conf"
        parameters: >
          AllowOverride All
                Options -Indexes +FollowSymLinks
                Require all denied
    http_listen:
      - port: 80
        ip: "*"
    extra_http_vhost_parameters: |
      RewriteEngine Off
    https_listen:
      - port: 443
        ip: "*"
    certificate_file: "/etc/ssl/certs/ssl-cert-snakeoil.pem"
    certificate_key_file: "/etc/ssl/private/ssl-cert-snakeoil.key"
    sslhonorcipherorder: "on"
    sslusestapling: "on"
    sslcompression: "off"
    sslstaplingcache: "shmcb:logs/stapling-cache(150000)"
    sslciphersuite: "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH"
    extra_https_vhost_parameters: |
      RewriteEngine On
          RewriteCond "%{TIME_HOUR}" ">=20" [OR]
          RewriteCond "%{TIME_HOUR}" "<07" 
          RewriteRule "^/rkt"     "-" [F]

Tests


In this directory you will setup the inventory, playbook and role to setup your role on a test instance like Vagrant and CI/CD system. I will setup a testing framework for roles in another post.

Include roles in your playbook.


Roles can be included in your playbook before your tasks are run. It looks like this:

---
# My webserver playbook

- hosts: webservers
  roles:
    - cloudrkt.apache

Extra


Remember that you can always begin with a small role, or even some tasks in a playbook if you don't know how to setup role with ail the settings like above. If you are comfortable with writing tasks and you are reusing a lot of your code to do the same it's time to setup roles.

  • Always try to make your role install and run without any configuration set by the user. It should produce the same installation as your favorite package manager does.
  • Make your role as complete as possible and configure settings using variables in your playbook, host_vars or group_vars and templates.
  • If you cannot create a huge template file to cover all the settings but just need a few settings to change, you can include your file or template from the role include like this:
- hosts: webservers
  roles:
    - { role: cloudrkt.apache, apache_server_external_template_file: path/to/template.j2 }

With this option you can copy the original file from the installation and use that as a baseline for your template file.

Role on out


You can find this role on the Ansible Galaxy In the meanwhile have fun with building and rolling out your Ansible roles.


Comments

comments powered by Disqus