Ansible - DevOps automation

Infrastructure Automation with Ansible

Infrastructure automation has become an integral part of the modern DevOps lifecycle. Manual server configuration, package installation, application deployment, and resource scaling are significant time sinks. This approach is prone to human error and reduces the reproducibility of processes. To solve these problems, Ansible, a powerful automation tool, is used.

Ansible is an open-source tool for automating system administration tasks. It is written in Python and does not require agents to be installed on managed hosts. It uses the YAML format to describe configurations and the SSH protocol to connect to servers. Thanks to its simple and readable syntax, Ansible has gained widespread adoption among large companies, including NASA, Twitter, and Red Hat.

Getting Started with Ansible

How Ansible Works

Ansible functions as a tool for configuration automation, server management, process orchestration, and application deployment. Its architecture is based on Python, and it uses YAML as a Domain-Specific Language (DSL) to describe tasks.

Key Advantages

Ansible's agentless architecture ensures simplicity of deployment and reduces maintenance overhead. The use of the SSH protocol for connecting to remote hosts guarantees secure data transmission. The YAML syntax makes configuration files human-readable and easy to maintain.

Extensibility through Python modules allows for the creation of custom solutions for specific tasks. Support for both Windows and Linux operating systems provides versatility in heterogeneous environments.

Installation Process

Ansible can be installed via the pip package manager:

pip install ansible

Alternatively, you can install it through the system's package manager:

sudo apt install ansible

Architecture and System Components

Core Ansible Components

Inventory Structure

The inventory is a file that describes managed hosts and their groupings:

[web]
web1 ansible_host=192.168.1.10
web2 ansible_host=192.168.1.11

[database]
db1 ansible_host=192.168.1.20
db2 ansible_host=192.168.1.21

The inventory can contain variables for groups and individual hosts. This allows for customizing Ansible's behavior for different servers without modifying the main playbook files.

Creating and Structuring Playbooks

Playbook Basics

A playbook is a script in YAML format that describes tasks and their execution order. Each playbook consists of one or more plays, which define actions for specific groups of hosts.

A basic playbook structure includes the following elements:

- name: Install and configure Nginx
  hosts: web
  become: true
  tasks:
    - name: Install Nginx package
      apt:
        name: nginx
        state: present
    
    - name: Start Nginx service
      service:
        name: nginx
        state: started
        enabled: true

Key Playbook Parameters

The name parameter provides a human-readable description of the task. The hosts directive specifies the target group of servers from the inventory. The tasks section contains the list of actions to be performed. The become option activates privilege escalation (sudo) to perform operations as the superuser.

Conditional Logic and Loops

Ansible supports conditional constructs and loops to create flexible scenarios:

- name: Install packages
  apt:
    name: "{{ item }}"
    state: present
  loop:
    - nginx
    - mysql-server
    - php-fpm
  when: ansible_os_family == "Debian"

Ansible Modules and Their Usage

Popular System Modules

Modules are units of logic that perform specific operations. Ansible comes with an extensive library of built-in modules for various tasks.

Practical Module Examples

The copy module transfers files from the control host to target servers:

- name: Copy configuration file
  copy:
    src: ./nginx.conf
    dest: /etc/nginx/nginx.conf
    owner: root
    group: root
    mode: '0644'
    backup: true

The template module allows the use of variables in configuration files:

- name: Generate virtual host
  template:
    src: vhost.j2
    dest: "/etc/nginx/sites-available/{{ domain_name }}"

Creating Custom Modules

Ansible allows the creation of custom modules in Python for specific tasks. Custom modules are placed in the library directory of the project or in system-wide Ansible paths.

Variables and Templating

Variable Types and Sources

Variables in Ansible provide flexibility and reusability for configurations. They can be defined in various places with different priorities.

Variables can be set directly in a playbook:

vars:
  app_port: 8080
  app_name: "mywebapp"
  database_host: "db.example.com"

Inventory variables are defined for groups of hosts or individual servers:

[web:vars]
nginx_port=80
ssl_enabled=true

[web]
web1 ansible_host=192.168.1.10 app_env=production
web2 ansible_host=192.168.1.11 app_env=staging

Working with Jinja2 Templates

The Jinja2 templating system is integrated into Ansible for generating configuration files. Templates support conditional logic, loops, and filters.

Content of the template.j2 file:

server {
    listen {{ nginx_port | default(80) }};
    server_name {{ server_name }};
    
    {% if ssl_enabled %}
    listen 443 ssl;
    ssl_certificate {{ ssl_cert_path }};
    ssl_certificate_key {{ ssl_key_path }};
    {% endif %}
    
    location / {
        proxy_pass http://127.0.0.1:{{ app_port }};
        proxy_set_header Host $host;
    }
}

Environment Variables and Facts

Ansible automatically gathers system information (facts) when connecting to hosts. This data is available as variables in playbooks:

- name: Conditionally install packages
  apt:
    name: "{{ item }}"
    state: present
  loop:
    - nginx
    - php-fpm
  when: ansible_distribution == "Ubuntu" and ansible_distribution_major_version|int >= 18

Roles and Project Organization

Role Structure and Creation

Roles provide modularity and reusability for Ansible code. They allow you to break down logic into separate components and simplify the maintenance of large projects.

A new role is created with the command:

ansible-galaxy init webserver

This command creates a standard directory structure:

webserver/
├── tasks/main.yml          # Main tasks for the role
├── handlers/main.yml       # Event handlers
├── templates/              # Jinja2 templates
├── files/                  # Static files
├── vars/main.yml           # Role variables
├── defaults/main.yml       # Default values
├── meta/main.yml           # Metadata and dependencies
└── README.md               # Role documentation

Using Roles in a Playbook

Roles are included in a playbook via the roles section:

- name: Configure web servers
  hosts: web
  become: true
  roles:
    - role: webserver
      vars:
        nginx_port: 8080
        ssl_enabled: true
    - role: monitoring
      when: enable_monitoring | default(false)

Role Dependencies

The meta/main.yml file defines a role's dependencies on other roles:

dependencies:
  - role: common
    vars:
      timezone: "UTC"
  - role: firewall
    when: enable_firewall | default(true)

Programmatic Interface

The ansible-runner Library

Ansible-runner provides a Python API for programmatically running playbooks and ad-hoc commands. This allows for integrating Ansible into custom applications and automation systems.

A basic usage example:

import ansible_runner

result = ansible_runner.run(
    private_data_dir='/tmp/ansible_workspace',
    playbook='deploy.yml',
    inventory='hosts.ini',
    extravars={'app_version': '1.2.3'}
)

print(f"Execution status: {result.status}")
print(f"Return code: {result.rc}")
print(f"Stats: {result.stats}")

Managing Inventory via the API

Programmatic inventory management allows for the dynamic generation of host lists:

from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager

loader = DataLoader()
inventory = InventoryManager(loader=loader, sources=['hosts.ini'])

# Get all hosts
all_hosts = inventory.get_hosts()
for host in all_hosts:
    print(f"Host: {host.name}, IP: {host.address}")

# Get hosts in a group
web_hosts = inventory.get_hosts(pattern='web')

Asynchronous Execution

Ansible-runner supports asynchronous execution for long-running operations:

import ansible_runner

# Asynchronous run
thread, runner = ansible_runner.run_async(
    playbook='deploy.yml',
    inventory='hosts.ini'
)

# Wait for completion
thread.join()
print(f"Result: {runner.status}")

Ansible Galaxy and the Ecosystem

Working with Collections

Ansible Galaxy is a central repository for roles and collections created by the community. Collections contain roles, modules, plugins, and other components.

Install a role from Galaxy:

ansible-galaxy install geerlingguy.mysql
ansible-galaxy install -r requirements.yml

A requirements.yml file for managing dependencies:

roles:
  - name: geerlingguy.nginx
    version: "3.1.0"
  - name: geerlingguy.mysql
    version: "4.0.0"

collections:
  - name: community.general
    version: ">=3.0.0"
  - name: ansible.posix

Creating Custom Collections

Ansible Galaxy supports the creation and publication of custom collections:

ansible-galaxy collection init mycompany.myapp

A collection's structure includes roles, modules, plugins, and documentation in a single package.

Integration with CI/CD Systems

GitHub Actions

Integrating Ansible with GitHub Actions automates deployment on repository changes:

name: Deploy Application
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v3
      
      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.9'
      
      - name: Install Ansible
        run: |
          pip install ansible
          ansible-galaxy install -r requirements.yml
      
      - name: Deploy application
        run: |
          ansible-playbook -i inventory/production deploy.yml
        env:
          ANSIBLE_HOST_KEY_CHECKING: false

Jenkins Pipeline

Integration with Jenkins allows for creating a pipeline to automate processes:

pipeline {
    agent any
    
    stages {
        stage('Checkout') {
            steps {
                git 'https://github.com/company/ansible-deploy.git'
            }
        }
        
        stage('Install Dependencies') {
            steps {
                sh 'pip install ansible'
                sh 'ansible-galaxy install -r requirements.yml'
            }
        }
        
        stage('Deploy') {
            steps {
                sh 'ansible-playbook -i inventory/production deploy.yml'
            }
        }
    }
}

GitLab CI

Configuration for GitLab CI/CD:

stages:
  - deploy

deploy_production:
  stage: deploy
  image: python:3.9
  before_script:
    - pip install ansible
    - ansible-galaxy install -r requirements.yml
  script:
    - ansible-playbook -i inventory/production deploy.yml
  only:
    - main

Security and Best Practices

Managing Secrets with Ansible Vault

Ansible Vault provides encryption for sensitive data in configuration files:

# Create an encrypted file
ansible-vault create secrets.yml

# Encrypt an existing file
ansible-vault encrypt database_passwords.yml

# Edit an encrypted file
ansible-vault edit secrets.yml

# Decrypt a file
ansible-vault decrypt secrets.yml

Using encrypted variables in a playbook:

- name: Configure database
  hosts: database
  vars_files:
    - secrets.yml
  tasks:
    - name: Create database user
      mysql_user:
        name: "{{ db_user }}"
        password: "{{ db_password }}"
        state: present

Security Principles

Applying the principle of least privilege means granting only the necessary permissions to perform tasks. Using SSH keys instead of passwords enhances authentication security.

Logging execution operations provides an audit trail of infrastructure changes. Regularly updating Ansible and its dependencies patches known vulnerabilities.

Restricting access to inventory and playbooks through version control systems with appropriate permissions prevents unauthorized changes.

Debugging and Troubleshooting

Verbose Output for Diagnostics

Ansible provides various levels of output verbosity for diagnosing problems:

# Basic debug level
ansible-playbook -v site.yml

# Detailed task information
ansible-playbook -vv site.yml

# Maximum verbosity
ansible-playbook -vvv site.yml

# Connection debugging
ansible-playbook -vvvv site.yml

Syntax Checking and Validation

Check a playbook's correctness before execution:

# Check YAML syntax
ansible-playbook --syntax-check deploy.yml

# Dry-run execution without making changes
ansible-playbook --check deploy.yml

# Show differences during a dry-run
ansible-playbook --check --diff deploy.yml

# Validate inventory
ansible-inventory --list

Error Handling Strategies

Ansible provides mechanisms for graceful error handling:

- name: Attempt to restart service
  service:
    name: nginx
    state: restarted
  ignore_errors: true
  register: restart_result

- name: Alternative action on error
  debug:
    msg: "Failed to restart Nginx: {{ restart_result.msg }}"
  when: restart_result.failed

Using blocks to group tasks with error handling:

- block:
    - name: Critical operation
      command: /usr/local/bin/critical-script.sh
    
    - name: Subsequent actions
      file:
        path: /tmp/success.flag
        state: touch
  
  rescue:
    - name: Actions on error
      debug:
        msg: "An error occurred in the block"
  
  always:
    - name: Cleanup resources
      file:
        path: /tmp/temp-data
        state: absent

Comparison with Alternatives

Analysis of Automation Tools

The Ansible Advantage

Its ease of learning makes Ansible accessible to teams with varying levels of expertise. The agentless architecture simplifies deployment and reduces overhead. Using SSH ensures security without additional setup.

The readability of YAML configurations facilitates code review and team collaboration. A rich ecosystem of modules covers most infrastructure automation tasks.

Python API Components

The ansible-core Library

Ansible-core provides the basic functionality for working with the system programmatically.

A practical example of working with DataLoader and InventoryManager:

from ansible.parsing.dataloader import DataLoader
from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager

# Initialize components
loader = DataLoader()
inventory = InventoryManager(loader=loader, sources=['hosts.ini'])
variable_manager = VariableManager(loader=loader, inventory=inventory)

# Get host information
for host in inventory.list_hosts():
    host_vars = variable_manager.get_vars(host=host)
    print(f"Host: {host.name}")
    print(f"Variables: {host_vars}")

The ansible-runner Module for Executing Operations

Ansible-runner provides a high-level interface for running playbooks and ad-hoc commands from Python applications.

An advanced example with result handling:

import ansible_runner
import json

# Execution configuration
config = {
    'inventory': 'hosts.ini',
    'playbook': 'deploy.yml',
    'extravars': {
        'app_version': '2.1.0',
        'environment': 'production'
    },
    'verbosity': 2
}

# Execute playbook
result = ansible_runner.run(**config)

# Analyze results
if result.status == 'successful':
    print("Deployment completed successfully")
    print(f"Hosts processed: {len(result.stats)}")
    for host, stats in result.stats.items():
        print(f"{host}: ok={stats.get('ok', 0)}, changed={stats.get('changed', 0)}")
else:
    print(f"Execution failed: {result.status}")
    
# Get execution events
for event in result.events:
    if event['event'] == 'runner_on_failed':
        print(f"Error in task: {event['event_data']['task']}")
        print(f"Message: {event['event_data']['res']['msg']}")

Creating Custom Modules

The ansible.module_utils module provides base classes for creating custom Ansible modules.

Example of creating a module to check web service availability:

#!/usr/bin/python

from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.urls import open_url
import json

def check_service_health(url, timeout):
    """Check web service availability"""
    try:
        response = open_url(url, timeout=timeout)
        if response.getcode() == 200:
            return True, "Service is available"
        else:
            return False, f"HTTP code: {response.getcode()}"
    except Exception as e:
        return False, f"Connection error: {str(e)}"

def main():
    module = AnsibleModule(
        argument_spec=dict(
            url=dict(type='str', required=True),
            timeout=dict(type='int', default=10),
            expected_status=dict(type='int', default=200)
        ),
        supports_check_mode=True
    )
    
    url = module.params['url']
    timeout = module.params['timeout']
    
    if module.check_mode:
        module.exit_json(changed=False, msg="Check mode")
    
    is_healthy, message = check_service_health(url, timeout)
    
    module.exit_json(
        changed=False,
        healthy=is_healthy,
        msg=message,
        url=url
    )

if __name__ == '__main__':
    main()

Managing Variables and Inventory

Components for programmatically working with variables and hosts:

from ansible.inventory.manager import InventoryManager
from ansible.vars.manager import VariableManager
from ansible.parsing.dataloader import DataLoader

class AnsibleInventoryManager:
    def __init__(self, inventory_sources):
        self.loader = DataLoader()
        self.inventory = InventoryManager(
            loader=self.loader, 
            sources=inventory_sources
        )
        self.variable_manager = VariableManager(
            loader=self.loader, 
            inventory=self.inventory
        )
    
    def get_host_variables(self, hostname):
        """Get variables for a specific host"""
        host = self.inventory.get_host(hostname)
        if host:
            return self.variable_manager.get_vars(host=host)
        return None
    
    def get_group_hosts(self, group_name):
        """Get a list of hosts in a group"""
        group = self.inventory.groups.get(group_name)
        if group:
            return [host.name for host in group.hosts]
        return []
    
    def add_dynamic_host(self, hostname, group_name, host_vars=None):
        """Dynamically add a host"""
        self.inventory.add_host(hostname, group=group_name)
        if host_vars:
            host = self.inventory.get_host(hostname)
            for key, value in host_vars.items():
                host.set_variable(key, value)

# Usage
inventory_manager = AnsibleInventoryManager(['hosts.ini'])
web_hosts = inventory_manager.get_group_hosts('web')
print(f"Web servers: {web_hosts}")

Practical Recommendations and Conclusion

Performance Optimization

To improve Ansible's performance, it is recommended to use parallel execution via the forks parameter. Configuring SSH connections with ControlMaster and ControlPersist reduces the overhead of establishing connections.

Using execution strategies like free or linear can optimize the order in which hosts are processed. Using fact caching speeds up subsequent playbook runs.

Structuring Projects

Organizing a project based on the principle of separation of concerns simplifies maintenance. Using separate directories for roles, inventory, and variables provides a logical structure.

Applying naming conventions for roles, variables, and files enhances code readability. Documenting roles and playbooks facilitates teamwork.

Monitoring and Logging

Setting up centralized logging for Ansible operations provides an audit trail of infrastructure changes. Using callback plugins allows for integrating Ansible with monitoring systems.

Ansible is a powerful and versatile tool for infrastructure automation. The combination of its simple YAML syntax, the flexibility of Python, an extensive module library, and an agentless architecture makes it an optimal choice for DevOps teams. Support for a programmatic interface via its Python API ensures integration with existing automation systems. For DevOps specialists and developers, Ansible remains one of the best solutions for configuration management, application deployment, and infrastructure scaling.