[PYTHON] [Small story] Two ways to correctly deep merge Ansible and nested dict variables

How to merge variables in Ansible?

Ansible uses Jinja2 for the variable Filter.

Do you sometimes want to deep merge variables including dictionaries cleanly? Today, I will introduce two ways to realize deep merge of variables in Ansible. What's different is that the method is different for Ansible 2.0 and above and below.

For Ansible version "> = 2.0"

A new ** combine filter ** has been implemented since Ansible 2.0. http://docs.ansible.com/ansible/playbooks_filters.html#combining-hashes-dictionaries

test.yml

- hosts: localhost
  gather_facts: no
  vars:
    dict:
      foo:
        bar: 1
    dict2:
      foo:
        baz: 2
      qux: 2
    # combining hashes/dictionaries (new in version 2.0)
    dict_combine: "{{ dict | combine(dict2, recursive=True) }}"
  tasks:
    - debug:
        var: dict_combine

If you do the above, you will get the following results.

$ ansible-playbook -i localhost, test.yml

PLAY [localhost] ***************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "dict_combine": {
        "foo": {
            "bar": 1,
            "baz": 2
        },
        "qux": 2
    }
}

It's very convenient. The contents of dict-> foo are properly merged. By the way, omitting "recursive" is just an update merge.

recursive=false

$ ansible-playbook -i localhost, test.yml

PLAY [localhost] ***************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "dict_combine": {
        "foo": {
            "baz": 2
        },
        "qux": 2
    }
}

By the way, the function equivalent to combine that omits recursive could be realized as follows by using update () even in the version before Ansible 2.0.

test.yml

- hosts: localhost
  gather_facts: no
  vars:
    dict:
      foo:
        bar: 1
    dict2:
      foo:
        baz: 2
      qux: 2
    # **[Note]**
    # jinja2 'do' tag need expression-statement extension
    # please set below to [default] section in ansible.cfg
    # jinja2_extensions=jinja2.ext.do
    dict_update: |
      {% do dict.update(dict2) %}
      {{ dict }}
  tasks:
    - debug:
        var: dict_update

For Ansible version "<2.0"

If you have to use Ansible 1.X for various reasons, it is a little inconvenient because you cannot use combine. Of course, you can import the source part of combine implemented in 2.0, but since it's a big deal, I'd like to create my own Filter plugin. http://docs.ansible.com/ansible/developing_plugins.html#filter-plugins

To use plugin, put the path (filter_plugins =) in ansible.cfg or put the custom file directly in the plugin directory of ansible itself and it will be read when ansible is executed.

/path-to/ansible/filter_plugins/dict_merge.py

from copy import deepcopy


def dict_merge(a, b):
    if not isinstance(b, dict):
        return b
    result = deepcopy(a)
    for k, v in b.iteritems():
        if k in result and isinstance(result[k], dict):
                result[k] = dict_merge(result[k], v)
        else:
            result[k] = deepcopy(v)
    return result


class FilterModule(object):
    def filters(self):
        return {'dict_merge': dict_merge}

This completes the plug-in creation. It's easier than you think. Let's test it.

test.yml

- hosts: localhost
  gather_facts: no
  vars:
    dict:
      foo:
        bar: 1
    dict2:
      foo:
        baz: 2
      qux: 2
    # custom filter plugin
    dict_merged: "{{ dict | dict_merge(dict2) }}"
  tasks:
    - debug:
        var: dict_merged

The execution result is as follows.

$ ansible-playbook -i localhost, test.yml

PLAY [localhost] ***************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "dict_merged": {
        "foo": {
            "bar": 1,
            "baz": 2
        },
        "qux": 2
    }
}

I was able to get the same result as combine. Of course, it also merges deeper dicts.

1. If you want to use deep merge quickly in X system, there is such a way, so why not consider it.

Other

In addition to the Filter plugin, Ansible has several plugins that you can create your own. http://docs.ansible.com/ansible/developing_plugins.html

Lookup and callbacks plugins are relatively informative and have information on people doing various things.

Let's enjoy your Ansible life!