Ansible Mitogen

Ansible Mitogen

2018-05-28 2 Par seuf

Today I discovered a new ansible strategy module that increase ansible performance a lot : Ansible Mitogen.

Mitogen is a python library for writing distributed self-replicating programs.

You can read a great article about this here :

https://sweetness.hmmz.org/2018-03-06-quadrupling-ansible-performance-with-mitogen.html

After some benchmark, I confirm : Mitogen is very fast ! I’ve divised my deployment by 2 :

For example, for a small playbook to deploy and configure 3 kafka nodes :

Before mitogen :

PLAY RECAP **********************************************************************************************************************************************************************************************
brok01 : ok=18 changed=0 unreachable=0 failed=0
brok02 : ok=18 changed=0 unreachable=0 failed=0
brok03 : ok=18 changed=0 unreachable=0 failed=0

Monday 28 May 2018 15:05:21 +0200 (0:00:02.680) 0:02:13.012 ************
===============================================================================
kafka : configuration "projet" ------------------------------------------------------------------------------------------------------------------------------------------------------------------ 11.85s
kafka : import ca erdf dans keystore ------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.10s
kafka : configuration kafka ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.51s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.04s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.03s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.68s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.67s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.22s
kafka : kafka directories ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2.19s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.15s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.14s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.13s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.08s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.99s
kafka : keystore jks ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.52s
java : installation de java ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.00s
kafka : kafka service rhel7 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.85s
kafka_metrics_reporter : copie du jar kafka metrics reporter ------------------------------------------------------------------------------------------------------------------------------------- 0.79s
kafka : kafka user ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.69s
kafka : kafka exists ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.66s

real 2m16,558s
user 0m37,302s
sys 0m4,845s

With Mitogen_linear strategy :

PLAY RECAP **********************************************************************************************************************************************************************************************
brok01 : ok=18 changed=0 unreachable=0 failed=0
brok02 : ok=18 changed=0 unreachable=0 failed=0
brok03 : ok=18 changed=0 unreachable=0 failed=0

Monday 28 May 2018 15:07:01 +0200 (0:00:01.775) 0:01:02.035 ************
===============================================================================
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.01s
kafka : configuration "projet" ------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.57s
kafka : import ca erdf dans keystore ------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.90s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.36s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.93s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.78s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.48s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.46s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.45s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.45s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.40s
Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.38s
kafka : configuration kafka ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.06s
java : installation de java ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.99s
kafka : keystore jks ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.70s
include_vars ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.58s
kafka : kafka directories ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.24s
kafka : kafka service rhel7 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.24s
kafka_metrics_reporter : copie du jar kafka metrics reporter ------------------------------------------------------------------------------------------------------------------------------------- 0.21s
Attend que le broker en question soit dans le cluster avant de restart un autre ------------------------------------------------------------------------------------------------------------------ 0.20s

real 1m5,575s
user 0m27,440s
sys 0m2,207s

All tasks are divided by 2.

Installation

Installation is super easy :

git clone https://github.com/dw/mitogen.git

Update ansible.cfg :

strategy_plugins = ~/git/seuf/mitogen/ansible_mitogen/plugins/strategy
strategy = mitogen_linear

That’s it !

You  can now run your playbooks faster !

Tips : I’ve updated my sudoers configuration file to allow the commands

deploy = (ALL) NOPASSWD:/usr/bin/python -c*