Ansible Mitogen
Today I discovered a new ansible strategy module that increase ansible performance a lot : Ansible Mitogen.
Mitogen is a python library for writing distributed self-replicating programs.
You can read a great article about this here :
https://sweetness.hmmz.org/2018-03-06-quadrupling-ansible-performance-with-mitogen.html
After some benchmark, I confirm : Mitogen is very fast ! I’ve divised my deployment by 2 :
For example, for a small playbook to deploy and configure 3 kafka nodes :
Before mitogen :
PLAY RECAP ********************************************************************************************************************************************************************************************** brok01 : ok=18 changed=0 unreachable=0 failed=0 brok02 : ok=18 changed=0 unreachable=0 failed=0 brok03 : ok=18 changed=0 unreachable=0 failed=0 Monday 28 May 2018 15:05:21 +0200 (0:00:02.680) 0:02:13.012 ************ =============================================================================== kafka : configuration "projet" ------------------------------------------------------------------------------------------------------------------------------------------------------------------ 11.85s kafka : import ca erdf dans keystore ------------------------------------------------------------------------------------------------------------------------------------------------------------- 8.10s kafka : configuration kafka ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.51s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.04s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.03s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.68s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.67s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.22s kafka : kafka directories ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2.19s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.15s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.14s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.13s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.08s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.99s kafka : keystore jks ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.52s java : installation de java ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.00s kafka : kafka service rhel7 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.85s kafka_metrics_reporter : copie du jar kafka metrics reporter ------------------------------------------------------------------------------------------------------------------------------------- 0.79s kafka : kafka user ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.69s kafka : kafka exists ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.66s real 2m16,558s user 0m37,302s sys 0m4,845s
With Mitogen_linear strategy :
PLAY RECAP ********************************************************************************************************************************************************************************************** brok01 : ok=18 changed=0 unreachable=0 failed=0 brok02 : ok=18 changed=0 unreachable=0 failed=0 brok03 : ok=18 changed=0 unreachable=0 failed=0 Monday 28 May 2018 15:07:01 +0200 (0:00:01.775) 0:01:02.035 ************ =============================================================================== Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.01s kafka : configuration "projet" ------------------------------------------------------------------------------------------------------------------------------------------------------------------- 4.57s kafka : import ca erdf dans keystore ------------------------------------------------------------------------------------------------------------------------------------------------------------- 3.90s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.36s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.93s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.78s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.48s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.46s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.45s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.45s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.40s Gathering Facts ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.38s kafka : configuration kafka ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.06s java : installation de java ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.99s kafka : keystore jks ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.70s include_vars ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.58s kafka : kafka directories ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.24s kafka : kafka service rhel7 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.24s kafka_metrics_reporter : copie du jar kafka metrics reporter ------------------------------------------------------------------------------------------------------------------------------------- 0.21s Attend que le broker en question soit dans le cluster avant de restart un autre ------------------------------------------------------------------------------------------------------------------ 0.20s real 1m5,575s user 0m27,440s sys 0m2,207s
All tasks are divided by 2.
Installation
Installation is super easy :
git clone https://github.com/dw/mitogen.git
Update ansible.cfg :
strategy_plugins = ~/git/seuf/mitogen/ansible_mitogen/plugins/strategy strategy = mitogen_linear
That’s it !
You can now run your playbooks faster !
Tips : I’ve updated my sudoers configuration file to allow the commands
deploy = (ALL) NOPASSWD:/usr/bin/python -c*
Hello. I inserted in sudoers.d/someuser
deploy = (ALL) NOPASSWD:/usr/bin/python -c*
It caused my sudoers error and I had to use recovery mode to fix. You had no problems ?
hello, you have to edit your sudoers with `sudoedit` (sudo sudoedit /etc/sudoers.d/someuser) to ensure there is no error syntax.
the syntax is explained in the mitogen doc : https://mitogen.networkgenomics.com/ansible_detailed.html