Tohuwabohu excorcism

User Tools

Site Tools

Trace: • shell_or_bash_scripts
devops:ansible

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
devops:ansible [2023/11/01 07:15] – removed - external edit (Unknown date) 127.0.0.1devops:ansible [2023/11/01 07:15] (current) – ↷ Page moved from camunda:devops:ansible to devops:ansible skipidar
Line 1: Line 1:
 +====== Ansible ======
 +
 +=== Experimenting environment ===
 +
 +Here is an environment for experimenting:
 +
 +<code>
 +https://github.com/skipidar/ExperimentsProject-4Ansible-usingVagrant/wiki
 +</code>
 +
 +
 +=== Configuration of ansible ===
 +
 +Add hosts here: **/etc/ansible/hosts**
 +<code>
 +local ansible_host=127.0.0.1   ansible_connection=local
 +
 +slave ansible_host=10.0.0.11 ansible_user=vagrant ansible_ssh_private_key_file=/home/vagrant/.ssh/key.myexperimental.openssh.ppk
 +</code>
 +
 +
 +Add the ssh **private-key** in openSSH form here, which will be used by ansible
 +<code>
 +/home/vagrant/.ssh/id_rsa
 +</code>
 +
 +Example:
 +<code>
 +-----BEGIN RSA PRIVATE KEY-----
 +MIIEoQIBAAKCAQEAxQRyoCBSijfLHpuQKR+0/nsO4p66r4xDre0nJZ8FgHwW+ATd
 +zRr/Mu6C5DgS1ENY9No8yOH56zqnqlhoQpTY5d+wypbkkF3AG3N0D0DrHK4CHjjP
 +NylHAm4+ATuU68agYOVTPycY19DXaAgQBqtJlRVHhMB9ZJ+ugAdKINjpW//8uwvP
 +HisH6GYbA5zWugNHmyfNLdYdJcdozTUprHFkRz6E2HyCxEuhurbcHtsEEfgCd93f
 +bHn2utRg24VIRFNZF24C6N/OSrsmdKbYsQV/VygHEo6VYM4DUOcz1nzU5f3f5k1p
 +itgBbyVqIID++XwxNCl8wAjUWFEHvA+xyf4ZywIBJQKCAQB/y4ifN498P92y/R89
 +RP/HqcR3UjPnfZOvFl6U8YcHO7vfJcBNsKWI0g+2oOmecN/Bavb+18S0QbjdMm0k
 +RN+3s7DyHK/u8LP2IWAJ5OuIOYTVVUgjx79bhcdavt0jUGgHf/6lgSTfBAFYPJTG
 +Dj2RIo9BWkN/iiwABLp2K9WjdHiYwMm2bQqZx2lB4oNPyfhdEUAmd2cdVCfH60mq
 +315KRjO9TgH3ORz43I62rFwSiEaP2VDWhG8nOiMp8wONhaES41uZ05pGA6F9cAdO
 +DvPpK4QcwZ8KZdkJQJBhXu3UvfIS1C6AZ7b08lf+ifU5shqZMerVPxY6RC6FzSRV
 +rs7tAoGBAPSrlQjDhHcCLFd/FyCuTaYQQhokbYRdeZ1bRj5Sv62XB6K8u6NeuKIB
 +4v3W24dnZb51xhk4SOFOXC2lxR0q56MxVsZwl+5vRJEt7wlVYy4yK8JH4r1shDZG
 +/FxQzIfkjxIxMoVhqKL2tdhlE+p0ZPZFRiTCvougXDCaMAfdlGrfAoGBAM4j+DoM
 +E8s5JlaUSIDws5f7axrOGy8lwWNzDtnHmVqoX/0Ck0cIcEhj1SzuYG64qtc5qL7p
 +ej9lU+eqaYWfPbLo+LnrIadaMPxQAG7DS0PILoPJAVgZl3PGUfjvk0kEH03YE8cj
 +OiRpefS9rbp2z1HtYQ30iQZzdmS+Q2c9ptqVAoGALkn5md/M8+unhizNBi7Qa4aJ
 +C91g09pVR0ielitUsiN9/DGLSGvdvckBbks+Sgyki9EXo+gNycK3fkH03AEyvgJq
 +XOTeeTelMDkfYqFzoPS8K6y8May/ETAGOvqHj1TBHx4QehllF+l8WV87HoS5J6w9
 +s+19/r15OaChTZifidcCgYEAgCQyhPLAMjhIQ6hIvtrQgRHoofW93wmhulxHgHUn
 ++hWjc8NUocb5sHVo1r2xkO9VcQgqoDBFEqbFdFUs1oWVDll4BN5FWjElXpKYfDQ1
 +tINbL1Nvicq4F4keMvXKQiUoN073iaBGvLBE5D6OloE0xDmdMjA5gIz2kaa0Rxh8
 +egkCgYB7QihLpX+8sDAyENjs5ylJgsOV8N6e2Twhqjy1B0j0SZoXToOB/Grohk5m
 +BZOsSCeSn4Qb+N8aX0EFXuUHCZVXsYnv8fpiZEYReXNij0mi5MbYby8eY24V9iuO
 +Xixof23aO5mPprsvg/s+GHfbXtZmApDi+wVoN9YNMN43ovxB+w==
 +-----END RSA PRIVATE KEY-----
 +</code>
 +
 +
 +
 +Add the ssh **public-key** in openSSH form here, which will be used by ansible
 +<code>
 +/home/vagrant/.ssh/authorized_keys
 +</code>
 +
 +
 +Example:
 +<code>
 +ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAxQRyoCBSijfLHpuQKR+0/nsO4p66r4xDre0nJZ8FgHwW+ATdzRr/Mu6C5DgS1ENY9No8yOH56zqnqlhoQpTY5d+wypbkkF3AG3N0D0DrHK4CHjjPNylHAm4+ATuU68agYOVTPycY19DXaAgQBqtJlRVHhMB9ZJ+ugAdKINjpW//8uwvPHisH6GYbA5zWugNHmyfNLdYdJcdozTUprHFkRz6E2HyCxEuhurbcHtsEEfgCd93fbHn2utRg24VIRFNZF24C6N/OSrsmdKbYsQV/VygHEo6VYM4DUOcz1nzU5f3f5k1pitgBbyVqIID++XwxNCl8wAjUWFEHvA+xyf4Zyw== rsa-key-20170928vagrant@master:~$
 +</code>
 +
 +
 +Configure the ssh-key permissions. Ansible will check them and if someone else can access the keys decine the usage of the key.
 +<code>
 +chmod -R 700 /home/vagrant/.ssh
 +</code>
 +
 +
 +
 +==== Debugging the connection ====
 +
 +Try connecting to the server using ssh
 +<code>
 +ssh ec2-user@48.114.120.127 -p 22 -i /root/vagrant-home/ssh/demokey.pem
 +</code>
 +
 +Try using the **-vvvv** parameter.
 +<code>
 +ansible slave -vvvv -m ping
 +</code>
 +==== Introduction ====
 +Here is a good introduction:
 +https://serversforhackers.com/c/an-ansible-tutorial
 +
 +
 +==== Commands ====
 +
 +Send a command to all ansible hosts
 +<code>
 +ansible all -m ping -u "ec2-user"
 +</code>
 +
 +
 +Send a command to slave only
 +<code>
 +ansible slave -m ping -u "ec2-user"
 +</code>
 +
 +
 +Execute a shell command remotely. \\
 +The module used here is shell - it is not that the shell is executed remotely, but using the argument **-m** an ansible **module** is used.
 +The **-a** flag is used to pass any **arguments** to the module
 +<code>
 +ansible all -s -m shell -a 'apt-get install nginx'
 +</code>
 +
 +
 +
 +==== Playbook ====
 +
 +Hierarchy:
 +
 +  * Playbook
 +    * Tasks - declarative describtion of as to be state
 +    * Handlers - secondary, after tasks
 +
 +=== Tasks ===
 +
 +Create the following playbook in a file addmycowsay.yaml
 +<code>
 +---
 +- hosts: local
 +  tasks:
 +   - name: install cowsay
 +     apt: pkg=cowsay state=installed update_cache=true
 +</code>
 +
 +And execute the playbook
 +<code>
 +ansible-playbook addmycowsay.yaml
 +</code>
 +
 +So the syntax is the following:
 +<code>
 +---
 +- hosts: <LOCATION>
 +  tasks:
 +   - name: <COMMENT>
 +     <MODULE>: pkg=<APT_PACKAGE> state=installed update_cache=true
 +</code>
 +
 +
 +=== Handlers ===
 +
 +<code>
 +---
 +- hosts: local
 +  tasks:
 +   - name: Install Nginx
 +     apt: pkg=nginx state=installed update_cache=true
 +     notify:
 +      - Start Nginx
 +
 +  handlers:
 +   - name: Start Nginx
 +     service: name=nginx state=started
 +</code>
 +
 +
 +==== Become non root ====
 +
 +To allow becoming non priviledged user to things must be set to True in **/etc/ansible/ansible.cfg**
 +
 +**Important**: The given settings must be uncommented at the right locations in ansible.cfg. It is insufficient to append those settings to ansible.cfg.
 +
 +<code>
 +allow_world_readable_tmpfiles = True
 +pipelining = True
 +</code>
 +
 +To uncomment them programmatically do
 +
 +<code>
 +sed -i 's/.*pipelining.*/pipelining = True/' /etc/ansible/ansible.cfg
 +sed -i 's/.*allow_world_readable_tmpfiles .*/allow_world_readable_tmpfiles = True/' /etc/ansible/ansible.cfg
 +</code>
 +
 +Here is an example playbook, which shows how to become the user "oracle"
 +<code>
 +# Setup the infrastructure for Faktura
 +- hosts: "myhost"
 +  become: yes
 +  become_method: sudo
 +  become_user: oracle
 +  vars:
 +    allow_world_readable_tmpfiles: true
 +  tasks:         
 +
 +
 +        # an error is thorwn when becoming unpriviledged user. Hence use sudo
 +        - name: install oracle client
 +          shell: |
 +                whoami
 +          args:
 +            chdir: /tmp/client
 +          environment:
 +            DISTRIB: /tmp/client
 +</code>
 +
 +
 +
 +==== Remove Old SSH Key Fingerprint ====
 +
 +When connecting to an unknown host - your are asked whether to trust it.
 +Linux then stores a fingerprint of the key associated with the host.
 +
 +If the host then changes, e.g. because of the replacement of the EC2 machine behind the host - connection will fail.
 +
 +To remove the fingerprint do:
 +<code>
 +ssh-keygen -R '45.157.214.126'
 +</code>
 +
 +
 +==== Extra vars alias arguments ====
 +
 +Pass external variables as arguments to the playbook, to e.g. define hosts outside of the playbook.
 +
 +<code>
 +ansible-playbook myplaybook.yaml --extra-vars "host=yourhost"
 +</code>
 +
 +And here is myplaybook:
 +<code>
 +- hosts: "{{ host }}"
 +  become: true
 +  become_method: sudo
 +  become_user: root
 +  
 +  ...
 +</code>
 +
 +
 +
 +==== Ansible behind proxy ====
 +
 +If you configure the Hosts to use corkscrew, then ansible will be able to reach the machines:
 +
 +Open /home/vagrant/.ssh/config, add
 +<code>
 +Host *
 +    ProxyCommand corkscrew 194.138.0.33 9400 %h %p
 +</code>
 +
 +
 +
 +
 +
 +==== Galaxy ====
 +You can use the galaxy (https://galaxy.ansible.com/intro) to share roles or download them.
 +
 +A role "cmacrae.sensu" might be downloaded using 
 +<code>
 +ansible-galaxy install cmacrae.sensu
 +</code>
 +
 +After that you can embed the role into your playbooks:
 +
 +<code>
 +---
 +- name: Install sensu
 +  hosts: slave
 +  become: true
 +  roles:
 +    - { role: cmacrae.sensu, sensu_master: true, sensu_include_dashboard: true  }
 +</code>
 +
 +
 +==== MODULE FAILURE - rc 127 ====
 +
 +When ssh works, but the ping fails with
 +<code>
 +$ ansible slaves -m ping -vvvv
 +
 +ansible 2.5.0
 +  config file = /etc/ansible/ansible.cfg
 +  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
 +  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
 +  executable location = /usr/bin/ansible
 +  python version = 2.7.12 (default, Nov 19 2016, 06:48:10) [GCC 5.4.0 20160609]
 +    "module_stdout": "",
 +    "msg": "MODULE FAILURE",
 +    "rc": 127
 +}
 +</code>
 +
 +Then it will help to ssh to the remote host and install python
 +<code>
 +ssh 34.242.178.166 -l ubuntu -i /home/vagrant/.ssh/DevOpsBootCamp.openssh.ppk
 +ubuntu@34.242.178.166: sudo apt install python
 +</code>
 +
 +Check https://github.com/ansible/ansible/issues/25941
 +
 +
 +==== AWX installation ====
 +The UI, which enables team management and delegation is available as the AWX project
 +https://github.com/ansible/awx
 +
 +
 +
 +
 +==== Importing and Including ====
 +
 +|include*|Is done evaluated at runtime|
 +|Import*|Is done at compile time. PREFER THAT.|
 +
 +
 +
 +
 +=== import_tasks ===
 +Can be used, to extract equal steps you want to reuse in a separate file.
 +
 +|import_tasks*| Allows passing variables. Requires the imported file to have only tasks, no "hosts" etc.|
 +
 +
 +Usecase: requirements for a Renderer/Viewer which needs the same preparations.
 +
 +common_requirements.yaml
 +It is important, that you dont have the header here. Only tasks
 +<code>
 +
 +  - name: Creates product configuration file
 +    file:
 +      path: /etc/apt/apt.conf.d/product
 +      state: touch
 +      owner: ubuntu
 +      group: ubuntu
 +      mode: 0700
 +
 +  - name: product config content
 +    copy:
 +      dest: "/etc/apt/apt.conf.d/product"
 +      content: |
 +        Acquire::https::repo.product.com {
 +          Verify-Peer "true";
 +          Verify-Host "true";
 +
 +          SslCert "/etc/ssl/product/MYCERT.crt";
 +          SslKey "/etc/ssl/product/MYKEY.key";
 +        };
 +</code>
 +
 +Renderer:
 +<code>
 +---
 +- hosts: "{{ host }}"
 +  become: true
 +  become_method: sudo
 +  become_user: root
 +  tasks:
 +
 +  - name: Install product requirements
 +    import_tasks: common_requirements.yml
 +    vars:
 +      host: aws
 +
 +  - name: Install renderer
 +    become: true
 +    apt: product_renderer
 +</code>
 +
 +Viewer:
 +<code>
 +---
 +- hosts: "{{ host }}"
 +  become: true
 +  become_method: sudo
 +  become_user: root
 +  tasks:
 +
 +  - name: Install product requirements
 +    import_tasks: common_requirements.yml
 +    vars:
 +      host: aws
 +
 +  - name: Install VIEWER, reusing the tasks
 +    become: true
 +    apt: product_viewer
 +</code>
 +
 +
 +=== import_playbook ===
 +
 +Unfortunately lacks the possibility to pass in variables.
 +Which makes it unusable
 +
 +
 +
 +==== Read remote files ====
 +To read remote file do use slurp
 +
 +
 +<code>
 +  - name: set facts
 +    set_fact:
 +      dest_timestamp: /tmp/progress/timestamp
 +      
 +      
 +  - name: Slurp timestamp file
 +    slurp:
 +      src: "{{ dest_timestamp }}"
 +    register: slurpfile
 +
 +  - name: set the timestamp fact
 +    set_fact:
 +      timestamp: "{{ slurpfile['content'] | b64decode }}"
 +
 +  - name: timestamp variable
 +    debug: msg="{{ timestamp }}"
 +      
 +      
 +</code>
 +
 +
 +==== Execute a shell script, as a user with a valid environment ====
 +Sometimes you face the issues, that the environment variables defined by some tool, installed for a special user - are not defined, when you execute the tool.
 +
 +To execute some installed software in a valid environment, \\
 +with /home/bashrc \\
 +with /etc/profile.d \\
 +etc
 +
 +use the following approach.
 +
 +<code>
 +      # unfortunately the environment defined by tool in /etc/profile.d - is not taken up by ansible. Explicitely requesting the interactive shell is necessary here
 +      - name: processing step
 +        shell:
 +          sudo -iu ubuntu yourtool.sh   > /tmp/logs/yourtool.log 2>&1
 +        args:
 +          executable: /bin/bash
 +
 +
 +       -i [command]
 +                   The -i (simulate initial login) option runs the shell specified in the passwd(5) entry of the target user as a login shell.  This means that login-specific
 +                   resource files such as .profile or .login will be read by the shell.  If a command is specified, it is passed to the shell for execution.  Otherwise, an
 +                   interactive shell is executed.  sudo attempts to change to that user's home directory before running the shell.  It also initializes the environment, leaving
 +                   DISPLAY and TERM unchanged, setting HOME, MAIL, SHELL, USER, LOGNAME, and PATH, as well as the contents of /etc/environment on Linux and AIX systems.  All other
 +                   environment variables are removed.
 +
 +</code>
 +See https://github.com/ansible/ansible/issues/4854
 +
 +UNfortunately the aproach with become_user - does not have the same result
 +