===== Puppet =====

The Learning VM TUtorial is here: https://kjhenner.gitbooks.io/puppet-quest-guide/content/quests/application_orchestrator.html

Te learning VM is available here: https://puppet.com/download-learning-vm


==== Glossary ====


|Resources |<WRAP> 
Describing node configuration works by declaring resources. Server provides the descriptions of relevant resources to the nodes then. 

Resource may describe
  * a single file
  * a package
  * a service daemons
  * a maintenance tasks

Abstract declaration of a resource "user"
<code>
user { 'root':
  ensure           => 'present',
  comment          => 'root',
  gid              => '0',
  home             => '/root',
  password         => '$1$PrY3W9V8$JTfCFGrmJ7tl7VwgXga14.',
  password_max_age => '99999',
  password_min_age => '0',
  shell            => '/bin/bash',
  uid              => '0',
}
</code>
</WRAP>|
|Classes |<WRAP> 
The class is the next level of abstraction above a resource. \\
Groups the resources. \\
Groups other classes.

Class may describe
  * describe entire system role as "database-server", "web application worker"
  * some abstract aspect as "cloud-tools", which is composed of many tools

To **define** a class - means to describe it in a module, so that it can be used on nodes. \\
To **declare** a class - means to apply the class to a node. So it will execute there and do its modifications, defined in the .pp file.

<fc #FF0000>Achtung: you only can apply a clas once to a node. So if you wish to use the same peace of code with ifferent parameters (e.g. create multiple users, folders...) use defined resource types</fc>

</WRAP>|
| defined resource types | like classes, but appliable to nodes multiple times  |
|node definition | The task of configuring which classes will be applied to a given node - is called **node classification**. \\ Node definitions are a puppet concept to write node classification down. |
|.pp manifests|<WRAP> 
Puppet language files.

Rules
  * Must use UTF8 encoding
  * May use Unix (LF) or Windows (CRLF) line breaks (note that the line break format also affects literal line breaks in strings)
</WRAP>|
|Modules|<WRAP>
A Puppet module is a self-contained bundle of all the Puppet code and other data needed to manage some aspect of your configuration. \\ You can download pre-built modules from the [[https://forge.puppet.com/|Puppet Forge]] or you can write your own modules.
</WRAP>|
|Puppet master | The puppet server which configures the puppet agents|
|Puppet agent| Installed on machines, which should be controlled by puppet (by puppet master)|




==== Commands ====

| puppet module search <MODULENAME> | Search module|
| puppet module install <MODULENAME> -v 5.16.1 | Install module|
| puppet agent --test \\ puppet agent -t| manually trigger puppet agent run. \\ Puppet run is the check for new configurations. \\ It usually occurs every 30 Minutes automatically|
| puppet resource user root | check user root as resource |
| puppet describe user | describe resource "user". Especially available attributes. |
| puppet apply -e "user { 'galatea': ensure => present, }" | Createa resource user |
| puppet apply --noop -e "user | --noop allows to try out what would be if your would do that on the node |
| puppet resource -e user galatea | Modify resourece user in VIM. (**ESC i** is editing. **:wq** is quit and save.) |
|<WRAP>
puppet resource package fortune-mod ensure=absent \\
puppet resource package cowsay ensure=absent provider=gem
</WRAP>| Uninstall a package resource from node |
|puppet master --configprint all \\ puppet config print graphdir |print configs|
|puppet module list|print module list|
|tree -L 2 -d /etc/puppetlabs/code/environments/production/modules/|<WRAP>
Print the modules as a tree, and only modules from one location, limiting modules to 2 levels
<code>
âââ cowsayings
â   âââ examples
â   âââ manifests
âââ graphite
â   âââ manifests
â   âââ spec
â   âââ templates
âââ vsftpd
    âââ examples
    âââ manifests
    âââ templates
    âââ tests
</code>
</WRAP>|
==== Class example in modules ====

== definition ==


Class **definition** - description of the class. Done before association with nodes.
<code>
# /etc/puppetlabs/code/environments/production/modules/cowsayings/manifests/cowsay.pp
class cowsayings::cowsay {
  package { 'cowsay':
    ensure => present,
    provider => 'gem',
  }
}

# /etc/puppetlabs/code/environments/production/modules/cowsayings/manifests/fortune.pp
class cowsayings::fortune {
  package { 'fortune-mod':
    ensure => present,
  }
}

# /etc/puppetlabs/code/environments/production/modules/cowsayings/manifests/init.pp
class cowsayings {
  include cowsaying::cowsay
  include cowsaying::fortune
}


Pefix **cowsayings** is the name of the module in this case.
Default path **manifests/init.pp** contains the default module's manifest **manifests/init.pp**. It contains the name of the module.

== declaration ==

Class **declaration** of a class "cowsay"  on a node.
<code>
# /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/cowsay.pp
include cowsayings::cowsay
</code>

Class **declaration** of a parent class "cowsayings" on a node. \\
Parent class "cowsayings" includes both of them
<code>
# /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/init.pp
include cowsayings
</code>


== Apply ==

Apply the class "cowsay" on the node
<code>
puppet apply /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/cowsay.pp
</code>

Apply the parent class "cowsayings" on the node, which would install both subclasses "cowsay", "fortune"
<code>
puppet apply /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/init.pp
</code>

== Parameterized class include ==
You can pass some parameters, when including external classes. \\
Here instead of writing **include poodle** and so accepting the defaults, 
we define the **$ensure to be true**. 

example of **NO parametrized** poodle class include
<code>
node 'beetle.example.com' {
    include basicstuff
    include poodle
}
</code>

example of **parametrized** poodle class include \\
Here **NO INCLUDE-keyword** was used.
<code>
node 'beetle.example.com' {
    include basicstuff
    class { 'poodle':
        $ensure => 'stopped'
    }
}
</code>


==== Defined Resource types ====

  * Use **define** keyword instead of **class**
  * **$title** is a special variable, which must be unique on a node and is mandatory
  * Binding of parameter variables ($content, $password) to values happens in parallel, meaning that you cannot use the value of one parameter to set another. The exception is the $title variable.


# /etc/puppetlabs/code/environments/production/modules/web_user/manifests/user.pp
<code>
define web_user::user (
  $content  = "<h1>Welcome to ${title}'s home page!</h1>",
  $password = undef,
) {
  $home_dir    = "/home/${title}"
  $public_html = "${home_dir}/public_html"
  user { $title:
    ensure   => present,
    password => $password,
  }
  file { [$home_dir, $public_html]:
    ensure => directory,
    owner  => $title,
    group  => $title,
    mode    => '0755',
  }
  file { "${public_html}/index.html":
    ensure  => file,
    owner   => $title,
    group   => $title,
    replace => false,
    content => $content,
    mode    => '0644',
  }
}
</code>

**Declaration** of users on a node. \\
'shelob', 'frodo' are the titles of the users, which may be references via the **$title** variable

# /etc/puppetlabs/code/environments/production/modules/web_user/examples/user.pp
<code>
web_user::user { 'shelob': }
web_user::user { 'frodo':
  content  => 'Custom Content!',
  password => pw_hash('sting', 'SHA-512', 'mysalt'),
}
</code>

==== Classes Metaparameter ====

Defines relationships between resources.

== before / required==
Define the ordering of reources, in which they have to be installed

SSH config File required package "openssh-server"
<ode>
file { '/etc/ssh/sshd_config':
    ensure     => present,
    source     => 'puppet:///modules/sshd/sshd_config',
    require    => Package['openssh-server'],
  }
</code>

Package "openssh-server" must be installed, before service "sshd" \\
**before** turns the requirenments around
<code>
package { 'openssh-server':
  ensure => present,
  before => Service['sshd'],
}
</code>


You can generate a graph, to check resource dependencies
<code>
# generate the dependency graph 
puppet apply sshd/examples/init.pp --noop --graph

# check where the .dot graphs are going to
puppet config print graphdir

#generate a png
dot -Tpng /opt/puppetlabs/puppet/cache/state/graphs/relationships.dot -o /var/www/quest/relationships.png
</code>
==== Files ====

You can distribute files via puppet. \\
Files under modules "files" dir are made available via puppet URL and may be used in class definitions. 

== Add File to module ==
To create a module, which will distribute a config file do: \\
add the config files to **/etc/puppetlabs/code/environments/production/modules/vimrc/files/vimrc**

== Create file distributing class ==

Creating a **class vimrc** to distribute file "vimrc"
<code>
class vimrc {
  file { '/root/.vimrc':
    ensure => present,
    source => 'puppet:///modules/vimrc/vimrc',
  }
}
</code>

The URL to file in puppet-module  has the form
<code>
puppet://{server hostname (optional)}/{mount point}/{remainder of path}
</code>

|server hostname|the name of the puppet server. Can omit that, when referencing the default server.|
|mount point|Where are the files? \\ **modules** is a shortcut, to the folders with puppet modules|
|remainder of path|Module name **vimrc**. \\ All files are always under **files** dir, so omit that. \\ **vimrc** is the name of the file. |

So path becomes

^File^URL^
| /etc/puppetlabs/code/environments/production/modules/vimrc/files/vimrc | <code>puppet:///modules/vimrc/vimrc</code> |


Make sure that puppet has **the rights** to change the file
<code>
chown pe-puppet:pe-puppet /etc/puppetlabs/code/environments/production/modules/vimrc/files/vimrc	
</code>

== Apply  ==
Create as usual a **init.pp** with **include vimrc**.

Install by
<code>
puppet apply vimrc/examples/init.pp
</code>

==== Default node definition ====

Manifest **site.pp** \\ 
in /etc/puppetlabs/code/environments/production/manifests/site.pp \\
defines the default node configuration.

==== Node setup ====

Puppet master provides a bash script for setting up nodes:
<code>
curl -k https://<master.example.com>:8140/packages/current/install.bash | sudo bash
</code>

Puppet MASTER keeps signed certificates of each Node, which is a part of the infrastructure.
To involve a node - sighn its certificate.

List all unsigned certificates of Nodes. Executable on master.
<code>
puppet cert list
</code>

Sign a certificate of node named **webserver.learning.puppetlabs.vm **
<code>
puppet cert sign webserver.learning.puppetlabs.vm
</code>

==== Variables ====

Definition
<code>
$doc_root = '/var/www/quest/'
</code>

Access
<code>
"${doc_root}hello.html"
</code>

<code>
class web {

  $doc_root = '/var/www/quest/'

  $english = 'Hello world!'
  $french = 'Bonjour le monde!'

  file { "${doc_root}hello.html":
    ensure => present,
    content => "<em>${english}</em>",
  }

  file { "${doc_root}bonjour.html":
    ensure => present,
    content => "<em>${french}</em>",
  }

}
</code>


==== Facts ====
Global variables, available via **facter** or in code (like variables).

Output facts
<code>
facter -p | less
facter operatingsystem 
</code>

Syntax to access facts in code
<code>
$::factname
</code>

<code>
class accounts ($user_name) {

  if $::operatingsystem == 'centos' {
    $groups = 'wheel'
  }
  elsif $::operatingsystem == 'debian' {
    $groups = 'admin'
  }
  else {
    fail( "This module doesn't support ${::operatingsystem}." )
  }

  notice ( "Groups for user ${user_name} set to ${groups}" )

  ... 

}
</code>

Apply manifest, with modified facts, via prefix:
<code>
FACTER_factname=new_value
</code>

Example
<code>
FACTER_operatingsystem=Debian puppet apply --noop accounts/examples/init.pp
</code>
==== Class parameters ====

**Define** a class, with parameters, which can be modifed later on.
<code>
class classname ( $parameter = 'default' ) {
  ...
}
<code>

<code>
class web ( $page_name="title", $message="message" ) {
</code>


**Declare** (include on a node) a class, with parameters. \\
Fill them with values.
<code>
class {'classname': 
  parameter => 'value',
}
<code>

<code>
class {'web': 
  page_name => 'hola',
  message   => 'Hola mundo!',
}
</code>
==== Puppet master (server) Configurations ====

You can retrieve ALL puppet configurations by executing 
<code>
puppet master --configprint all
</code>

Or just single lines by doing 
<code>
puppet master --configprint agent_catalog_run_lockfile 
puppet master --configprint agent_disabled_lockfile 
...
puppet master --configprint modulepath
...
</code>

Example configuration
<code>
agent_catalog_run_lockfile = /opt/puppetlabs/puppet/cache/state/agent_catalog_run.lock
agent_disabled_lockfile = /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
allow_duplicate_certs = false
always_cache_features = true
app_management = true
archive_file_server = learning.puppetlabs.vm
archive_files = true
autoflush = true
autosign = /etc/puppetlabs/puppet/autosign.conf
basemodulepath = /etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
bindaddress = 0.0.0.0
binder_config = 
bucketdir = /opt/puppetlabs/puppet/cache/bucket
ca = true
ca_name = Puppet CA: learning.puppetlabs.vm
ca_port = 8140
ca_server = learning.puppetlabs.vm
ca_ttl = 157680000
cacert = /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem
cacrl = /etc/puppetlabs/puppet/ssl/ca/ca_crl.pem
cadir = /etc/puppetlabs/puppet/ssl/ca
cakey = /etc/puppetlabs/puppet/ssl/ca/ca_key.pem
capass = /etc/puppetlabs/puppet/ssl/ca/private/ca.pass
caprivatedir = /etc/puppetlabs/puppet/ssl/ca/private
capub = /etc/puppetlabs/puppet/ssl/ca/ca_pub.pem
catalog_cache_terminus = store_configs
catalog_terminus = compiler
cert_inventory = /etc/puppetlabs/puppet/ssl/ca/inventory.txt
certdir = /etc/puppetlabs/puppet/ssl/certs
certificate_revocation = true
certname = learning.puppetlabs.vm
cfacter = false
classfile = /opt/puppetlabs/puppet/cache/state/classes.txt
client_datadir = /opt/puppetlabs/puppet/cache/client_data
clientbucketdir = /opt/puppetlabs/puppet/cache/clientbucket
clientyamldir = /opt/puppetlabs/puppet/cache/client_yaml
code = ""
codedir = /etc/puppetlabs/code
color = ansi
confdir = /etc/puppetlabs/puppet
config = /etc/puppetlabs/puppet/puppet.conf
config_file_name = puppet.conf
config_version = ""
configprint = all
configtimeout = 120
csr_attributes = /etc/puppetlabs/puppet/csr_attributes.yaml
csrdir = /etc/puppetlabs/puppet/ssl/ca/requests
daemonize = true
data_binding_terminus = hiera
default_file_terminus = rest
default_manifest = ./manifests
default_schedules = true
deviceconfig = /etc/puppetlabs/puppet/device.conf
devicedir = /opt/puppetlabs/puppet/cache/devices
diff = diff
diff_args = -u
digest_algorithm = md5
disable_per_environment_manifest = false
disable_warnings = ["deprecations"]
dns_alt_names = ""
document_all = false
environment = production
environment_data_provider = none
environment_timeout = 0
environmentpath = /etc/puppetlabs/code/environments
evaltrace = false
external_nodes = none
factpath = /opt/puppetlabs/puppet/cache/lib/facter:/opt/puppetlabs/puppet/cache/facts
facts_terminus = yaml
fileserverconfig = /etc/puppetlabs/puppet/fileserver.conf
filetimeout = 15
forge_authorization = 
freeze_main = false
genconfig = false
genmanifest = false
graph = false
graphdir = /opt/puppetlabs/puppet/cache/state/graphs
group = pe-puppet
hiera_config = /etc/puppetlabs/code/hiera.yaml
hostcert = /etc/puppetlabs/puppet/ssl/certs/learning.puppetlabs.vm.pem
hostcrl = /etc/puppetlabs/puppet/ssl/crl.pem
hostcsr = /etc/puppetlabs/puppet/ssl/csr_learning.puppetlabs.vm.pem
hostprivkey = /etc/puppetlabs/puppet/ssl/private_keys/learning.puppetlabs.vm.pem
hostpubkey = /etc/puppetlabs/puppet/ssl/public_keys/learning.puppetlabs.vm.pem
http_connect_timeout = 120
http_debug = false
http_keepalive_timeout = 4
http_proxy_host = none
http_proxy_password = none
http_proxy_port = 3128
http_proxy_user = none
http_read_timeout = 
ignorecache = false
ignoremissingtypes = false
ignoreschedules = false
keylength = 4096
lastrunfile = /opt/puppetlabs/puppet/cache/state/last_run_summary.yaml
lastrunreport = /opt/puppetlabs/puppet/cache/state/last_run_report.yaml
ldapattrs = all
ldapbase = ""
ldapclassattrs = puppetclass
ldapparentattr = parentnode
ldappassword = ""
ldapport = 389
ldapserver = ldap
ldapssl = false
ldapstackedattrs = puppetvar
ldapstring = (&(objectclass=puppetClient)(cn=%s))
ldaptls = false
ldapuser = ""
libdir = /opt/puppetlabs/puppet/cache/lib
localcacert = /etc/puppetlabs/puppet/ssl/certs/ca.pem
log_level = notice
logdir = /var/log/puppetlabs/puppet
manage_internal_file_permissions = true
manifest = /etc/puppetlabs/code/environments/production/manifests
masterhttplog = /var/log/puppetlabs/puppet/masterhttp.log
masterport = 8140
max_deprecations = 10
max_errors = 10
max_warnings = 10
maximum_uid = 4294967290
mkusers = false
module_groups = base+pe_only
module_repository = https://forgeapi.puppetlabs.com
module_skeleton_dir = /opt/puppetlabs/puppet/cache/puppet-module/skeleton
module_working_dir = /opt/puppetlabs/puppet/cache/puppet-module
modulepath = /etc/puppetlabs/code/environments/production/modules:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
name = master
node_cache_terminus = write_only_yaml
node_name = cert
node_name_fact = ""
node_name_value = learning.puppetlabs.vm
node_terminus = classifier
noop = false
onetime = false
ordering = manifest
passfile = /etc/puppetlabs/puppet/ssl/private/password
path = none
pidfile = /var/run/puppetlabs/master.pid
plugindest = /opt/puppetlabs/puppet/cache/lib
pluginfactdest = /opt/puppetlabs/puppet/cache/facts.d
pluginfactsource = puppet:///pluginfacts
pluginsignore = .svn CVS .git
pluginsource = puppet:///plugins
pluginsync = true
postrun_command = ""
preferred_serialization_format = pson
prerun_command = ""
preview_outputdir = /opt/puppetlabs/puppet/cache/preview
priority = 
privatedir = /etc/puppetlabs/puppet/ssl/private
privatekeydir = /etc/puppetlabs/puppet/ssl/private_keys
profile = false
publickeydir = /etc/puppetlabs/puppet/ssl/public_keys
puppetdlog = /var/log/puppetlabs/puppet/puppetd.log
report = true
report_port = 8140
report_server = learning.puppetlabs.vm
reportdir = /opt/puppetlabs/puppet/cache/reports
reports = puppetdb
reporturl = http://localhost:3000/reports/upload
req_bits = 4096
requestdir = /etc/puppetlabs/puppet/ssl/certificate_requests
resourcefile = /opt/puppetlabs/puppet/cache/state/resources.txt
rest_authconfig = /etc/puppetlabs/puppet/auth.conf
route_file = /etc/puppetlabs/puppet/routes.yaml
rundir = /var/run/puppetlabs
runinterval = 1800
serial = /etc/puppetlabs/puppet/ssl/ca/serial
server = learning.puppetlabs.vm
server_datadir = /opt/puppetlabs/puppet/cache/server_data
show_diff = false
signeddir = /etc/puppetlabs/puppet/ssl/ca/signed
skip_tags = ""
splay = false
splaylimit = 1800
srv_domain = puppetlabs.vm
ssl_client_ca_auth = 
ssl_client_header = HTTP_X_CLIENT_DN
ssl_client_verify_header = HTTP_X_CLIENT_VERIFY
ssl_server_ca_auth = 
ssldir = /etc/puppetlabs/puppet/ssl
statedir = /opt/puppetlabs/puppet/cache/state
statefile = /opt/puppetlabs/puppet/cache/state/state.yaml
static_catalogs = true
storeconfigs = true
storeconfigs_backend = puppetdb
strict_environment_mode = false
strict_hostname_checking = false
strict_variables = false
summarize = false
supported_checksum_types = ["md5", "sha256"]
syslogfacility = daemon
tags = ""
trace = false
trusted_oid_mapping_file = /etc/puppetlabs/puppet/custom_trusted_oid_mapping.yaml
trusted_server_facts = false
use_cached_catalog = false
use_srv_records = false
usecacheonfailure = true
user = pe-puppet
vardir = /opt/puppetlabs/puppet/cache
waitforcert = 120
yamldir = /opt/puppetlabs/puppet/cache/yaml
</code>



===== Orchestration =====

Details are here: https://kjhenner.gitbooks.io/puppet-quest-guide/content/quests/application_orchestrator.html

Orchestration allows to install dependent applications in the right order.

It works as following:
  - setup the node to know the orchestrator
  - create an Application, which may combine multiple nodes to a single semantical unit (MySQL Server, Apache with PHP application )
    - setup orchestrator user with the rights
    - setup an **artificial, public resource** to share the data between nodes (**in ruby**)
    - define, which node **produces** data to fill artificial, public resource. Which node consumes data from artificial, public resource.
    - declare the resource
    - in site.pp declare - which part of the application will be installed on which node.

{{http://i520.photobucket.com/albums/w327/schajtan/2016-04-25_14-27-49_zpsphhbuy9w.png}}