devops:puppet

Puppet

The Learning VM TUtorial is here: https://kjhenner.gitbooks.io/puppet-quest-guide/content/quests/application_orchestrator.html

Te learning VM is available here: https://puppet.com/download-learning-vm

Glossary

Resources	Describing node configuration works by declaring resources. Server provides the descriptions of relevant resources to the nodes then. Resource may describe a single file a package a service daemons a maintenance tasks Abstract declaration of a resource “user” user { 'root': ensure => 'present', comment => 'root', gid => '0', home => '/root', password => '$1$PrY3W9V8$JTfCFGrmJ7tl7VwgXga14.', password_max_age => '99999', password_min_age => '0', shell => '/bin/bash', uid => '0', }
Classes	The class is the next level of abstraction above a resource. Groups the resources. Groups other classes. Class may describe describe entire system role as “database-server”, “web application worker” some abstract aspect as “cloud-tools”, which is composed of many tools To define a class - means to describe it in a module, so that it can be used on nodes. To declare a class - means to apply the class to a node. So it will execute there and do its modifications, defined in the .pp file. <fc #FF0000>Achtung: you only can apply a clas once to a node. So if you wish to use the same peace of code with ifferent parameters (e.g. create multiple users, folders…) use defined resource types</fc>
defined resource types	like classes, but appliable to nodes multiple times
node definition	The task of configuring which classes will be applied to a given node - is called node classification. Node definitions are a puppet concept to write node classification down.
.pp manifests	Puppet language files. Rules Must use UTF8 encoding May use Unix (LF) or Windows (CRLF) line breaks (note that the line break format also affects literal line breaks in strings)
Modules	A Puppet module is a self-contained bundle of all the Puppet code and other data needed to manage some aspect of your configuration. You can download pre-built modules from the Puppet Forge or you can write your own modules.
Puppet master	The puppet server which configures the puppet agents
Puppet agent	Installed on machines, which should be controlled by puppet (by puppet master)

Commands

puppet module search <MODULENAME>	Search module
puppet module install <MODULENAME> -v 5.16.1	Install module
puppet agent –test puppet agent -t	manually trigger puppet agent run. Puppet run is the check for new configurations. It usually occurs every 30 Minutes automatically
puppet resource user root	check user root as resource
puppet describe user	describe resource “user”. Especially available attributes.
puppet apply -e “user { 'galatea': ensure ⇒ present, }”	Createa resource user
puppet apply –noop -e “user	–noop allows to try out what would be if your would do that on the node
puppet resource -e user galatea	Modify resourece user in VIM. (ESC i is editing. :wq is quit and save.)
puppet resource package fortune-mod ensure=absent puppet resource package cowsay ensure=absent provider=gem	Uninstall a package resource from node
puppet master –configprint all puppet config print graphdir	print configs
puppet module list	print module list
tree -L 2 -d /etc/puppetlabs/code/environments/production/modules/	Print the modules as a tree, and only modules from one location, limiting modules to 2 levels âââ cowsayings âÂ Â âââ examples âÂ Â âââ manifests âââ graphite âÂ Â âââ manifests âÂ Â âââ spec âÂ Â âââ templates âââ vsftpd âââ examples âââ manifests âââ templates âââ tests

Class example in modules

definition

Class definition - description of the class. Done before association with nodes.

# /etc/puppetlabs/code/environments/production/modules/cowsayings/manifests/cowsay.pp
class cowsayings::cowsay {
  package { 'cowsay':
    ensure => present,
    provider => 'gem',
  }
}

# /etc/puppetlabs/code/environments/production/modules/cowsayings/manifests/fortune.pp
class cowsayings::fortune {
  package { 'fortune-mod':
    ensure => present,
  }
}

# /etc/puppetlabs/code/environments/production/modules/cowsayings/manifests/init.pp
class cowsayings {
  include cowsaying::cowsay
  include cowsaying::fortune
}


Pefix **cowsayings** is the name of the module in this case.
Default path **manifests/init.pp** contains the default module's manifest **manifests/init.pp**. It contains the name of the module.

== declaration ==

Class **declaration** of a class "cowsay"  on a node.
<code>
# /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/cowsay.pp
include cowsayings::cowsay

Class declaration of a parent class “cowsayings” on a node.
Parent class “cowsayings” includes both of them

# /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/init.pp
include cowsayings

Apply

Apply the class “cowsay” on the node

puppet apply /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/cowsay.pp

Apply the parent class “cowsayings” on the node, which would install both subclasses “cowsay”, “fortune”

puppet apply /etc/puppetlabs/code/environments/production/modules/cowsayings/examples/init.pp

Parameterized class include

You can pass some parameters, when including external classes.
Here instead of writing include poodle and so accepting the defaults, we define the $ensure to be true.

example of NO parametrized poodle class include

node 'beetle.example.com' {
    include basicstuff
    include poodle
}

example of parametrized poodle class include
Here NO INCLUDE-keyword was used.

node 'beetle.example.com' {
    include basicstuff
    class { 'poodle':
        $ensure => 'stopped'
    }
}

Defined Resource types

Use define keyword instead of class
$title is a special variable, which must be unique on a node and is mandatory
Binding of parameter variables ($content, $password) to values happens in parallel, meaning that you cannot use the value of one parameter to set another. The exception is the $title variable.

# /etc/puppetlabs/code/environments/production/modules/web_user/manifests/user.pp

define web_user::user (
  $content  = "<h1>Welcome to ${title}'s home page!</h1>",
  $password = undef,
) {
  $home_dir    = "/home/${title}"
  $public_html = "${home_dir}/public_html"
  user { $title:
    ensure   => present,
    password => $password,
  }
  file { [$home_dir, $public_html]:
    ensure => directory,
    owner  => $title,
    group  => $title,
    mode    => '0755',
  }
  file { "${public_html}/index.html":
    ensure  => file,
    owner   => $title,
    group   => $title,
    replace => false,
    content => $content,
    mode    => '0644',
  }
}

Declaration of users on a node.
'shelob', 'frodo' are the titles of the users, which may be references via the $title variable

# /etc/puppetlabs/code/environments/production/modules/web_user/examples/user.pp

web_user::user { 'shelob': }
web_user::user { 'frodo':
  content  => 'Custom Content!',
  password => pw_hash('sting', 'SHA-512', 'mysalt'),
}

Classes Metaparameter

Defines relationships between resources.

before / required

Define the ordering of reources, in which they have to be installed

SSH config File required package “openssh-server” <ode> file { '/etc/ssh/sshd_config':

  ensure     => present,
  source     => 'puppet:///modules/sshd/sshd_config',
  require    => Package['openssh-server'],
}

</code>

Package “openssh-server” must be installed, before service “sshd”
before turns the requirenments around

package { 'openssh-server':
  ensure => present,
  before => Service['sshd'],
}

You can generate a graph, to check resource dependencies

# generate the dependency graph 
puppet apply sshd/examples/init.pp --noop --graph

# check where the .dot graphs are going to
puppet config print graphdir

#generate a png
dot -Tpng /opt/puppetlabs/puppet/cache/state/graphs/relationships.dot -o /var/www/quest/relationships.png

Files

You can distribute files via puppet.
Files under modules “files” dir are made available via puppet URL and may be used in class definitions.

Add File to module

To create a module, which will distribute a config file do:
add the config files to /etc/puppetlabs/code/environments/production/modules/vimrc/files/vimrc

Create file distributing class

Creating a class vimrc to distribute file “vimrc”

class vimrc {
  file { '/root/.vimrc':
    ensure => present,
    source => 'puppet:///modules/vimrc/vimrc',
  }
}

The URL to file in puppet-module has the form

puppet://{server hostname (optional)}/{mount point}/{remainder of path}

server hostname	the name of the puppet server. Can omit that, when referencing the default server.
mount point	Where are the files? modules is a shortcut, to the folders with puppet modules
remainder of path	Module name vimrc. All files are always under files dir, so omit that. vimrc is the name of the file.

So path becomes

File	URL
/etc/puppetlabs/code/environments/production/modules/vimrc/files/vimrc	puppet:///modules/vimrc/vimrc

Make sure that puppet has the rights to change the file

chown pe-puppet:pe-puppet /etc/puppetlabs/code/environments/production/modules/vimrc/files/vimrc

Apply

Create as usual a init.pp with include vimrc.

Install by

puppet apply vimrc/examples/init.pp

Default node definition

Manifest site.pp
in /etc/puppetlabs/code/environments/production/manifests/site.pp
defines the default node configuration.

Node setup

Puppet master provides a bash script for setting up nodes:

curl -k https://<master.example.com>:8140/packages/current/install.bash | sudo bash

Puppet MASTER keeps signed certificates of each Node, which is a part of the infrastructure. To involve a node - sighn its certificate.

List all unsigned certificates of Nodes. Executable on master.

puppet cert list

Sign a certificate of node named webserver.learning.puppetlabs.vm

puppet cert sign webserver.learning.puppetlabs.vm

Variables

Definition

$doc_root = '/var/www/quest/'

Access

"${doc_root}hello.html"

class web {

  $doc_root = '/var/www/quest/'

  $english = 'Hello world!'
  $french = 'Bonjour le monde!'

  file { "${doc_root}hello.html":
    ensure => present,
    content => "<em>${english}</em>",
  }

  file { "${doc_root}bonjour.html":
    ensure => present,
    content => "<em>${french}</em>",
  }

}

Facts

Global variables, available via facter or in code (like variables).

Output facts

facter -p | less
facter operatingsystem

Syntax to access facts in code

$::factname

class accounts ($user_name) {

  if $::operatingsystem == 'centos' {
    $groups = 'wheel'
  }
  elsif $::operatingsystem == 'debian' {
    $groups = 'admin'
  }
  else {
    fail( "This module doesn't support ${::operatingsystem}." )
  }

  notice ( "Groups for user ${user_name} set to ${groups}" )

  ... 

}

Apply manifest, with modified facts, via prefix:

FACTER_factname=new_value

Example

FACTER_operatingsystem=Debian puppet apply --noop accounts/examples/init.pp

Class parameters

Define a class, with parameters, which can be modifed later on.

class classname ( $parameter = 'default' ) {
  ...
}
<code>

<code>
class web ( $page_name="title", $message="message" ) {

Declare (include on a node) a class, with parameters.
Fill them with values.

class {'classname': 
  parameter => 'value',
}
<code>

<code>
class {'web': 
  page_name => 'hola',
  message   => 'Hola mundo!',
}

Puppet master (server) Configurations

You can retrieve ALL puppet configurations by executing

puppet master --configprint all

Or just single lines by doing

puppet master --configprint agent_catalog_run_lockfile 
puppet master --configprint agent_disabled_lockfile 
...
puppet master --configprint modulepath
...

Example configuration

agent_catalog_run_lockfile = /opt/puppetlabs/puppet/cache/state/agent_catalog_run.lock
agent_disabled_lockfile = /opt/puppetlabs/puppet/cache/state/agent_disabled.lock
allow_duplicate_certs = false
always_cache_features = true
app_management = true
archive_file_server = learning.puppetlabs.vm
archive_files = true
autoflush = true
autosign = /etc/puppetlabs/puppet/autosign.conf
basemodulepath = /etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
bindaddress = 0.0.0.0
binder_config = 
bucketdir = /opt/puppetlabs/puppet/cache/bucket
ca = true
ca_name = Puppet CA: learning.puppetlabs.vm
ca_port = 8140
ca_server = learning.puppetlabs.vm
ca_ttl = 157680000
cacert = /etc/puppetlabs/puppet/ssl/ca/ca_crt.pem
cacrl = /etc/puppetlabs/puppet/ssl/ca/ca_crl.pem
cadir = /etc/puppetlabs/puppet/ssl/ca
cakey = /etc/puppetlabs/puppet/ssl/ca/ca_key.pem
capass = /etc/puppetlabs/puppet/ssl/ca/private/ca.pass
caprivatedir = /etc/puppetlabs/puppet/ssl/ca/private
capub = /etc/puppetlabs/puppet/ssl/ca/ca_pub.pem
catalog_cache_terminus = store_configs
catalog_terminus = compiler
cert_inventory = /etc/puppetlabs/puppet/ssl/ca/inventory.txt
certdir = /etc/puppetlabs/puppet/ssl/certs
certificate_revocation = true
certname = learning.puppetlabs.vm
cfacter = false
classfile = /opt/puppetlabs/puppet/cache/state/classes.txt
client_datadir = /opt/puppetlabs/puppet/cache/client_data
clientbucketdir = /opt/puppetlabs/puppet/cache/clientbucket
clientyamldir = /opt/puppetlabs/puppet/cache/client_yaml
code = ""
codedir = /etc/puppetlabs/code
color = ansi
confdir = /etc/puppetlabs/puppet
config = /etc/puppetlabs/puppet/puppet.conf
config_file_name = puppet.conf
config_version = ""
configprint = all
configtimeout = 120
csr_attributes = /etc/puppetlabs/puppet/csr_attributes.yaml
csrdir = /etc/puppetlabs/puppet/ssl/ca/requests
daemonize = true
data_binding_terminus = hiera
default_file_terminus = rest
default_manifest = ./manifests
default_schedules = true
deviceconfig = /etc/puppetlabs/puppet/device.conf
devicedir = /opt/puppetlabs/puppet/cache/devices
diff = diff
diff_args = -u
digest_algorithm = md5
disable_per_environment_manifest = false
disable_warnings = ["deprecations"]
dns_alt_names = ""
document_all = false
environment = production
environment_data_provider = none
environment_timeout = 0
environmentpath = /etc/puppetlabs/code/environments
evaltrace = false
external_nodes = none
factpath = /opt/puppetlabs/puppet/cache/lib/facter:/opt/puppetlabs/puppet/cache/facts
facts_terminus = yaml
fileserverconfig = /etc/puppetlabs/puppet/fileserver.conf
filetimeout = 15
forge_authorization = 
freeze_main = false
genconfig = false
genmanifest = false
graph = false
graphdir = /opt/puppetlabs/puppet/cache/state/graphs
group = pe-puppet
hiera_config = /etc/puppetlabs/code/hiera.yaml
hostcert = /etc/puppetlabs/puppet/ssl/certs/learning.puppetlabs.vm.pem
hostcrl = /etc/puppetlabs/puppet/ssl/crl.pem
hostcsr = /etc/puppetlabs/puppet/ssl/csr_learning.puppetlabs.vm.pem
hostprivkey = /etc/puppetlabs/puppet/ssl/private_keys/learning.puppetlabs.vm.pem
hostpubkey = /etc/puppetlabs/puppet/ssl/public_keys/learning.puppetlabs.vm.pem
http_connect_timeout = 120
http_debug = false
http_keepalive_timeout = 4
http_proxy_host = none
http_proxy_password = none
http_proxy_port = 3128
http_proxy_user = none
http_read_timeout = 
ignorecache = false
ignoremissingtypes = false
ignoreschedules = false
keylength = 4096
lastrunfile = /opt/puppetlabs/puppet/cache/state/last_run_summary.yaml
lastrunreport = /opt/puppetlabs/puppet/cache/state/last_run_report.yaml
ldapattrs = all
ldapbase = ""
ldapclassattrs = puppetclass
ldapparentattr = parentnode
ldappassword = ""
ldapport = 389
ldapserver = ldap
ldapssl = false
ldapstackedattrs = puppetvar
ldapstring = (&(objectclass=puppetClient)(cn=%s))
ldaptls = false
ldapuser = ""
libdir = /opt/puppetlabs/puppet/cache/lib
localcacert = /etc/puppetlabs/puppet/ssl/certs/ca.pem
log_level = notice
logdir = /var/log/puppetlabs/puppet
manage_internal_file_permissions = true
manifest = /etc/puppetlabs/code/environments/production/manifests
masterhttplog = /var/log/puppetlabs/puppet/masterhttp.log
masterport = 8140
max_deprecations = 10
max_errors = 10
max_warnings = 10
maximum_uid = 4294967290
mkusers = false
module_groups = base+pe_only
module_repository = https://forgeapi.puppetlabs.com
module_skeleton_dir = /opt/puppetlabs/puppet/cache/puppet-module/skeleton
module_working_dir = /opt/puppetlabs/puppet/cache/puppet-module
modulepath = /etc/puppetlabs/code/environments/production/modules:/etc/puppetlabs/code/modules:/opt/puppetlabs/puppet/modules
name = master
node_cache_terminus = write_only_yaml
node_name = cert
node_name_fact = ""
node_name_value = learning.puppetlabs.vm
node_terminus = classifier
noop = false
onetime = false
ordering = manifest
passfile = /etc/puppetlabs/puppet/ssl/private/password
path = none
pidfile = /var/run/puppetlabs/master.pid
plugindest = /opt/puppetlabs/puppet/cache/lib
pluginfactdest = /opt/puppetlabs/puppet/cache/facts.d
pluginfactsource = puppet:///pluginfacts
pluginsignore = .svn CVS .git
pluginsource = puppet:///plugins
pluginsync = true
postrun_command = ""
preferred_serialization_format = pson
prerun_command = ""
preview_outputdir = /opt/puppetlabs/puppet/cache/preview
priority = 
privatedir = /etc/puppetlabs/puppet/ssl/private
privatekeydir = /etc/puppetlabs/puppet/ssl/private_keys
profile = false
publickeydir = /etc/puppetlabs/puppet/ssl/public_keys
puppetdlog = /var/log/puppetlabs/puppet/puppetd.log
report = true
report_port = 8140
report_server = learning.puppetlabs.vm
reportdir = /opt/puppetlabs/puppet/cache/reports
reports = puppetdb
reporturl = http://localhost:3000/reports/upload
req_bits = 4096
requestdir = /etc/puppetlabs/puppet/ssl/certificate_requests
resourcefile = /opt/puppetlabs/puppet/cache/state/resources.txt
rest_authconfig = /etc/puppetlabs/puppet/auth.conf
route_file = /etc/puppetlabs/puppet/routes.yaml
rundir = /var/run/puppetlabs
runinterval = 1800
serial = /etc/puppetlabs/puppet/ssl/ca/serial
server = learning.puppetlabs.vm
server_datadir = /opt/puppetlabs/puppet/cache/server_data
show_diff = false
signeddir = /etc/puppetlabs/puppet/ssl/ca/signed
skip_tags = ""
splay = false
splaylimit = 1800
srv_domain = puppetlabs.vm
ssl_client_ca_auth = 
ssl_client_header = HTTP_X_CLIENT_DN
ssl_client_verify_header = HTTP_X_CLIENT_VERIFY
ssl_server_ca_auth = 
ssldir = /etc/puppetlabs/puppet/ssl
statedir = /opt/puppetlabs/puppet/cache/state
statefile = /opt/puppetlabs/puppet/cache/state/state.yaml
static_catalogs = true
storeconfigs = true
storeconfigs_backend = puppetdb
strict_environment_mode = false
strict_hostname_checking = false
strict_variables = false
summarize = false
supported_checksum_types = ["md5", "sha256"]
syslogfacility = daemon
tags = ""
trace = false
trusted_oid_mapping_file = /etc/puppetlabs/puppet/custom_trusted_oid_mapping.yaml
trusted_server_facts = false
use_cached_catalog = false
use_srv_records = false
usecacheonfailure = true
user = pe-puppet
vardir = /opt/puppetlabs/puppet/cache
waitforcert = 120
yamldir = /opt/puppetlabs/puppet/cache/yaml

Orchestration

Details are here: https://kjhenner.gitbooks.io/puppet-quest-guide/content/quests/application_orchestrator.html

Orchestration allows to install dependent applications in the right order.

It works as following:

setup the node to know the orchestrator
create an Application, which may combine multiple nodes to a single semantical unit (MySQL Server, Apache with PHP application )
1. setup orchestrator user with the rights
2. setup an artificial, public resource to share the data between nodes (in ruby)
3. define, which node produces data to fill artificial, public resource. Which node consumes data from artificial, public resource.
4. declare the resource
5. in site.pp declare - which part of the application will be installed on which node.

Table of Contents

Puppet

Glossary

Commands

Class example in modules

definition

Apply

Parameterized class include

Defined Resource types

Classes Metaparameter

before / required

Files

Add File to module

Create file distributing class

Apply

Default node definition

Node setup

Variables

Facts

Class parameters

Puppet master (server) Configurations

Orchestration