file:/ required for create-stack or validate-stack, but no for deployIf using nested-stacks first you need a bucket, into which you will package nested stacks.
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS template for SiteWise demo
Resources:
MyS3SubstackBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: my-alf-s3-package-bucket-2023-12-05
AccessControl: Private
Tags:
- Key: Purpose
Value: CF stacks bucket
MyBucketPolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Ref MyS3SubstackBucket
PolicyDocument:
Statement:
- Sid: AllowCloudFormationAccess
Effect: Allow
Principal:
Service: cloudformation.amazonaws.com
Action: s3:*
Resource: !Join
- ''
- - 'arn:aws:s3:::'
- !Ref MyS3SubstackBucket
- /*
now deploy the bucket
FILENAME="seed.cloudformation.yaml" STACKNAME="MySeedStackName" REGION="eu-west-1" # validate aws cloudformation validate-template --template-body file://$FILENAME # check the change set aws cloudformation deploy --stack-name $STACKNAME --template-file $FILENAME --region $REGION --no-execute-changeset # execute via "deploy" which automatically creates / updates stack aws cloudformation deploy --stack-name $STACKNAME --template-file $FILENAME --region $REGION # delete stack # aws cloudformation delete-stack --stack-name $STACKNAME
parent1.cloudformation.yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: Provision a SG
Parameters:
VpcIdParameter:
Type: String
Default: "vpc-01eb7fd6f29cea57b"
packageBucket:
Type: String
Default: "my-alf-s3-package-bucket-2023-12-05"
Resources:
SubStack1:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: "substack.helloworld.cloudformation.yaml"
Parameters:
VpcId: !Ref VpcIdParameter
substack.helloworld.cloudformation.yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: Provision a SG
Parameters:
VpcId:
Type: String
Resources:
MySecurityGroup:
Type: AWS::EC2::SecurityGroup
Properties:
GroupDescription: MySecurityGroup
VpcId: !Ref VpcId
SecurityGroupIngress:
- IpProtocol: tcp
FromPort: 80
ToPort: 80
CidrIp: 0.0.0.0/0 # Example: Allowing HTTP traffic from anywhere (Please adjust for your use case)
Tags:
- Key: Name
Value: MySecurityGroup
now you can package the stack
set -e FILENAME="parent1.cloudformation.yaml" STACKNAME="MyParentStackName" REGION="eu-west-1" PACKAGEBUCKET="my-alf-s3-package-bucket-2023-12-05" # validate # aws cloudformation validate-template --template-body file://$FILENAME # package uploading substacks rm packaged-root-template.yaml aws cloudformation package --template-file $FILENAME --s3-bucket $PACKAGEBUCKET --output-template-file packaged-root-template.yaml --region $REGION # check the change set, dont execute : "no-execute-changeset" # aws cloudformation deploy --stack-name $STACKNAME --template-file $FILENAME --region $REGION --no-execute-changeset # arn of change set is printed, here arn:aws:cloudformation:eu-west-1:913372342854:changeSet/awscli-cloudformation-package-deploy-1701783364/21bb1e0c-a0ea-41ca-9edd-5a7ab989b3a5 # can see change-set # aws cloudformation describe-change-set --change-set-name arn:aws:cloudformation:eu-west-1:913372342854:changeSet/awscli-cloudformation-package-deploy-1701783364/21bb1e0c-a0ea-41ca-9edd-5a7ab989b3a5 --region $REGION # can continue via # aws cloudformation execute-change-set --change-set-name arn:aws:cloudformation:eu-west-1:913372342854:changeSet/awscli-cloudformation-package-deploy-1701783364/21bb1e0c-a0ea-41ca-9edd-5a7ab989b3a5 --region $REGION # execute via "deploy" which automatically creates / updates stack aws cloudformation deploy --stack-name $STACKNAME --template-file packaged-root-template.yaml --region $REGION --parameter-overrides VpcIdParameter="vpc-01eb7fd6f29cea57b" # delete stack # aws cloudformation delete-stack --stack-name $STACKNAME
User parameters.
The resources, which should be created, like EC2 Instances, SecurityGroups.
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-template-resource-type-ref.html
What is printed inside the AWS console, after the cloud formation script is completed.
Initiation is described inside of the following block: AWS::CloudFormation::Init
This block is NOT executed on the instance automatgically. Instead a special script must be executed within the UserData block.
"UserData": {
"Fn::Base64": { "Fn::Join":["", [
"#!/bin/bash -ex\n",
"\n",
"# Install the cfn-init and cfn-signal scripts \n",
"apt-get update\n",
"apt-get -y install python-setuptools\n",
"mkdir aws-cfn-bootstrap-latest\n",
"curl https://s3.amazonaws.com/cloudformation-examples/aws-cfn-bootstrap-latest.tar.gz | tar xz -C aws-cfn-bootstrap-latest --strip-components 1\n",
"easy_install aws-cfn-bootstrap-latest\n",
"\n",
"# Trigger the default configset described in AWS::CloudFormation::Init \n",
"/usr/local/bin/cfn-init --stack ", { "Ref":"AWS::StackName" }, " --resource WebServerInstance" ," --region ", { "Ref": "AWS::Region" }," --configset InstallAndRun" , "\n",
"\n",
"# Signal the status from cfn-init to release the CreationPolicy \n",
"/usr/local/bin/cfn-signal --stack ", { "Ref":"AWS::StackName" }, " --resource WebServerInstance" ," --region ", { "Ref": "AWS::Region" }," --exit-code $? \n"
]]}
For details see: https://gist.github.com/skipidar/a3966bbaf733de429c676f6e910b3bc4
This is an amazon provided script, which triggers the “AWS::CloudFormation::Init” block. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-init.html
"/usr/local/bin/cfn-init --stack ", { "Ref":"AWS::StackName" }, " --resource WebServerInstance" ," --region ", { "Ref": "AWS::Region" }," --configset InstallAndRun"
The most interesting usage of this script is, that it may be triggered directly on the named instance, to do the debugging.
This script sends a signal to a WaitCondition or to a CreationPolicy.
It in pair with the cfn-init it signals the completion of an instance creation.
/usr/local/bin/cfn-signal --stack ", { "Ref":"AWS::StackName" }, " --resource WebServerInstance" ," --region ", { "Ref": "AWS::Region" }," --exit-code $?
The creation of the resources may happen in parallel. Cloud formation waits if resources are marked as a dependent.
You can easily define the order, by specifying the DependsOn attribute.
Instance initiation is triggered from UserData. So UserData comes first.
"AWS::CloudFormation::Init" : {
"config" : {
"packages" : {
:
},
"groups" : {
:
},
"users" : {
:
},
"sources" : {
:
},
"files" : {
:
},
"commands" : {
:
},
"services" : {
:
}
A lot is necessary to download from S3 using a role autehnticaiton.
A role (here alf-digital-s3) is required with following settings:
Here are the details: https://aws.amazon.com/blogs/devops/authenticated-file-downloads-with-cloudformation/
{
"Parameters": {
...
"RoleS3ReaderName": {
"Description": "IAM role for S3 access",
"Type": "String",
"Default": "myIamS3RoleWithTrustToEc2Service",
"ConstraintDescription": "Must be a valid IAM role, with read access to S3 files."
}
},
"Resources": {
"WebServerInstance": {
"Type": "AWS::EC2::Instance",
"Metadata": {
"AWS::CloudFormation::Authentication": {
"default": {
"type": "S3",
"buckets": ["mybucketname"],
"roleName": {
"Ref": "RoleS3ReaderName"
}
}
},
"AWS::CloudFormation::Init": {
"configSets": {
"myConfigSetsHere": ["myconfigset"]
},
"myconfigset": {
"files": {
"/root/alf.digital.seed.zip": {
"source": "http://mybucketname.s3-eu-central-1.amazonaws.com/path/to/myarchive.zip",
"authentication": "default",
"mode": "000644",
"owner": "root",
"group": "root"
}
}
}
}
},
"CreationPolicy": {
"ResourceSignal": {
"Count": "1",
"Timeout": "PT15M"
}
},
"Properties": {
...
"IamInstanceProfile": {
"Ref": "InstanceProfileS3"
}
}
},
"InstanceProfileS3": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [{
"Ref": "RoleS3ReaderName"
}]
}
}
},
}
To reference cloudformation arguments in commands - use the “env”
Resources:
Parameters:
S3Prefix:
AllowedPattern: >-
^s3://[/a-zA-Z0-9_-]*[a-zA-Z0-9_-]$
Description: The s3://..... prefix to the bucket and folder, where the temp files are located.
Type: String
...
04_execute:
command: !Sub |
ansible-playbook /ssh-jump-host/sshJumpServer.ansible.yml -v --extra-vars "host=local" --extra-vars s3prefix="${S3Prefix}" >> "/ssh-jump-host/sshJumpServer.ansible.yml.log"
When creating a windows-machine - it is possible to join the domain via a PowerShell script, instead of using the wizard inside the Amazon console.
It is useful if some initiation of the domain must be done. Since the domain-joining via the Wizard seems to happen AFTER the user-data are executed - to be able to modify the Active Directory from the user-data we must manually join the domain.
For that we do the following:
During the script, after the domain join the machine will be restarted. So the execution of the script will be stopped. In order to go on with the userdata script execution after the restart one should use <persist>true</persist>. This will make the script persistent and executed after the restart again.
<powershell>
echo "START" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
#install snapin
Install-WindowsFeature -Name AD-Domain-Services,GPMC -IncludeManagementTools -Restart
##### JOIN THE DOMAIN #####
#Retrieve the AWS instance ID, keep trying until the metadata is available
$instanceID = "null"
while ($instanceID -NotLike "i-*") {
Start-Sleep -s 3
$instanceID = invoke-restmethod -uri http://169.254.169.254/latest/meta-data/instance-id
echo "Waiting until the instance metadata is available" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
}
echo "Instance metadata is available" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
# set the domain-own DNS servers to the network adapter, so that the domain name can be resolved and we can join
echo "Setting DNS for the Ethernet adapter" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
$adapter = Get-NetAdapter -Name "Ethernet 2"
Set-DnsClientServerAddress -InterfaceAlias $adapter.Name -ServerAddresses ("21.1.2.28","21.1.3.84")
echo "Setting DNS done" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
# join the domain now
echo "Joining the domain block start" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
$domain = "basicm.local"
$username = "$domain\admin"
$password = "TonoGada65" | ConvertTo-SecureString -AsPlainText -Force
$cred = New-Object -typename System.Management.Automation.PSCredential($username, $password)
Try {
$partOfDomain = (Get-WmiObject -Class Win32_ComputerSystem).PartOfDomain
if ( $partOfDomain ) {
echo "Already part of the domain. Wont trigger domain join" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
}
Else {
echo "Not yet part of the domain - start joining" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
Rename-Computer -NewName $instanceID -Force
Start-Sleep -s 5
Add-Computer -DomainName basicm.local -OUPath "OU=basicm,dc=basicm,dc=local" -Options JoinWithNewName,AccountCreate -Credential $cred -Force -Restart -erroraction 'stop'
}
}
Catch{
echo $_.Exception | Out-File c:\Windows\Temp\error-joindomain.txt -Append
}
echo "Joining the domain block done" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
# wait until the machine joins the domain
echo "Now waiting for the instance to be part of the domain" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
$partOfDomain = $False
while ( -not $partOfDomain ) {
Start-Sleep -s 3
$partOfDomain = (Get-WmiObject -Class Win32_ComputerSystem).PartOfDomain
echo "Waiting for the instance to join the domain" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
}
echo "Instance joined the domain '$env:userdomain'" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
# add a basic user
echo "Now modify the Active Directory" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
Try {
echo "Adding new OU" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
New-ADOrganizationalUnit -Name Groups -Path "OU=basicm,dc=basicm,dc=local" -Credential $cred
NEW-ADGroup –name "OpenVpnUsers" –groupscope Global –path "OU=Groups,OU=basicm,dc=basicm,dc=local" -Credential $cred
echo "Adding new user" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
New-ADUser -GivenName First -Surname User -Name first.user -Path "ou=Users,ou=basicm,dc=basicm,dc=local" -PasswordNeverExpires $True -ChangePasswordAtLogon $False -Description "The initial OpenVPN user" -Credential $cred
$user = Get-ADUser "CN=first.user,ou=Users,ou=basicm,dc=basicm,dc=local" -Credential $cred
echo "Setting a new password" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
Set-ADAccountPassword $user -NewPassword (ConvertTo-SecureString "123AbC!!!!" -AsPlainText -force) -Reset -Credential $cred
Enable-ADAccount -Identity $user -Credential $cred
echo "Adding new member to the groups" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
$group = Get-ADGroup "CN=OpenVpnUsers,OU=Groups,OU=basicm,dc=basicm,dc=local" -Credential $cred
Add-ADGroupMember $group -Members $user -Credential $cred
$group = Get-ADGroup "CN=Admins,OU=AWS Delegated Groups,dc=basicm,dc=local" -Credential $cred
Add-ADGroupMember $group -Members $user -Credential $cred
echo "Adding a system user" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
New-ADUser -Name s000001 -Path "ou=Users,ou=basicm,dc=basicm,dc=local" -PasswordNeverExpires $True -ChangePasswordAtLogon $False -Description "System user for LDAP connections" -Credential $cred
$user = Get-ADUser "CN=s000001,ou=Users,ou=basicm,dc=basicm,dc=local" -Credential $cred
echo "Set the password and enable" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
Set-ADAccountPassword $user -NewPassword (ConvertTo-SecureString "komumisa76!" -AsPlainText -force) -Reset -Credential $cred
Enable-ADAccount -Identity $user -Credential $cred
}
Catch{
echo $_.Exception | Out-File c:\Windows\Temp\error-joindomain.txt -Append
}
# disable the userdata running task scheduler task, now when we are done
echo "Now Disable the task" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
Try {
& SCHTASKS /Change /DISABLE /TN "\Amazon Ec2 Launch - Userdata Execution"
}
Catch{
echo $_.Exception | Out-File c:\Windows\Temp\error-joindomain.txt -Append
}
echo "Script done" | Out-File c:\Windows\Temp\logs-joindomain.txt -Append
</powershell>
<persist>true</persist>
To convert the above powershell script to a cloudformation capable UserData block do:
Result:
{
"AWSTemplateFormatVersion": "2010-09-09",
"Description" : "Creates a Windows server, joins it to the domain 'basic.local' using the domain admin, creates an initial directory users and some directory groups.",
"Parameters": {
"Subnet": {
"Type": "AWS::EC2::Subnet::Id",
"Description": "A private, shared subnet."
},
"SecGroup": {
"Type": "AWS::EC2::SecurityGroup::Id",
"Description": "A private, shared security group."
},
"KeyName": {
"Description": "Name of an existing EC2 KeyPair to enable SSH access to the instances",
"Type": "AWS::EC2::KeyPair::KeyName",
"ConstraintDescription": "must be the name of an existing EC2 KeyPair."
},
"DomainDnsServer1": {
"Type": "String",
"Description": "The DNS Server 1 of the directory. Check the directory service > Directory > DNS Address "
},
"DomainDnsServer2": {
"Type": "String",
"Description": "The DNS Server 2 of the directory. Check the directory service > Directory > DNS Address"
},
"DomainAdminPassword": {
"Type": "String",
"Description": "The password of the existing domain user 'admin', which was defined during the directory creation.",
"Default": "TonoGada65"
},
"FirstUserPassword": {
"Type": "String",
"Description": "The password for the example domain user 'first.user', which will be created. Use him e.g. for RDP connection. ",
"Default": "123AbC!!!!"
},
"S000001UserPassword": {
"Type": "String",
"Description": "The password for the system domain user 'S000001', which will be created. Use him e.g. for LDAP/AD joining.",
"Default": "komumisa76!"
}
},
"Resources": {
"MicrosoftWindowsAdManager": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-96b824ef",
"UserData": {
"Fn::Base64": {
"Fn::Join": ["",
[" <powershell> \n ",
" \n ",
" echo \"START\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" \n ",
" #install snapin \n ",
" Install-WindowsFeature -Name AD-Domain-Services,GPMC -IncludeManagementTools -Restart \n ",
" \n ",
" ##### JOIN THE DOMAIN ##### \n ",
" \n ",
" #Retrieve the AWS instance ID, keep trying until the metadata is available \n ",
" $instanceID = \"null\" \n ",
" while ($instanceID -NotLike \"i-*\") { \n ",
" Start-Sleep -s 3 \n ",
" $instanceID = invoke-restmethod -uri http://169.254.169.254/latest/meta-data/instance-id \n ",
" echo \"Waiting until the instance metadata is available\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" } \n ",
" echo \"Instance metadata is available\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" \n ",
" \n ",
" # set the domain-own DNS servers to the network adapter, so that the domain name can be resolved and we can join \n ",
" echo \"Setting DNS for the Ethernet adapter\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" $adapter = Get-NetAdapter -Name \"Ethernet 2\" \n ",
" Set-DnsClientServerAddress -InterfaceAlias $adapter.Name -ServerAddresses (\"",{ "Ref" : "DomainDnsServer1" },"\",\"",{ "Ref" : "DomainDnsServer2" },"\") \n ",
" echo \"Setting DNS done\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" \n ",
" \n ",
" # join the domain now \n ",
" echo \"Joining the domain block start\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" $domain = \"basic.local\" \n ",
" $username = \"$domain\\admin\" \n ",
" $password = \"",{ "Ref" : "DomainAdminPassword" },"\" | ConvertTo-SecureString -AsPlainText -Force \n ",
" $cred = New-Object -typename System.Management.Automation.PSCredential($username, $password) \n ",
" \n ",
" Try { \n ",
" $partOfDomain = (Get-WmiObject -Class Win32_ComputerSystem).PartOfDomain \n ",
" if ( $partOfDomain ) { \n ",
" echo \"Already part of the domain. Wont trigger domain join\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" } \n ",
" Else { \n ",
" echo \"Not yet part of the domain - start joining\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" Rename-Computer -NewName $instanceID -Force \n ",
" Start-Sleep -s 5 \n ",
" Add-Computer -DomainName basic.local -OUPath \"OU=basic,dc=basic,dc=local\" -Options JoinWithNewName,AccountCreate -Credential $cred -Force -Restart -erroraction 'stop' \n ",
" } \n ",
" \n ",
" } \n ",
" Catch{ \n ",
" echo $_.Exception | Out-File c:\\Windows\\Temp\\error-joindomain.txt -Append \n ",
" } \n ",
" echo \"Joining the domain block done\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" \n ",
" \n ",
" \n ",
" # wait until the machine joins the domain \n ",
" echo \"Now waiting for the instance to be part of the domain\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" $partOfDomain = $False \n ",
" while ( -not $partOfDomain ) { \n ",
" Start-Sleep -s 3 \n ",
" $partOfDomain = (Get-WmiObject -Class Win32_ComputerSystem).PartOfDomain \n ",
" echo \"Waiting for the instance to join the domain\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" } \n ",
" echo \"Instance joined the domain '$env:userdomain'\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" \n ",
" \n ",
" # add an initial user \n ",
" echo \"Now modify the Active Directory\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" Try { \n ",
" echo \"Adding new OU\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" New-ADOrganizationalUnit -Name Groups -Path \"OU=basic,dc=basic,dc=local\" -Credential $cred \n ",
" NEW-ADGroup –name \"OpenVpnUsers\" –groupscope Global –path \"OU=Groups,OU=basic,dc=basic,dc=local\" -Credential $cred \n ",
" \n ",
" \n ",
" \n ",
" echo \"Adding new user\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" New-ADUser -GivenName First -Surname User -Name first.user -Path \"ou=Users,ou=basic,dc=basic,dc=local\" -PasswordNeverExpires $True -ChangePasswordAtLogon $False -Description \"The initial OpenVPN user\" -Credential $cred \n ",
" $user = Get-ADUser \"CN=first.user,ou=Users,ou=basic,dc=basic,dc=local\" -Credential $cred \n ",
" \n ",
" echo \"Setting a new password\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" Set-ADAccountPassword $user -NewPassword (ConvertTo-SecureString \"",{ "Ref" : "FirstUserPassword" },"\" -AsPlainText -force) -Reset -Credential $cred \n ",
" Enable-ADAccount -Identity $user -Credential $cred \n ",
" \n ",
" echo \"Adding new member to the groups\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" $group = Get-ADGroup \"CN=OpenVpnUsers,OU=Groups,OU=basic,dc=basic,dc=local\" -Credential $cred \n ",
" Add-ADGroupMember $group -Members $user -Credential $cred \n ",
" $group = Get-ADGroup \"CN=Admins,OU=AWS Delegated Groups,dc=basic,dc=local\" -Credential $cred \n ",
" Add-ADGroupMember $group -Members $user -Credential $cred \n ",
" \n ",
" echo \"Adding a system user\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" New-ADUser -Name s000001 -Path \"ou=Users,ou=basic,dc=basic,dc=local\" -PasswordNeverExpires $True -ChangePasswordAtLogon $False -Description \"System user for LDAP connections\" -Credential $cred \n ",
" $user = Get-ADUser \"CN=s000001,ou=Users,ou=basic,dc=basic,dc=local\" -Credential $cred \n ",
" \n ",
" echo \"Set the password and enable\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" Set-ADAccountPassword $user -NewPassword (ConvertTo-SecureString \"",{ "Ref" : "S000001UserPassword" },"\" -AsPlainText -force) -Reset -Credential $cred \n ",
" Enable-ADAccount -Identity $user -Credential $cred \n ",
" } \n ",
" Catch{ \n ",
" echo $_.Exception | Out-File c:\\Windows\\Temp\\error-joindomain.txt -Append \n ",
" } \n ",
" \n ",
" \n ",
" \n ",
" \n ",
" # disable the userdata running task scheduler task, now when we are done \n ",
" echo \"Now Disable the task\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" Try { \n ",
" & SCHTASKS /Change /DISABLE /TN \"\\Amazon Ec2 Launch - Userdata Execution\" \n ",
" } \n ",
" Catch{ \n ",
" echo $_.Exception | Out-File c:\\Windows\\Temp\\error-joindomain.txt -Append \n ",
" } \n ",
" \n ",
" \n ",
" echo \"Script done\" | Out-File c:\\Windows\\Temp\\logs-joindomain.txt -Append \n ",
" \n ",
" </powershell> \n ",
" <persist>true</persist> "]]
}
},
"NetworkInterfaces": [{
"AssociatePublicIpAddress": "true",
"DeviceIndex": "0",
"SubnetId": {
"Ref": "Subnet"
},
"GroupSet": [{
"Ref": "SecGroup"
}]
}],
"KeyName": { "Ref" : "KeyName" },
"Tags": [{
"Key": "Name",
"Value": "AMicrosoftAdManager"
},
{
"Key": "Purpose",
"Value": "To manage the Active Directory Users. Should be stopped the most of the time."
}],
"InstanceType": "t2.small",
"BlockDeviceMappings": [{
"DeviceName": "/dev/sda1",
"Ebs": {
"VolumeSize": "30",
"VolumeType": "gp2"
}
}]
}
}
}
}
To validate the local template use the aws command. The validation includes simple dependency checks too.
aws cloudformation validate-template --template-body "file:////mnt/d/1PROJEKTE/AWS/alf.digital/cloudFormation/alf.digital.template"
The logs on an Instance are here: /var/log/
| Human readable logs of the whole instance initiation | /var/log/cloud-init-output.log |
| Human readable logs of the cfn-init call | /var/log/cfn-init.log |
| Code from AWS::CloudFormation::Init block of a Cloud Formation template on Linux systems | /var/lib/cfn-init/data/metadata.json |
You can repeat the initiation part of the instance by triggering the cfn-init script.
/usr/local/bin/cfn-init -v --stack WebS<erver1 --resource WebServerInstance --region eu-central-1 --configset InstallAndRun && cat /var/log/cfn-init.log
The user data are located under /var/lib/cloud/instance
Here is the complete troubleshooting guide: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html
The scripts, created by userData are located under: /var/lib/cloud/instance/scripts
You can try to re run them.
Here is an example creates a security group, an EC2 Instance and associate the group with the instance. It has parameters: - the instance type - VPC Id - Server subnet
https://gist.github.com/skipidar/a8e86fbeccd51e6fb96f720ddd3885f1
Here are some examples by AWS
Exporting Stack Output Values vs. Using Nested Stacks:
Stack A Export:
"Outputs" : {
"PublicSubnet" : {
"Description" : "The subnet ID to use for public web servers",
"Value" : { "Ref" : "PublicSubnet" },
"Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-SubnetID" }}
},
"WebServerSecurityGroup" : {
"Description" : "The security group ID to use for public web servers",
"Value" : { "Fn::GetAtt" : ["WebServerSecurityGroup", "GroupId"] },
"Export" : { "Name" : {"Fn::Sub": "${AWS::StackName}-SecurityGroupID" }}
}
}
Stack B Import
"Resources" : {
"WebServerInstance" : {
"Type" : "AWS::EC2::Instance",
"Properties" : {
"InstanceType" : "t2.micro",
"ImageId" : "ami-a1b23456",
"NetworkInterfaces" : [{
"GroupSet" : [{"Fn::ImportValue" : {"Fn::Sub" : "${NetworkStackNameParameter}-SecurityGroupID"}}],
"AssociatePublicIpAddress" : "true",
"DeviceIndex" : "0",
"DeleteOnTermination" : "true",
"SubnetId" : {"Fn::ImportValue" : {"Fn::Sub" : "${NetworkStackNameParameter}-SubnetID"}}
}]
}
}
}
AWS specific parameters:
Example of a custom definition in CLoudFormation. EFS with Fargate in this example
CustomTaskDefinition:
Type: 'Custom::TaskDefinition'
Version: '1.0'
Properties:
ServiceToken: !GetAtt 'CustomResourceFunction.Arn'
TaskDefinition: {
executionRoleArn: !Ref CustomTaskDefinitionRole,
containerDefinitions: [
{
name: "navvis",
image: !Ref NavvisDockerImage,
memoryReservation: 2000,
logConfiguration: {
logDriver: "awslogs",
options: {
awslogs-group: !Ref CustomLogsGroup,
awslogs-datetime-format: "%Y-%m-%d %H:%M:%S.%L",
awslogs-region: !Ref 'AWS::Region',
awslogs-stream-prefix: !Sub '${ProjectName}'
}
},
portMappings: [
{
hostPort: 8080,
protocol: "tcp",
containerPort: 8080
}
],
command: [],
"environment": [
{
"name": "INSTANCE_NAME",
"value": !Ref 'BuildingID'
},
{
"name": "INSTANCE_PORT",
"value": "8080"
},
{
"name": "STORAGE_PATH",
"value": "/mnt"
}
]
,
mountPoints: [
{sourceVolume: "myefs", containerPath: "/mnt"}
]
}
],
family: "navvis",
taskRoleArn: !Ref CustomTaskDefinitionRole, #required for EFS permissions
requiresCompatibilities: ["FARGATE"],
cpu: "256",
memory: "2048",
networkMode: "awsvpc",
volumes: [
{
name: "myefs",
efsVolumeConfiguration: {
fileSystemId: {'Fn::ImportValue': !Sub '${ProjectName}-arc-${TargetAwsAccount}-efsid-for-${BuildingID}----${ScanID}'}
}
},
]
}
CustomResourceFunction:
Type: 'AWS::Lambda::Function'
Properties:
Code:
ZipFile: |
const aws = require('aws-sdk')
const response = require('cfn-response')
const ecs = new aws.ECS({apiVersion: '2014-11-13'})
exports.handler = function(event, context) {
console.log("REQUEST RECEIVED:\n" + JSON.stringify(event))
if (event.RequestType === 'Create' || event.RequestType === 'Update') {
ecs.registerTaskDefinition(event.ResourceProperties.TaskDefinition, function(err, data) {
if (err) {
console.error(err);
response.send(event, context, response.FAILED)
} else {
console.log(`Created/Updated task definition ${data.taskDefinition.taskDefinitionArn}`)
response.send(event, context, response.SUCCESS, {}, data.taskDefinition.taskDefinitionArn)
}
})
} else if (event.RequestType === 'Delete') {
ecs.deregisterTaskDefinition({taskDefinition: event.PhysicalResourceId}, function(err) {
if (err) {
if (err.code === 'InvalidParameterException') {
console.log(`Task definition: ${event.PhysicalResourceId} does not exist. Skipping deletion.`)
response.send(event, context, response.SUCCESS)
} else {
console.error(err)
response.send(event, context, response.FAILED)
}
} else {
console.log(`Removed task definition ${event.PhysicalResourceId}`)
response.send(event, context, response.SUCCESS)
}
})
} else {
console.error(`Unsupported request type: ${event.RequestType}`)
response.send(event, context, response.FAILED)
}
}
Handler: 'index.handler'
MemorySize: 128
Role: !GetAtt 'CustomResourceRole.Arn'
Runtime: 'nodejs10.x'
Timeout: 30
CustomResourceRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Principal:
Service: 'lambda.amazonaws.com'
Action: 'sts:AssumeRole'
Policies:
- PolicyName: 'customresource'
PolicyDocument:
Statement:
- Effect: Allow
Action:
- 'ecs:DeregisterTaskDefinition'
- 'ecs:RegisterTaskDefinition'
Resource: '*'
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource: '*'
- Effect: Allow
Action:
- 'iam:PassRole'
Resource: '*' # replace with value of taskRoleArn
CustomLogsGroup:
Type: 'AWS::Logs::LogGroup'
Properties:
LogGroupName: !Sub "${ProjectName}-custom"
RetentionInDays: 14
NavvisServiceFargate:
Type: 'AWS::ECS::Service'
Properties:
Cluster: {'Fn::ImportValue': !Sub '${ProjectName}-arc-${TargetAwsAccount}-ecscluster'}
LaunchType: FARGATE
DesiredCount: 1
PlatformVersion: "1.4.0" # by default on 30th June 2020 still pick up automatically the version 1.3.0 and not really the latest (same behaviour on web console) but to have the EFS we need the 1.4.0
DeploymentConfiguration:
MinimumHealthyPercent: 0 # setting to 0 as otherwise, if there are not enough resources (e.g. eni per instance) - the service will not be able to tear down the old container to replace it by the new one
MaximumPercent: 250
TaskDefinition: !Ref CustomTaskDefinition
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: DISABLED
SecurityGroups:
- {'Fn::ImportValue': !Sub '${ProjectName}-arc-${TargetAwsAccount}-codebuildfargate-sg'}
Subnets:
- {'Fn::ImportValue': !Sub '${PlatformPrefix}-PrivateSubnet1AID'}
- {'Fn::ImportValue': !Sub '${PlatformPrefix}-PrivateSubnet1BID'}
- {'Fn::ImportValue': !Sub '${PlatformPrefix}-PrivateSubnet1CID'}
LoadBalancers:
- ContainerName: 'navvis'
ContainerPort: 8080
TargetGroupArn: !Ref TargetGroupNavvisHTTP
SchedulingStrategy: "REPLICA"