Cloud-init runcmd not working in CloudFormation UserData.

Question

I have the following UserData in my CloudFormation template:

```
            "UserData": {
                    "Fn::Base64": {
                        "Fn::Join": [
                            "",
                            [
                                "#cloud-config
runcmd:
- sudo wget --directory-prefix -O prep_unused_disk.sh /opt/ABCops/sbin https://xxxxxxyyyyyyy-public.s3.amazonaws.com/scripts/common/prep_unused_disk.sh
- sudo chmod 755 /opt/ABCops/sbin/prep_unused_disk2.sh
- sudo /opt/ABCops/sbin/prep_unused_disk2.sh
",
                                "output: { all : '| tee -a /var/log/cloud-init-output.log'}
",
                                {
                                    "Fn::Join": [ " ",
                                        [
                                            "hostname:",
                                            { "Fn::Join": [ "", [ { "Ref": "AWS::StackName" }, "-HOST01
" ] ] }
                                        ]
                                    ]
                                },
                                {
                                    "Fn::Join": [ " ",
                                        [
                                            "fqdn:",
                                            { "Fn::Join": [ "", [ { "Ref": "AWS::StackName" }, "-HOST01.tm.ABCops.net
" ] ] }
                                        ]
                                    ]
                                },
                                { "Fn::Join": [ " ", [ "rh_subscription:
" ] ] },
                                {
                                    "Fn::Join": [ "",
                                        [
                                            "    activation-key: ",
                                            {
                                                "Fn::FindInMap": [
                                                    "PatchManagementPatchGroup",
                                                    { "Ref": "PatchManagementWave" },
                                                    {
                                                        "Fn::FindInMap": [
                                                            "PatchManagementLifecycle",
                                                            { "Ref": "ABCEnvironment" },
                                                            { "Ref": "AWS::AccountId" }
                                                        ]
                                                    }
                                                ]
                                            },
                                            "_", { "Ref": "OSDistro" }, "_",
                                            {
                                                "Fn::FindInMap": [
                                                    "PatchManagementLifecycle",
                                                    { "Ref": "ABCEnvironment" },
                                                    { "Ref": "AWS::AccountId" }
                                                ]
                                            }, "-AK"
                                        ]
                                    ]
                                },
                                { "Fn::Sub": "
    org: ABC-TM
" }
                            ]
                        ]
                    }
                },
```
What I am finding is that the first part of that UserData where it download a script from S3 then tries to run it does not work.  The second part of that UserData where it registers the host to Foreman/Katello DOES work.  I also have console=tty1 in my grub line.   Regardless of any of this I am seeing NOTHING in /var/log/cloud-init.log or /var/log/cloud-init-output.log.  Its as if this section of code has not run at all.  I only know the host is registered to Foreman/Katello by logging in there and looking.

Previously, I had the runcmd calling a local script in the AMI but that too gave me no indication that it was running and if it was if it worked.  So I changed it to a wget so at least I could validate that the script was being downloaded.

What am I doing wrong?  Why is it not logging anything?  If I remove one of the hyphens I will get in the logs that a yaml error happened so cloud-init is reading it.

Answer

Hello,

I tested a sample CloudFormation Template using the syntax above and was seeing the same behaviour. The commands under `runcmd` were not running. Then I noticed that the syntax used in `Fn::Join` for commands is a bit incorrect.

For example consider the following sample template, with commands that create a `prep_unused_disk.sh` script file and executes it. In the below example you will see that each command is separated and is a string in list of  `Fn::Join`. 
```
"UserData": {
    "Fn::Base64": {
        "Fn::Join": [
            "",
            [
                "#cloud-config
",
                "runcmd:
",
                "- sudo echo '#!/bin/bash' > prep_unused_disk.sh && echo 'echo "Hello, World!"' >> prep_unused_disk.sh
",
                "- sudo chmod 755 prep_unused_disk.sh
",
                "- sudo sh prep_unused_disk.sh
",
                "output: { all : '| tee -a /var/log/cloud-init-output.log'}
",
                {
                    "Fn::Join": [ " ",
                        [
                            "hostname:",
                            { "Fn::Join": [ "", [ { "Ref": "AWS::StackName" }, "-HOST01
" ] ] }
                        ]
                    ]
                },
                {
                    "Fn::Join": [ " ",
                        [
                            "fqdn:",
                            { "Fn::Join": [ "", [ { "Ref": "AWS::StackName" }, "-HOST01.tm.ABCops.net
" ] ] }
                        ]
                    ]
                },
                { "Fn::Join": [ " ", [ "rh_subscription:
" ] ] },
                {
                    "Fn::Join": [ "",
                        [
                            "    activation-key: ",
                            {
                                "Fn::FindInMap": [
                                    "PatchManagementPatchGroup",
                                    { "Ref": "PatchManagementWave" },
                                    {
                                        "Fn::FindInMap": [
                                            "PatchManagementLifecycle",
                                            { "Ref": "ABCEnvironment" },
                                            { "Ref": "AWS::AccountId" }
                                        ]
                                    }
                                ]
                            },
                            "_", { "Ref": "OSDistro" }, "_",
                            {
                                "Fn::FindInMap": [
                                    "PatchManagementLifecycle",
                                    { "Ref": "ABCEnvironment" },
                                    { "Ref": "AWS::AccountId" }
                                ]
                            }, "-AK"
                        ]
                    ]
                },
                { "Fn::Sub": "
    org: ABC-TM
" }
            ]
        ]
    }
}
```
After running the template with above UserData, I was able to view the log "Hello, World!" in  "/var/log/cloud-init-output.log".

Hope this helps.

Cloud-init runcmd not working in CloudFormation UserData.

Relevant content