How can I send user-data output to the console logs on an EC2 instance running RHEL 7 or RHEL 8?

3 minute read
0

I want to troubleshoot my Amazon Elastic Compute Cloud (Amazon EC2) Linux RHEL 7 or RHEL 8 instance bootstrap. I want to log the user-data invocation and then ship it to the console logs.

Short description

To troubleshoot issues on your EC2 instance bootstrap, add code to your user-data bash script that redirects all output to /var/log/user-data.log and /dev/console. You can do this even if you can't access the instance through SSH. After you run the code, your user-data invocation logs appear in your console.

Note: This resolution is for RHEL 7 and RHEL 8 only. For information on Amazon Linux and Amazon Linux 2, see How can I send user-data output to the console logs on an EC2 instance running Amazon Linux or Amazon Linux 2?

Resolution

  1. Open the Amazon EC2 console.

  2. Launch a new instance and SSH into it.

  3. Edit the GRUB_CMDLINE_LINUX line in /etc/default/grub and change "console=ttyS0,115200n8 console=tty0" to "console=tty1 console=ttyS0":

    RHEL 7

    # cat /etc/default/grub
    GRUB_TIMEOUT=1
    GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
    GRUB_DEFAULT=saved
    GRUB_DISABLE_SUBMENU=true
    GRUB_TERMINAL_OUTPUT="console"
    GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0 net.ifnames=0 rd.blacklist=nouveau crashkernel=auto"
    GRUB_DISABLE_RECOVERY="true"

    RHEL 8

    # cat /etc/default/grub
    GRUB_TIMEOUT=1
    GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
    GRUB_DEFAULT=saved
    GRUB_DISABLE_SUBMENU=true
    GRUB_TERMINAL_OUTPUT="console"
    GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0 net.ifnames=0 rd.blacklist=nouveau nvme_core.io_timeout=4294967295 crashkernel=auto"
    GRUB_DISABLE_RECOVERY="true"
    GRUB_ENABLE_BLSCFG=true
  4. Recreate the /boot/grub2/grub.cfg file:

    RHEL 7

    # grub2-mkconfig -o /boot/grub2/grub.cfg
    Generating grub configuration file ...
    Found linux image: /boot/vmlinuz-3.10.0-1062.1.2.el7.x86_64
    Found initrd image: /boot/initramfs-3.10.0-1062.1.2.el7.x86_64.img
    Found linux image: /boot/vmlinuz-0-rescue-026767dbe06a4910a5ce3bd0def903c0
    Found initrd image: /boot/initramfs-0-rescue-026767dbe06a4910a5ce3bd0def903c0.img
    done

    RHEL 8

    # grub2-mkconfig -o /boot/grub2/grub.cfg
    Generating grub configuration file ...
    done
  5. Stop the instance.

  6. Create a new AMI from the modified instance.

  7. Launch a new instance from the new AMI.

  8. To redirect the user-data output console, run the following command:

    #!/bin/bash -xe
    exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1
      yum -y update
      echo "Hello from user-data!"

    The following line redirects the user-data output:

    exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

    You receive a console output that's similar to the following example:

    RHEL 7:

      subscription-manager.x86_64 0:1.24.26-3.el7_8
      subscription-manager-rhsm.x86_64 0:1.24.26-3.el7_8
      subscription-manager-rhsm-certificates.x86_64 0:1.24.26-3.el7_8
      sudo.x86_64 0:1.8.23-9.el7
      systemd.x86_64 0:219-73.el7_8.8
      systemd-libs.x86_64 0:219-73.el7_8.8
      systemd-sysv.x86_64 0:219-73.el7_8.8
      teamd.x86_64 0:1.29-1.el7
      tuned.noarch 0:2.11.0-8.el7
      tzdata.noarch 0:2020a-1.el7
      util-linux.x86_64 0:2.23.2-63.el7
      yum.noarch 0:3.4.3-167.el7
      yum-utils.noarch 0:1.1.31-54.el7_8
    
    Replaced:
      iwl7265-firmware.noarch 0:22.0.7.0-72.el7
    
    Complete!
    + echo 'Hello from user-data!'
    Hello from user-data!

    RHEL 8:

    Installed:
      grub2-tools-efi-1:2.02-82.el8_2.1.x86_64
    
      kernel-4.18.0-193.13.2.el8_2.x86_64
    
      kernel-core-4.18.0-193.13.2.el8_2.x86_64
    
      kernel-modules-4.18.0-193.13.2.el8_2.x86_64
    
      linux-firmware-20191202-97.gite8a0f4c9.el8.noarch
    
    Complete!
    +
    echo 'Hello from user-data!'
    Hello from user-data!

    Note: Because your user-data is visible in the console, be sure that you don't include confidential information in the data that you send.

Related information

Run commands on your Linux instance at launch

AWS OFFICIAL
AWS OFFICIALUpdated 5 months ago