Unable to use any tools under /sbin in cloud-init script

0

I have a cloud init script which attaches a volume, creates a partition on said volume and then mounts that partition. For that I use parted and mkfs as usual. The image is based on Amazon Linux 2023 and it's AMI id is ami-005ad02a0f1729234 (questdb version 8.2.3).

The script is like this (stripped to essential part):

#!/bin/bash
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
INSTANCE_ID=$(curl -H"X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 attach-volume --device /dev/xvdc --instance-id $INSTANCE_ID --volume-id $VOLUME_ID
echo "volume attached, waiting for device to become available"
while ! lsblk | grep xvdc; do
    sleep 0.1
done
parted -s -a opt /dev/xvdc mklabel gpt
parted -s -a opt /dev/xvdc mkpart primary ext4 0% 100%
mkfs -t ext4 /dev/xvdc1
mount /dev/xvdc1 /mnt
  • The above will result in parted: command not found.
  • If I change to /sbin/parted, the mkfs is not found
  • If I change mkfs to /sbin/mkfs then I get .../sbin/mkfs.ext4: no such file or directory
  • If I edit path at the top of the scrip export PATH="/sbin/:/usr/sbin/:$PATH" it has no effect
  • If I call which parted I get no parted in ((null))
  • If I run with #!/usr/bin/env bash it has no effect

The only workaround is to switch paths to /usr/sbin/ and not use mkfs but use the filesystem-specific program directly. I.e. this works.

#!/bin/bash
TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
INSTANCE_ID=$(curl -H"X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
aws ec2 attach-volume --device /dev/xvdc --instance-id $INSTANCE_ID --volume-id $VOLUME_ID
echo "volume attached, waiting for device to become available"
while ! lsblk | grep xvdc; do
    sleep 0.1
done
/usr/sbin/parted -s -a opt /dev/xvdc mklabel gpt
/usr/sbin/parted -s -a opt /dev/xvdc mkpart primary ext4 0% 100%
/usr/sbin/mkfs.ext4 /dev/xvdc1
mount /dev/xvdc1 /mnt

I am creating the instance with the following CDK code:

    const instance = new ec2.Instance(this, "Instance", {
      instanceType: instanceType,
      machineImage: ec2.MachineImage.lookup({
        name: `questdb-${cfg.questdbVersion}*`,
      }),
      vpc: vpc,
      availabilityZone: vpc.availabilityZones[0],
      vpcSubnets: {
        subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
      },
      propagateTagsToVolumeOnCreation: true,
      securityGroup,
    });
    const volume = new ec2.Volume(
      this,
      `QuestdbVolume${hashFromString(`${cfg.questdbVersion}${cfg.snapshotId ?? ""}`, 4)}`,
      {
        encrypted: true,
        availabilityZone: vpc.availabilityZones[0],
        snapshotId: cfg.snapshotId,
        size: cdk.Size.gibibytes(cfg.dbSize),
        iops: 16000,
        throughput: 1000,
        volumeType: ec2.EbsDeviceVolumeType.GENERAL_PURPOSE_SSD_GP3,
      },
    );
    instance.applyCloudFormationInit(
      ec2.CloudFormationInit.fromElements(
        ec2.InitFile.fromFileInline("/opt/init/init.sh", path.join(__dirname, "init", "init.sh"), {
          mode: "000755",
        }),
        ec2.InitCommand.shellCommand("/opt/init/init.sh", {
          env: {
            VOLUME_ID: volume.volumeId
          },
        }),
      ),
    );

I found absolutely nothing online about this. Is this something specific to this QuestDB image? Why is documentation not mentioning anything about that?

1 Answer
1
Accepted Answer

On Amazon Linux 2023 and specifically in your QuestDB-based AMI, utilities like parted, mkfs.ext4, etc., are located in /usr/sbin and /sbin, but those directories may not be in PATH during execution unless explicitly set.

However, simply setting export PATH=... in the script often has no effect when using InitCommand, because each command runs in a fresh shell.

Solution: Use full paths for commands

To avoid issues, use the full absolute paths for the tools. Here’s a cleaned-up version of your script that works reliably:

#!/bin/bash

TOKEN=$(curl -X PUT "http://169.254.169.254/latest/api/token" \
  -H "X-aws-ec2-metadata-token-ttl-seconds: 21600")

INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" \
  http://169.254.169.254/latest/meta-data/instance-id)

aws ec2 attach-volume --device /dev/xvdc --instance-id "$INSTANCE_ID" --volume-id "$VOLUME_ID"

echo "volume attached, waiting for device to become available"

while ! lsblk | grep xvdc; do
    sleep 0.1
done

/usr/sbin/parted -s -a opt /dev/xvdc mklabel gpt
/usr/sbin/parted -s -a opt /dev/xvdc mkpart primary ext4 0% 100%
/usr/sbin/mkfs.ext4 /dev/xvdc1

mkdir -p /mnt
mount /dev/xvdc1 /mnt

Alternative: Use a wrapper script

If you want to avoid using full paths everywhere and ensure PATH is available, wrap your logic in a script and invoke it using bash -l (login shell), which sources /etc/profile and sets up the environment.

Modify InitCommand like this:

ec2.InitCommand.shellCommand("bash -l -c '/opt/init/init.sh'", {
  env: {
    VOLUME_ID: volume.volumeId,
  },
}),

This ensures the system paths from login shells are respected (/sbin, /usr/sbin, etc.).

profile picture
answered 17 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions