bash Command not executing on Ec2 from Lambda

0

I'm trying to run certain bash on a Ec2 instance that I create in a Lambda function.

code

const instanceParams = {
      ImageId: "ami-0c101f26f147fa7fd",
      InstanceType: "t2.micro",
      MinCount: 1,
      MaxCount: 1,
      IamInstanceProfile: {
        Arn: iparn
      },
      UserData: Buffer.from(
        `#!/bin/bash
    export NANO_ID="${nanoId}" 
    echo $NANO_ID > test2.txt
    aws s3 cp ./test2.txt s3://s3storebucketName/test2.txt
    
    aws s3 cp s3://s3storebucketName/Script.js . > test3.txt 2>&1
    aws s3 cp ./test3.txt s3://s3storebucketName/test3.txt
    
    sudo yum install nodejs npm -y > test1.txt 2>&1
    aws s3 cp ./test1.txt s3://s3storebucketName/test1.txt
    
    npm install @aws-sdk/client-dynamodb @aws-sdk/client-s3 > test4.txt 2>&1
    aws s3 cp ./test4.txt s3://s3storebucketName/test4.txt
    
    node Script.js > test5.txt 2>&1
    aws s3 cp ./test5.txt s3://s3storebucketName/test5.txt
    `
      ).toString("base64")
    };
      const data = await ec2.send(new RunInstancesCommand(instanceParams));

above is the code for my instanceParams and how i call my RunInstancesCommand. The bash command that I have given in the UserData. It only runs till before sudo yum install nodejs npm -y > test1.txt 2>&1 meaning there is some issue with the suco yum install command. iparn has the role of AdministratorAccess, and it also has the security group with range of ports and protocols set to all.

I need help in installing nodejs and npm onto my ec2 instance. How should I go about it. what is the easiest way to get the packages installed.

I have to run it in this way. I shouldnt be SSHing into Ec2 via Lambda.

I tried using dnf instead of yum.

EDIT:

Above was part of the below code

"use strict";

// lambda/ScriptFunction.js
import {
  EC2Client,
  RunInstancesCommand,
  waitForInstanceRunning,
  TerminateInstancesCommand,
  CreateKeyPairCommand
} from "@aws-sdk/client-ec2";

import {
  IAMClient,
  GetRoleCommand,
  CreateInstanceProfileCommand,
  AddRoleToInstanceProfileCommand,
  ListInstanceProfilesCommand,
  GetInstanceProfileCommand
} from "@aws-sdk/client-iam";

export async function handler(event) {
  console.log("------------------------");
  try {
    for (const record of event.Records) {
      if (record.eventName === "INSERT") {
        await handleInsert(record);
      }
    }
    console.log("------------------------");
    return "Success!";
  } catch (error) {
    console.error(error);
    console.log("------------------------");
    return "Error";
  }
}

async function handleInsert(record) {
  console.log("Handling INSERT Event");
  const newImage = record.dynamodb.NewImage;
  const nanoId = newImage.nanoId.S;
  const res = await createEC2(nanoId);
  console.log(res);
  console.log("Done handling INSERT Event");
}

async function getInstanceProfileArn(iam, instanceProfileName) {
  try {
    const params = {
      InstanceProfileName: instanceProfileName
    };
    const data = await iam.send(new GetInstanceProfileCommand(params));
    console.log(data);
    return data.InstanceProfile.Arn;
  } catch (error) {
    console.error("Error arn:", error);
    throw error;
  }
}

async function createEC2(nanoId) {
  const ec2 = new EC2Client({});
  const iam = new IAMClient({});
  try {
    const ipname = "Ec2adminRole-instance-profile";
    const iparn = await getInstanceProfileArn(iam, ipname);
    console.log(iparn);
    console.log("Role added to instance profile successfully.");
    const instanceParams = {
      ImageId: "ami-0a699202e5027c10d",
      InstanceType: "t2.micro",
      MinCount: 1,
      MaxCount: 1,
      IamInstanceProfile: {
        Arn: iparn
      },
      UserData: Buffer.from(
        `#!/bin/bash
        update yum -y > test0.txt
        aws s3 cp ./test0.txt s3://s3samplebucket/test0.txt
        export NANO_ID="${nanoId}" 
        echo $NANO_ID > test2.txt
        aws s3 cp ./test2.txt s3://s3samplebucket/test2.txt
        aws s3 cp s3://s3samplebucket/Script.js . > test3.txt 2>&1
        aws s3 cp ./test3.txt s3://s3samplebucket/test3.txt
        yum install -y nodejs npm > test1.txt 2>&1
        aws s3 cp ./test1.txt s3://s3samplebucket/test1.txt
        npm install @aws-sdk/client-dynamodb @aws-sdk/client-s3 > test4.txt 2>&1
        aws s3 cp ./test4.txt s3://s3samplebucket/test4.txt
        node Script.js > test5.txt 2>&1
        aws s3 cp ./test5.txt s3://s3samplebucket/test5.txt`
      ).toString("base64")
    };
    const data = await ec2.send(new RunInstancesCommand(instanceParams));
    const instanceId = data.Instances[0].InstanceId;
    console.log(instanceId);
    await waitForInstanceRunning(
      { client: ec2, maxWaitTime: 300000000 },
      { InstanceIds: [instanceId] }
    );
    await new Promise((resolve) => setTimeout(resolve, 6e3));
    await ec2.send(
      new TerminateInstancesCommand({ InstanceIds: [instanceId] })
    );
    return {
      statusCode: 200,
      body: data
    };
  } catch (error) {
    console.error("Error creating EC2 instance:", error);
    throw error;
  }
}

The output on the text files is as such: test2.txt and test3.txt are created and the expected data is seen. test0.txt doesnt output anything, its an empty file. test4.txt and test5.txt are seen the s3 bucket mean they arent created.

I tried use the EventBridge to see if I cloud log the output of each command but it is just triggered event. Doesnt really say anything.

3 Answers
2
Accepted Answer

The nodejs package is not available in the default Amazon Linux 2 repositories. You may need to enable the Amazon Linux 2 Extra repository before installing nodejs. You can refer to this guide for detailed instructions: How to Install and Configure Node.js on EC2 Instance Amazon Linux 2

profile picture
EXPERT
answered a month ago
profile pictureAWS
EXPERT
reviewed a month ago
  • Thank you this was helpful, But on subsequent use it doesnt work.

    I tried to implement it with below

    UserData: Buffer.from( #!/bin/bash yum update -y yum install gcc-c++ make -y curl -sL https://rpm.nodesource.com/setup_18.x | sudo bash - yum update -y yum install nodejs -y node -v > test.txt aws s3 cp ./test.txt s3://Justans3samplebucket/test.txt ).toString("base64")

    The first time i ran it, it work, if i run the lambda again it isnt working anymore. The first time in test.txt, it had output the node version.

  • The nodejs package is not available in the default Amazon Linux 2 repositories.

    The AMI on the second line of the script is AL2023

    $ aws ec2 describe-images --region us-east-1 --image-ids ami-0c101f26f147fa7fd
    {
        "Images": [
            {
                "VirtualizationType": "hvm",
                "Description": "Amazon Linux 2023 AMI 2023.4.20240319.1 x86_64 HVM kernel-6.1",
    

    Both packages should be available https://docs.aws.amazon.com/linux/al2023/release-notes/all-packages-AL2023.3.html

  • This is the image-ids he is using ami-0a699202e5027c10d

  • This is the image-ids he is using ami-0a699202e5027c10d

    Well spotted!

    The first code block has ImageId: "ami-0c101f26f147fa7fd" which is AL2023.

    The second code block has ImageId: "ami-0a699202e5027c10d" which is Amazon Linux 2.

    nodejs and npm are not available in the default repos on AL2 (you have to install the EPEL repo):

    $ grep -i pretty /etc/os-release
    PRETTY_NAME="Amazon Linux 2"
    $ sudo yum list nodejs npm
    Loaded plugins: extras_suggestions, langpacks, priorities, update-motd
    230 packages excluded due to repository priority protections
    Available Packages
    nodejs.x86_64                   1:16.20.2-1.el7                             epel
    npm.x86_64                      1:8.19.4-1.16.20.2.1.el7                    epel
    $
    

    @ThatDevguy can you clarify the version of Amazon Linux please?

  • With this should work already:

    UserData: Buffer.from(
     `#!/bin/bash
     sudo yum update -y
     sudo yum install gcc-c++ make -y
    curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | sudo bash
    export NVM_DIR="$HOME/.nvm"
    [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
    nvm install 14
    node -v > test.txt
     aws s3 cp ./test.txt s3://simples3bucket/test.txt`
     ).toString("base64")
    
1

Hello.

I think it can be installed using the method described in the document below.
So I thought it would be a good idea to add the following command to the user data.
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-up-node-on-ec2-instance.html

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm install --lts
node -e "console.log('Running Node.js ' + process.version)"
profile picture
EXPERT
answered a month ago
profile picture
EXPERT
reviewed a month ago
  • I tried this. I just get the output that node is not installed.

  • By the way, what kind of problem is displayed when executing "yum install"? For example, if there is a timeout error, there is a possibility that there is no network route to access the repository from EC2, resulting in an error. If you are running EC2 in a private subnet, I think you will need a communication route to the NAT Gateway. If it is a public subnet, you will need to check whether a public IP address is assigned to EC2.

  • In the respective test file, there is no content being printed. Above i have shown how i have created the EC2. I'm very new to this, I dont know how to set that up. How would I have to do that?

  • If you have not made any special subnet settings, EC2 is probably running in the default subnet. You can check the user data execution log using the following command to check if there are any errors.

    cat /var/log/cloud-init-output.log
    
  • By the way, the first line of the user data looks like the following, but the correct name is "yum update".

    update yum -y > test0.txt
    
1

User Data script will run as the root user anyway, so there should be no need to run it with sudo.

Try dropping sudo so it's just yum install nodejs npm -y > test1.txt 2>&1 and see if that gets you further.

There is still an issue then look at whether test1.txt gets created; and if it does get created are you able to post its contents here?

Your AMI ami-0c101f26f147fa7fd is Amazon Linux 2023 and both packages are definitely available https://docs.aws.amazon.com/linux/al2023/release-notes/all-packages-AL2023.3.html

profile picture
EXPERT
Steve_M
answered a month ago
profile picture
EXPERT
reviewed a month ago
  • I tried doing it without sudo too. It dint make any difference. Test1.txt is just the output the value i set to nano_Id. Yes both the packages is suppose to work but i dont understand why it is not working.

  • Test1.txt is just the output the value i set to nano_Id

    It's test2.txt that is set to ${NANO_ID}, test1.txt should be the output of the yum install command, and should be copied to the bucket (if it exists).

    I'm wondering if the EC2 is able to reach the YUM repos that the nodejs & npm packages need to be retrieved from?

  • Yes sorry you are right. For the Test1.txt. thats where its stops adding files to my s3 bucket. I just see test0, test2, test3, i dont see the rest. So I dont know what the output is or what the issue is.

  • Sounds like the yum command is never completing.

    When the EC2 is provisioned, does it have a public IP address, and is it provisioned in a public subnet (that is, a subnet whose route table has an entry for 0.0.0.0/0 destination Internet Gateway)? You should be able to tell this from the EC2 section of AWS Console.

    Also its security group will need to allow (at least) 80/tcp and 443/tcp outbound.

    The EC2 instance needs to be able to reach the YUM repositories on the internet and pull down the packages from there, and install them.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions