By using AWS re:Post, you agree to the AWS re:Post Terms of Use

How do I assign a static private IP address to the primary node of my Amazon EMR cluster?

2 minute read
0

I want to create a static private IP address and associate the IP address with the primary node of an Amazon EMR cluster.

Resolution

To create a static private IP address and associate the IP address with your cluster's primary node, use a bootstrap action.

Complete the following steps:

  1. Confirm that your AWS Identity and Access Management (IAM) policy allows EMR_EC2_DefaultRole and ec2:AssignPrivateIpAddresses permissions.
  2. Sign in to GitHub on the GitHub website.
  3. Download the assign_private_ip.py assign_private_ip.py script on the GitHub website.
    Note: If you use security configurations to launch your cluster with only IMDSv2 allowed, then download version 2 of the script assign_private_ip_v2.py on the GitHub website.
  4. Save the script in an Amazon Simple Storage Service (Amazon S3) bucket.
  5. Specify the script as a custom bootstrap action, and then use the script to launch your cluster. You can also run the script as an Amazon EMR step.
    Note: The script requires a private IP address from the CIDR range of your subnet and an AWS Region as an argument. The script attaches that private IP address to the network interface (eth0) of the primary node. The script also configures the network settings to redirect all traffic from the secondary IP address to the primary IP address. It uses the s3://BUCKET_NAME/assign_private_ip.py PRIVATE_IP us-east-1 format.
  6. To find the new IP address, open the Amazon Elastic Compute Cloud (Amazon EC2) console.
  7. Select the Amazon EC2 instance that's the primary node of the EMR cluster. You can find the new IP address on the Networking tab, in the Secondary private IPs field.

Related information

How do I determine whether to use a bootstrap action or a step on an Amazon EMR cluster?

AWS OFFICIAL
AWS OFFICIALUpdated 24 days ago
5 Comments

Just one correction in the above steps,

When you pass the Bootstrap Action script, it requires to two input arguments. If missing any one of below might cause index out of range exception in BA failure.

  1. Private IP address
  2. Region where the cluster gets deployed.

Example:

s3://<bucket-name>/assign_private_ip.py <your private IP> us-east-1
profile pictureAWS
SUPPORT ENGINEER
replied a year ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied a year ago

Thanks for the update

profile pictureAWS
SUPPORT ENGINEER
replied a year ago

The assign_private_ip.py is outdated - does not support EMR 7.0.0. on Amazon Linux 2023: IMDSv2 not implemented.

replied 9 months ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

profile pictureAWS
MODERATOR
replied 9 months ago