Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

How do I assign a static private IP address to the primary node of my Amazon EMR cluster?

2 minute read
0

I want to create a static private IP address and associate the IP address with the primary node of an Amazon EMR cluster.

Resolution

To create a static private IP address and associate the IP address with your cluster's primary node, use a bootstrap action.

Complete the following steps:

  1. Confirm that your AWS Identity and Access Management (IAM) policy allows EMR_EC2_DefaultRole and ec2:AssignPrivateIpAddresses permissions.
  2. Sign in to GitHub on the GitHub website.
  3. Download the assign_private_ip.py assign_private_ip.py script on the GitHub website.
    Note: If you use security configurations to launch your cluster with only IMDSv2 allowed, then download version 2 of the script assign_private_ip_v2.py on the GitHub website.
  4. Save the script in an Amazon Simple Storage Service (Amazon S3) bucket.
  5. Specify the script as a custom bootstrap action, and then use the script to launch your cluster. You can also run the script as an Amazon EMR step.
    Note: The script requires a private IP address from the CIDR range of your subnet and an AWS Region as an argument. The script attaches that private IP address to the network interface (eth0) of the primary node. The script also configures the network settings to redirect all traffic from the secondary IP address to the primary IP address. It uses the s3://BUCKET_NAME/assign_private_ip.py PRIVATE_IP us-east-1 format.
  6. To find the new IP address, open the Amazon Elastic Compute Cloud (Amazon EC2) console.
  7. Select the Amazon EC2 instance that's the primary node of the EMR cluster. You can find the new IP address on the Networking tab, in the Secondary private IPs field.

Related information

How do I determine whether to use a bootstrap action or a step on an Amazon EMR cluster?

Topics
Analytics
Tags
Amazon EMR
Language
English
AWS OFFICIALUpdated a year ago
5 Comments

Just one correction in the above steps,

When you pass the Bootstrap Action script, it requires to two input arguments. If missing any one of below might cause index out of range exception in BA failure.

  1. Private IP address
  2. Region where the cluster gets deployed.

Example:

s3://<bucket-name>/assign_private_ip.py <your private IP> us-east-1
AWS
SUPPORT ENGINEER
replied 2 years ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

AWS
MODERATOR
replied 2 years ago

Thanks for the update

AWS
SUPPORT ENGINEER
replied 2 years ago

The assign_private_ip.py is outdated - does not support EMR 7.0.0. on Amazon Linux 2023: IMDSv2 not implemented.

replied 2 years ago

Thank you for your comment. We'll review and update the Knowledge Center article as needed.

AWS
MODERATOR
replied 2 years ago

Relevant content