AWS Glue Job Not Releasing ENIs After Completion

Question

I have an AWS Glue resource in the us-east-2 region. The Glue job runs daily, and I have integrated my DocumentDB connection into it. Each time the job runs, it creates Elastic Network Interfaces (ENIs).

Currently, my Glue job has created around 1,200 ENIs in an available state. As a result, I encountered the error:
"You have reached the maximum ENI limit."

However, most of these ENIs remain in the available state, while only a few are in in-use status, which are being used by other AWS services.

I have two questions:

When the Glue job runs, it creates ENIs. But once the job is completed, shouldn't these ENIs be automatically deleted?
    I noticed that if I manually delete some ENIs in the available state, my Glue job runs successfully. Why aren’t the ENIs automatically deleted after the Glue job completes?

re:Post Agent · Answer

You've encountered an important issue with AWS Glue and Elastic Network Interfaces (ENIs). Let's address your questions:

1. Yes, typically when an AWS Glue job completes, the ENIs created for that job should be automatically released and deleted. This is the expected behavior under normal circumstances.

2. The fact that ENIs are accumulating in an available state after your Glue jobs complete indicates that there might be an issue with the cleanup process. This is not the normal behavior and suggests a potential problem that needs to be investigated.

There are a few possible reasons for this situation:

1. Job Configuration: Ensure that your Glue job is properly configured to release resources after completion. Check the job parameters and settings to make sure there's nothing preventing the proper cleanup.

2. VPC Configuration: If your Glue job is running in a VPC, make sure the VPC settings are correct and not interfering with the ENI release process.

3. AWS Glue Service Issue: There could be a service-side issue with AWS Glue that's preventing the automatic cleanup of ENIs. This is less likely but still possible.

4. Permissions: Verify that the IAM role associated with your Glue job has the necessary permissions to release ENIs.

To address this issue:

1. Contact AWS Support: Given the scale of the problem (1,200 accumulated ENIs), it's advisable to open a support ticket with AWS. They can investigate if there's a service-side issue or provide specific guidance for your account.

2. Monitor Job Status: Ensure your jobs are completing successfully. Failed or timed-out jobs might not properly release resources.

3. Implement a Cleanup Process: As a temporary workaround, you could implement an automated process to clean up unused ENIs periodically. However, be cautious and ensure you're only deleting ENIs that are truly not in use.

4. Review Job Configurations: Double-check your Glue job configurations, especially any custom scripts or code, to ensure they're not inadvertently preventing ENI release.

Remember, manually deleting available ENIs is a temporary solution. The root cause needs to be identified and addressed to prevent this issue from recurring and to ensure your Glue jobs operate efficiently without manual intervention.
      
**Sources**  
[Scale AWS Glue jobs by optimizing IP address consumption and expanding network capacity using a private NAT gateway | AWS Big Data Blog](https://aws.amazon.com/blogs/big-data/scale-aws-glue-jobs-by-optimizing-ip-address-consumption-and-expanding-network-capacity-using-a-private-nat-gateway/)  
[I got the error like "Elastic network interface limit is exceeded. Please increase your elastic network interface limit" | AWS re:Post](https://repost.aws/questions/QUaQzzenzKTACFZhUEj5nRPw/i-got-the-error-like-elastic-network-interface-limit-is-exceeded-please-increase-your-elastic-network-interface-limit)

AWS Glue Job Not Releasing ENIs After Completion

Add your answer

Relevant content