By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

"Resource Unavailable" in AWS Glue

0

I have around 80 AWS Glue jobs concurrently at peak in a given time window ( have already raised the default limit of 50 concurrent jobs to 150 ) but came to a scenario where the AWS Glue took 36 mins to respond back with "Resource Unavailable" error. These jobs run standard worker types and max capacity between 2-5 and the account limit on number of DPU(s) has already been bumped to 600.

Does any one have idea around the possible reasons for it and is it normally recommended to handle this error and resubmit the AWS Glue job ?

Topics
Analytics
Tags
AWS Glue
Language
English
AWS
EXPERT
Dipankar_G
asked 4 years ago685 views
1 Answer
0
Accepted Answer

For any AWS Glue jobs, connections and development endpoints that you use, check that your cluster has not run out of elastic network interfaces.

As per the online docs this could be due to not enough ENIs available. I have experienced this as well. Increasing number of automated retries in an AWS Glue job can also automatically take care of this error.

AWS
Vivek_S
answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content