- Newest
- Most votes
- Most comments
Yes, Amazon EMR does have timeout mechanisms for job execution, but they vary depending on the specific EMR service and configuration you're using.
For EMR Serverless, each job run has a set timeout duration. By default, this timeout is 12 hours. However, you can configure this timeout setting to a value that meets your specific job requirements using the executionTimeoutMinutes property when starting your job run.
For EMR clusters running on EC2, there isn't a built-in execution timeout for individual steps. However, you can implement an auto-termination policy for the entire cluster. This policy allows you to specify an idle timeout, which determines how long the cluster can remain idle before it automatically terminates. The idle timeout can be set between a minimum of 60 seconds (1 minute) and a maximum of 604,800 seconds (7 days).
It's worth noting that for EMR on EKS (Elastic Kubernetes Service), there is a default "cluster timeout" of 15 minutes for jobs in the SUBMITTED state. If a job cannot start and remains in the SUBMITTED state for 15 minutes, it will time out. This timeout can be overridden using configuration settings.
In summary, while there isn't a universal timeout for all EMR executions, there are various timeout mechanisms available depending on the specific EMR service and configuration you're using. You can configure these timeouts to suit your job requirements.
Sources
Running jobs from the AWS CLI - Amazon EMR
Common errors when running jobs - Amazon EMR
Using an auto-termination policy for Amazon EMR cluster cleanup - Amazon EMR
Relevant content
asked 2 years ago
asked 3 years ago
asked 4 years ago
- AWS OFFICIALUpdated 2 years ago
