- Mais recentes
- Mais votos
- Mais comentários
The reason why your experiencing this delay is because Glue does not support cloudwatch event for notifying the step functions with the latest status. Same is the case with EMR as well. Currently, by default the polling schedule is every 1 minute for the first 10 minutes, then every 5 minutes thereafter. Therefore, if the job is taking more than 10 minutes to complete it's execution the you can expect a delay of an average of 2.5 minutes with 5 minutes being the worst case. The only way is we poll its status by making Describe* api call to EMR/Glue up to every 5 minutes. The step function team knows about this issue and are trying to implement a solution.
The workaround that you can implement on your end is to make use of a Lambda function to make Describe API calls to describe EMR/Glue job status more often than Step Functions does.
If you require in depth assistance about this issue then I would advice you to raise a support case with the Technical Support team of Step Functions.
Conteúdo relevante
- AWS OFICIALAtualizada há 2 anos
- AWS OFICIALAtualizada há 7 meses
Thank you for responding. We'll implement a workaround for now.