Hello everyone,
we are facing a major issue regarding AWS Glue's workflows. For some reason starting from today 01/02/2024, we have noticed that some ANY and ALL triggers will activate and trigger their assigned job multiple times, forcing runs to fail with Max concurrent runs exceeded. Moreover, some ANY triggers, seem to completely skip their intended target and trigger the last job of the Workflow, which by no means is target for said trigger.
This issue affects more than one workflows as well as more than one triggers. Please note that we have not performed any actions the past few days on those specific triggers or workflows.
Has anyone else faced a similar issue recently, do you have any idea what is going wrong? Is there any chance that Workflows are not supported by AWS? (I read something among these lines in StackOverflow)
AWS OFFICIALUpdated 2 years ago
We are facing the same problem. Triggers seem to start glue jobs multiple times. Any news from the AWS support?
If this is a sudden change in behavior as it sounds, please open a support ticket or provide the job worflow run id here so it can be investigated