AWs trigger EventBatchingCondition/BatchWindow is not optional
Hi team,
I have a glue workflow :
trigger (type = "EVENT") => trigger a glue job (to take data from S3 and push them to MySQL RDS)
I configured the glue Triggering criteria to kickoff the glue job after 5 events were received.
in the console it says :
Specify the number of events received or maximum elapsed time before firing this trigger.
Time delay in seconds (optional)
on AWS documentation it says also it's not required :
BatchWindow
Window of time in seconds after which EventBridge event trigger fires. Window starts when first event is received.
Type: Integer
Valid Range: Minimum value of 1. Maximum value of 900.
Required: No
So I want only my trigger to be triggered only and only after 5 events are received and not depending on: Time delay in seconds (optional).
actually, the Time delay in seconds (optional) is set to 900 by default and my job is started after 900s even if there are no 5 events received.
that's not the behaviour we want. We want ONLY the job to be started after x events are received. I tried via the console to edit the trigger and remove the 900s for the Time delay in seconds (optional) input but I can't save it until I put a value on it.
it says it's optional but it doesn't seem to be.
is there a workaround to make the trigger not take account of Time delay in seconds (optional)? and only be launched when it received x events and nothing else.
right now the behaviour I have is that my job is triggered after 900s, we want to eliminate this case and let the job be triggered only and only if there is x event received and nothing else.
how can I make the Time delay in seconds (optional) input optional, because now the console forces me to put a value in there?
thank you.
Hello,
Firstly as mentioned in 1, BatchSize is required and BatchWindow is optional. If BatchWindow is omitted, the window defaults to 900 seconds, which is the maximum window size.
Secondly, as mentioned on glue console under Time delay in seconds “The time delay starts when the first event is received”.
Therefore, Glue trigger expects all the events to happen with in a maximum of 900 seconds from the start of first event.
————————
Reference:
1 https://docs.aws.amazon.com/glue/latest/dg/starting-workflow-eventbridge.html
================
Thank you and have a nice day!
Relevant questions
Glue WorkFlow - Max concurrent runs exceeded
asked 5 months agotrigger glue job from s3
Accepted Answerasked a month agoWill AWS Glue Blueprints support trigger types other than a schedule?
Accepted Answerasked 3 months agoCall a glue Job from within another without using vpc endppoints or SG
asked 3 months agobest practice to move ETL reated files
asked a month agoCDK AWS trigger with type EVENT
asked a month agoGlue Jobs & Multiple tables
Accepted Answerasked 4 years agoadding EventBatchingCondition to CDK trigger
asked a month agoAWs trigger EventBatchingCondition/BatchWindow is not optional
asked 23 days agoHow to create a Glue Workflow programmatically?
Accepted Answer