AWs trigger EventBatchingCondition/BatchWindow is not optional
I have a glue workflow :
trigger (type = "EVENT") => trigger a glue job (to take data from S3 and push them to MySQL RDS)
I configured the glue Triggering criteria to kickoff the glue job after 5 events were received.
in the console it says :
Specify the number of events received or maximum elapsed time before firing this trigger.
Time delay in seconds (optional)
on AWS documentation it says also it's not required :
BatchWindow Window of time in seconds after which EventBridge event trigger fires. Window starts when first event is received. Type: Integer Valid Range: Minimum value of 1. Maximum value of 900. Required: No
So I want only my trigger to be triggered only and only after 5 events are received and not depending on: Time delay in seconds (optional).
actually, the Time delay in seconds (optional) is set to 900 by default and my job is started after 900s even if there are no 5 events received.
that's not the behaviour we want. We want ONLY the job to be started after x events are received. I tried via the console to edit the trigger and remove the 900s for the Time delay in seconds (optional) input but I can't save it until I put a value on it.
it says it's optional but it doesn't seem to be.
is there a workaround to make the trigger not take account of Time delay in seconds (optional)? and only be launched when it received x events and nothing else.
right now the behaviour I have is that my job is triggered after 900s, we want to eliminate this case and let the job be triggered only and only if there is x event received and nothing else.
how can I make the Time delay in seconds (optional) input optional, because now the console forces me to put a value in there?
Firstly as mentioned in 1, BatchSize is required and BatchWindow is optional. If BatchWindow is omitted, the window defaults to 900 seconds, which is the maximum window size.
Secondly, as mentioned on glue console under Time delay in seconds “The time delay starts when the first event is received”.
Therefore, Glue trigger expects all the events to happen with in a maximum of 900 seconds from the start of first event.
Thank you and have a nice day!
Glue WorkFlow - Max concurrent runs exceededasked 5 months ago
trigger glue job from s3Accepted Answer
Will AWS Glue Blueprints support trigger types other than a schedule?Accepted Answerasked 3 months ago
Call a glue Job from within another without using vpc endppoints or SGasked 3 months ago
best practice to move ETL reated files
CDK AWS trigger with type EVENT
Glue Jobs & Multiple tablesAccepted Answerasked 4 years ago
adding EventBatchingCondition to CDK trigger
AWs trigger EventBatchingCondition/BatchWindow is not optionalasked 23 days ago
How to create a Glue Workflow programmatically?Accepted AnswerEXPERTasked 2 years ago