how to trigger a step function from a s3 object notification?
Can one directly invoke a step function directly from a s3 object notification ? for example , when object is created. also, does it work similar to lambda , when hooked up to the s3 object notification. for each object created , will it start a new instance of step function or can we make it such that all of the object notification goes to the same instance of step function, and as i understand step function are state machines , so say i have two states enabled and disabled. when object is created, state machine/step function will move to enable state , and for any other s3 object created notification that comes after the first one , as the step function is already in a enabled state , it won't do anything unless it is in a disabled state. once the files are processed it will move to disabled state.
To invoke a Step Function when an object is posted to S3, you'll need to create a Lambda function that starts an execution of the state machine. Each execution will be separate and independent for each uploaded object.
There isn't a concept of "enabled" and "disabled" for state machine executions - they are either running and in some state, or terminated.
@Michael_F - thanks. follow up question - if we need a lambda inbetween, it should be possible to check if state machine is already running/or in some state, right? instead of creating another instance of the state machine. and only start/run the same instance , if it is in terminated state.
There are a few ways to go about that. The naive way would be to probe all the running executions, but the DescribeExecution API method is eventually consistent, and that could lead to errors. Instead, I would recommend keeping a list of object processing states in DynamoDB (using the S3 object ID as the primary key).
There is no direct integration between S3 and StepFunctions. You can send the S3 notification to a Lambda function, as suggest by @Michael_F, or a no code solution, by sending the event to EventBridge and setting a rule with a StepFunction target.
Saying that, I am not sure that StepFunctions is your right solution. It seems that you want to run some process on multiple files. If that is the case, you will not be able to "attach" into a running state machine. There may be some options using the Wait For Task Token integration pattern, but I am not sure it will solve what you are trying to do.
I think it would be best if you could explain your use case better.
Conventions for Drawing Arrows in Architectural DiagramsAccepted AnswerEXPERTasked 2 years ago
Can I specify GET URL path parameter in step function?asked 3 months ago
Intermittend InvalidSignature error when retrieving object from S3 Object Lambda accesspoint in lambda@edge functionasked 2 months ago
AWS Lambda Function Triggers on S3 Event, But only Once in About 30 Minsasked 2 years ago
S3 object and lambda function in step machine with the asynchronous express workflowsAccepted Answerasked 4 months ago
how to trigger a step function from a s3 object notification?asked a month ago
Coordinating Step functions from App Flow -> Event Bridge -> DynamoDBasked 4 months ago
S3: Configure object lock with replication (CRR) on aws s3 bucketsasked 24 days ago
Step Function to Send Email on Error/Successasked 4 months ago
Return Value from Lambda function triggered by SQS to individual clientAccepted Answerasked 4 months ago