By using AWS re:Post, you agree to the Terms of Use
/how to trigger a step function from a s3 object notification?/

how to trigger a step function from a s3 object notification?

0

Can one directly invoke a step function directly from a s3 object notification ? for example , when object is created. also, does it work similar to lambda , when hooked up to the s3 object notification. for each object created , will it start a new instance of step function or can we make it such that all of the object notification goes to the same instance of step function, and as i understand step function are state machines , so say i have two states enabled and disabled. when object is created, state machine/step function will move to enable state , and for any other s3 object created notification that comes after the first one , as the step function is already in a enabled state , it won't do anything unless it is in a disabled state. once the files are processed it will move to disabled state.

2 Answers
3

To invoke a Step Function when an object is posted to S3, you'll need to create a Lambda function that starts an execution of the state machine. Each execution will be separate and independent for each uploaded object.

There isn't a concept of "enabled" and "disabled" for state machine executions - they are either running and in some state, or terminated.

You can find an experimental CDK solution construct that creates an S3 bucket connected to a Step Functions state machine here.

EXPERT
answered a month ago
  • @Michael_F - thanks. follow up question - if we need a lambda inbetween, it should be possible to check if state machine is already running/or in some state, right? instead of creating another instance of the state machine. and only start/run the same instance , if it is in terminated state.

  • There are a few ways to go about that. The naive way would be to probe all the running executions, but the DescribeExecution API method is eventually consistent, and that could lead to errors. Instead, I would recommend keeping a list of object processing states in DynamoDB (using the S3 object ID as the primary key).

0

There is no direct integration between S3 and StepFunctions. You can send the S3 notification to a Lambda function, as suggest by @Michael_F, or a no code solution, by sending the event to EventBridge and setting a rule with a StepFunction target.

Saying that, I am not sure that StepFunctions is your right solution. It seems that you want to run some process on multiple files. If that is the case, you will not be able to "attach" into a running state machine. There may be some options using the Wait For Task Token integration pattern, but I am not sure it will solve what you are trying to do.

I think it would be best if you could explain your use case better.

EXPERT
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions