Passing data between two ecs tasks using step functions

Question

There are two fargate tasks that runs on scheduled time . I want to pass the output of one ecs task as an input to another using step functions. But I am getting stucked here . The problem I am facing is with the key used in both the tasks which need to be same as the key used in state machine defination in ResultPath / OutputPath.
Lets say I am passing a list in first task output like 
return ( "news_items": news_items }
The ResultPath I am using is $.news_items
But in the second task it is not receiving the correct input 
Please help me with this issue

Ahmed · Answer

I believe you need first to check the size of the data that needs to be sent. If the data is small (I believe the threshold is 256K), you can send the data within the step function. However, if the data is more than that, you will need to use middleware (as suggested by the answer above). In other words, in the latter case, you need to make the first task write the data as output to s3 and the next task reads the data as input from the same s3.

myronix88 · Answer

I think you can use S3 as a middleware: https://nuvalence.io/insights/aws-step-function-integration-with-ecs-or-fargate-tasks-data-in-and-out/

Passing data between two ecs tasks using step functions

Relevant content