StepFunctions/Lambda and ECS

Hello All

Here is what I am attempting to do using StepFunctions/Lambda/ECS+Fargate+python+docker

I have a job that is queuing a request and polling to see if the request is complete. Once complete, the file for the request needs to be downloaded. All the above steps, except downloading file is done using Lambda and Choices in Step functions. For the actual download I am using a container as the download can take over 15 mins.

Question: What I need to do is send the Job_ID from The final "Is File Ready?" Choice to the Fargate/ECS Container so that the python script in the container downloads the file and puts it in S3. The final choice, is basically checking if the file status is Ready or Pending. If Ready, the file is ready to download, else keep looping after 10 mins sleep and check the status again.

Is this the correct thought process: Somehow in the "Is File Ready?" state the output should include the json that the ECSTaskRun can consume? and this json output should have the Job_id as an evn parameter? Once this is done, the python script can read the job_id as env param, os.environ.get('job_id), and do the remaining items?

Or, am I on the wrong track here.

Also I am deploying everything in Python CDK.

Thanks for the feedback.