How to fetch Batch Parameters which get passed when a new file is uploaded into s3 using python

Question

Once a new file is uploaded onto s3 bucket an event is triggered which has the s3 file name and is passed onto AWS Batch into Batch parameters, I want to get the batch parameters value in my python code, so i can select that file which is uploaded and can use that file

Answer

Hello.

You can share the S3 path etc. from EventBridge to AWS Batch by following the steps in the document below.  
https://docs.aws.amazon.com/batch/latest/userguide/batch-cwe-target.html#cwe-input-transformer

On the AWS Batch side, you can share the S3 path etc. with Python by executing a command in the job definition.  
https://docs.aws.amazon.com/batch/latest/userguide/create-job-definition-Fargate.html  
https://docs.aws.amazon.com/batch/latest/userguide/job_definition_parameters.html

For example, suppose you set the following input transformer in EventBridge as shown below.  
https://repost.aws/knowledge-center/batch-target-eventbridge-rules  
```
{"S3BucketValue":"$.detail.requestParameters.bucketName","S3KeyValue":"$.detail.requestParameters.key"}
```
Set the parameters passed to the batch job as follows.  
```
{"Parameters" : {"S3bucket": , "S3key": }}
```
The AWS Batch job definition command passes S3 information to Python as follows.  
```
"command": ["python", "main.py", "Ref::S3bucket", "Ref::S3key"]
```
You should be able to check S3 information from the arguments by using the Python code as shown below.  
```
import sys

s3_bucket_name = sys.argv[1]
s3_key = sys.argv[2]
```

How to fetch Batch Parameters which get passed when a new file is uploaded into s3 using python

Relevant content