How to wait to trigger Lambda function until specific files are uploaded

Question

I'm new to AWS and working on Lambda function triggered by S3. I have two json files and need to use the both data to send queue to SQS. For example, two files, "file_a.json" and "file_b.json" are to be uploaded to S3 at the same time.

```
file_a.json
{"data": "123"}

file_b.json
{"data": "456"}
```

Then, Lambda will send queue message like ["123", "456"] which is created with the data from both files.

However, S3 invokes Lambda function every time the single file is uploaded and this makes it hard to send queue to SQS as I wrote above. I'm thinking if S3 can wait until the two files are uploaded to trigger Lambda, it can solve the issue. If anyone had faced the similar situation, I would like to know how to resolve this problem. Thanks!

Accepted Answer

You will need to put this logic inside your Lambda function. Something like this:

```
If (my companion file exists in the bucket) then
    if (my file datetime is newer that my companion file) then
        parse files and add data to queue
    end
end
```

Answer

Hi, short Answer. That is not possible. The S3 Lambda Trigger sends an Event for **every** Object.

You can send the message to SQS and than trigger the Lambda function via the SQS Trigger. **S3 -> SQS -> Lambda -> SQS**
The SQS Trigger offers a configurable Batch size and Batch window. That can work if you can guarantee that only these two files trigger the S3 Events in the defined Batch window.

How to wait to trigger Lambda function until specific files are uploaded

Contenuto pertinente