How to wait to trigger Lambda function until specific files are uploaded

0

I'm new to AWS and working on Lambda function triggered by S3. I have two json files and need to use the both data to send queue to SQS. For example, two files, "file_a.json" and "file_b.json" are to be uploaded to S3 at the same time.

file_a.json
{"data": "123"}

file_b.json
{"data": "456"}

Then, Lambda will send queue message like ["123", "456"] which is created with the data from both files.

However, S3 invokes Lambda function every time the single file is uploaded and this makes it hard to send queue to SQS as I wrote above. I'm thinking if S3 can wait until the two files are uploaded to trigger Lambda, it can solve the issue. If anyone had faced the similar situation, I would like to know how to resolve this problem. Thanks!

2回答
3
承認された回答

You will need to put this logic inside your Lambda function. Something like this:

If (my companion file exists in the bucket) then
    if (my file datetime is newer that my companion file) then
        parse files and add data to queue
    end
end
profile pictureAWS
エキスパート
kentrad
回答済み 2年前
profile picture
エキスパート
レビュー済み 9ヶ月前
profile pictureAWS
エキスパート
レビュー済み 2年前
0

Hi, short Answer. That is not possible. The S3 Lambda Trigger sends an Event for every Object.

You can send the message to SQS and than trigger the Lambda function via the SQS Trigger. S3 -> SQS -> Lambda -> SQS The SQS Trigger offers a configurable Batch size and Batch window. That can work if you can guarantee that only these two files trigger the S3 Events in the defined Batch window.

AWS
Marco
回答済み 2年前
profile picture
エキスパート
レビュー済み 9ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ