Can there be duplicate message in standard sqs triggered via S3 notification event for the same file uploaded multiple times.

Yes, it is possible for duplicate messages to be published to an SQS Standard Queue triggered via S3 event notification for the same file upload. Here's a detailed breakdown of the different scenarios you asked about:

1. S3 Bucket with Versioning Disabled:

Duplicate Messages for the Same File (Same Name, Same or Different Content): Yes, if you upload a file with the same name (test.txt) multiple times, each time the S3 event notification will trigger a new message to the SQS Standard Queue, even if the content is identical. This is because, in S3 with versioning disabled, each upload of the file overwrites the previous one. The SQS Standard Queue does not guarantee exactly-once message delivery; it guarantees at least once delivery. Therefore, it is possible for the same event (file upload) to result in duplicate messages in the queue.

ETag and Sequencer Values: The ETag for the file will remain the same if the file content is exactly the same between uploads (because it’s essentially a hash of the content).

The Sequencer is a unique identifier generated by S3 for each event. It ensures that each event is unique. Even if the file name and content are identical, the sequencer will be different for each upload event.

If the File Has Different Content:

The ETag will change because the ETag is tied to the content.

The Sequencer will also be different for each upload event. : 2. S3 Bucket with Versioning Enabled

Duplicate Messages for the Same File (Same Name, Same or Different Content):

With versioning enabled, each upload of the file test.txt creates a new version of the file. Even if you upload a file with the same name, S3 will generate a new version ID, and each upload will trigger a new event notification to the SQS Standard Queue. Hence, multiple messages will be sent to the queue for each upload, and each will reference a different version of the file.

Again, as SQS Standard Queue guarantees at least once delivery, duplicate messages for the same event are possible.

ETag and Sequencer Values:

The ETag will be the same only if the file content is exactly the same across uploads (because the ETag is content-based). The Sequencer will always be different because each upload generates a unique event, even when the file content is identical.

Versioning Impact:

If versioning is enabled, the version ID of the file will differ for each upload (even if the content is identical). The S3 event notification sent to SQS will include the version ID, making it clear which version of the file the message refers to.

If different content is uploaded, the ETag, Sequencer, and Version ID will all be different.

Summary:

Duplicate Messages:

Regardless of whether versioning is enabled or disabled, duplicate messages may appear in an SQS Standard Queue due to the "at least once" delivery guarantee. ETag:

Remains the same for the same file content but changes if the content changes.

Sequencer:

Always unique for each event, even for identical content. With Versioning Enabled:

Each file upload creates a new version, and S3 sends a separate notification for each version, even if the file name is the same.

Yes, it is possible to receive duplicate messages in an Amazon SQS standard queue triggered via S3 event notification for the same file uploaded multiple times, even with the same name and content. This is due to the at-least-once delivery guarantee of SQS Standard queues.

Duplicate Messages: You can receive duplicate messages if the same file (same or different content) is uploaded multiple times. SQS Standard queues allow duplicates.

ETag and Sequencer: ETag may remain the same for the same content file, but the sequencer will always be unique for each S3 event. So, for multiple uploads of the same file, the sequencer will differ, even if the ETag matches.

Versioning Enabled: If versioning is enabled on the S3 bucket, each upload creates a new version of the file, and the ETag and sequencer will both differ for each version.