Video Sync Issue with S3 API - Updated Videos Not Consistently Appearing in List

Question

Hello Guys,

We are facing a recurring issue with video synchronization in our application that uses S3. The issue occurs frequently, and we would appreciate your assistance in resolving it.

When we update videos in our S3 bucket, we can see the videos in the bucket, but when we call the API to get the list of videos, some updated videos are not shown. This issue persists for 2-3 days, after which the updated video list appears correctly when we call the API. This happens at a ratio where 3 out of 5 times, we can successfully retrieve the correct list, while 2 out of 5 times, the list does not include the newly uploaded videos.

We are wondering if this could be related to any caching mechanisms being used by AWS. Could you please advise on how to resolve this issue so that the video list consistently reflects the latest updates? For your reference, I am also sharing the S3 API we are using.

package name= package : '@aws-sdk/client-s3'
method using to retrieve list from S3 = 
import {  ListObjectsV2Command } from '@aws-sdk/client-s3';

const params = {
  Bucket: "Videos",
  MaxKeys: 1000,
  Prefix: prefix,
};
const command = new ListObjectsV2Command(params);

await this.s3Client.send(command);

We appreciate your help in addressing this matter.

Ujjawal_S · Answer

S3 delivers strong [read-after-write](https://aws.amazon.com/blogs/aws/amazon-s3-update-strong-read-after-write-consistency/) consistency automatically for all applications, without changes to performance or availability. Amazon S3 achieves high availability by replicating data across multiple servers within AWS data centers. If a PUT request is successful, your data is safely stored. Any read (GET or LIST request) that is initiated following the receipt of a successful PUT response will return the data written by the PUT request.

So, it would be very highly unlikely that s3 will show this kind of behaviour. The API seems fine to me, in order to troubleshoot the issue further you can enable [debugging]( https://docs.aws.amazon.com/AmazonS3/latest/userguide/get-request-ids.html#sdk-request-ids) and share the request ID with AWS Support.

- Also share the screenshot and timestamp as when you uploaded/write the object and when you are performing the List Request.

Video Sync Issue with S3 API - Updated Videos Not Consistently Appearing in List

답변 추가

관련 콘텐츠