S3 Versioning enabling after PutObject operation

0

I have created a lambda function that puts JSON files into a S3 bucket every 10 mins. I created the S3 bucket with versioning disabled

The files will normally be overwriting the existing files. I noticed that the S3 bucket was growing in size and found that multiple versions of the files were being stored. I suspended the versioning on the S3 bucket, but it seems to enable everytime the lambda function runs. I have tried changing the lambda function to delete the file before deleting it, but it ends up with either a "Delete marker" version or it doesn't get deleted.

function saveObject(key, data, options) {
    return new Promise((resolve, reject) => {
        const params = {
            ...s3BaseParams,
            Key: key
        };

        s3.deleteObject(params, (error) => {
            if (error && error.code !== 'NoSuchKey') {
                console.error(error);
                reject(error);
                return;
            }

            s3.putObject({
                ...params,
                ...(options || {}),
                Body: data
            }, (error, data) => {
                if (error) {
                    console.error(error);
                    reject(error);
                    return;
                }

                resolve();
            });
        });
    });
}

What am I missing so that the S3 bucket doesn't save different versions of the files?

I am using a serverless (cloud formation) resource to create the bucket (extract below)

  Resources:
    LakeTempsBucket:
      Type: 'AWS::S3::Bucket'
      Properties:
        BucketName: ${self:custom.env.BUCKET_NAME}
        VersioningConfiguration:
          Status: Suspended
        PublicAccessBlockConfiguration:
          BlockPublicAcls: true
          BlockPublicPolicy: true
2 Answers
0

Once you enable versioning on s3 bucket, you can't disable s3 versioning. It can only be suspended.

What does it mean by suspended versioning:

You can add objects to versioning-suspended buckets in Amazon S3 to create the object with a null version ID or overwrite any object version with a matching version ID. After you suspend versioning on a bucket, Amazon S3 automatically adds a null version ID to every subsequent object stored thereafter (using PUT, POST, or COPY) in that bucket.

Refer second document in the reference section for more details about, how adding objects work in a suspended versioning bucket.

Reference:

Working with objects in a versioning-suspended bucket

Adding objects to versioning-suspended buckets

Hope it helps.

Comment here if you have additional questions, happy to help.

Abhishek

profile pictureAWS
EXPERT
answered 7 months ago
  • I never knowingly enabled versioning, so I would have thought versioning would have been disabled by default and should remain disabled/suspended if you don't enable it or you suspend it.

0

Hello.

Even if a past version overwrites a remaining object, the past version will not be deleted.
Therefore, you need to completely delete the deletion marker and past versions before uploading.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/AddingObjectstoVersionSuspendedBuckets.html

Another option to reduce the storage capacity of a bucket with versioning enabled is to set a lifecycle rule to delete old versions.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.html

profile picture
EXPERT
answered 7 months ago
profile pictureAWS
EXPERT
reviewed 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions