S3 Versioning enabling after PutObject operation

0

I have created a lambda function that puts JSON files into a S3 bucket every 10 mins. I created the S3 bucket with versioning disabled

The files will normally be overwriting the existing files. I noticed that the S3 bucket was growing in size and found that multiple versions of the files were being stored. I suspended the versioning on the S3 bucket, but it seems to enable everytime the lambda function runs. I have tried changing the lambda function to delete the file before deleting it, but it ends up with either a "Delete marker" version or it doesn't get deleted.

function saveObject(key, data, options) {
    return new Promise((resolve, reject) => {
        const params = {
            ...s3BaseParams,
            Key: key
        };

        s3.deleteObject(params, (error) => {
            if (error && error.code !== 'NoSuchKey') {
                console.error(error);
                reject(error);
                return;
            }

            s3.putObject({
                ...params,
                ...(options || {}),
                Body: data
            }, (error, data) => {
                if (error) {
                    console.error(error);
                    reject(error);
                    return;
                }

                resolve();
            });
        });
    });
}

What am I missing so that the S3 bucket doesn't save different versions of the files?

I am using a serverless (cloud formation) resource to create the bucket (extract below)

  Resources:
    LakeTempsBucket:
      Type: 'AWS::S3::Bucket'
      Properties:
        BucketName: ${self:custom.env.BUCKET_NAME}
        VersioningConfiguration:
          Status: Suspended
        PublicAccessBlockConfiguration:
          BlockPublicAcls: true
          BlockPublicPolicy: true
2 Risposte
0

Once you enable versioning on s3 bucket, you can't disable s3 versioning. It can only be suspended.

What does it mean by suspended versioning:

You can add objects to versioning-suspended buckets in Amazon S3 to create the object with a null version ID or overwrite any object version with a matching version ID. After you suspend versioning on a bucket, Amazon S3 automatically adds a null version ID to every subsequent object stored thereafter (using PUT, POST, or COPY) in that bucket.

Refer second document in the reference section for more details about, how adding objects work in a suspended versioning bucket.

Reference:

Working with objects in a versioning-suspended bucket

Adding objects to versioning-suspended buckets

Hope it helps.

Comment here if you have additional questions, happy to help.

Abhishek

profile pictureAWS
ESPERTO
con risposta 8 mesi fa
  • I never knowingly enabled versioning, so I would have thought versioning would have been disabled by default and should remain disabled/suspended if you don't enable it or you suspend it.

0

Hello.

Even if a past version overwrites a remaining object, the past version will not be deleted.
Therefore, you need to completely delete the deletion marker and past versions before uploading.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/AddingObjectstoVersionSuspendedBuckets.html

Another option to reduce the storage capacity of a bucket with versioning enabled is to set a lifecycle rule to delete old versions.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/how-to-set-lifecycle-configuration-intro.html

profile picture
ESPERTO
con risposta 8 mesi fa
profile pictureAWS
ESPERTO
verificato 8 mesi fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande