Compression while upload in S3

Question

Hello,

Does S3 has this feature: when one needs to upload from Management Console a huge file in S3 (5GB), the user to upload it compressed (500MB) and S3 to automatically decompress and list the 5GB file (the upload would be much faster for highly compressible files: txt, json, csv, ....) ?   
Or is there a command to manually decompress a file in S3 ?   
Does such commands exist in CLI ?

Thank you,  
Mihai

Accepted Answer

Hi,

Amazon S3 doesn't have native features to compress/decompress data. But, Amazon S3 Object Lambda will enable you to use AWS Lambda functions to process data as it is being retrieved from S3. This includes the ability to compress, filter, mask, redact, and modify the format of the data. Please check the blog below for better understanding:

https://aws.amazon.com/blogs/aws/introducing-amazon-s3-object-lambda-use-your-code-to-process-data-as-it-is-being-retrieved-from-s3/

With respect to the CLI option, there was a question on that previously, pls have a look below:

https://repost.aws/questions/QUkg2RAWBES2Kvhi-da6P9vg/is-there-a-mechanism-for-automatically-compressing-objects-loaded-to-s3

Thanks

Compression while upload in S3

Relevanter Inhalt