AWS S3 tar.gz compressed files issue after download from S3

0

Hello,

I would like to check with you guys here in AWS forum the issue that I discovered a while ago. Basically I'm trying to test our backup recovery process. Below is my simple flow of our backing up process.

Backup Flow
ec2 -> uploads backup files compressed using tar.gz to S3 -> S3(backup bucket).
NOTE: I used AWS CLI for moving files from ec2 to s3.

However when I tried to download the tar.gz file from S3, considering that on the AWS Console S3 file list it's has a correct file extension of tar.gz e.g. backup1.tar.gz BUT after the download, I've noticed that my backup file turns to backup1.tar (not tar.gz) so when I tried to decompress the backup its not usable or readable etc.
I'm not sure if it's on my OS level error but overall it will make a huge confusion to the user if the backup process is working fine or not.

質問済み 7年前3963ビュー
6回答
0

Why don't you use native AWS tool for creating snapshots? Why you make the process of backup your EC2 instance so complex?)

Moreover you can automate all this by using any backup vendors tools. For example, Cloudberry Backup Server Edition.

AlViKo
回答済み 7年前
0

I'm having the same problems...

I did aws s3 sync some/local/folder s3://my-special-bucket/some/local/folder
and some of the files that got uploaded were .tar.gz files. Well, AWS decided that those should receive the metadata Content-Type: applicaton/x-tar instead of Content-Type: application/x-gzip. Highly aggravating. I had to write a script to list every file in my bucket and then fix the metadata for any file with the .tar.gz extension.

回答済み 5年前
0

I have yet to see anyone explain how to download a tar.gz from an S3 bucket without AWS changing the format to a .tar and changing the config of the files. I need help to figure out how to down load a large (1.7 GB) tar.gz file.

DSimon
回答済み 4年前
0

Hello All,
I have the same issue, anyone has any solution or suggestion?

回答済み 4年前
0

Hi anjalitrambadia, DSimon,

We took a quick look over the sync issues and wanted to get some more information.

If you wouldn't mind creating an issue on the AWS-CLI's Github (https://github.com/aws/aws-cli/), that will help us better track this. Including the version of your aws cli (aws --version) and a brief set up steps for reproduction will be really useful.

Thanks,
AWS-CLI team

Edited by: nateprewitt on May 12, 2020 6:11 PM

回答済み 4年前
0

You should use --content-encoding "application/x-compressed-tar" option during S3 uploading:
aws s3 cp --content-encoding "application/x-compressed-tar".......

回答済み 3年前
profile picture
エキスパート
レビュー済み 16日前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ