Multi Download from Amazon S3 (Glacier Deep Archive)

Question

Hello Community,

I am having an issue with downloading multiple files from S3. My files (photos) were stored in Intelligent Storage and made their way all the way down to Glacier Deep Archive. I have now restored the files and I can download them individually, but since there are thousands of photographs I do not want to do this one by one.

I followed some steps to use the AWS CLI to download the photos:

```
 aws s3 cp --recursive s3:/// 
```
As well as just to copy it to a new S3 bucket

```
aws s3 cp --recursive  s3:///  s3:///
```
The error message I see for both of the above is

```
Object is of storage class GLACIER. Unable to perform copy operations on GLACIER objects. You must restore the object to be able to perform the operation. See aws s3 copy help for additional parameter options to ignore or force these transfers.
```
So although all of the files have been restored, they are still showing as GLACIER when I try to copy them using the CLI, but I can download them instantly one by one using the UI.

Please can you advise how I will be able to bulk download these files?

Thanks, 
Billy

Answer

To perform a recursive copy for an entire prefix of objects that have be restored from the Glacier storage class and overwrite existing objects with the Amazon S3 Standard storage class, you can try to running the following command:
> aws s3 cp s3://awsexamplebucket/dir1/ s3://awsexamplebucket/dir1/ --storage-class STANDARD --recursive --force-glacier-transfer

To perform a recursive copy for an entire prefix of objects that have been restored from the Glacier storage class to the Standard storage class in another bucket, you can try running the following command:
> aws s3 cp s3://awsexamplebucket1/dir1/ s3://awsexamplebucket2/dir1/ --storage-class STANDARD --recursive --force-glacier-transfer

For more information about how to download multiple objects that have been restored from GLACIER, please refer to the following link: https://aws.amazon.com/premiumsupport/knowledge-center/restore-s3-object-glacier-storage-class/

If the temporary copy of the object is no longer available, you will have to restore the objects once more before you will be able to successfully run the the above commands. You can refer to the following link for step-by-step instructions on how to restore thousands of objects:
https://aws.amazon.com/premiumsupport/knowledge-center/s3-batch-operation-initiate-restore/

Multi Download from Amazon S3 (Glacier Deep Archive)

Relevant content