How do I copy my Amazon EBS snapshot data to my Amazon S3 bucket?

Short description

You can't copy Amazon EBS snapshots directly to an Amazon S3 bucket that you manage. To copy the snapshot data to your S3 bucket, create a volume from the snapshot. Then, attach the volume to an Amazon Elastic Compute Cloud (Amazon EC2) Linux instance. Finally, use the AWS Command Line Interface (AWS CLI) to transfer the data.

To store snapshots that you infrequently access, see Archive Amazon EBS snapshots.

Resolution

Note: If you receive errors when you run AWS CLI commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

Important: It's a best practice to protect your data with encryption. Make sure that your S3 bucket has the correct access controls.

To copy the contents of your Amazon EBS snapshot to an S3 bucket, complete the following steps:

1. Create an Amazon EBS volume from the snapshot.

2. Launch an Amazon EC2 Linux instance in the same Availability Zone as your volume.

3. Attach the volume to the instance as a secondary volume.

4. Connect to your Linux instance using SSH or Session Manager, a capability of AWS Systems Manager.

5. Grant your EC2 instance access to your S3 bucket.

6. To view your available disk devices, run the following command:

   lsblk

   Example output:

   NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
   xvda    202:0    0   15G  0 disk
   └─xvda1 202:1    0   15G  0 part /
   xvdf    202:0    0   15G  0 disk
   └─xvdf1 202:1    0   15G  0 part

   Note: Nitro-based instances show Amazon EBS volumes as NVMe block devices with the nvme[0-26]n1 disk name.
   Example output on a Nitro-based instance:

   NAME          MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
   nvme0n1       259:0    0    8G  0 disk
   └─nvme0n1p1   259:1    0    8G  0 part /
   └─nvme0n1p128 259:2    0    1M  0 part
   nvme1n1       259:3    0  100G  0 disk
   └─nvme1n1p1   259:4    0  100G  0 part

7. To mount the volume to your instance, run the following command:

   sudo mount /dev/xvdf1 /mnt

   Note: Replace /dev/xvdf1 with the partition name of your volume.

8. Turn on the Extra Packages for Enterprise Linux (EPEL) repository, and then install the pv package to monitor progress during tar archive creation. To install pv, run the following command:

   sudo $(command -v dnf || command -v yum || command -v apt) install pv

9. To copy the Amazon EBS volume data to your S3 bucket, run the following command:

   SIZE=$(( $(sudo du -sk /mnt | awk '{print $1}') * 1024 ))
   
   sudo tar c /mnt \
       | pv -s "$SIZE" \
       | gzip \
       | tee >(md5sum > backup1.md5) \
       | aws s3 cp - "s3://MY-BUCKET/backup1.tar.gz" \
           --expected-size "$SIZE" \
           --storage-class STANDARD

   Note: Replace MY-BUCKET with the name of your S3 bucket and backup1 with the name of your file. The command creates a compressed archive from the /mnt directory and uploads the archive to your S3 bucket. The command also generates an MD5 checksum for data integrity verification and uses the AWS CLI multipart upload for large files.

10. (Optional) To verify the checksum, run the following command:

    aws s3 cp s3://MY-BUCKET/backup1.tar.gz - | md5sum -c backup1.md5

    Note: Replace MY-BUCKET with the name of your S3 bucket and backup1 with the name of your file.

11. Open the Amazon S3 console to confirm that the compressed file uploaded to your S3 bucket.

12. To unmount the volume, run the following command:

    sudo umount /mnt

13. Detach the Amazon EBS volume from the Linux instance.

14. Delete the volume, and then terminate your instance.