Sync from S3 to local pricing comparison

0

Hi I am wondering which of the following two ways to sync files from a S3 bucket to local is cheaper (assume 10GB of new data per day and 1TB of data constantly in the bucket):

  1. perform aws s3 sync s3://bucketname /local/path
  2. use a S3 file system on a t2.micro bastion EC2 instance in the same region. Do rsync from the bastion server and transfer files from S3 file system to local

I have difficulty estimating the pricing for S3 solutions. If there is an even better approach than the above to sync from S3 to local, I am happy to know as well. I appreciate any suggestions from you!

asked 2 years ago989 views
2 Answers
0

There is no cost to transfer data from a bucket it the same region as your EC2 instance, regardless of whether it is a bastion or not. If you are retrieving data from a bucket in another region, you will pay the inter-region data transfer costs.

See: https://aws.amazon.com/s3/pricing/

profile pictureAWS
answered 2 years ago
0

First Approach:

With AWS S3 sync, You can split the transfer into multiple mutually exclusive operations to improve the transfer time by multi-threading. you can run multiple, parallel instances of aws s3 sync using the AWS CLI. You can create more upload threads while using the --exclude and --include parameters for each instance of the AWS CLI. To know more, refer link {2}

Note: The --exclude and --including parameters are processed on the client-side. Therefore, note that resources on your local machine might affect the performance of the ope

**Pricing **

aws s3 sync transfer data OUT from Amazon S3 to the internet in order to retrieve it on your local machine. you will be billed for all the data transfer bandwidth.

Check out the Data Transfer tab on S3 pricing page. Refer link {3}

Second Approach: (GUI based approach)

AWS Transfer for SFTP, a fully-managed, highly-available SFTP service. You simply create a server, set up user accounts, and associate the server with one or more Amazon Simple Storage Service (Amazon S3) buckets. You have fine-grained control over user identity, permissions, and keys. You can also use IAM policies to control the level of access granted to each user. You can also make use of your existing DNS name and SSH public keys, making it easy for you to migrate to Transfer for SFTP.

You can use any existing FTP/SFTP tool like Filezilla, WinSCP to connect to the S3 bucket. To know more, refer {1}

Reference Links:

  1. https://aws.amazon.com/blogs/aws/new-aws-transfer-for-sftp-fully-managed-sftp-service-for-amazon-s3/

  2. https://aws.amazon.com/premiumsupport/knowledge-center/s3-large-transfer-between-buckets/

  3. https://aws.amazon.com/s3/pricing/

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions