By using AWS re:Post, you agree to the Terms of Use
/AWS Glue for SQL Server(on-premises) Data Migration/

AWS Glue for SQL Server(on-premises) Data Migration


Curious to know about, can we use AWS Glue to migrate 24 TB data from MS SQL Server(On-prem) to S3?How much time will it take (average) and how much effective it would be from other options? And would it be costly from other options of data transfer(like AWS DataSync, Snowball)?

1 Answers

The choice of migration of data is often based on other requirements. Do you need to keep it in sync after initial migration? Or can you snapshot it, migrate the data, hydrate the SQL Server RDS instance, and then resume? Do you have a specific amount of time you want to accomplish this migration within?

Database migration service (DMS) can either transfer the data while the database is running, then provide that sync before switch over. It is more expensive than options like DataSync (or just S3 multi-part upload), which is mainly constrained by your bandwidth out of the source(s) and your company's internet connectivity. Sometimes, Snowball is the fastest way to get the data into S3 because of those bottlenecks. In some cases, I've seen those network bottlenecks impact DMS performance on highly mutating databases.

answered 23 days ago
  • Actually the data migration is happening on Snowflake on Aws cloud, so the option of DMS is not valid here. Only doing the historical data migration, no sync up after that. And time limit is maximum 1 day.

  • The the time needed will be based on the internet capability of your uploading instance and your overall internet connectivity. Multi-part/multi-thread will help (DataSync does this as does the AWS CLI). You can then compare the estimated time to the time you think getting a Snowball device, getting it installed and sending it back will take.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions