Questions on Data lake using DMS
A customer is implementing a data lake and i am assessing using DMS for pulling data into s3 from on an premise MS SQL DB. As part of this i would like to understand the following points on DMS:
- Other users of DMS - widely used?
- Examples of using DMS with SQL Server Change Tracking
- Volumetrics - how much can it handle
- Running over VPN - any issues that you're aware of?
- Known issues/limitations of DMS (big binary objects etc)
- What ports/access (accounts) does it require for the on prem SQL server
- What is the overhead on the on prem SQL server?
- 최신
- 최다 투표
- 가장 많은 댓글
-
Other users of DMS - widely used? yes, > 100,000 moved to aws using DMS so far
-
Volumetrics - how much can it handle? Never hit a limit of not keeping up yet.
-
Running over VPN - any issues that you're aware of? No, other than if using CDC then the CDC change tables could get larger if there’s not enough bandwidth to send the data to DMS instance in the cloud.
-
Known issues/limitations of DMS (big binary objects etc)? All documented: https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.SQLServer.html
-
What ports/access (accounts) does it require for the on prem SQL server? 1433, sysadmin access to read the logs and CDC tables
-
What is the overhead on the on prem SQL server? Depends. Transactional repl will add more info to the transaction log, will produce bigger logs. CDC tables will take more space in the customer database, and add some sqlagent jobs to do cleanup.
