Cost analysis for replication of data

Question

Which service among AWS DMS, AWS Glue and AWS Kinesis is cheap and efficient for replication of large amount data between two AWS account?

My main focus is to lower the cost for replication of data.
Can anyone please answer this with respect to cost?

Accepted Answer

It depend on various factors,
1. How much latency is accepted from source to target?
2. Replicating data using CDC or Logical technology ? This also depends on accepted latency by the business.
3. Are all the tables have Primary Key for CDC or Audit column for Logical replication?
4. How much impact on the performance on the source and the target system?

Glue  is like ETL service, Kinesis is streaming service and DMS is database replication tool.
They can be fit in alternatively. If you want to perform data transformation while replicating to other target or if you want replicate read once and write at multiple target. Glue or Kinesis might best fit. They can be easily integrate with other services. They are highly available.
Also if you are planning huge load every night, 4 hour or something like Glue might good. But if you needs data real time and wants to make transformation DMS + Kinesis might fit better.

DMS is one of the simplest solution of you wants to replicate data on going basis with some level of transformation., It cost might be less, but if you wants to replicate huge amount of data in batches or on-going, you might needs to choose higher compute capacity and also think about source system should also meets the requirement of replication.

Would be better if you can share more on the use case to conclude better cost effective solution.

Answer

Hello,

Based on your goal of minimizing costs for replicating large amounts of data between AWS accounts, I would recommend evaluating AWS Database Migration Service (AWS DMS). Here are a few factors to consider:

* AWS DMS supports continuous replication of data stored in databases like MySQL, Oracle, PostgreSQL, MongoDB, and Amazon S3. This allows data to be replicated in near real-time with very low latency.
* DMS replication is done natively within the AWS network, so there are no data transfer charges between accounts like there would be for transferring data across the internet. This keeps replication costs very low.
* DMS has a pay-as-you-go pricing model where you only pay for the amount of data replicated. It can replicate petabytes of data cost-effectively.
* Other services like AWS Glue or Kinesis may incur higher costs for processing or transferring large amounts of data between accounts on an ongoing basis compared to DMS' native replication.
* DMS also supports migrating existing database snapshots between accounts as a one-time data migration, which could help lower initial setup costs.

**I'd recommend testing DMS replication between your accounts to get a sense of the costs based on your specific data volumes and workload**

Thanks

Cost analysis for replication of data

Relevant content