How does DMS handle source tables without primary keys?

0

Use case: dms full+cdc sync from DB2 tables to redshift and postgres

DMS redshift documentation says with redshift as a target, both source and target table need to have a primary key, or else it wont batch load with copy command. Tables with PK are using COPY appropriately, but a db2 table without a PK is doing this:

update tbl set col1='a', col2='b', col3=true 
where col1='a' and col2='b' and col3=false

Can I get around this by using define-primary-key table mapping rule (assuming my source data has a practical unique index)?

So if my source DMS table doesn't have a primary key, how does it work with postgres? I'm assuming it creates a table with no primary key and uses the same type of update as above. But then if I use define-primary-key table mapping rule DMS will all of a sudden switch to updating records based on only the primary key?

asked 5 months ago369 views
2 Answers
0

If a PK is defined then the PK columns are used to look up rows in target and when matched then update else insert from source. If no PK is defined then, as you have seen, all columns from source need to be used, which can be more resource intensive operation.

profile pictureAWS
answered 5 months ago
  • But what if the source does not have a primary key but you add a primary key to the target in a dms transformation? The docs for redshift say source and target need primary key, the docs for postgres only mentioned target.

  • If the source does not have a primary key but you add a primary key to the target in a DMS transformation then the PK you defined will be used during updates on target table.

    For large tables it is highly recommended to have PK. For small tables not having PK could be okay but a good OLTP database design always includes PK for each table.

    DMS will still proceed without PK's but you will see slow performance especially if the source data is changing frequently.

  • @MilindOke, thanks for confirming it will act that way with postgres as a target. will it behave that way with redshift as a target? The docs for redshift as target say BOTH source and target need a primary key.

0

You can use the AWS Schema Conversion Tool to with DMS to help make this mapping. If no direct conversion is possible, AWS SCT will give you an assessment report with a list of possible actions for you might take take. With SCT, you can manage your keys, map data types , and perform manual conversions. https://docs.aws.amazon.com/SchemaConversionTool/latest/userguide/CHAP_Converting.DW.html

profile pictureAWS
answered 5 months ago
  • I know I can use define-primary-key rule-action to define a primary key in a transformation rule. I’m asking how dms acts when I do that.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions