1 Answer
- Newest
- Most votes
- Most comments
0
Hello There,
Based on the requirement shared, I have tried the replicate the same at my end and was able to achieve the below.
-
Created a database (employee) at the source instance with two tables X & Y.
- Create table tableX(createdby int, createddate datetime);
- Create table tableY(createddate datetime, createdby int);
-
Setup DMS RI & endpoints.
-
Configured the task with below mapping rule. * Selection rule: where schema name is like 'employee' and Source table name is like 'table%', include
- Transformation rules: where schema name is like 'employee' and Source table name is like 'table%', rename
- Action:Rename:FinalData
- This would create a new table 'finaldata', in the target database 'employee'.
- Transformation rules: where schema name is like 'employee' and Source table name is like 'table%', rename
-
To test this further, I have executed below insert queries.
* Insert into tableY values(CURRENT_TIME(),20); * Insert into tableY values(CURRENT_TIME(),20); * Insert into tableX values('12',CURRENT_TIME()); * Insert into tableX values('12',CURRENT_TIME());
-
On the destination, the misplaced columns are handled, as indicated in the below result.
select * from finaldata;
createdby | createddate |
---|---|
20 | 2023-02-18 07:43:31 |
20 | 2023-02-18 07:43:35 |
12 | 2023-02-18 07:43:46 |
12 | 2023-02-18 07:43:52 |
Hence, I would suggest you to try the above setup at your end and test it further.
Relevant content
- asked a year ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
Thanks for the post it is very helpful. This might work for full load. However, when the CDC files land in the cdc-file folder there is no column name. I am not seeing where, in the transformation above, the column order has been reversed for TableY to match the expected structure in the target, so no errors occur during cdc ongoing replication.
Just to be clear all databases have a “Full-Load and Ongoing” Task that pulls data from on-prem to S3 targets. So, in the example above there would be four S3 target locations: S3:Bucket/Coke/dbo/TableX, S3:Bucket/Pepsi/dbo/TableX, S3:Bucket/Coke/cdc-files S3:Bucket/Pepsi/cdc-files
These buckets above are also the Source of an S3->Single RDS instance where a SourceID column is added to every table with the value of “Coke” or “Pepsi” or whatever other database is in the mix.