DMR with kafka multi-topic

Yes, it is possible to create a single AWS Database Migration Service (DMS) task where data from a single source table can be migrated to different Kafka topics based on source column data. This feature is supported in AWS DMS engine versions 3.4.6 and higher.

To achieve this, you can use object mapping rules in your DMS task configuration. Here's how you can approach this:

1. Use the kafka-target-topic attribute in your object mapping rules to specify different Kafka topics for different data subsets from the same source table.

2. Utilize the rule-action of "map-record-to-record" to define how the data should be mapped.

3. Create multiple object mapping rules, each with a different kafka-target-topic value.

4. Use selection rules or filters within each object mapping rule to determine which records go to which topic based on the source column data.

For example, you could create rules that send data to different topics based on a specific column value or range of values. This allows you to partition your data across multiple Kafka topics within a single DMS task.

Keep in mind that you'll need to carefully design your mapping rules to ensure that all data is correctly routed to the appropriate Kafka topics. Also, make sure your Kafka cluster is properly configured to handle multiple topics and that the necessary topics are created before starting the migration task.

This approach eliminates the need to create multiple separate tasks when you need to migrate data from the same source table to different Kafka target topics, making your migration process more efficient and easier to manage.
Sources
Using Apache Kafka as a target for AWS Database Migration Service - AWS Database Migration Service
AWS DMS release notes - AWS Database Migration Service