DMS(CDC) to Kafka Payload format

1

Hi, Currently am using DMS to capture db changes and sending them to Kafka. Am only interested in data (json values without "data" keyword included) and not metadata. I am following the user guide and specifically the below configuration but not working. Could you please assist. So am only interested in flat json payload with only the data values inserted to table. Example: { "prop1":"value1","prop2":"value2"}

{ "rules": [ { "rule-type": "selection", "rule-id": "1", "rule-name": "1", "rule-action": "include", "object-locator": { "schema-name": "BW3", "table-name": "INT_TRANSACTIONS" } }, { "rule-type": "object-mapping", "rule-id": "2", "rule-name": "DefaultMapToKafka", "rule-action": "map-record-to-record", "object-locator": { "schema-name": "BW3", "table-name": "INT_TRANSACTIONS" } } ] }

1 Antwort
1
Akzeptierte Antwort

Hi,

If I understand correctly, you want only flat json file without the:

  1. data keyword
  2. metadata section

Currently, there is no available way for DMS to migrate with above requirement however, I was able to find internal feature request raised for option removing metadata. There will be no ETA for when this can be available.

Best option for right now is to clean the database as needed before ingest into the consumer.

AWS
SUPPORT-TECHNIKER
Kevin_Z
beantwortet vor 2 Jahren
profile picture
EXPERTE
überprüft vor einem Monat
  • Thanks, By cleaning db as needed however, still the above keywords will be included in payload , which is something we dont need.

  • Hi, sorry for the typo, I mean to say clean the data from Kafka before ingesting into a consumer.

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen