DMS - PostgreSQL to MSK

Question

I am working on POC to migrate from Debezium to AWS DMS for PostgreSQL. PostgreSQL as Source and Kafka as Destination (AWS MSK). In Debezium i used to see before and after image for updates, but in DMS for Updates, i see only after values in Data section for the same table.

I could not find any documentation on how to get before image. Is this expected behavior or do i need to update any settings to get before image.

issue 2, table on which CDC is enabled, i have 3 jsonb columns. when i run update on the rows where jsonb have data, in kafka i see messages either jsonb column not getting populated at all (not returning those columns in kafka message) , in some cases 1 jsonb column is populated.

i tried with full LOB or Limited LOB options, going as far as 10mb for Maximum LOB size. is there anything i need to change to get LOB populated in Kafka?

I am using PostgresSQL 14.4 version, DMS Replication Instance engine Version 3.5.1 and AWS MSK 2.3.1

Any info would be appreciated

Answer

Hi,

On first issue, to get before-image updates with Kafka, see https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.BeforeImage.html

On issue #2, you could try to remap you jsonb onto a CLOB to see if you get better results. See https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.PostgreSQL.html.

```
MapJsonbAsClob

By default, AWS DMS maps JSONB to NCLOB. You can specify MapJsonbAsClob to let PostgreSQL migrate the 
JSONB type as CLOB.

Example: --postgre-sql-settings='{"MapJsonbAsClob": "true"}'
```

Best,

Didier

Answer

anyone else any idea how jsonb issue can be resolved

DMS - PostgreSQL to MSK

관련 콘텐츠