CDC DMS task - removing Op column

Question

i have a DMS Replication task  that is full+ongoing replication. the source table dont have update or delete operation , only insert for new records. the task is adding Op field with the I (value for insert).

i dont need this field. the initial load csv does not have this field , but the files create after each insert contain it. that cause glue in the next step of the pipeline to create the wront schema and after that some of the values are not in the correct field when quering the data

i have tried using transformation and drop this column , but that didn't work

```
        {
            "rule-type": "transformation",
            "rule-id": "185351232",
            "rule-name": "185351232",
            "rule-target": "column",
            "object-locator": {
                "schema-name": "%",
                "table-name": "%",
                "column-name": "Op"
            },
            "rule-action": "remove-column",
            "value": null,
            "old-value": null
        },
```

any idea what i can do to remove it ?

hyp · Answer

Hi,

How about to use rule-action: remove-column[1]?

[1] https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.html#CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Transformations.Limitations

subhashr · Answer

Removing opcolumn  in cdc is not possible 
But you can add it in fullload by using eca IncludeOpForFullLoad

Giovanni Lauria · Answer

Here is the IncludeOpForFullLoad option

https://aws.amazon.com/blogs/database/filter-delete-operations-with-aws-dms/

CDC DMS task - removing Op column

Relevanter Inhalt