How does AWS DMS table selection rules handle overlap or conflict?

Question

Hello! I'm reading through the docs that talk about AWS DMS selection rules and wildcards:

1. https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Selections.html
2. https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TableMapping.SelectionTransformation.Wildcards.html

I have a use-case where we're currently pulling in a few dozen unnecessary tables in a snapshot load from source (Postgres) to destination (Athena DataLake). I want to exclude MOST tables matching a certain pattern (`%_aud`, to be specific), but then there's one table matching that same pattern I want to explicitly keep (let's say its called `foo_aud`).

I'd like to not have to write explicit exclusion rules for the ~20 or so other tables, so I'm hoping I can do something like this: (rules are specified in JSON in a cloudformation file)

```json
{
    "rules": [
// ...our other selection rules...
        {
            "rule-type": "selection",
            "rule-id": "21",
            "rule-name": "excludeaudit",
            "object-locator": {
                "schema-name": "myschema",
                "table-name": "%_aud"
            },
            "rule-action": "exclude"
        },
        {
            "rule-type": "selection",
            "rule-id": "22",
            "rule-name": "includefooaudit",
            "object-locator": {
                "schema-name": "myschema",
                "table-name": "foo_aud"
            },
            "rule-action": "include"
        }
    ]
}
```

What will happen in the above example? Will DMS correctly exclude all `%_aud` tables, but then include `foo_aud`? Does the rule ordering matter - e.g. if I swap rules 22 and 21? (How is rule priority managed?) Or is there some other way I can achieve this?

Thank you!

Answer

This should work for you. The order of the rule ids in the Mapping Rules of any AWS DMS tasks does not influence the priority unless you use "load-order" in your mapping rules.

Since its only 1 table that you plan to explicitly include, you can try "explicit" as the "rule-action" as well. Something like below:

{
    "rules": [
        {
            "rule-type": "selection",
            "rule-id": "1",
            "rule-name": "1",
            "object-locator": {
                "schema-name": "NewCust",
                "table-name": "Customer"
            },
            "rule-action": "explicit"
        }
    ]
}

I would say, go ahead and test both out. Either or both should work.

How does AWS DMS table selection rules handle overlap or conflict?

Relevanter Inhalt