Source: postgres
Target: OpenSearch
I have a text
column called description
in one of my postgres tables. Per the documentation, this data type is mapped to a NCLOB
. Since OpenSearch does not not offer LOB support, my description
is missing in my OpenSearch documents.
I tried using the mapping rule bellow, but does not seem to be doing anything
{
"rule-type": "transformation",
"rule-id": "3",
"rule-name": "3",
"rule-target": "column",
"object-locator": {
"schema-name": "public",
"table-name": "jobs",
"column-name": "description"
},
"rule-action": "change-data-type",
"data-type": {
"type": "string",
"length": 500
}
}
When i check the logs i see the following
Column 'description' is unsupported in table def 'public.jobs' since the LOB support is disabled
However, i do have LOB enabled under task settings:
"TargetMetadata": {
"ParallelApplyBufferSize": 0,
"ParallelApplyQueuesPerThread": 0,
"ParallelApplyThreads": 0,
"TargetSchema": "",
"InlineLobMaxSize": 0,
"ParallelLoadQueuesPerThread": 0,
"SupportLobs": true,
"LobChunkSize": 10,
"TaskRecoveryTableEnabled": false,
"ParallelLoadThreads": 0,
"BatchApplyEnabled": false,
"FullLobMode": true,
"LimitedSizeLobMode": false,
"LoadMaxFileSize": 0,
"ParallelLoadBufferSize": 0
},
Is that transformation rule supposed to work? Or will any LOB column be skipped because OpenSearch does not have LOB support?
Any way to make this work?
Thanks!
How does it handle varchars? in the same table i have a
name
column which is avarchar(255)
and that get's mapped to astring
type in OpenSearch. Is there no way to migrate atext
type from postgres to OpenSearch?Yeah, it's confusing how DMS handles varchar even it is not mentioned in the documentation as supported source data type! I'm stuck in the same problem, did you find any solution?