AWS DMS + OpenSearch + Index templates

0

I'm migrating some data from postgres to OpenSearch, but i'm struggling with migrating a set of coordinates. In postgres i have latitude and longitude, and i know DMS does not support geo types when using OpenSearch as a target. I wanted to work around this by using an ingest pipeline and an index template:

PUT _ingest/pipeline/my-pipeline-id
{
  "description": "My optional pipeline description",
  "processors": [
    {
      "set": {
        "field": "location.lon",
        "value": "{{{longitude}}}"
      }
    },
    {
      "set": {
        "field": "location.lat",
        "value": "{{{latitude}}}"
      }
    }
  ]
}
PUT /_index_template/jobs_template
{
  "index_patterns": [
    "jobs*"
  ],
  "template": {
    "settings": {
      "index.default_pipeline": "my-pipeline-id"
    },
    "mappings":{
      "properties": {
        "location": {
          "type": "geo_point"
        }
      }
    }
  }
}

I first tested having only the pipeline without the mappings and that part works. However, when i add a mapping to the template, and re run the migration task i get the following error

2022-01-20T15:23:01 [TARGET_LOAD     ]E:  Elasticsearch:FAILED SourceTable:jobs TargetIndex:jobs Operation:INSERT_ENTRY RecordPKKey:93904e3c-5565-4469-94d6-e58fbecdc5a3 RecordPKID:217D5CE32D4EC983FE2C3CFD6048821EA2A95F3658122A80A7EEB3A6088EA89CES HttpCode:400 ESErrorResponse:  
{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Rejecting mapping update to [jobs] as the final mapping would have more than 1 type: [_doc, doc]"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Rejecting mapping update to [jobs] as the final mapping would have more than 1 type: [_doc, doc]"
    },
    "status": 400
}

Is there any way to make this work? Any way to create a geo point so I can perform geo queries in OpenSearch?

Keine Antworten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen