By using AWS re:Post, you agree to the Terms of Use

AWS DMS + OpenSearch + Index templates

0

I'm migrating some data from postgres to OpenSearch, but i'm struggling with migrating a set of coordinates. In postgres i have latitude and longitude, and i know DMS does not support geo types when using OpenSearch as a target. I wanted to work around this by using an ingest pipeline and an index template:

PUT _ingest/pipeline/my-pipeline-id
{
  "description": "My optional pipeline description",
  "processors": [
    {
      "set": {
        "field": "location.lon",
        "value": "{{{longitude}}}"
      }
    },
    {
      "set": {
        "field": "location.lat",
        "value": "{{{latitude}}}"
      }
    }
  ]
}
PUT /_index_template/jobs_template
{
  "index_patterns": [
    "jobs*"
  ],
  "template": {
    "settings": {
      "index.default_pipeline": "my-pipeline-id"
    },
    "mappings":{
      "properties": {
        "location": {
          "type": "geo_point"
        }
      }
    }
  }
}

I first tested having only the pipeline without the mappings and that part works. However, when i add a mapping to the template, and re run the migration task i get the following error

2022-01-20T15:23:01 [TARGET_LOAD     ]E:  Elasticsearch:FAILED SourceTable:jobs TargetIndex:jobs Operation:INSERT_ENTRY RecordPKKey:93904e3c-5565-4469-94d6-e58fbecdc5a3 RecordPKID:217D5CE32D4EC983FE2C3CFD6048821EA2A95F3658122A80A7EEB3A6088EA89CES HttpCode:400 ESErrorResponse:  
{
    "error": {
        "root_cause": [
            {
                "type": "illegal_argument_exception",
                "reason": "Rejecting mapping update to [jobs] as the final mapping would have more than 1 type: [_doc, doc]"
            }
        ],
        "type": "illegal_argument_exception",
        "reason": "Rejecting mapping update to [jobs] as the final mapping would have more than 1 type: [_doc, doc]"
    },
    "status": 400
}

Is there any way to make this work? Any way to create a geo point so I can perform geo queries in OpenSearch?