I'm migrating some data from postgres to OpenSearch, but i'm struggling with migrating a set of coordinates.
In postgres i have latitude
and longitude
, and i know DMS does not support geo types when using OpenSearch as a target.
I wanted to work around this by using an ingest pipeline and an index template:
PUT _ingest/pipeline/my-pipeline-id
{
"description": "My optional pipeline description",
"processors": [
{
"set": {
"field": "location.lon",
"value": "{{{longitude}}}"
}
},
{
"set": {
"field": "location.lat",
"value": "{{{latitude}}}"
}
}
]
}
PUT /_index_template/jobs_template
{
"index_patterns": [
"jobs*"
],
"template": {
"settings": {
"index.default_pipeline": "my-pipeline-id"
},
"mappings":{
"properties": {
"location": {
"type": "geo_point"
}
}
}
}
}
I first tested having only the pipeline without the mappings and that part works. However, when i add a mapping to the template, and re run the migration task i get the following error
2022-01-20T15:23:01 [TARGET_LOAD ]E: Elasticsearch:FAILED SourceTable:jobs TargetIndex:jobs Operation:INSERT_ENTRY RecordPKKey:93904e3c-5565-4469-94d6-e58fbecdc5a3 RecordPKID:217D5CE32D4EC983FE2C3CFD6048821EA2A95F3658122A80A7EEB3A6088EA89CES HttpCode:400 ESErrorResponse:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [jobs] as the final mapping would have more than 1 type: [_doc, doc]"
}
],
"type": "illegal_argument_exception",
"reason": "Rejecting mapping update to [jobs] as the final mapping would have more than 1 type: [_doc, doc]"
},
"status": 400
}
Is there any way to make this work? Any way to create a geo point so I can perform geo queries in OpenSearch?