1 Answer
- Newest
- Most votes
- Most comments
0
Basically there two ways, I did this using dynamic mapping with dynamic documents
Define a Hidden Column
In your Glue Catalog table, add a hidden column named _id
. This column will store the custom index ID you want to use for each record.
Access the Value in Glue Script
Within your Glue job script (Python or Scala), access the value of the hidden _id
column for each record being processed.
Enable Dynamic Documents
When configuring the OpenSearch data sink in your Glue job, set the "Dynamic Documents" option to "true" within the connection properties. This allows Glue to dynamically add the _id field with your custom value during data insertion.
The other way is pre defined index mapping, let me know if you want to know about it.
Here are the references that might help you
answered a month ago
Relevant content
- asked a year ago
- asked 9 months ago
- asked 2 years ago
- AWS OFFICIALUpdated 10 months ago
- AWS OFFICIALUpdated 10 months ago
- AWS OFFICIALUpdated a year ago
I would like to know about pre defined index mapping.Do you have an example?
When configuring the OpenSearch data sink in your Glue job, set the "Dynamic Documents" option to "true" within the connection properties. This allows Glue to dynamically add the _id field with your custom value during data insertion. Can you share an example for this