dynamo zero etl pipeline configuration examples for templating/extracting nested values

0

Hi,

I am setting up my pipeline configuration for our dynamodb table's zero-etl integration pipeline with our opensearch domain, but am struggling to find any examples for how to solve the following issue:

Our table has a ListAttribute, which is a list of objects. Each object (sometimes) has a id : value pair. objects = [ object 1: {id: 123, other_id: 123455, blah blah other key values we don’t need} , object 2: {id: 456, other_id: 1355, blah blah other key values we don’t need} , ]

Whats the best way to configure our dynamo ingestion pipeline configuration so that we not only index on every other attribute in the row, but also create an additional index on a list of all of the concatenations of “id#other_id”, extracted from each of the objects nested in the list? Am i supposed to add this inside a "processor" section or inside a template_content section? There doesn't seem to be thorough documentation for processing this kind of data, so I appreciate the help i can get! Thanks.

clara
gefragt vor einem Monat140 Aufrufe
1 Antwort
0

template_type and template_content in the pipeline configuration are used to define explicit mapping rules whereas Processors perform an action on your data, such as filtering, transforming, or enriching.

Having said that, if you could you provide how a sample row in your Dynamo DB table looks like at your end and what is your expectation on how the OpenSearch Document should be indexed for that particular row in the index.

Also, you can also reach out AWS Support in case if you need further assistance.

AWS
SUPPORT-TECHNIKER
Navya T
beantwortet vor einem Monat
profile picture
EXPERTE
überprüft vor einem Monat

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen