Does Firehose support CRUD operations on OpenSearch

Question

Hi,

We are designing a data indexing pipeline to OpenSearch using Firehose, like :
data source ----(putRecordBatch)----> firehose ---(Lambda transformation)--> OpenSearch

New data added to data store will be indexed via this flow in near-real time. There is requirement of both creating new documents, and doing partial update on existing documents(ie, updating select attributes of a specified document id)

Does Firehose integration with OpenSearch support Upsert(Update if document is seen in OpenSearch, else insert)?

Thanks

Answer

Amazon Kinesis Data Firehose is a delivery streaming service and could not perform CURD operation to Opensearch. Currently, it dose not support Upsert / doing partial update on existing documents as a unique document ID is been generated by firehose while inserting the data to Opensearch. Thus, everytime data will flow new records will be created.

Does Firehose support CRUD operations on OpenSearch

Contenuto pertinente