- Newest
- Most votes
- Most comments
The answer you referenced is still valid. When using Kinesis Firehose with Lambda transformations, it is not possible to map one input record to multiple output records. Kinesis Firehose expects to receive exactly one output record for each input record when using Lambda transformations.
The Lambda function used for transformation in Firehose can modify the content of the record, but it cannot create new records or split a single record into multiple records. This limitation is inherent to how Firehose processes data with Lambda transformations.
If you need to split a single record containing multiple items into separate records, you would need to perform this splitting operation before the data reaches Firehose. For example, you could do this in the source application that's sending data to Kinesis Streams, or you could use a separate Lambda function triggered by Kinesis Streams to split the records before they are sent to Firehose.
It's important to note that while you can't create multiple records from one during the Firehose Lambda transformation, you can still perform other types of transformations on the data. This includes modifying the content, enriching the data with additional information, or even dropping records based on certain conditions. However, the one-to-one relationship between input and output records must be maintained in the Lambda transformation step of Firehose.
Sources
Can I split a batch/list of records to individual records while using Firehose Lambda Transformations | AWS re:Post
