Firehose delivery stream writes every record exactly 2 times

0

Events are sent once from a lambda function to a firehose delivery stream without any transformation or conversion. observing the data on s3 every record is written twice.

the lambda uses "putRecordBatch" to send the object to FDS.

example:

{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}
{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}

"id" is unique

I understand that duplication might happen but not to 100% of the data.

how can we figure out the cause of this duplication if it is abnormal.

lime
已提問 5 年前檢視次數 547 次
1 個回答
0

pushRecord was called with a callback and the invoked again vai .promise()

lime
已回答 5 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南