Firehose delivery stream writes every record exactly 2 times

0

Events are sent once from a lambda function to a firehose delivery stream without any transformation or conversion. observing the data on s3 every record is written twice.

the lambda uses "putRecordBatch" to send the object to FDS.

example:

{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}
{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}

"id" is unique

I understand that duplication might happen but not to 100% of the data.

how can we figure out the cause of this duplication if it is abnormal.

lime
已提问 5 年前547 查看次数
1 回答
0

pushRecord was called with a callback and the invoked again vai .promise()

lime
已回答 5 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则