Firehose delivery stream writes every record exactly 2 times

0

Events are sent once from a lambda function to a firehose delivery stream without any transformation or conversion. observing the data on s3 every record is written twice.

the lambda uses "putRecordBatch" to send the object to FDS.

example:

{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}
{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}

"id" is unique

I understand that duplication might happen but not to 100% of the data.

how can we figure out the cause of this duplication if it is abnormal.

lime
gefragt vor 5 Jahren548 Aufrufe
1 Antwort
0

pushRecord was called with a callback and the invoked again vai .promise()

lime
beantwortet vor 5 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen