Firehose delivery stream writes every record exactly 2 times

0

Events are sent once from a lambda function to a firehose delivery stream without any transformation or conversion. observing the data on s3 every record is written twice.

the lambda uses "putRecordBatch" to send the object to FDS.

example:

{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}
{"e":"pv","url":"https://foo.com", "id":"1552c074-8432-43e9-a1e5-7e5206ed8790"}
{"e":"pv","url":"https://baz.com", "id":"ebb18ce2-fe8d-4c1b-8d2e-3338762c2c8a"}

"id" is unique

I understand that duplication might happen but not to 100% of the data.

how can we figure out the cause of this duplication if it is abnormal.

lime
posta 5 anni fa547 visualizzazioni
1 Risposta
0

pushRecord was called with a callback and the invoked again vai .promise()

lime
con risposta 5 anni fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande