Firehose only delivers 99% of records to s3

0

I have set up a firehose (direct put) to deliver messages to a s3 bucket.

Unfortunately, regardless of the number of message to be delivery, Firehose always delivers n-500 messages.

In screenshot attached, we can see what happens when attempting to deliver 500000 records, it gets stuck at 4995000. Same happens when I attempt to deliver 200000 messages, it gets stuck at 199500

I have replaced the firehose, change the instances, even tried to do the same from different aws accounts, but the result is the same.

Anyone who could shed some light on this would be very much appreciated.

Sincerley,

Josep Maria

Edited by: JosepMaria on Sep 27, 2020 5:53 AM

Edited by: JosepMaria on Sep 27, 2020 5:57 AM

demandé il y a 4 ans243 vues
1 réponse
1
Réponse acceptée

Modify code by adding :

"initialPositionon":"START_OF_FILE",

répondu il y a 3 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions