How can I implement exponential backoff and retries in KDA(running Flink 1.13)

0

I saw that in Flink 1.15, this was added(https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/state/task_failure_recovery/) but I need to handle the scenario of my source and sink(which are Kinesis Data Streams) going away(and returning a UnknownHostException). I know it's not a thing that will happen very often with KDS but we happened to run into this one time. Our Flink App is written in Java. I see several config constants but was thinking about making this exception recoverable and using this consumer config(SHARD_GETRECORDS_RETRIES). I'm not sure on the producer side(not sure if the KPL will do the retry for me).

profile picture
asked 2 years ago281 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions