How to measure the lag between AWS Redshift MV and MSK topics in an architecture involving AWS Redshift streaming ingestion from MSK?

0

In our architecture we have data from AWS MSK being ingested into AWS Redshift using the Amazon Redshift Streaming Ingestion feature. In this approach, we define an external schema where we have the Materialized Views (MVs) being defined on the topics in MSK cluster. These MVs have auto-refresh turned on.

At any given point in time, what are the design options to measure the lag between the data in the MSK topics and the Redshift Materialized Views i.e. how far the Redshift MVs are the from the kafka topics respectively?

  • Any inputs on this would be appreciated?

MK
質問済み 6ヶ月前114ビュー
回答なし

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ