How to measure the lag between AWS Redshift MV and MSK topics in an architecture involving AWS Redshift streaming ingestion from MSK?

0

In our architecture we have data from AWS MSK being ingested into AWS Redshift using the Amazon Redshift Streaming Ingestion feature. In this approach, we define an external schema where we have the Materialized Views (MVs) being defined on the topics in MSK cluster. These MVs have auto-refresh turned on.

At any given point in time, what are the design options to measure the lag between the data in the MSK topics and the Redshift Materialized Views i.e. how far the Redshift MVs are the from the kafka topics respectively?

  • Any inputs on this would be appreciated?

MK
已提问 6 个月前114 查看次数
没有答案

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则