How to measure the lag between AWS Redshift MV and MSK topics in an architecture involving AWS Redshift streaming ingestion from MSK?

0

In our architecture we have data from AWS MSK being ingested into AWS Redshift using the Amazon Redshift Streaming Ingestion feature. In this approach, we define an external schema where we have the Materialized Views (MVs) being defined on the topics in MSK cluster. These MVs have auto-refresh turned on.

At any given point in time, what are the design options to measure the lag between the data in the MSK topics and the Redshift Materialized Views i.e. how far the Redshift MVs are the from the kafka topics respectively?

  • Any inputs on this would be appreciated?

MK
preguntada hace 6 meses114 visualizaciones
No hay respuestas

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas