How does MSK Replicator work

0

if i am using MSK Replicator to replicate MSK topic data and consumer offset from cluster X to cluster Y and suddenly cluster X crashes, so i recreate a new cluster and i prefer use the same name X, how will the replicator work under this use case? and if i create a new MSK replicator to replicate data from cluster Y to cluster X, will this replicator copy data from Y to new cluster X?

And also does MSK replicator support source cluster is 2.8.2.tired and destination cluster is 3.6.0?

debug9
gefragt vor 4 Monaten760 Aufrufe
1 Antwort
0

Hi, debug9

When using MSK Replicator to replicate MSK topic data and consumer offset from cluster X to cluster Y, and if cluster X crashes and you recreate a new cluster with the same name X, here's how the MSK Replicator would work:

Replicator Behavior for Recreated Cluster X:
    - If you recreate cluster X with the same name, MSK Replicator may not automatically resume replication to the new cluster X. MSK Replicator maintains a mapping between topics and consumer groups in the source and destination clusters. Recreating a cluster with the same name may not be automatically recognized as the same cluster by the replicator.
    - You may need to update the MSK Replicator configuration to point to the new cluster X, ensuring that the replication configuration is aligned with the new cluster's details.

Creating a New MSK Replicator to Replicate Data from Cluster Y to New Cluster X:
    - If you create a new MSK Replicator to replicate data from cluster Y to the new cluster X, the replicator will copy data from Y to the new cluster X as configured in the replication configuration.
    - This process is independent of the previous replication from old cluster X to cluster Y.

MSK Replicator Version Compatibility:
    - MSK Replicator's version compatibility depends on the versions of Apache Kafka running on the source and destination clusters.
    - From my understanding, it's recommended to use MSK Replicator with clusters running the same Kafka version to ensure compatibility and proper replication.

https://aws.amazon.com/about-aws/whats-new/2023/11/amazon-msk-support-apache-kafka-version-3-6-0-tiered-storage/

profile picture
hoylem
beantwortet vor 4 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen