How do I troubleshoot the errors I get when I try to create an Amazon MSK Replicator?

Resolution

When you create an MSK Replicator, you might receive one or more types of error messages. Identify the error and follow the instructions associated with it to troubleshoot it.

Error: "Invalid parameter kafkaCluster: The subnets in the source and target Amazon MSK clusters have partial overlap. Keep it distinct or have complete overlap."

You get this error when your source and target cluster's subnets partially overlap. This means that you must choose the same subnet for both the source and target cluster. Or, you must choose a distinct subnet for each MSK cluster.

Error: "There is an issue with your replicator Code: InvalidInput.InvalidKafkaCluster Message: The specified Amazon MSK Cluster has less than 3 brokers. Cluster ARN: arn:aws:kafka:us-east-1:012345678911:cluster/mTls/ad3edd97-d6d9-479c-a886-fb56f3749b01-11."

You get this error when either your source or destination MSK cluster has less than three brokers configured. If you use two Availability Zones (AZs) for your MSK cluster, then create a total of four brokers in two Availability Zones for the MSK Replicator.

Error: "There is an issue with your replicator Code: InvalidInput.InvalidKafkaCluster Message: IAM Auth is not enabled for the Amazon MSK Cluster arn:aws:kafka:us-east-1:012345678911:cluster/targetMSK/1e01c82c-66d0-4d09-8bae-6f47e8ad9c81-11."

You get this error when your source MSK cluster has AWS Identity and Access Management (IAM) authentication but your target cluster doesn't. Make sure both your source and target clusters have activated authentication.

Error: "There is an issue with your replicator Code: InvalidInput.InvalidVpcConfig Message: The subnet(s) you have provided for Amazon MSK Clusters do not belong to the same Availability Zones."

If your source MSK cluster Availability Zone and your target MSK cluster Availability Zone don't correlate then you receive the error.

To resolve this error, make sure that the Availability Zones for source and target MSK clusters overlap. However, the subnet definition for both the MSK clusters must be either kept distinct from each other or have a complete overlap.

Error: "There is an issue with your Replicator Code: InvalidInput.UnsupportedBrokerInstanceType Message: The Broker Instance Type is not supported for cluster ARN: arn:aws:kafka:us-east-2:012345678911:cluster/Cross-region/d65a6e2e-c6ee-4805-a5cf-5328ac349a94-6, please use a different Instance Type."

Kafka T3.small instance type isn't supported for MSK Replicator. Make sure that your source and target MSK clusters have an instance type of Kafka M5.large, or higher. For more information, see Updating the broker type using the AWS Management Console.

Error: "The MSK source cluster must have a cluster-based permissions policy."

This error appears because you activated multi-Virtual Private Cloud (multi-VPC) in your source MSK cluster for cross-Region but didn't define the cluster resource-based policy.

Make sure your cross-region MSK Replicator has sufficient permissions to use AWS PrivateLink to connect with your source MSK cluster. Follow these steps to update your resource-based cluster policy:

1. Log in to the Amazon MSK console.
2. In the left panel, choose Clusters. Then, choose the Properties tab.
3. Under Security settings, choose Edit cluster policy
4. Replace the Principal with your account ID.

For more information regarding policy permissions, see Permissions for multi-VPC private connectivity.

Error: "There is an issue with your Replicator Code: ConnectivityError.SourceBrokerUnreachable Message: The source Kafka broker is not reachable, please check network connectivity."

This error occurs when your source MSK cluster's security group doesn't have the required inbound and outbound rules.

For example, for your source security group (sg-0f2c249b8675a8d9f) and your target security group (sg-0bc6196de6bc0f028), the rules are as follows:

Inbound Rule:  
~~~~~~~~~~~~~~  
- Allow 'All traffic' from both cluster Security Group sg-0f2c249b8675a8d9f and sg-0bc6196de6bc0f028.  
  
Outbound Rule:  
~~~~~~~~~~~~~~  
- Allow all traffic to 0.0.0.0/0. Or, you can also provide VPC CIDR range of Target MSK cluster.

For more information, see Choose your source cluster.

Error: "There is an issue with your replicator Code: ConnectivityError.TargetBrokerUnreachable Message: The target Kafka broker is not reachable, please check network connectivity."

You get this error when your target MSK cluster's security group doesn't have the required inbound and outbound rules.

For example, for your source security group (sg-0f2c249b8675a8d9f) and your target security group (sg-0bc6196de6bc0f028), the rules are as follows:

Inbound Rule:  
~~~~~~~~~~~~~~  
- Allow 'All traffic' from both cluster Security Group sg-0f2c249b8675a8d9f and sg-0bc6196de6bc0f028.  
  
Outbound Rule:  
~~~~~~~~~~~~~~  
- Allow all traffic to 0.0.0.0/0. Or, you can also provide VPC CIDR range of Source MSK cluster.

For more information, see Choose your target cluster.

Related information

Requirements and considerations for creating an Amazon MSK Replicator

Troubleshooting MSK Replicator