Hi Team,
I am trying to Debezium and MSK for CDC.
I have MySql Auraora serverless database
I'm trying to connect the all (MySQL aurora serverless Database, Debezium, my MSK Cluster) for data change capture
I'm trying to follow this blog :
Introducing Amazon MSK Connect
in this blog, we use connector capacity type = auto Scaled
but in Amazon Managed Streaming for Apache Kafka, Developer Guide, it says :
Important
The Debezium MySQL connector plugin supports only one task and does not work with autoscaled capacity mode for Amazon MSK Connect. You should instead use provisioned capacity mode and set worker count equal to one in your connector configuration. To learn more about the capacity modes for MSK Connect, see Connector capacity.
Developer Guide
at this point, I enable to successfully create an MSK connector (MSK Connect)
not sure which approach is the right one
1 - are the MSK and MSK connectors compatible with MySQL aurora serverless ?
2 - is there any other article that includes all the steps/constraints on how to create an MSK cluster and MSK connector with Auror and Debezium?
3 - does the MSK connector need internet access from within the VPC to be successfully created?
now I have this error when trying to create the connector :
Code: InvalidInput.InvalidConnectorConfiguration
Message: The connector configuration is invalid. Message: Failed to find any class that implements Connector and which name matches io.debezium.connector.mysql.MySqlConnector, available connectors are:.........
[Worker-07edsd2458791sds1545] org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches io.debezium.connector.mysql.MySqlConnector, available connectors are: PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='2.7.1', encodedVersion=2.7.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='2.7.1', encodedVersion=2.7.1, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='2.7.1', encodedVersion=2.7.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}
Note: my VPC is private only and doesn't have access to the internet
Thank you for your Help!
Thank you! Does my VPC need internet access to create the connector?
I have this error :
[Worker-02457def14578] java.net.UnknownHostException: b-1.mycluster.p225487.x2.kafka.region.amazonaws.com