MSK Connect for CDC on My SQL aurora

Question

Hi Team,

I am trying to Debezium and MSK for CDC.

I have MySql Auraora serverless database 
I'm trying to connect the all (MySQL aurora serverless Database, Debezium, my MSK Cluster) for data change capture

I'm trying to follow this blog :

[Introducing Amazon MSK Connect](https://aws.amazon.com/blogs/aws/introducing-amazon-msk-connect-stream-data-to-and-from-your-apache-kafka-clusters-using-managed-connectors/)

in this blog, we use connector capacity type = auto Scaled

but in Amazon Managed Streaming for Apache Kafka, Developer Guide, it says :

> Important
The Debezium MySQL connector plugin supports only one task and does not work with autoscaled capacity mode for Amazon MSK Connect. You should instead use provisioned capacity mode and set worker count equal to one in your connector configuration. To learn more about the capacity modes for MSK Connect, see Connector capacity.

[Developer Guide](https://docs.aws.amazon.com/msk/latest/developerguide/mkc-debeziumsource-connector-example.html)

at this point, I enable to successfully create an MSK connector (MSK Connect)

not sure which approach is the right one

1 - are the MSK and MSK connectors compatible with MySQL aurora **serverless** ?

2 - is there any other article that includes all the steps/constraints on how to create an MSK cluster and MSK connector with Auror and Debezium?

3 - does the MSK connector need internet access from within the VPC to be successfully created?

now I have this error when trying to create the connector :

```
Code: InvalidInput.InvalidConnectorConfiguration
Message: The connector configuration is invalid. Message: Failed to find any class that implements Connector and which name matches io.debezium.connector.mysql.MySqlConnector, available connectors are:.........
```

```
[Worker-07edsd2458791sds1545] org.apache.kafka.connect.errors.ConnectException: Failed to find any class that implements Connector and which name matches io.debezium.connector.mysql.MySqlConnector, available connectors are: PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSinkConnector, name='org.apache.kafka.connect.file.FileStreamSinkConnector', version='2.7.1', encodedVersion=2.7.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.file.FileStreamSourceConnector, name='org.apache.kafka.connect.file.FileStreamSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorCheckpointConnector, name='org.apache.kafka.connect.mirror.MirrorCheckpointConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorHeartbeatConnector, name='org.apache.kafka.connect.mirror.MirrorHeartbeatConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.mirror.MirrorSourceConnector, name='org.apache.kafka.connect.mirror.MirrorSourceConnector', version='1', encodedVersion=1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockConnector, name='org.apache.kafka.connect.tools.MockConnector', version='2.7.1', encodedVersion=2.7.1, type=connector, typeName='connector', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSinkConnector, name='org.apache.kafka.connect.tools.MockSinkConnector', version='2.7.1', encodedVersion=2.7.1, type=sink, typeName='sink', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.MockSourceConnector, name='org.apache.kafka.connect.tools.MockSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.SchemaSourceConnector, name='org.apache.kafka.connect.tools.SchemaSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSinkConnector, name='org.apache.kafka.connect.tools.VerifiableSinkConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}, PluginDesc{klass=class org.apache.kafka.connect.tools.VerifiableSourceConnector, name='org.apache.kafka.connect.tools.VerifiableSourceConnector', version='2.7.1', encodedVersion=2.7.1, type=source, typeName='source', location='classpath'}
```

Note: my VPC is private only and doesn't have access to the internet

Thank you for your Help!

Answer

This error is not related to auto-scale of MSK Connect. Aurora support different engines, so for some of them, maybe autoscale will be relevant, but for your case, just use "provisioned" and choose 1 worker.
As for the error, in most cases it means that your Custom Plugin is packaged incorrectly. Please ensure you executed these steps:

```
$ tar xzf debezium-connector-mysql-1.6.1.Final-plugin.tar.gz
$ cd debezium-connector-mysql
$ zip -9 ../debezium-connector-mysql-1.6.1.zip *
$ cd ..
```

MSK Connect for CDC on My SQL aurora

관련 콘텐츠