Max plugins per MSK connector = 1, but plugins is an array?
Hi there,
I'm attempting to deploy an MSK Connector through the AWS CLI. This connector needs to refer to 2 plugins, 1 which is the Elasticsearch sink connector and another which is the Avro converter as the data in the topic is Avro-encoded (and the Elasticsearch zip archive doesn't include the Confluent Avro converter)
I have registered the 2x zip packages in MSK Connect Plugins fine, but when I come to create the connector & refer to both plugins, I get the following error:
An error occurred (BadRequestException) when calling the CreateConnector operation: Invalid parameter plugins: The number of plugins per connector must be 1.
Which is misleading as the "plugins" section of the CLI create-connector command, refers to an array of plugins for a connector:
https://docs.aws.amazon.com/cli/latest/reference/kafkaconnect/create-connector.html (see --plugins)
Am I missing something, should we be able to pass multiple plugins in when creating an MSK connector?
Thanks
This is not an answer to your question but just a note. You can only select 1 plugin in the console (GUI) when creating a connector so even though the API allows an array, perhaps, it's a limitation right now that only 1 plugin can be used.
I have not found a way of deleting a custom plugin. Have you?
At present, only one custom plugin is allowed for each connector, as you see via the console.
The API does provide an array of plugins, which exists to allow for possible future changes, but can only contain one element today.
Thanks.
Yeah same here - I've not found a way of deleting a custom plugin, either through the Console or the CLI.
Do we know if this functionality is coming soon? (@aws)
Ok thanks for the update. It might be worth a small update to the docs to mention the limit of 1 that is currently in place?
Relevant questions
MSK Connect - Exposing Connector Specific Metrics
asked 5 months agoIs there a way to update connector configuration using MSK-connect API?
asked 5 months agoMSK Connect Debezium Connector using AWS Glue Schema Registry for AVRO serialization
asked 5 months agoGlue to Snowflake connector, is it mandatory?
Accepted Answerasked 4 months agoCloudwatch Connector V2 Upgrade Deployment Fails
Accepted Answerasked 3 years agoCopying data from IBM MQ to Amazon MSK
asked 8 months agoMax plugins per MSK connector = 1, but plugins is an array?
asked 8 months agoMSK connect connector deployment
asked 12 days agoMSK Connect - Failed to create using in-built AWSServiceRoleForKafkaConnect role
asked 5 months agoHow to reset offset of a consumer group created through MSK connect
asked 2 months ago