Transfer messsages from AmazonMQ to Amazon MSK

Question

How do i transfer messages from ActiveMQ to AmazonMSK?
1) I have created a cluster in MSK
2) I have a queue in ActiveMQ existing and also have a message 
3) I have created a topic as well in MSK
I have created a source connector in MSK and this is my connector configuration 
{
  "connector.class": "io.confluent.connect.activemq.ActiveMQSourceConnector",
  "activemq.url": "ssl://b-e9f4e3c0-7f14-463d-b87a-...............-1.mq.us-.......amazonaws.com:ABCD",
  "confluent.topic.bootstrap.servers": "b-1.clicluster.ABCD.kafka.us-east-1.amazonaws.com:ABCD,b-3.clicluster.ABCD.kafka.us-east-1.amazonaws.com:ABCD,b-2.clicluster.ABCD.kafka.us-east-1.amazonaws.com:ABCD",
  "tasks.max": "5",
  "transforms.InsertField.static.value": "activemq",
  "transforms": "InsertField",
  "jms.destination.name": "air-canada.lgw.gatwick",
  "transforms.InsertField.static.field": "source",
  "confluent.license": "",
  "key.converter.schemas.enable": "false",
  "transforms.InsertField.type": "org.apache.kafka.connect.transforms.InsertField$Value",
  "confluent.topic.security.protocol": "SSL",
  "value.converter.schemas.enable": "false",
  "kafka.topic": "KafkaTest",
  "activemq.password": "ABCD",
  "value.converter": "org.apache.kafka.connect.storage.StringConverter",
  "jms.destination.type": "queue",
  "key.converter": "org.apache.kafka.connect.storage.StringConverter",
  "activemq.username": "ABCD"
} 
When i create the connector on UI, this is the error I get on the console There is an issue with the connector.
Code: KafkaConnect.ConnectorNotReady
Message: The connector is not ready. Message: Connector himaas has 5 failed tasks out of 5.

I am not able to read messages as Kafka consumer from ActiveMQ. Any help would be appreciated.

re:Post Agent · Answer

To transfer messages from Amazon MQ for ActiveMQ to Amazon MSK, you're on the right track using a Kafka Connect source connector. However, there are several issues in your current setup that need to be addressed:

1. **Connection Configuration**: Your ActiveMQ URL format appears to be correct (using SSL), but you need to ensure you're using the correct port number. For SSL connections to ActiveMQ, the port is typically 61617.

2. **Authentication**: Verify that the username and password for ActiveMQ are correct and that the user has permissions to access the specified queue.

3. **Connector Class**: The connector class you're using (`io.confluent.connect.activemq.ActiveMQSourceConnector`) is from Confluent's commercial connector. If you're not using Confluent Platform with a valid license, this could be causing issues. You might need to:
- Provide a valid Confluent license, or
- Use an open-source alternative connector for ActiveMQ

4. **SSL Configuration**: When connecting to Amazon MQ with SSL, you may need additional SSL configuration parameters such as trust store settings.

5. **Error Diagnosis**: The error "Connector has 5 failed tasks out of 5" indicates all your connector tasks are failing. You should check the connector logs for more specific error messages that can help diagnose the exact issue.

Here's a revised connector configuration approach:

- Ensure you have the correct ActiveMQ broker endpoint and port
- Verify your MSK cluster bootstrap servers are correct
- Add SSL trust store configuration if needed
- Check if you need a valid Confluent license or switch to an open-source connector
- Verify network connectivity between your MSK Connect environment and the Amazon MQ broker (they should be in the same VPC or have proper network routing)

If you're using the Confluent connector, you'll need a valid license or consider using an alternative open-source ActiveMQ connector compatible with Kafka Connect.  
**Sources**  
[Amazon MQ for ActiveMQ brokers - Amazon MQ](https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/amazon-mq-basic-elements.html)  
[Using Amazon MQ for ActiveMQ - Amazon MQ](https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/working-with-activemq.html)  
[Connecting a Java application to your Amazon MQ broker - Amazon MQ](https://docs.aws.amazon.com/amazon-mq/latest/developer-guide/amazon-mq-connecting-application.html)

Transfer messsages from AmazonMQ to Amazon MSK

Aggiungi la tua risposta

Contenuto pertinente