Copying data from IBM MQ to Amazon MSK

0

I am trying to use the Kafka Connect source connector for copying data from IBM MQ into Amazon MSK. I was able to create the custom plugin in MSK, but when creating the connector it is transitioning to failed state.
The custom plugin i build from. https://github.com/ibm-messaging/kafka-connect-mq-source
For the connector configuration i am using
connector.class=com.ibm.eventstreams.connect.mqsource.MQSourceConnector
mq.connection.name.list=<host>(1414)
mq.channel=<channel name>
tasks.max=1
mq.queue.manager=<name>
mq.queue=<queue name>
mq.channel.name=<channel name>
kafka.topic=<topic name>
value.converter=org.apache.kafka.connect.storage.StringConverter
key.converter=org.apache.kafka.connect.storage.StringConverter
mq.connection.mode=client
The configuration works and i am able to copy messages from IBM MQ to Apache Kafka installed on EC2. It is not working from IBM MQ to Amazon MSK. Are there any limitations on what source connectors you use with Amazon MSK?

질문됨 3년 전1060회 조회
2개 답변
0

You can use most connectors with MSK Connect, as long as they do not need to actively listen themselves. In the case of the MQ connector, this should work for you. The issue is most likely connected to the networking setup and the ability of the connector to find your MQ instances. Since networking can be very dependent on your environment it is difficult to be specific, and you may be able to get more details by sharing the logs and specifics with the AWS support team. However, some common challenges include name resolution, and making sure your host names are resolvable from the connector's perspective.

답변함 3년 전
0

Thank you for your response. I don't believe it is a networking issue as i tried the reachability analyzer from the VPC end point where the IBM MQ is installed and the AWS Kafka Connect ENI and it is reachable. I also enabled the cloudwatch log monitoring and don't see any errors in the logs. So i am not sure if there is an issue with the connector configuration and if the properties specified are correct as i don't see I also don't see any message saying connectivity is established.

답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠