- Newest
- Most votes
- Most comments
Hello,
Now AWS Glue supports to process data from Kafka using IAM authentication menthod. For this you need to create a connection glue and select IAM authentication menthod.
Currently, AWS Glue does not integrate with Amazon MSK clusters configured to use AWS IAM access control.
You can connect to MSK clusters using IAM Access Control from other AWS services, such as Amazon EMR and Amazon Kinesis Data Analytics. In these cases, you configure the Kafka client to use MSK IAM authentication.
Also, AWS Lambda has its own integration with MSK clusters using IAM Access Control, through an Event Source Mapping. By taking this approach, the Kafka client is configured for you.
Hello,
I have published a blog which discuss about how to integrate AWS Glue ETL job with MSK serverless using IAM authentication. Hope it helps.
Relevant content
- Accepted Answerasked a year ago
- asked 2 years ago
- asked 6 months ago
- How do I troubleshoot common issues when using my Amazon MSK cluster with SASL/SCRAM authentication?
AWS OFFICIALUpdated 2 years ago - AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
Hello
No unfortunately MSK connector with IAM authentication for Glue is not working for me. Though in the console it appears but when I choose that option it does not work. Seems a bug in Glue console. Checked in the AWS Documentation. Could not find any mention of MSK connector with IAM authentication for Glue