Questions tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
In our architecture we have data from AWS MSK being ingested into AWS Redshift using the Amazon Redshift Streaming Ingestion feature. In this approach, we define an external schema where we have the...
0
answers
0
votes
113
views
asked 6 months agolg...
When I am using both the WHL file which contains all the packages and the glue connection which contains the broker of Kafka, it is giving an below mentioned error.
Error:
Glue ETL Marketplace -...
1
answers
0
votes
241
views
asked 6 months agolg...
We are observing that on the consumer side whenever consumer `max.poll.interval.ms` is breached while AWS Maintenance is going on then the consumer stops consuming messages from the topic. Basically,...
1
answers
0
votes
331
views
asked 6 months agolg...
Hello,
I am exploring the use of MSK Serverless for a project and would like to ensure the security of data in transit. I am aware that standard MSK clusters utilize TLS 1.2 encryption for data in...
1
answers
0
votes
188
views
asked 6 months agolg...
Is it possible to use MSK connect in AWS and connect to a Confluent Managed Broker for the connect cluster?
Essentially set the CONNECT_BOOTSTRAP_SERVERS to broker from a VPC peered Confluent Kafka...
0
answers
0
votes
100
views
asked 6 months agolg...
We have a MSK cluster with prometheus monitoring enabled. We are monitoring the consumer lag on different topics on a grafana dashboard. The metrics are collected by a prometheus docker every minute...
1
answers
0
votes
921
views
asked 6 months agolg...
Hello, i am having some trouble for a bit to setup event patterns for EKS and MSK. I want a more detailed monitoring but have been trying to at least get the cluster state change as an event but...
1
answers
0
votes
585
views
asked 6 months agolg...
I created an msk cluster with Unauthenticated and IAM authentication but when I try to contact the brokers from my spring boot app it fails. I entered the bootstrap-server endpoints in the...
0
answers
0
votes
66
views
asked 7 months agolg...
I've upgraded AWS MSK service from 3.4.0 to 3.5.1, without changing any other configurations. After upgrade I see a lot of DEBUG message in CloudWatch Logs for AWS MSK Partition:
```
[2023-10-10...
1
answers
0
votes
103
views
asked 7 months agolg...
Hi there!
I have a lambda function triggered by a kafka topic from an MSK serverless cluster. I am adding thousands of messages at a time in very short bursts (~7000 messages in each batch) and am...
2
answers
0
votes
452
views
asked 7 months agolg...
MSK provides multiple security option like IAM Access control, Mutual TLS authentication and SASL/SCRAM and AWS recommends to use IAM Access control. But i am not clear in which usecase i need to use...
1
answers
0
votes
514
views
asked 7 months agolg...
As I am new to using AWS streaming technologies (such as: Kafka, Kinesis, Flink, etc.) it has been challenging to wrap my mind around all that is available and to establish a proper approach despite...
3
answers
0
votes
315
views
asked 7 months agolg...