Questions tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
We are observing that on the consumer side whenever consumer `max.poll.interval.ms` is breached while AWS Maintenance is going on then the consumer stops consuming messages from the topic. Basically,...
1
answers
0
votes
324
views
asked 6 months agolg...
Hello,
I am exploring the use of MSK Serverless for a project and would like to ensure the security of data in transit. I am aware that standard MSK clusters utilize TLS 1.2 encryption for data in...
1
answers
0
votes
186
views
asked 6 months agolg...
Is it possible to use MSK connect in AWS and connect to a Confluent Managed Broker for the connect cluster?
Essentially set the CONNECT_BOOTSTRAP_SERVERS to broker from a VPC peered Confluent Kafka...
0
answers
0
votes
99
views
asked 6 months agolg...
We have a MSK cluster with prometheus monitoring enabled. We are monitoring the consumer lag on different topics on a grafana dashboard. The metrics are collected by a prometheus docker every minute...
1
answers
0
votes
916
views
asked 6 months agolg...
Hello, i am having some trouble for a bit to setup event patterns for EKS and MSK. I want a more detailed monitoring but have been trying to at least get the cluster state change as an event but...
1
answers
0
votes
579
views
asked 6 months agolg...
I created an msk cluster with Unauthenticated and IAM authentication but when I try to contact the brokers from my spring boot app it fails. I entered the bootstrap-server endpoints in the...
0
answers
0
votes
66
views
asked 7 months agolg...
I've upgraded AWS MSK service from 3.4.0 to 3.5.1, without changing any other configurations. After upgrade I see a lot of DEBUG message in CloudWatch Logs for AWS MSK Partition:
```
[2023-10-10...
1
answers
0
votes
101
views
asked 7 months agolg...
Hi there!
I have a lambda function triggered by a kafka topic from an MSK serverless cluster. I am adding thousands of messages at a time in very short bursts (~7000 messages in each batch) and am...
2
answers
0
votes
448
views
asked 7 months agolg...
MSK provides multiple security option like IAM Access control, Mutual TLS authentication and SASL/SCRAM and AWS recommends to use IAM Access control. But i am not clear in which usecase i need to use...
1
answers
0
votes
511
views
asked 7 months agolg...
As I am new to using AWS streaming technologies (such as: Kafka, Kinesis, Flink, etc.) it has been challenging to wrap my mind around all that is available and to establish a proper approach despite...
3
answers
0
votes
312
views
asked 7 months agolg...
Is there an option to collect the Kafka headers when consuming a Kafka topic in kinesis firehose? If not I think it would be a great improvement.
2
answers
0
votes
264
views
asked 7 months agolg...
I am working on POC to migrate from Debezium to AWS DMS for PostgreSQL. PostgreSQL as Source and Kafka as Destination (AWS MSK). In Debezium i used to see before and after image for updates, but in...
2
answers
0
votes
388
views
asked 7 months agolg...