Questions tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Amazon MSK provides a metric named SumOffsetLag, described in the [documentation](https://docs.aws.amazon.com/msk/latest/developerguide/metrics-details.html) as
> The aggregated offset lag for all...
1
answers
0
votes
375
views
asked 5 months agolg...
In our architecture we have data from AWS MSK being ingested into AWS Redshift using the Amazon Redshift Streaming Ingestion feature. In this approach, we define an external schema where we have the...
0
answers
0
votes
114
views
asked 6 months agolg...
When I am using both the WHL file which contains all the packages and the glue connection which contains the broker of Kafka, it is giving an below mentioned error.
Error:
Glue ETL Marketplace -...
1
answers
0
votes
245
views
asked 6 months agolg...
We are observing that on the consumer side whenever consumer `max.poll.interval.ms` is breached while AWS Maintenance is going on then the consumer stops consuming messages from the topic. Basically,...
1
answers
0
votes
337
views
asked 6 months agolg...
Hello,
I am exploring the use of MSK Serverless for a project and would like to ensure the security of data in transit. I am aware that standard MSK clusters utilize TLS 1.2 encryption for data in...
1
answers
0
votes
194
views
asked 6 months agolg...
Is it possible to use MSK connect in AWS and connect to a Confluent Managed Broker for the connect cluster?
Essentially set the CONNECT_BOOTSTRAP_SERVERS to broker from a VPC peered Confluent Kafka...
0
answers
0
votes
102
views
asked 6 months agolg...
We have a MSK cluster with prometheus monitoring enabled. We are monitoring the consumer lag on different topics on a grafana dashboard. The metrics are collected by a prometheus docker every minute...
1
answers
0
votes
942
views
asked 6 months agolg...
Hello, i am having some trouble for a bit to setup event patterns for EKS and MSK. I want a more detailed monitoring but have been trying to at least get the cluster state change as an event but...
1
answers
0
votes
599
views
asked 7 months agolg...
I created an msk cluster with Unauthenticated and IAM authentication but when I try to contact the brokers from my spring boot app it fails. I entered the bootstrap-server endpoints in the...
0
answers
0
votes
68
views
asked 7 months agolg...
I've upgraded AWS MSK service from 3.4.0 to 3.5.1, without changing any other configurations. After upgrade I see a lot of DEBUG message in CloudWatch Logs for AWS MSK Partition:
```
[2023-10-10...
1
answers
0
votes
109
views
asked 7 months agolg...
Hi there!
I have a lambda function triggered by a kafka topic from an MSK serverless cluster. I am adding thousands of messages at a time in very short bursts (~7000 messages in each batch) and am...
2
answers
0
votes
464
views
asked 7 months agolg...
MSK provides multiple security option like IAM Access control, Mutual TLS authentication and SASL/SCRAM and AWS recommends to use IAM Access control. But i am not clear in which usecase i need to use...
1
answers
0
votes
532
views
asked 7 months agolg...