Questions tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
**What I'm trying to do:**
I want to run the Python file in EC2 to send data to MSK, and I want to save the data in MSK in Parquet format in S3 through the Sink connector.
**Problem:**
An error...
0
answers
0
votes
348
views
asked 6 months agolg...
I am running into an issue with my MSK cluster's broker prometheus metrics. The JMX metrics endpoint constantly returns 429 (too many requests) errors when prometheus attempts to scrape the /metrics...
1
answers
1
votes
540
views
asked 6 months agolg...
![Enter image description here](/media/postImages/original/IMsCdSahucQjqNNIkdr0VPDw)
![Enter image description here](/media/postImages/original/IM8yhsUwYIQi6-xqogiklOuw)
Hi
AWS Kafka not working...
0
answers
0
votes
338
views
asked 6 months agolg...
I want to add Confluent Cloud Apache Kafka as a Data source in AWS ETL job to read data stream from Kafka topic.
I created a cluster, topic, AWS SQS source connector and AWS S3 sink connector in...
1
answers
0
votes
363
views
asked 6 months agolg...
I'm using an IoT rule to publish IoT data with Managed Service Kafka (MSK). However, I frequently encounter the following error, causing messages to be missing.
It generally works well, but what...
0
answers
0
votes
352
views
asked 6 months agolg...
Amazon MSK provides a metric named SumOffsetLag, described in the [documentation](https://docs.aws.amazon.com/msk/latest/developerguide/metrics-details.html) as
> The aggregated offset lag for all...
1
answers
0
votes
501
views
asked 6 months agolg...
In our architecture we have data from AWS MSK being ingested into AWS Redshift using the Amazon Redshift Streaming Ingestion feature. In this approach, we define an external schema where we have the...
0
answers
0
votes
118
views
asked 7 months agolg...
When I am using both the WHL file which contains all the packages and the glue connection which contains the broker of Kafka, it is giving an below mentioned error.
Error:
Glue ETL Marketplace -...
1
answers
0
votes
270
views
asked 7 months agolg...
We are observing that on the consumer side whenever consumer `max.poll.interval.ms` is breached while AWS Maintenance is going on then the consumer stops consuming messages from the topic. Basically,...
1
answers
0
votes
404
views
asked 7 months agolg...
Hello,
I am exploring the use of MSK Serverless for a project and would like to ensure the security of data in transit. I am aware that standard MSK clusters utilize TLS 1.2 encryption for data in...
1
answers
0
votes
233
views
asked 7 months agolg...
Is it possible to use MSK connect in AWS and connect to a Confluent Managed Broker for the connect cluster?
Essentially set the CONNECT_BOOTSTRAP_SERVERS to broker from a VPC peered Confluent Kafka...
0
answers
0
votes
111
views
asked 8 months agolg...
We have a MSK cluster with prometheus monitoring enabled. We are monitoring the consumer lag on different topics on a grafana dashboard. The metrics are collected by a prometheus docker every minute...
1
answers
0
votes
1026
views
asked 8 months agolg...