Questions tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I've upgraded AWS MSK service from 3.4.0 to 3.5.1, without changing any other configurations. After upgrade I see a lot of DEBUG message in CloudWatch Logs for AWS MSK Partition:
```
[2023-10-10...
1
answers
0
votes
134
views
asked 8 months agolg...
Hi there!
I have a lambda function triggered by a kafka topic from an MSK serverless cluster. I am adding thousands of messages at a time in very short bursts (~7000 messages in each batch) and am...
2
answers
0
votes
511
views
asked 8 months agolg...
MSK provides multiple security option like IAM Access control, Mutual TLS authentication and SASL/SCRAM and AWS recommends to use IAM Access control. But i am not clear in which usecase i need to use...
1
answers
0
votes
615
views
asked 8 months agolg...
As I am new to using AWS streaming technologies (such as: Kafka, Kinesis, Flink, etc.) it has been challenging to wrap my mind around all that is available and to establish a proper approach despite...
3
answers
0
votes
346
views
asked 8 months agolg...
Is there an option to collect the Kafka headers when consuming a Kafka topic in kinesis firehose? If not I think it would be a great improvement.
2
answers
0
votes
310
views
asked 8 months agolg...
I am working on POC to migrate from Debezium to AWS DMS for PostgreSQL. PostgreSQL as Source and Kafka as Destination (AWS MSK). In Debezium i used to see before and after image for updates, but in...
2
answers
0
votes
451
views
asked 8 months agolg...
We're using MSK for asynchronous messaging. Somoetimes some issues happen in consuming some messages and I need to know what messages are currently inside msk. Is there a GUI tool (from aws or 3rd...
Accepted AnswerAmazon Managed Streaming for Apache Kafka (Amazon MSK)
2
answers
0
votes
1043
views
asked 8 months agolg...
Lets say I have a kafka cluster having three brokers running in a different subnets.
out of them only one is public and other two are private. public is attached to a NAT gateway, and private subnets...
1
answers
0
votes
379
views
asked 8 months agolg...
I am able to send the data by using below code in python and not receiving the message at consumer end. I have enabled all the authentications. I am able to send and received with kafka commands. i...
0
answers
0
votes
89
views
asked 9 months agolg...
When I try to update MSK cluster config to the latest version I get an error
```
The number of partitions per broker is above the recommended limit. Add more brokers and rearrange the partitions per...
3
answers
0
votes
1902
views
asked 9 months agolg...
I want to find the Ip of machine running Source Connector running on MSK Connect , so that It can be whitelisted on source side, otherwise I have to whitelist all Ips on source side ?
is there...
1
answers
0
votes
209
views
asked 9 months agolg...
I have a running MSK connector which talks with a Kafka running on ec2-instance in a created VPN with public subnets.
when the dynamo connector is trying to access the dynamodb table getting the below...
2
answers
0
votes
346
views
asked 9 months agolg...