Questions tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi there!
I have a lambda function triggered by a kafka topic from an MSK serverless cluster. I am adding thousands of messages at a time in very short bursts (~7000 messages in each batch) and am...
2
answers
0
votes
495
views
asked 8 months agolg...
MSK provides multiple security option like IAM Access control, Mutual TLS authentication and SASL/SCRAM and AWS recommends to use IAM Access control. But i am not clear in which usecase i need to use...
1
answers
0
votes
581
views
asked 8 months agolg...
As I am new to using AWS streaming technologies (such as: Kafka, Kinesis, Flink, etc.) it has been challenging to wrap my mind around all that is available and to establish a proper approach despite...
3
answers
0
votes
337
views
asked 8 months agolg...
Is there an option to collect the Kafka headers when consuming a Kafka topic in kinesis firehose? If not I think it would be a great improvement.
2
answers
0
votes
299
views
asked 8 months agolg...
I am working on POC to migrate from Debezium to AWS DMS for PostgreSQL. PostgreSQL as Source and Kafka as Destination (AWS MSK). In Debezium i used to see before and after image for updates, but in...
2
answers
0
votes
431
views
asked 8 months agolg...
We're using MSK for asynchronous messaging. Somoetimes some issues happen in consuming some messages and I need to know what messages are currently inside msk. Is there a GUI tool (from aws or 3rd...
Accepted AnswerAmazon Managed Streaming for Apache Kafka (Amazon MSK)
2
answers
0
votes
993
views
asked 8 months agolg...
Lets say I have a kafka cluster having three brokers running in a different subnets.
out of them only one is public and other two are private. public is attached to a NAT gateway, and private subnets...
1
answers
0
votes
364
views
asked 8 months agolg...
I am able to send the data by using below code in python and not receiving the message at consumer end. I have enabled all the authentications. I am able to send and received with kafka commands. i...
0
answers
0
votes
88
views
asked 8 months agolg...
When I try to update MSK cluster config to the latest version I get an error
```
The number of partitions per broker is above the recommended limit. Add more brokers and rearrange the partitions per...
3
answers
0
votes
1708
views
asked 8 months agolg...
I want to find the Ip of machine running Source Connector running on MSK Connect , so that It can be whitelisted on source side, otherwise I have to whitelist all Ips on source side ?
is there...
1
answers
0
votes
203
views
asked 8 months agolg...
I have a running MSK connector which talks with a Kafka running on ec2-instance in a created VPN with public subnets.
when the dynamo connector is trying to access the dynamodb table getting the below...
2
answers
0
votes
335
views
asked 8 months agolg...
When configuring an IOT rule action for MSK, I get the following error.
> 1 validation error detected: Value '[]' at 'topicRulePayload.actions.1.member.kafka.headers' failed to satisfy constraint:...
1
answers
0
votes
539
views
asked 9 months agolg...