AWS Builder Center: Learn, Build and Connect with builders in the AWS community

AWS Builder Center is the official home for builders on AWS. Share and read what others are working on, follow people who inspire you, explore training and workshops, and find tools to support what you're building.

Articles tagged with Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Amazon MSK makes it easy to ingest and process streaming data in real time with fully managed Apache Kafka.

Content language: English

Filter articles

Select tags to filter

Sort by

Sort by most recent

Browse through articles or filter your results using the tools displayed.

24 results

"Topic not present in metadata after 60000 ms" on Amazon MSK...

Hyangmok BaekSUPPORT ENGINEER

published 7 days ago0 votes41 views

When a Kafka producer fails with TimeoutException: Topic ... not present in metadata after 60000 ms, most people check whether the topic exists or whether the network is broken. But this error also oc...

ARTICLE

Authenticate to Amazon MSK with Google OAuth/OIDC via AWS We...

Masudur Rahaman SayemEXPERT

published 20 days ago0 votes59 views

This post shows how to authenticate a Kafka client to Amazon MSK using a Google OAuth/OIDC identity, with no long-lived AWS keys.

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

How much do Kafka transactions cost you on Amazon MSK Expres...

Hyangmok BaekSUPPORT ENGINEER

published a month ago0 votes71 views

Cluster sizing for MSK Express brokers is usually calculated from the official per-broker throughput quotas. But workloads that use Kafka transactions hit a different bottleneck - commit processing - ...

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Five operational differences to check before adopting Amazon...

Hyangmok BaekSUPPORT ENGINEER

published a month ago0 votes105 views

Express brokers offer up to 3x throughput per broker, 20x faster scaling, and no storage management. But if you operate them the way you operate Standard brokers, you will run into unexpected behavior...

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Why adding consumers won't fix your consumer lag on Amazon M...

Hyangmok BaekSUPPORT ENGINEER

published a month ago1 votes102 views

When consumer lag spikes, the usual reaction is to add consumers or partitions. But when lag piles up on a single partition, scaling out does not help and can even make things worse: the cause is not ...

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Why doesn't kafka-delete-records reduce consumer lag on Amaz...

Hyangmok BaekSUPPORT ENGINEER

published a month ago1 votes119 views

When a consumer gets stuck on a problematic message, a common reaction is to delete that message from the topic. However, after running kafka-delete-records.sh, the consumer lag number stays exactly t...

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Why raising socket.receive.buffer.bytes on Amazon MSK does n...

Jeongho KimSUPPORT ENGINEER

published a month ago1 votes78 views

You raise `socket.receive.buffer.bytes` on your Amazon MSK brokers to 1 MiB or 2 MiB, but the TCP send window your producers observe and their throughput stay the same. This article explains why the b...

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

After receiving an Amazon MSK security patching notification...

Jeongho KimSUPPORT ENGINEER

published a month ago0 votes81 views

For operators who, after receiving an Amazon MSK security patching notification, want to know whether the patch will affect their cluster and whether the schedule can be moved or skipped. It covers ho...

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

What to do when you receive the AWS_KAFKA_HIGH_RISK_CONFIG_R...

Jeongho KimSUPPORT ENGINEER

published 2 months ago0 votes189 views

This article explains the AWS Health event `AWS_KAFKA_HIGH_RISK_CONFIG_RF_EQUALS_MINISR` for Amazon MSK and how to respond. It is a proactive notification that your cluster has topics where the replic...

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Understanding the MSK Connect Service Execution Role: When I...

Jeongho KimSUPPORT ENGINEER

published 4 months ago0 votes140 views

This article explains when the MSK Connect service execution role is actually used, why worker log delivery does not use this role, and how to accurately determine whether the role is in use.

ARTICLE

Amazon Managed Streaming for Apache Kafka (Amazon MSK)AWS Identity and Access Management

How do I resolve "context deadline exceeded" timeout errors ...

Jeongho KimSUPPORT ENGINEER

published 5 months ago0 votes293 views

When monitoring Amazon MSK clusters using JMX exporter with Prometheus, you may encounter "context deadline exceeded" timeout errors on port 11001. This article explains how to identify and resolve th...

ARTICLE

Analytics Amazon Managed Streaming for Apache Kafka (Amazon MSK)Monitoring & Logging

Best Practices for AWS Glue Schema Registry in Large-Scale A...

Masudur Rahaman SayemEXPERT

published 7 months ago1 votes403 views

Production streaming applications across Apache Flink, Spark Streaming, Kafka Streams, Kinesis Client Library and AWS Lambda might experience throttling on AWS Glue Schema Registry's GetSchemaVersion ...

ARTICLE

Amazon Managed Service for Apache Flink AWS Lambda AWS Glue Amazon Kinesis Data Streams Amazon Managed Streaming for Apache Kafka (Amazon MSK)

1
2
Page size
12 / page