Intermittent high latency (4+ minutes) in Kinesis GetRecords calls - How to handle timeouts effectively?

Question

We're experiencing intermittent high latency issues with our Kinesis implementation:

**Setup**:
- Kinesis stream with 64 shards
- 40 consumers polling continuously
- Using @aws-sdk/client-kinesis v3.74.0
- Latency-sensitive application
- both poller and consumers are in eu-west region

**Issue**:
We've observed that some GetRecords calls are taking approximately 4 minutes to complete. This is happening intermittently and causing significant latency spikes in our processing.

**Questions**:
1. Why would GetRecords calls take 4 minutes to complete sometimes? Is this a known issue?
2. Is there a way to set a requestTimeout in the AWS SDK for Kinesis specifically? The documentation isn't clear about timeout configuration.
3. If we implement our own timeout mechanism, would subsequent GetRecords calls be likely to succeed, or would they face the same issue?
4.  ** What's the recommended approach to handle such scenarios in latency-sensitive applications?**

Any insights into the cause and potential solutions would be greatly appreciated

Praveen Muppala · Answer

Hello Samriddhi,

To address the latency issues, consider implementing detailed monitoring, sharding scaling, balancing, and record size monitoring. In Amazon Kinesis, each **GetRecords** request retrieves up to **1MB** of data. If your records exceed this limit, multiple requests will be needed, increasing latency.

The following resources provide insights into mitigating latency issues in Kinesis:

- [Troubleshooting Kinesis Data Stream Latency Issues](https://repost.aws/knowledge-center/kinesis-data-stream-latency-issues)  
- [AWS Documentation: Low-Latency Streaming with Kinesis](https://docs.aws.amazon.com/streams/latest/dev/kinesis-low-latency.html)  
- [Kinesis API Reference: GetRecords](https://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetRecords.html)  
- [AWS Blog: Scaling Kinesis Data Streams](https://aws.amazon.com/blogs/big-data/under-the-hood-scaling-your-kinesis-data-streams/)

Intermittent high latency (4+ minutes) in Kinesis GetRecords calls - How to handle timeouts effectively?

Add your answer

Relevant content