- Newest
- Most votes
- Most comments
Based on your description, there are several potential causes for the sudden timeouts in your Lambda function after years of stable operation:
-
Network Configuration Issues: The ETIMEDOUT error can occur when there's a connection failure because the host didn't respond within a specific period. This could indicate a network path issue between your Lambda function and DynamoDB.
-
Downstream Service Problems: DynamoDB might be experiencing intermittent issues or throttling. Even though your function is simple, if DynamoDB is not responding in a timely manner, your function will time out.
-
Cold Start Issues: If your function's invocation pattern has changed recently, you might be experiencing more cold starts than before, which could lead to timeouts during the initialization phase.
-
Resource Constraints: Your Lambda function might not have enough memory or CPU allocated to handle the current workload efficiently.
-
VPC Configuration: If your Lambda is in a VPC, check that all subnets have valid routes to NAT Gateways or Transit Gateways, and that network ACLs and security groups are properly configured.
To troubleshoot this issue, I recommend:
- Increase the function's memory allocation, which also increases proportional CPU power
- Add detailed logging to pinpoint where exactly the timeout is occurring
- Use AWS X-Ray to trace requests and identify bottlenecks
- Monitor CloudWatch metrics for your Lambda function and DynamoDB to identify any resource constraints or unusual patterns
- Check if there have been any AWS service issues in your region during the timeouts
- Implement proper error handling and retry logic in your function to handle temporary failures
Since this is a sudden change after years of stability, it's likely that something in the environment has changed, rather than an issue with your code. It could be related to AWS infrastructure changes, increased load, or changes in how DynamoDB is responding to your requests.
Sources
Troubleshoot ETIMEDOUT error in my Lambda function | AWS re:Post
Lambda Timeout Errors | AWS re:Post
Troubleshoot invocation issues in Lambda - AWS Lambda
answered 10 months ago
Why Lambda Suddenly Started Timing Out
For a Lambda → DynamoDB → API Gateway flow that has worked for years, intermittent timeouts are almost always caused by one of these:
1. Hidden DynamoDB Latency / Auto-Retries
Even small, occasional throttles or network jitter cause the AWS SDK to quietly retry in the background. This can push execution time from 300ms → 30s unexpectedly.
Trigger patterns:
- Hot partition / repeatedly writing to same key
- Increased traffic on table or GSI
- DynamoDB p95/p99 latency spikes
2. API Gateway Integration Timeout Mismatch
API Gateway has a ~29s hard limit. If Lambda finishes after API Gateway's timeout, you will see:
- Lambda logs show "finished normally"
- API Gateway returns timeout
Check the Integration Timeout on API Gateway.
3. Lambda Returns but Event Loop Is Still Busy (Node/Python)
If your code leaves:
- open sockets
- long-lived connections
- un-awaited promises / background tasks
→ Lambda appears to “complete,” but still runs until timeout.
Common when database clients are recreated inside handler or not properly reused.
4. Occasional DNS / TLS Connection Stall to DynamoDB
Rare but real; first request sometimes blocks → SDK retries → timeout.
Fixable with shorter SDK timeouts and AbortController/Config(read_timeout=...).
How to Diagnose Quickly (15–20 minutes)
| Step | Action | Interpretation |
|---|---|---|
| 1 | Invoke Lambda directly (no API Gateway) | If still slow → problem is Lambda/DynamoDB. |
| 2 | Enable X-Ray for Lambda + API GW | Look for where the time is spent: handler vs DynamoDB. |
| 3 | Add log timestamps & getRemainingTimeInMillis() before/after the DynamoDB call | Shows if the slowdown is in DynamoDB or after handler finish. |
| 4 | Check DynamoDB metrics: ThrottledRequests, p95 Write latency | Spikes = cause. |
| 5 | Check API Gateway Integration Timeout | Must be < Lambda timeout. |
Quick Fixes Based on Diagnosis
| If Diagnosis Shows… | Do This |
|---|---|
| DynamoDB call is slow | Reduce retries + add request timeouts; check hot partition patterns; increase WCU/On-Demand capacity. |
| Lambda returns but still times out | Ensure single shared DynamoDB client outside handler; ensure all Promises awaited; set callbackWaitsForEmptyEventLoop=false if Node. |
| API Gateway times out first | Increase API GW timeout or reduce Lambda timeout; keep Lambda ≤ 90% of API GW timeout. |
| Occasional network stalls | Add SDK timeouts (read_timeout, connect_timeout) and fail fast logic. |
Minimum Logging to Add Immediately
Inside handler:
console.log("start", context.getRemainingTimeInMillis()); await ddb.put(...); console.log("after ddb", context.getRemainingTimeInMillis()); return response;
Look for which log gap consumes time.
What Information Would Help Diagnose Faster
Share these five items (sanitized is fine/preferred):
-
Language + runtime (Node, Python, etc.)
-
The DynamoDB access pattern (what is your partition key?)
-
Lambda CloudWatch “REPORT” logs for:
- fast execution
- slow (timeout) execution
-
DynamoDB CloudWatch metrics (screenshots):
ThrottledRequestsSuccessfulRequestLatency p95/p99
-
Your API Gateway Integration Timeout setting
With these, I should be able to pinpoint the root cause precisely and provide more exact fix (single code change or configuration adjustment).
If any part of this explanation was helpful in diagnosing the issue, feel free to let me know — happy to help further.
answered 10 months ago
Relevant content
asked 3 years ago
asked 2 years ago
asked a year ago
- AWS OFFICIALUpdated 4 months ago
- AWS OFFICIALUpdated 2 months ago

To specify: