Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

High latency and unstable response with DeepSeek-V3.1 in Bedrock (us-west-2)

1

Hello AWS Team,

I’m experiencing high latency and unstable response times when invoking DeepSeek-V3.1 via Amazon Bedrock in the Oregon (us-west-2) region.

  • The latency is significantly higher compared to other models in the same setup.

  • The delay (time-to-first-token) has become worse and more frequent recently — sometimes exceeding several minutes before the stream starts.

  • When I run the same workload in the Tokyo (ap-northeast-1) region, the responses are much faster and more consistent.

To confirm it’s not caused by my application, I also tested directly in the Amazon Bedrock > Chat / Text playground, simply asking:

“Hi there!”

However, even this simple test did not return any response after more than 10 minutes.

Could you please clarify:

  1. Is there any known issue or capacity limitation affecting DeepSeek-V3.1 in us-west-2?

  2. Is the inference infrastructure for DeepSeek-V3.1 models deployed differently between regions?

  3. Are there any best-practice recommendations to reduce latency for this model?

Thank you in advance!

Topics
Machine Learning & AIGenerative AI on AWS
Tags
Amazon Bedrock
Language
English

asked a year ago573 views

2 Answers
0

Based on the information available, there are several factors that could be contributing to the high latency and unstable response times you're experiencing with DeepSeek-V3.1 in the us-west-2 region.

While there's no specific mention of known issues with DeepSeek-V3.1 in us-west-2, the latency differences you're observing between regions (us-west-2 vs. ap-northeast-1) could be related to varying demand levels. DeepSeek-V3.1 is supported in multiple regions including us-west-2 and ap-northeast-1 for foundation model inference.

To address your latency issues, here are some recommendations:

  1. Consider regional factors: The distance between your application and the Amazon Bedrock endpoint can impact latency. Also, Demand for the model at given point of time may vary between regions. Since you're seeing better performance in ap-northeast-1, you might want to continue using that region if it's feasible for your use case.

  2. Adjust timeout settings: If you're using the AWS Java SDK, you might need to increase the socketTimeout value (default is 30 seconds) to accommodate longer processing times. A value of 480 seconds is recommended for large context lengths.

We request you to refer sources provided below to optimize the bedrock model latency.

If the issue persists, especially with simple prompts in the Bedrock playground, it could indicate a regional capacity limitation due to demand or other issues that would require AWS support to investigate further. So, We request you to reach out to AWS Support via support case, this will help us to investigate the issue in detail and provide recommendations.

Sources
[+]https://repost.aws/knowledge-center/bedrock-improve-performance-latency [+]https://aws.amazon.com/blogs/machine-learning/optimizing-ai-responsiveness-a-practical-guide-to-amazon-bedrock-latency-optimized-inference/

answered a year ago

AWS
SUPPORT ENGINEER

reviewed a year ago

AWS
SUPPORT ENGINEER

revised a year ago

0

Yea, I can confirm that DeepSeek v3.1 models are unusable on AWS Bedrock. I've tried both US-East-2 and US-West2 both to no avail. I waited probably 30 minutes each time, ran my prompt attempts through console and terminal... nothing. Huge flop from AWS's dev side, why do these guys actually just suck with making developer tools now?

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content