Skip to content

Bedrock prompt cache models

0

Hello,

Which of the Amazon foundation models support prompt cache at this moment ?

I found a contradiction about Amazon Titan prompt cache. According to this code Amazon Titan can do cache: var modelId = "amazon.titan-text-premier-v1:0";. According to this document only Amazon Nova can do cache.

Thank you,

asked 2 years ago1.1K views

2 Answers
1

Hi,

Page https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html give the full list of supported model:

Nova comes at the top of this table but use scroll bar on the right to see the other models.

Excerpt below:

Enter image description here

Best,

Didier

EXPERT

answered 2 years ago

EXPERT

reviewed 2 years ago

  • Thank you for your answer, I'm sorry for the misunderstanding, when I said only Nova, I was talking only about the Amazon foundation models. Titan is not in this list, and from Amazon is only Nova.

1

Based on the most recent information available, the Amazon foundation models that currently support prompt caching are:

  1. Amazon Nova Micro
  2. Amazon Nova Lite
  3. Amazon Nova Pro

It's important to note that there appears to be a discrepancy in the information you've found. While the code example you referenced suggests that Amazon Titan Text Premier might support prompt caching, the official documentation indicates that only the Amazon Nova models currently have this capability.

As for the Anthropic models, Claude 3.5 Haiku and Claude 3.5 Sonnet v2 also support prompt caching.

Given the conflicting information, it's best to rely on the most recent official documentation. The support for prompt caching may be evolving, and features can change over time. If you need the most up-to-date and accurate information, I recommend checking the latest Amazon Bedrock documentation or contacting AWS support directly.

It's worth noting that prompt caching is a relatively new feature in Amazon Bedrock, and its availability may be expanding to more models over time. The feature is designed to significantly reduce costs and latency for supported models by caching frequently used prompts across multiple API calls.
Sources
Amazon Bedrock announces preview of prompt caching - AWS
Reduce costs and latency with Amazon Bedrock Intelligent Prompt Routing and prompt caching (preview) | AWS News Blog
Prompt Caching - Amazon Bedrock

answered 2 years ago

EXPERT

reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.