スキップしてコンテンツを表示

Bedrock prompt cache models

0

Hello,

Which of the Amazon foundation models support prompt cache at this moment ?

I found a contradiction about Amazon Titan prompt cache. According to this code Amazon Titan can do cache: var modelId = "amazon.titan-text-premier-v1:0";. According to this document only Amazon Nova can do cache.

Thank you,

質問済み 2年前1121ビュー

2回答
1

Hi,

Page https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html give the full list of supported model:

Nova comes at the top of this table but use scroll bar on the right to see the other models.

Excerpt below:

Enter image description here

Best,

Didier

エキスパート

回答済み 2年前

エキスパート

レビュー済み 2年前

  • Thank you for your answer, I'm sorry for the misunderstanding, when I said only Nova, I was talking only about the Amazon foundation models. Titan is not in this list, and from Amazon is only Nova.

1

Based on the most recent information available, the Amazon foundation models that currently support prompt caching are:

  1. Amazon Nova Micro
  2. Amazon Nova Lite
  3. Amazon Nova Pro

It's important to note that there appears to be a discrepancy in the information you've found. While the code example you referenced suggests that Amazon Titan Text Premier might support prompt caching, the official documentation indicates that only the Amazon Nova models currently have this capability.

As for the Anthropic models, Claude 3.5 Haiku and Claude 3.5 Sonnet v2 also support prompt caching.

Given the conflicting information, it's best to rely on the most recent official documentation. The support for prompt caching may be evolving, and features can change over time. If you need the most up-to-date and accurate information, I recommend checking the latest Amazon Bedrock documentation or contacting AWS support directly.

It's worth noting that prompt caching is a relatively new feature in Amazon Bedrock, and its availability may be expanding to more models over time. The feature is designed to significantly reduce costs and latency for supported models by caching frequently used prompts across multiple API calls.
Sources
Amazon Bedrock announces preview of prompt caching - AWS
Reduce costs and latency with Amazon Bedrock Intelligent Prompt Routing and prompt caching (preview) | AWS News Blog
Prompt Caching - Amazon Bedrock

回答済み 2年前

エキスパート

レビュー済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

関連するコンテンツ