AWS Bedrock - Anthropic Claude V2 Max tokens

0

When invoking the model ("modelId": "anthropic.claude-v2") using boto3 version 1.28.59 with "max_tokens_to_sample" higher than 8191. I get an error: botocore.errorfactory.ValidationException: An error occurred (ValidationException) when calling the InvokeModel operation: Malformed input request: 60000 is not less or equal to 8191, please reformat your input and try again.

One one the reasons I am looking into Claude 2 is its 100K tokens limit. Do I need to request getting 100K?

Thank you in advance! Max

Max M
已提問 8 個月前檢視次數 4951 次
2 個答案
2
已接受的答案

My bad. The documentation (https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html#model-parameters-claude) clearly states that the longest output is 8K. The context size can be 100K, but the length of the response cannot exceed 8K. My use case was to decompose a document into Q&As, so my output is about the same size as the input and I will need to chunk documents into smaller pieces.

Max

Max M
已回答 8 個月前
profile picture
專家
已審閱 23 天前
profile picture
專家
已審閱 3 個月前
0

The input/output sizes can change and https://docs.anthropic.com/claude/reference/input-and-output-sizes contains the latest info as of this writing :)

AWS
已回答 4 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南