By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Using bedrock: claude3 Haiku how can I enable more than 4096 output tokens?

0

The model boasts handling 200k tokens, I would like more than 4k output tokens, doesn't even work when using the API.

Topics
Machine Learning & AIGenerative AI on AWS
Tags
Machine Learning & AIAmazon Bedrock
Language
English
HMG_Tasha
asked 23 days ago308 views
1 Answer
0

Hello.

As far as I can see from the document below, I think the Max output is currently fixed at "4096".
So I don't think the output token can be higher than 4096 at the moment.
https://docs.anthropic.com/en/docs/models-overview#model-comparison

profile picture
EXPERT
Riku_Kobayashi
answered 23 days ago
profile pictureAWS
EXPERT
Didier_Durand
reviewed 23 days ago
  • Didier_Durand EXPERT
    23 days ago

    Hi HMG Tasha, to verify it, you can always use a very long string as prompt and invoke LLM with it: you will get an ValidationException error giving you the maximum authorized length in the message.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content