2 Respostas
- Mais recentes
- Mais votos
- Mais comentários
2
Hi,
Do you set the value of max tokens to its maximum (=2048) when calling via API? The default value is lower (=300 if I recall correctly)
Best
Didier
1
Hi Didier,
Thank you for the response. I am not noticing any cutoffs anymore. I wish I had written down wordcounts so that we did have a decent approximation of the tokens because you bring up a great point. I don't believe we were hitting the limits (I had some other examples which fall well below the 300 token mark) but the good news is that Playground is working well based on all my tests today and in the latter example my code was not getting the last streaming chunk that contained the stop sequence which is also fixed.
Thanks again.
respondido há 8 meses
Conteúdo relevante
- AWS OFICIALAtualizada há 3 meses
- AWS OFICIALAtualizada há 2 anos
- AWS OFICIALAtualizada há 8 meses
If it's helpful, a good rule of thumb is that 3 words is equal to about 4 tokens. So formulas you can use are...
token count = word count * 1.33 world count = token count * 0.75
Hello there!
Im facing the same issue with the AI21Labs Jurassic-2 Ultra model. Can you help me out? How do you solve this problem and how do you use the stop sequences?
Best, Hasan.