Bedrock FM API chunk token streaming

Question

I notice when I use titan and claude with streaming. The results come out in chunk, compared to one token at a time. 
Is there a way to control the chunk size, or it is determined in real-time on your side.

Thanks!

Answer

Currently there is no way to control the length of the chunking responses. More details on what parameters you can control are available in the documentation at [InvokeModelWithResponseStream](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html) and [Inference parameters for foundation models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters.html)

Bedrock FM API chunk token streaming

Contenuto pertinente