The number of output and input tokens in Bedrock responses

For the retrieve_and_generate API in Amazon Bedrock, there isn't a direct way to access the number of input and output tokens in the response structure, unlike the invoke_agent API where you can parse the EventCompletion to get this information.

The retrieve_and_generate API doesn't provide explicit token count information in its response. This means that, unfortunately, it's not possible to directly obtain the number of input and output tokens from the retrieve_and_generate API response.

However, it's worth noting that token usage information is generally available for other Bedrock operations. For example, when using the InvokeModel or Converse APIs, you can typically find token usage details in the response. These details usually include the number of input tokens, output tokens, and total tokens used for the request.

If token count information is crucial for your use case with retrieve_and_generate, you might need to consider alternative approaches or use other APIs that provide this information explicitly. You could also reach out to AWS support for more specific guidance on this particular API's behavior regarding token counts.

Sources
TokenUsage - Amazon Bedrock
Generate responses in the console using playgrounds - Amazon Bedrock
Chat with a model in the Amazon Bedrock in SageMaker Unified Studio chat playground - Amazon SageMaker Unified Studio