Total number of tokens in a session

In Amazon Bedrock, there isn't a built-in way to retrieve the total number of input and output tokens for an entire session automatically. You would need to keep track of these totals in your own code.

After each interaction with the model, Amazon Bedrock provides token usage information for that specific request. This includes the number of input tokens, output tokens, and total tokens processed. To get the cumulative total for a session, you would need to sum these values after each interaction.

Here's how you could approach this:

1. Initialize variables to store the cumulative token counts at the start of your session.

2. After each request to the model, retrieve the token usage information from the response.

3. Add the token counts from each request to your cumulative totals.

4. Continue this process for each interaction in the session.

This method allows you to maintain an ongoing tally of the total tokens used throughout the session. It's important to note that the token counts provided by Amazon Bedrock for each request are accurate and should be used for tracking usage, especially for billing purposes.

Remember that if you're using features like prompt caching, only the new tokens processed will count towards your usage and limits, not the cached tokens. This can affect how you calculate your total token usage for the session.

By manually tracking these totals, you can also implement your own session management features, such as limiting the total number of tokens per session or adjusting your prompts based on accumulated token usage.
Sources
Tokens processed in Amazon Bedrock | AWS re:Post
TokenUsage - Amazon Bedrock
Does usage of prompt caching impact utilization of the tokens-per-minute service limit? | AWS re:Post