What comprises the token count in Invoke_model API

The token count in the InvokeModel API for Amazon Bedrock includes both input and output tokens, but it's important to understand how these are calculated and what they represent.

For the input tokens, the count typically includes the entire prompt or query sent to the model. This means it encompasses the user's input as well as any context or instructions provided to the model.

For the output tokens, the count represents the tokens generated by the model in its response.

The InvokeModel response structure includes fields that provide this information:

1. inputTextTokenCount: This field shows the number of tokens in the prompt or input text.
2. results: This contains an array with one item, which includes a tokenCount field representing the number of tokens in the model's response.

It's important to note that these token counts are specific to the model's processing of your direct input and its generated output. They do not include additional processing steps like guardrails calls or additional agent prompts.

If you're using more complex workflows involving agents or multiple API calls, each individual call to InvokeModel will have its own token counts. The cumulative token usage for a conversation or multi-step process would be the sum of these individual counts.

For accurate tracking of token usage, especially for billing purposes, you should add up the inputTextTokenCount and tokenCount (from the results) for each InvokeModel call in your workflow. This will give you the total tokens processed by the Bedrock model.

Remember that token counts can vary between different models and may be handled differently in various API calls within the Bedrock service. Always refer to the specific documentation for the model and API you're using for the most accurate information on token counting and usage.

Sources
Amazon Titan Text models - Amazon Bedrock
Tokens processed in Amazon Bedrock | AWS re:Post