Cohere R on Bedrock does not include "meta" in response

Question

When I use Cohere R/Cohere R+ with the `invoke` endpoint on Bedrock, the response object includes the properties `response_id`, `text`, `generation_id`, `chat_history`, and `finish_reason`, but there is no `meta` or `token_count` field as mentioned in the [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-cohere-command-r-plus.html), which I would like to have in order to get the `input_tokens` and `output_tokens` values. There are no errors in my request as far as I can tell. (`finish_reason` is `COMPLETE` and there is a proper response in `text`.) Is there something that needs to be included in the request in order to get the `meta` field?

Didier Durand · Accepted Answer

Hi.

For Bedrock models, those fields are http headers, not in the response body itself. Look at this example of mine that I just run:

```
{
    "ResponseMetadata": {
        "RequestId": "b930f59e-9fb1-46b9-93ac-b8a1ed57277c",
        "HTTPStatusCode": 200,
        "HTTPHeaders": {
            "date": "Mon, 03 Jun 2024 12:44:39 GMT",
            "content-type": "application/json",
            "content-length": "1204",
            "connection": "keep-alive",
            "x-amzn-requestid": "b930f59e-9fb1-46b9-93ac-b8a1ed57277c",
            "x-amzn-bedrock-invocation-latency": "3029",
            "x-amzn-bedrock-output-token-count": "225",
            "x-amzn-bedrock-input-token-count": "18"
        },
        "RetryAttempts": 0
    },
    "contentType": "application/json",
    "body": ""
}
```

Best,

Didier

Cohere R on Bedrock does not include "meta" in response

Add your answer

Relevant content