Is Bedrock Converse API prompt caching expected to hit when only the structured output schema is changed?

Question

When using Bedrock's Converse API with prompt caching and structured output, it looks like only changing the schema between requests causes a prompt caching miss. Is this expected behavior that should be documented or is this a bug?

```
require "aws-sdk-bedrockruntime"
require "json"
require "faker"

client = Aws::BedrockRuntime::Client.new(region: "us-west-2")

TEXT = Array.new(60) { Faker::Lorem.paragraph(sentence_count: 5) }.join("

")

SYSTEM_PROMPT = <<~PROMPT
Summarize the text below:

PROMPT

SCHEMA_A = JSON.generate({
type: "object",
properties: { summary: { type: "string" }, key_points: { type: "array", items: { type: "string" } } },
required: ["summary", "key_points"],
additionalProperties: false
})

SCHEMA_B = JSON.generate({
type: "object",
properties: { topic: { type: "string" } },
required: ["topic"],
additionalProperties: false
})

def call(client, label, schema)
resp = client.converse(
    model_id: "us.anthropic.claude-sonnet-4-6",
    system: [
    { text: SYSTEM_PROMPT },
    { cache_point: { type: "default" } }
    ],
    messages: [{ role: "user", content: [{ text: "Summarize this text." }] }],
    inference_config: { temperature: 0 },
    output_config: {
    text_format: {
        type: "json_schema",
        structure: {
        json_schema: { schema: schema, name: label }
        }
    }
    }
)
u = resp.usage
puts "#{label}: input=#{u.input_tokens} cached_hit=#{u.cache_read_input_tokens}
cache_created=#{u.cache_write_input_tokens}"
end

puts "
1. Warm up cache with Schema A"
call(client, "schema_a", SCHEMA_A)

puts "2. Same schema — expect a cache HIT"
call(client, "schema_a", SCHEMA_A)

puts "3. Different schema — expect a cache HIT but it is MISS"
call(client, "schema_b", SCHEMA_B)

puts "4. Back to Schema A — expect a cache HIT"
call(client, "schema_a", SCHEMA_A)
```

Produces

```
1. Warm up cache with Schema A
schema_a: input=11 cached_hit=0
cache_created=2586
2. Same schema — expect cache HIT
schema_a: input=11 cached_hit=2586
cache_created=0
3. Different schema — expect cache HIT but is MISS (bug)
schema_b: input=11 cached_hit=0
cache_created=2543
4. Back to Schema A — expect cache HIT
schema_a: input=11 cached_hit=2586
cache_created=0
```

re:Post Agent · Answer

This is expected behavior, not a bug. When you change the JSON schema structure in your structured output configuration, it invalidates the prompt cache. This is by design because the schema is part of the constrained decoding process that affects how the model generates tokens.

Structured outputs on Amazon Bedrock use a grammar-based approach where the JSON schema is compiled into a grammar that constrains token generation. These compiled grammars are cached per account for 24 hours from first access. However, changing the JSON schema structure invalidates this cache because it fundamentally changes the constraints on the model's output generation.

In your example, when you switch from SCHEMA_A (with summary and key_points fields) to SCHEMA_B (with only a topic field), you're changing the schema structure itself, which causes the cache miss you're observing in step 3.

It's worth noting that if you only changed the `name` or `description` fields in your schema while keeping the structure the same, the cache would not be invalidated. But structural changes to the schema—like different properties, types, or requirements—will cause a cache miss.

This behavior makes sense from a technical perspective: the schema directly influences the token generation process through constrained decoding, so different schemas require different processing paths and cannot share the same cached state.  
**Sources**  
[Structured outputs on Amazon Bedrock: Schema-compliant AI responses | Artificial Intelligence](https://aws.amazon.com/blogs/machine-learning/structured-outputs-on-amazon-bedrock-schema-compliant-ai-responses/)

Is Bedrock Converse API prompt caching expected to hit when only the structured output schema is changed?

Add your answer

Relevant content