How to Create a CloudWatch Dashboard to Monitor Amazon Bedrock Inference Workloads

This guide walks through creating a CloudWatch dashboard to monitor Amazon Bedrock inference workloads. It covers usage metrics, performance, reliability, quota tracking, and cost visibility.

Important: Track metrics per model. Quotas (RPM, TPM), pricing, throttling, and burndown rates are all model-specific.

---

What the Automatic Dashboard Already Provides

AWS provides a free, zero-setup automatic dashboard for Bedrock. Before building a custom dashboard, understand what it already covers.

Access: CloudWatch → Dashboards → Automatic Dashboards tab → Bedrock

---

Metrics to Monitor

Usage Metrics

Metric	Statistic	What It Tells You
Invocations	Sum (1-min period)	Requests per minute (RPM)
InputTokenCount	Average	Typical prompt size per request
InputTokenCount	Maximum	Largest prompt in a single request
OutputTokenCount	Average	Typical response size per request
OutputTokenCount	Maximum	Largest response in a single request
InputTokenCount	Sum (hourly/daily)	Total input tokens — maps to cost
OutputTokenCount	Sum (hourly/daily)	Total output tokens — larger cost driver

Performance Metrics

Metric	Statistic	What It Tells You
TimeToFirstToken	p50, p99	User-perceived responsiveness for streaming
InvocationLatency	p50, p99	End-to-end time from request to last token

Tip: If InvocationLatency is high but TTFT is normal → long response, not a problem. If TTFT is high → actual latency degradation.

Reliability Metrics

Metric	Statistic	What It Tells You
InvocationThrottles	Sum	Hitting quota — requests being dropped
InvocationClientErrors	Sum	4xx — malformed prompts, context window exceeded
InvocationServerErrors	Sum	5xx — service-side failures

Quota Capacity Metrics

Metric	Statistic	What It Tells You
EstimatedTPMQuotaUsage	Sum (1-min)	Token quota consumption with burndown
% of Quota	Math: m1/QUOTA*100	Headroom before throttling

Cache Efficiency (If Using Prompt Caching)

Metric	Statistic	What It Tells You
CacheReadInputTokens	Sum	Cached tokens — cheaper, no TPM impact
CacheWriteInputTokens	Sum	Tokens written to cache — DO count against TPM
Cache Hit Rate	Math expression	Higher = better cost efficiency

---

Dashboard Setup — Step by Step

Step 1: Create a New Dashboard

1. Open CloudWatch console: https://console.aws.amazon.com/cloudwatch
2. Left navigation → click Dashboards
3. Click Create dashboard
4. Name: Bedrock-Monitoring
5. Click Create dashboard

Step 2: Add Throttles Widget (Put This First)

Red alert widget — if non-zero, you're losing requests.

1. Add widget → Line → Metrics
2. Navigate: AWS/Bedrock → By Model Id
3. Select InvocationThrottles for each model
4. Graphed metrics: Statistic = Sum, Period = 1 Minute
5. Label each line with the model name
6. Create widget

Step 3: Add RPM Widget

1. Add widget → Line → Metrics
2. AWS/Bedrock → By Model Id → select Invocations
3. Statistic = Sum, Period = 1 Minute
4. Create widget
5. Add quota line: Edit → Graph options → Horizontal annotations → Value = RPM quota, Color = Red

Step 4: Add Input Token Count (Avg & Peak)

1. Add widget → Line → Metrics
2. AWS/Bedrock → By Model Id → select InputTokenCount
3. Add the metric a second time (duplicate)
4. Line 1: Statistic = Average, Label = "Avg Input Tokens"
5. Line 2: Statistic = Maximum, Label = "Peak Input Tokens"
6. Create widget

Step 5: Add Output Token Count (Avg & Peak)

1. Same as Step 4 but select OutputTokenCount
2. Line 1: Average → "Avg Output Tokens"
3. Line 2: Maximum → "Peak Output Tokens"

Step 6: Add Time to First Token

Note: Only emitted for streaming APIs (ConverseStream, InvokeModelWithResponseStream).

1. Add widget → Line → Metrics
2. AWS/Bedrock → By Model Id → select TimeToFirstToken
3. Add metric a second time
4. Line 1: Statistic = p50, Label = "TTFT median"
5. Line 2: Statistic = p99, Label = "TTFT worst case"
6. Create widget

Step 7: Add Error Rate Widget

1. Add widget → Line → Metrics
2. Select: InvocationThrottles, InvocationClientErrors, InvocationServerErrors
3. All set to: Sum, Period = 1 Minute
4. Create widget

Step 8: Add TPM Quota Usage

1. Add widget → Line → Metrics
2. Select EstimatedTPMQuotaUsage, Statistic = Sum, Period = 1 Minute
3. Create widget
4. Edit → Add math expression: m1 / YOUR_TPM_QUOTA * 100
5. Add horizontal annotations: 80% (orange), 100% (red)

Step 9: Add Daily Token Totals (Cost)

Multiply by per-token pricing for cost estimate.

1. Add widget → Number (single value) → Metrics
2. InputTokenCount: Sum, Period = 1 Day
3. OutputTokenCount: Sum, Period = 1 Day
4. Create widget

Step 10: Save

Click Save at the top right. Dashboard is live.

---

Recommended Alarms

Create at: CloudWatch → Alarms → Create alarm → Select metric → Set threshold.

Alarm	Condition	Priority
Throttles > 0	Any throttle in 1-min window	HIGH — requests dropped
RPM > 80% of quota	Sustained 3 minutes	WARNING
TPM > 80% of quota	Sustained 3 minutes	WARNING
Server errors > 5 in 5 min	5xx spike	WARNING
Client errors > 10	Error rate spike	WARNING — app bug
TTFT p99 > 5 seconds	Latency degradation	WARNING — UX impact
Invocations = 0	No traffic 5 min (biz hrs)	HIGH — app down

---

Best Practices

Practice	Why
Track per model	Quotas/pricing/throttling are per-model
Throttles widget at the top	Most important signal
Add quota limit annotations	Makes spikes immediately actionable
Monitor TTFT, not just latency	TTFT = user-perceived speed
Optimize maxTokens	High values reserve quota upfront → throttles
Start with automatic dashboard	Free baseline, then customize
Separate prod vs dev	Avoid masking patterns
Alarm at 80%, not 100%	At 100% you're already throttled
Track daily token sums	Cost estimation without waiting for billing

---

Important: Metric Visibility in CloudWatch

InvocationThrottles and other error/event metrics only appear in CloudWatch after the event has actually occurred. CloudWatch does not show metrics that have never been emitted.

What this means for your dashboard:

• If you search for InvocationThrottles and get "no metrics found" — that's good news: you haven't been throttled yet.
• Build your dashboard with the metrics that are available (Invocations, TokenCounts, Latency, TPM). Error and throttle metrics will appear automatically when those events occur.
• You can still create alarms on these metrics — they will stay in INSUFFICIENT_DATA state until first emitted, then activate.

Metric	When It First Appears
Invocations	After first successful request
InputTokenCount / OutputTokenCount	After first successful request
TimeToFirstToken	After first successful streaming request
EstimatedTPMQuotaUsage	After first successful request
InvocationThrottles	ONLY after a request has been throttled
InvocationClientErrors	ONLY after a 4xx error occurs
InvocationServerErrors	ONLY after a 5xx error occurs