Overall p90 Latency Among Many Latency Metrics

Question

Hi re-Post community,

I'm in process of building a dashboard inspired by this [AWS post](https://aws.amazon.com/builders-library/building-dashboards-for-operational-visibility/).

In the post, there's a concept of displaying overall latency as widget, where latency is calculated among all APIs. I'm struggling to find a way in cloudwatch(I'm using CDK) to do this.

To be more specific,
I have the following metrics from Step functions and "ExecutionTime" latency metrics is provided by AWS Step function by default.

```javascript
m1 - namespace="AWS/States", Metrics="ExecutionTime", Dimension="StateMachineARN1....."
m2 - namespace="AWS/States", Metrics="ExecutionTime", Dimension="StateMachineARN2....."
m3 - namespace="AWS/States", Metrics="ExecutionTime", Dimension="StateMachineARN3....."
```

But I don't know how do I do proper math to get to aggregate them and then do percentile 90.
Or maybe I'm doing it wrong.

Thanks,

Answer

Based on the description you could use a Metric Math 'AVG' function with p90 statistic (you may utilize appropriate function that best fits your model). See sample dashboard source which uses p90 over a 1 hour (3600) period. When approaching to build an 'overall' latency you'd want to ensure the Step functions have a degree of correlation that fulfills a common business goal (otherwise you may have to re-evaluate the effectiveness for such a combined aggregation). This blog post could be useful: https://aws.amazon.com/blogs/mt/what-is-observability-and-why-does-it-matter-part-1/

{
    "metrics": [
        [ { "expression": "AVG(METRICS())", "label": "Expression1", "id": "e1", "stat": "p90" } ],
        [ "AWS/States", "ExecutionTime", "StateMachineArn", "StateMachineARN1", { "id": "m1" } ],
        [ "...", "ExecutionTime", "StateMachineArn", "StateMachineARN1", { "id": "m2" } ],
        [ "...", "ExecutionTime", "StateMachineArn", "StateMachineARN1", { "id": "m3" } ]
    ],
    "view": "timeSeries",
    "stacked": false,
    "region": "us-east-2",
    "stat": "p90",
    "period": 3600
}

Overall p90 Latency Among Many Latency Metrics

Relevant content