How can I make Prometheus metrics collected by AWS Cloudwatch appear as metrics in Cloudwatch?

0

I have a Java service running in Fargate which exports Prometheus metrics via an /actuator/prometheus endpoint. It is my understanding that the only way to get Prometheus metrics into Cloudwatch is to deploy a Cloudwatch Agent in the Fargate cluster, and have the agent collect the metrics.

After much struggle, I have managed to get the Cloudwatch Agent deployed to the cluster. It also seems to collect metrics exported by the service.

However, the metrics ends up in a log group named /aws/ecs/containerinsights/{name}/prometheus. An example of a single log line can be found below.

How do I get those metrics into the "metrics" section of Cloudwatch? Right now, it seems that the metrics only makes it into a log, which provides no real value. I'd like to be able to browse and graph the metrics. I am unable to find these metrics in the Container Insights or the metrics section.

{
    "ClusterName": "[...]",
    "ECS_PROMETHEUS_NAME": "[...]",
    "ECS_PROMETHEUS_PATH": "/actuator/prometheus",
    "ECS_PROMETHEUS_PORT": "9082",
    "LaunchType": "FARGATE",
    "StartedBy": "ecs-svc/8880037603679832257",
    "TaskClusterName": "[...]",
    "TaskDefinitionFamily": "[...]",
    "TaskGroup": "service:[...]",
    "TaskId": "a57784efbb124bcfa4d3291cc51f4694",
    "TaskRevision": "442",
    "Timestamp": "1697115231324",
    "Version": "0",
    "area": "heap",
    "container_name": "[...]",
    "id": "G1 Eden Space",
    "instance": "10.1.177.38:9082",
    "job": "[...]",
    "jvm_memory_committed_bytes": 170917888,
    "jvm_memory_max_bytes": -1,
    "jvm_memory_used_bytes": 11534336,
    "prom_metric_type": "gauge"
}
質問済み 7ヶ月前691ビュー
1回答
1

Hi, seems like that log event does not contain a special field called CloudWatchMetrics, which is needed to materialize metrics from logs. This format is called EMF (Embedded metric format).

The reason why it's missing could be that your agent configuration does not include "emf_processor" and "metric_declaration" ? Could you check that?

An example configuration can be found in this yaml file: https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/ecs-task-definition-templates/deployment-mode/replica-service/cwagent-prometheus/cloudformation-quickstart/cwagent-ecs-prometheus-metric-for-awsvpc.yaml , which is linked from this doc: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/ContainerInsights-Prometheus-install-ECS.html

AWS
回答済み 7ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ