How to push prometheus histogram metrics to cloudwatch

0

I am using cloudwatch agent to push gitlab-runner metrics to CloudWatch metrics. The gitlab-runner provides a histogram metric for how long jobs take and also provides this metric for several configured runners, which than looks like this:

gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="30"} 0
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="60"} 1
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="300"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="600"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="1800"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="3600"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="7200"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="10800"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="18000"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="36000"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="+Inf"} 9
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="30"} 0
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="60"} 0
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="300"} 2
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="600"} 4
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="1800"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="3600"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="7200"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="10800"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="18000"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="36000"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="+Inf"} 5

Now, I want to push this metric to CW so that I can build a dashboard from this, but separate it for: a) The runner (so the runner label) b) The bucket (so le=30,60,....)

How do I have to configure CloudWatch agent to archive this? The main problem I have is, how can I push the gitlab_runner_job_duration_seconds_bucket for the different values of le as different metrics?

Thanks!

Nathan
preguntada hace 7 meses349 visualizaciones
1 Respuesta
0

I think what you need to do here is send the logs to Cloudwatch as they are and then use Cloudwatch itself rather than the Cloudwatch agent to seperate out the data elements you need into Metrics

You might be able to use Metric Filters to achieve the result you are looking for

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringLogData.html

AWS
Barry M
respondido hace 7 meses
  • What do you mean? CloudWatch Agent sends the logs to Cloudwatch for me. How would I do that without Cloudwatch Agent?

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas