How to push prometheus histogram metrics to cloudwatch

0

I am using cloudwatch agent to push gitlab-runner metrics to CloudWatch metrics. The gitlab-runner provides a histogram metric for how long jobs take and also provides this metric for several configured runners, which than looks like this:

gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="30"} 0
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="60"} 1
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="300"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="600"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="1800"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="3600"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="7200"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="10800"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="18000"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="36000"} 9
gitlab_runner_job_duration_seconds_bucket{runner="Tr2JygoJs",system_id="s_91b8bee3abf2",le="+Inf"} 9
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="30"} 0
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="60"} 0
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="300"} 2
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="600"} 4
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="1800"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="3600"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="7200"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="10800"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="18000"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="36000"} 5
gitlab_runner_job_duration_seconds_bucket{runner="xNY81T-sW",system_id="s_91b8bee3abf2",le="+Inf"} 5

Now, I want to push this metric to CW so that I can build a dashboard from this, but separate it for: a) The runner (so the runner label) b) The bucket (so le=30,60,....)

How do I have to configure CloudWatch agent to archive this? The main problem I have is, how can I push the gitlab_runner_job_duration_seconds_bucket for the different values of le as different metrics?

Thanks!

Nathan
posta 7 mesi fa350 visualizzazioni
1 Risposta
0

I think what you need to do here is send the logs to Cloudwatch as they are and then use Cloudwatch itself rather than the Cloudwatch agent to seperate out the data elements you need into Metrics

You might be able to use Metric Filters to achieve the result you are looking for

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringLogData.html

AWS
Barry M
con risposta 7 mesi fa
  • What do you mean? CloudWatch Agent sends the logs to Cloudwatch for me. How would I do that without Cloudwatch Agent?

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande