AWS CloudWatch metrics to OpenSearch

Question

I'm working on Centralized Logging of my environnement AWS presented in this guide:https://docs.aws.amazon.com/solutions/latest/centralized-logging/overview.html

I'm using the same architecture as presented in AWS Doc.

I'm searching the possibility to send metrics from CloudWatch of each account to OpenSearch in centralized account. I was thinking of a Lambda function that retrieves each X minutes the data with get_metric_data ? I'm collecting the metrics with CloudWatch Agent

Is there an another method to do it ?

Thank you

Answer

There is an opensource projects which sends all CW metrics and logs for your Amazon OpenSearch Service deployment to a centralized opensearch. Have a look at this and see if this helps: https://github.com/aws-samples/amazon-opensearch-service-monitor

Answer

CloudWatch [Metric Streams](https://aws.amazon.com/blogs/aws/cloudwatch-metric-streams-send-aws-metrics-to-partners-and-to-your-apps-in-real-time/) is the easiest way to continually get metric data sent out of an account.

In your case, the simplest setup for that would be to send from your accounts to a central S3 bucket in your central account. From there you can have a Lambda that triggers whenever S3 files get uploaded to the bucket - read those S3 files and log the content to the Lambda's log output as [EMF](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html) - a simple JSON format that creates metrics.

So pipeline is basically: **remote account -> Metric Stream -> central account S3 bucket -> Lambda -> CW Logs as EMF -> CW Metrics**

AWS CloudWatch metrics to OpenSearch

관련 콘텐츠