Creating metric filters encompassing multiple log groups


I'm trying to create a metric filter that scans for ERRORS between a bunch of log groups. I see a lot of articles that show how to create a metric filter for a single log group but we have >100 log groups so it would be horrible to manually add it. I'm also open to new suggestions if I'm approaching this incorrectly, still learning how to use Cloudwatch.

1개 답변
수락된 답변

Cloudwatch Logs Insights will allow you to filter on up to 20 log groups at a time. If you need to aggregate across a larger number of groups, or with more complex queries, then the suggested approach is to use subscriptions to send logs from each log group to a Kinesis stream, where you could do further processing either with Kinesis Analytics or Lambda. Alternatively, you could stream your log groups into an OpenSearch cluster.

답변함 2년 전
profile picture
검토됨 한 달 전
profile picture
검토됨 4달 전
  • Ah got it, so what we currently do is use subscriptions on two of our log groups but I was wondering if there was a different way instead of creating a subscription for each log group manually and then further process it with Lambda.

    I'll check out OpenSearch cluster, to see if that is something feasible and within our budget. Thank you!

  • The best way to deal with large configurations is always automation. Ideally you would use a complete Infrastructure as Code solution such as CloudFormation or Terraform to bring these resources under control. If that's not possible, the next-best thing is to write a small script in the language of your choice, using either the AWS CLI or an AWS SDK, to find all of the log groups and modify them to suit your needs. Python is a popular language used for small scripts like this.

    If this answer / comment was helpful, please upvote it! Thanks :)

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠