Skip to content

Metric filter for specific log-stream

0

Hello,

I have a challenge right now that I can't solve after reading a lot of documentations and articles in the internet. In CloudWatch I have a log group "my_app", inside I have 2 log-streams: "app-log" and "connect-log". I need to create a metric filter that will work only on "app-log" log-stream. How can I reach it?

asked 2 years ago1.3K views

2 Answers
0

Creating a metric filter that works only on a specific log stream within a log group in CloudWatch is not directly supported. Metric filters are applied at the log group level and process all log streams within that group.

However, you can achieve a similar result by incorporating the log stream name into your filter pattern. Here's how you can approach this:

  1. When creating your metric filter, use a filter pattern that includes the log stream name as part of the matching criteria.

  2. Assuming your log events include the log stream name, you can create a filter pattern like this:

    [..., stream="app-log", ...]

    This pattern will only match log events that contain "app-log" as the stream name.

  3. If your log events don't explicitly include the stream name, you may need to modify your logging configuration to include this information in each log entry.

  4. When creating the metric filter in the CloudWatch console or via AWS CLI, use this expanded filter pattern that includes the stream name check.

  5. Configure the rest of your metric filter as usual, specifying the metric namespace, name, and value to extract.

This approach will effectively create a metric that only increments based on log events from the "app-log" stream, even though the filter is applied to the entire "my_app" log group.

Remember that this method relies on the log stream name being part of your log data. If this isn't currently the case, you may need to adjust your application's logging configuration to include this information.

While this solution doesn't directly filter at the log stream level, it provides a workaround to achieve the desired result of creating metrics based on a specific log stream within a log group.
Sources
Create a metric filter for a log group - Amazon CloudWatch Logs
Filter pattern syntax for metric filters - Amazon CloudWatch Logs
Example: Count log events - Amazon CloudWatch Logs

answered 2 years ago

EXPERT

reviewed 2 years ago

0

To solve the issue, you can use the stream_name field in your filter pattern. CloudWatch automatically adds metadata fields to your logs, including stream_name By including stream_name= app-log in your filter pattern, you ensure the metric only counts events from that specific stream The ... in the pattern means "match any remaining fields"

something like this should help

Create a metric filter that matches only events from the specific log stream

aws cloudwatch put-metric-filter
--log-group-name "my_app"
--filter-name "AppLogFilter"
--filter-pattern "[timestamp, stream_name= app-log, ...]"
--metric-transformations
metricName=YourMetricName,
metricNamespace=YourNamespace,
metricValue=1,
unit=Count

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.