CloudWatch metric filter

Question

I want to filter out the some metrics from the cloud watch logs that are published from EC2 instance and i have already metric filter but when i am filtering out the keyword "Err" with process name "pro_col_iww_yww_post300932059" it is giving correct result. But i want to filter keyword  Err with 4 or 6 defined process name then how would i create that filter pattern.

What I need - : i want to search Err keyword with process names -   pro_col_iww_yww_post300932059,
pro_col_wqw_wtw_post300123222,
pro_col_wsw_jww_post300121271,
pro_col_fww_lww_post300481214.

Question - I want to filter keyword Err with process name as mentioned above(these are fixed name) from the log events which have Err.

Accepted Answer

Hi,
sorry for the late reply. As we don't have an example of your logs and don't know your seletion criteria, we are flying blind and cannot be sure what would be the best possible syntax. Please find below an example that I hope could unblock you:
Considering the following log messages:
```
2023-12-07 Some info for pro_col_iww_yww_post300932059 A message info
2023-12-07 Some info for pro_col_wqw_wtw_post300123222 Another message with other info
2023-12-07 Some info for pro_col_wsw_jww_post300121271 Again a message with other info
2023-12-07 Some info for pro_col_abc_jww_post300121271 Reject this message because the prefix abc is not allowed
2023-12-07 Some info for pro_col_wsw_jww_post123ABC123 Reject this message because the suffix id contains letters
2023-12-07 An Error message, we want it too
```

We want to keep messages 1, 2, 3 and 6 as per the comments I've added to the messages.
One possible way (there are multiple ways, this is just an option) to filter those messages would be:
```
%Err|pro_col_[iwsq]{3}_[wtyj]{3}_post[\d]{9}%
```

Explanation:
- the filter starts with % and ends with % => this is a Regex filter (no double quotes when using a regex filter!)
- the filter has a first option then | as an OR then a second option
- the second option to match has nested patterns: 3 letters exactly out of i,w,s,q for one part, 3 letters exactly from a different set of letters for the next part, and 9 decimals exactly at the end of the pattern we are matching on

In the examples you give above you seem to be using | but inside double quotes so this wouldn't be recognized as an OR, unless it's an actual character you're trying to match? Also, I don't see % around your expression if it's a regex you intend to craft, % is mandatory if it's a regex.
I hope this example and explanation will fill the gap with what you need to be successful with your query.

Answer

Hi,
it might help if you shared your existing filter. If you look into the second link shared by Tom B and look at the space delimited log events example, it shows a way to link together conditions that seems to match your need. As indicated on that page: You also can use the logical operators AND (&&) and OR (||) to create compound expressions.

Note however that you can also use wildcards. In the same example I'm referring to, you can see some filters such as request =*.html*, status_code = 4* - it might also be useful in your case if you want to match an unbounded list of process names?

Answer

You might be able to create a metric filter by using some of these examples: https://docs.aws.amazon.com/securityhub/latest/userguide/cloudwatch-controls.html#cloudwatch-5

I'm not positive if that format only works for CloudTrail logs but it may help.

Here is a link to the filter and pattern syntax: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#matching-terms-unstructured-log-events

The CloudTrail is in the JSON format but using the unstructured log events syntax might help.

CloudWatch metric filter

Relevant content