Problem using fileds with "@" with CloudWatch Metric Filters

The issue you're experiencing with CloudWatch Metric Filters for Glue job logs is related to the special character "@" in field names. This is a known consideration when working with CloudWatch Logs, particularly for log groups that use fields starting with "@" such as "@message" or "@log".

To resolve this issue, you need to use a specific syntax when referencing these fields in your metric filter patterns. For fields that contain non-alphanumeric characters other than the "@" symbol or the period (.), you must surround the field name with backtick characters (`).

In your case, since you're working with the "/aws-glue/jobs/logs-v2" log group and trying to use fields like "@message", you should enclose these field names in backticks in your filter pattern. For example:

Instead of using: { @message = * } You should use: { @message = * }

This syntax tells CloudWatch Logs to treat the entire string within the backticks as a single field name, including the "@" symbol.

The same approach should be applied to other fields in that log group that start with "@", such as "@log". Always enclose them in backticks when defining your metric filter patterns.

By using this syntax, you should be able to create metric filters that correctly process the Glue job logs and trigger your alarms as expected. This approach allows you to work with these special field names while maintaining the functionality of your CloudWatch alarms and metric filters.

Remember to test your updated filter patterns to ensure they're matching the log events as intended. You can use the CloudWatch console to test the pattern against example log events to verify that it's working correctly before applying it to your metric filter and alarm configuration.

Sources
Fields that contain special characters - Amazon CloudWatch Logs
Filter pattern syntax for metric filters - Amazon CloudWatch Logs

Hello.

I think the following logs are recorded for Glue Job, but what is the reason for including "@message" etc. in the filter?
I thought that if you want to filter by error, just set the filter pattern to "ERROR".

24/11/14 22:05:26 INFO Executor: Starting executor ID 8 on host yyy.yyy.yyy.yyy