How can you filter by json property in cloudwatch log?

Question

I have json being logged in cloudwatch log. I am trying to filter by specific json property value. I am using the dot notation. It never finds what I know is there.

This is in the log

{ "requestId": "abc123", "apiId": "123456"}

I am filtering with this

{$.requestId="abc123"}

It never finds the log item. Is this supposed to work?

Answer

Try to run it from Logs Insights and using smth like `filter requestId="abc123"`
Some doc is here: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_AnalyzeLogData_RunSampleQuery.html

Answer

Hi Andyn,

That `{$.requestId="abc123"}` looks like a valid filter for JSON formatted logs, and yes it is supported and it should work if the log event(s) are in the log group. You can check more details from (1).

I just checked the 'log event' and 'filter pattern' you have provided which is working as expected on my end with replication. Have you checked the retention policy of the Log Group ? and whether the log event(s) that should be matching are recent(not past the Retention policy period)?

For instance, to check whether your filter pattern is working or not you can setup a test Log Group/Log Stream and create log event using the Console.  The steps are -> CW Console -> Log Groups -> go into the Log Stream -> Action -> Create log event. Then you can test with adjusting the filter with matching and non-matching patterns to see if the filter works or not.

If you are still encountering problems filtering the log events, please try with specific and short time ranges rather than just filtering with default settings on the console.  Further, if you have a Premium Support subscription consider creating Technical Support case under CloudWatch Logs service so that we can assist you in a private setting.

(1) Filter and pattern syntax - Matching terms in JSON log events - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html#matching-terms-events-json

Thanks

How can you filter by json property in cloudwatch log?

Relevant content