Any chance I can get some help on this Parser / Filter issue? :)
Subscription Log Filters support only space delimited and JSON parsing. Also, be aware that Log Filter and Log Insights have different syntax.
However, I believe the best use of Subscription Log Filter is to filter log events, and not to parse them. Note that in your example, you attempt to retrieve every field. If your intent is to send all data in the log event to ElasticSearch, then you need not include a log filter at-all. Try clicking the ‘Test pattern’ button with an empty Subscription filter pattern to see this working.
Finally, when you create your log subscription to ElasticSearch, a Lambda function will be created on your behalf to handle the log event. If you need your log event parsed, say to conform to a specific schema for ElasticSearch, you may be better served parsing the log entry in the Lambda function, where you’ll have the full power of the programming language of your choice.
Thanks for the reply! I was under the impression a filter was mandatory to stream logs to ES. Definitely sending everything through is better, and adding login/parsing rules to Lambda also helps.
Archiving CloudWatch logs no with data lossasked 5 months ago
Are we able to export only parts of the Amazon CloudWatch logs to Amazon S3?Accepted Answerasked 2 years ago
Metric filter for logs JSON data with @ property namesasked a year ago
Log Subscription Filter To Opensearchasked a year ago
CloudWatch logs are not reaching Splunk for the new lambda with AWS Firehose/Lambda integrationasked 6 months ago
sending data from Cloudwatch 1 log stream of 20 to Firehoseasked 2 months ago
Subscription filter on comma-delimited (CSV) logasked 2 years ago
JSON logs not seen as JSON by Subscription filter?asked 2 years ago
Metric Filter Creation from CloudWatch Logasked 5 months ago
How can you filter by json property in cloudwatch log?asked a year ago