Subscription filter on comma-delimited (CSV) log

Hello,

Subscription Log Filters support only space delimited and JSON parsing. Also, be aware that Log Filter and Log Insights have different syntax.

However, I believe the best use of Subscription Log Filter is to filter log events, and not to parse them. Note that in your example, you attempt to retrieve every field. If your intent is to send all data in the log event to ElasticSearch, then you need not include a log filter at-all. Try clicking the ‘Test pattern’ button with an empty Subscription filter pattern to see this working.

Finally, when you create your log subscription to ElasticSearch, a Lambda function will be created on your behalf to handle the log event. If you need your log event parsed, say to conform to a specific schema for ElasticSearch, you may be better served parsing the log entry in the Lambda function, where you’ll have the full power of the programming language of your choice.

Best,
Kevin

Any chance I can get some help on this Parser / Filter issue? :)

Hi Kevin,

Thanks for the reply! I was under the impression a filter was mandatory to stream logs to ES. Definitely sending everything through is better, and adding login/parsing rules to Lambda also helps.

Thanks!