sending data from Cloudwatch 1 log stream of 20 to Firehose

0

Hi.

I want to send data from a cloudwatch logstream of a log group that posses 20 different log streams. The idea is to only send the data from that specific log stream and not the whole log group, but the I can only make a subscription to the whole log group. How can I separate it?

1 Answer
1

Hello,

The Kinesis Firehose Subscription Filters are created to filter the incoming log events that are applied on the entire log group and not on per log stream basis. Unfortunately, this subscription filtering is currently restricted to per log group level. [1]

In order to achieve your use-case to forward log events only from a specific log stream to a destination, I suggest you using a lambda function or writing a script that runs the command get_log_events[2] every 5 minutes. Through the Getlogevents CLI command, you can obtain the logs of a specific log stream and then push these logs to the firehose using the script. We recommend reaching out to AWS Support or your AWS point of contact for any production workloads related guidance.

Let me know if this answers your questions. Thank you for your interest in re:Post community.

Regards, Tejaswini

Reference documents: [1] Real-time processing of log data with subscriptions - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Subscriptions.html [2] GetLogevent CLI command - https://docs.aws.amazon.com/cli/latest/reference/logs/get-log-events.html

AWS
answered 2 years ago
AWS
EXPERT
reviewed 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions