Are we able to export only parts of the Amazon CloudWatch logs to Amazon S3?

Question

A customer wants to export only parts of the logs from CloudWatch to S3, is there a way to do that?

Reason is that the logGroup is very big and they are  only interested in the data for the last months.

Is there a way from AWS to delete older log streams periodically?
To save some costs on storage?

Bruno_M · Accepted Answer

The pattern you describe is quite common, what you want to do is create a subscription filter for your cloudwatch log group.

Ideally this is the pattern [CW Log Group Filter] -->[Kinesis Firehose] -->[Delivery Stream] -->[S3]

Read the following guide for an example:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#FirehoseExample

As to your question on a way to periodically delete log streams, CW Log Group retention periods are configurable between 1 day up to 10 years.

Relevant content