By using AWS re:Post, you agree to the Terms of Use

Are we able to export only parts of the Amazon CloudWatch logs to Amazon S3?

0

A customer wants to export only parts of the logs from CloudWatch to S3, is there a way to do that?

Reason is that the logGroup is very big and they are only interested in the data for the last months.

Is there a way from AWS to delete older log streams periodically? To save some costs on storage?

1 Answer
0
Accepted Answer

The pattern you describe is quite common, what you want to do is create a subscription filter for your cloudwatch log group.

Ideally this is the pattern [CW Log Group Filter] -->[Kinesis Firehose] -->[Delivery Stream] -->[S3]

Read the following guide for an example:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html#FirehoseExample

As to your question on a way to periodically delete log streams, CW Log Group retention periods are configurable between 1 day up to 10 years.

answered 2 years ago
  • is it mandatory to use both kinesis firehose and datastreams or can we achieve by using one to save cost?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions