Why are my CloudWatch logs failing to export to S3 buckets?

Resolution

To troubleshoot tasks that failed during creation, check the following settings:

• Region – Confirm that your CloudWatch Logs log streams and S3 buckets are in the same Region.
• S3 bucket policies – By default, all S3 buckets and objects are private. Only the resource owner (the AWS account that created the bucket) can access the bucket and any objects it contains. Use bucket policies to set access permissions on the S3 buckets to CloudWatch Logs.

Note: When you set export permissions on your S3 bucket, you must specify the account IDs of the accounts that can export logs to your bucket. List these accounts under the aws:SourceAccount key. However, aws:SourceAccount can't be added within the s3:GetBucketAcl action.

• S3 bucket prefixes – When you set the S3 bucket policy, it's a best practice to include a randomly generated string as the prefix for the bucket. If you use a prefix, you must specify the randomly generated string in the S3 bucket prefix settings when you create the export task. Otherwise, the export task creation fails.
• AWS Identity and Access Management (IAM) policies – Confirm that the IAM user (IAM role) who created the export task has full access to Amazon S3 and CloudWatch Logs.
• Resource quotas – The CloudWatch Logs service quota allows only one running or pending export task per account per Region. This quota can't be changed. Be sure that you are operating within the allowed quota.
• Type of server-side encryption – Be sure that you're using a supported type of server-side encryption. You can export to S3 buckets that are encrypted with AES-256 and SSE-KMS.

To troubleshoot tasks that failed after creation, check the Time Range setting. If you export log streams with large amounts of data and specify a long time range, then the export task might fail. In this case, specify a shorter time range.

Note: It might take up to 12 hours for the logs to be available for exporting. The export task itself can take some time. For real-time processing or continuously archiving new data to S3, use subscription filters. You can stream to Amazon Kinesis Data Firehose and set Amazon S3 as the target. For archiving historical data to S3, export your data to Amazon S3.

Related information

I configured Amazon CloudWatch to export log data to Amazon S3, but the log data is either missing or not valid. How do I resolve this issue?

How do I retrieve log data from CloudWatch Logs?

Exporting log data to Amazon S3

Using CloudWatch Logs subscription filters