You will want something like the following. Note that it has a deliberate sleep
command in it to try and avoid API throttling. This isn't perfect - I could probably do better by checking the return value from the delete_log_stream
call but it will only be an issue if you're deleting many logs.
It also checks for a keyword (in this case "keyword") to skip those logs. And it sets the retention for log groups that don't have a retention period set to 7 days.
What the middle does is delete logs older than the retention time. They should already be deleted but there are cases where the retention period is set after logs have been created and those logs are retained.
This doesn't completely answer your question but it gives you something to start with.
import boto3
import time
logs = boto3.client('logs')
def lambda_handler(event, context):
logGroups = logs.describe_log_groups()['logGroups']
for group in logGroups:
if 'keyword' not in group['logGroupName']: continue
daysRetention = group.get('retentionInDays', 0)
if daysRetention != 7:
logs.put_retention_policy(logGroupName=group['logGroupName'], retentionInDays=7)
continue
maxRetention = time.time()-(daysRetention*86400)
logStream = logs.describe_log_streams(logGroupName=group['logGroupName'])['logStreams']
for stream in logStream:
if (stream['creationTime']/1000) < maxRetention:
print(f'Deleting: {region} {group["logGroupName"]} {stream["logStreamName"]}')
logs.delete_log_stream(logGroupName=group['logGroupName'], logStreamName=stream['logStreamName'])
time.sleep(0.2)
Relevant questions
AWS Lambda@Edge created using AWS CDK doesn't put Log to CloudWatch
Accepted Answerasked 4 months agoMy Lambda function is not getting invoked all of a sudden.
asked 5 months agoDelete CloudWatch Log Groups using AWS Lambda function.
asked a month agoHow divide Log Group by each Lambda function alias?
asked 2 years agoCan I use CloudWatch Logs to trigger a Lambda function?
Accepted Answerasked 2 years agoSpecific Cloudwatch log groups not responding to queries
asked 6 months agoCloudwatch Log Insights doesn't find logs from the first Lambda Invocation
asked 2 months agoAWS Lambda Function Triggers on S3 Event, But only Once in About 30 Mins
asked 2 years agoAuthorizer function not logging to CloudWatch
asked 3 years agoRetrieve or store AWS Step function Execution history older than 90 days.
asked 2 years ago
I would change the
stream['creationTime']
tostream[lastIngestionTime]
^^^ What he said. ;-)