Saltar al contenido
Al usar AWS re:Post, aceptas las AWS re:Post Términos de uso
AWS re:Postre:Post
Acerca de re:Post

Log events missing in cloudwatch export in S3 bucket using Create export task function

0

I am trying to create a daily backup of cloudwatch log to S3 bucket in log Archive account using lambda function calling cloud watch create export task. The export is happening successfully but I found there were few log event missing in log export log file for the given time range, though I can see log events in Cloudwatch stream in cloudwatch console.Also I can see the missing events TimeStamp and Ingestion time both are in the define time range. I tested both way, manual export using console menu and using Lambda. same result.

Can anyone suggest why this is happening. does anyone else also faced similar issue?

Temas
Administración y gobierno
Etiquetas
Amazon CloudWatch
Idioma
English

preguntada hace un año173 visualizaciones

1 Respuesta
0

This could be becuase CloudWatch export tasks use the log ingestion time (when CloudWatch received the log), NOT the event timestamp shown in the log message. This is the most common reason for missing events. Export Range Uses ingestion time only. Common scenarios causing missing logs would be either Late Arriving Logs or Logs Still Being Ingested. One solution would be Add a Buffer Period and wait 5-10 minutes after the time period before running the export.

ref: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/S3Export.html

respondido hace un año

AWS
EXPERTO

revisado hace un año

  • ANJU RANI
    hace un año

    I understand, What I found is, though the ingestion time mentioned in CloudWatch is in my export task range, still log event missed in export. Additionally when i added buffer period of 10 minute, the the results are: example in my export log, one event is of time 23:58:01 and next event in log is from next date 00:05:02 but I can see almost 10 event in between these time range as missing if compared to Cloudwatch log events in Console. Not sure why this is happening.

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Contenido relevante