cloudwatch agent log custom date time format

0

Hi I'm currently using aws cloudwatch agent for pushing custom log to aws.

Sample log line:

v:1,dt:2023-12-22T11:22:33+00:00,active:1,idle:34,totalRequest:363944,slow:76,traffic:1.5

Cloud watch config (/etc/awslogs/awslogs/conf)

[sample-log]
datetime_format = "dt:%Y-%m-%dT%H:%M:%S+00:00"
file = /path/to/log
log_group_name = log-group-name
log_stream_name = {instance_id}
retention_in_days = 90

My question are:

  1. The date format must be quoted, if not log group will not created (error in parsing ?). Why is it must be quoted ? is there a documentation about it ?
  2. Even when quoted, log group is created, but log entry still not recognizing timestamp in the log line, it is using log fetch time.

I found it very hard and confusing, many trial error and guesses, are there any detail documentation about datetime_format parsing ? is there a way to test an expression with a matching line (tool/code) ?

I even have tried using python class DateTimeParser from AWS parser.py source.

in ec2 server with aws log agent installed:
yum list installed | grep aws
repoquery -l aws-cli-plugin-cloudwatch-logs
copy file /usr/lib/python2.7/site-packages/cwlogs/parser.py
class DateTimeParser

The syntax parsed correctly for the expression vs log line, but when applied to aws log daemon, it wont run or wont recognize properly.

coolpie
feita há 10 meses984 visualizações
1 Resposta
0

Hi, yes, there is a doc re. datetime_format for CloudWatch agent: see https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html and search for datetime_format.

Best,

Didier

profile pictureAWS
ESPECIALISTA
respondido há 10 meses
  • hi Didier, yes i've already read that doc, and also have spent many time googling, but so far there are no other detailed documentation about dateformat parsing. Thats why i have to run AWS python source code to verify it. (question updated: how to get the parser file)

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas