cloudwatch agent log custom date time format

0

Hi I'm currently using aws cloudwatch agent for pushing custom log to aws.

Sample log line:

v:1,dt:2023-12-22T11:22:33+00:00,active:1,idle:34,totalRequest:363944,slow:76,traffic:1.5

Cloud watch config (/etc/awslogs/awslogs/conf)

[sample-log]
datetime_format = "dt:%Y-%m-%dT%H:%M:%S+00:00"
file = /path/to/log
log_group_name = log-group-name
log_stream_name = {instance_id}
retention_in_days = 90

My question are:

  1. The date format must be quoted, if not log group will not created (error in parsing ?). Why is it must be quoted ? is there a documentation about it ?
  2. Even when quoted, log group is created, but log entry still not recognizing timestamp in the log line, it is using log fetch time.

I found it very hard and confusing, many trial error and guesses, are there any detail documentation about datetime_format parsing ? is there a way to test an expression with a matching line (tool/code) ?

I even have tried using python class DateTimeParser from AWS parser.py source.

in ec2 server with aws log agent installed:
yum list installed | grep aws
repoquery -l aws-cli-plugin-cloudwatch-logs
copy file /usr/lib/python2.7/site-packages/cwlogs/parser.py
class DateTimeParser

The syntax parsed correctly for the expression vs log line, but when applied to aws log daemon, it wont run or wont recognize properly.

coolpie
已提問 10 個月前檢視次數 986 次
1 個回答
0

Hi, yes, there is a doc re. datetime_format for CloudWatch agent: see https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AgentReference.html and search for datetime_format.

Best,

Didier

profile pictureAWS
專家
已回答 10 個月前
  • hi Didier, yes i've already read that doc, and also have spent many time googling, but so far there are no other detailed documentation about dateformat parsing. Thats why i have to run AWS python source code to verify it. (question updated: how to get the parser file)

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南