Add additional fields to Logs Insights from python logger

0

Hi.

I'm filtering my logs from a Fargate container using Cloud Watch Insights with something like this

fields @timestamp, @message
| sort @timestamp desc
| filter @message like /(ERROR:root:)/
| limit 20

I know that lambda add additionals fields like @requestId, but not sure if is possible create custom additional fields in some way with the python logger, for example the log level and do something like this:

fields @timestamp, @level, @message
| sort @timestamp desc
| filter @level = 'ERROR'
| limit 20

It's possible to add custom fields like lambda does?

CGarces
demandé il y a 2 ans1935 vues
1 réponse
0

Cloudwatch Logs Insights has fields auto discovery feature. It automatically discovers fields from some of AWS services, like Lambda as you mentioned. It also discovers all fields in JSON documents. So one of the solutions could be to output logging messages in JSON format. If you are using python, you can configure your logger to use python-json-logger, or implement json logging yourself.

Let's say we have this kind of messages in Cloudwatch logs:

{
    "levelname": "ERROR",
    "message": "Some kind of exception occurred",
    "funcName": "function_name",
    "filename": "app.py"
}

Then you can filter by json fields in CW Logs Insights like this:

fields @timestamp, @message
| filter levelname="ERROR" and funcName="function_name"
| sort @timestamp desc
| limit 20

You can also have nested json in your log messages and filter based on those. Another important part about json logs is to have consistent schema for logging messages across the whole application.

répondu il y a 2 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions