By using AWS re:Post, you agree to the Terms of Use

Questions tagged with Log Analysis

Sort by most recent
  • 1
  • 12 / page

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Why are aggregate results in a Log Insights query nonsensical (count < count_distinct for the same variable)?

The following log insights query on a single log group returns negative numbers for the variable `@distinct_unique_keys_delta`: ```sql parse @message /(?<@unique_key>Processing key: \w+\/[\w=_-]+\/\w+\.\d{4}-\d{2}-\d{2}-\d{2}\.[\w-]+\.\w+\.\w+)/ | filter @message like /Processing key: \w+\/[\w=_-]+\/\w+\.\d{4}-\d{2}-\d{2}-\d{2}\.[\w-]+\.\w+\.\w+/ | stats count(@unique_key) - count_distinct(@unique_key) as @distinct_unique_keys_delta by datefloor(@timestamp, 1d) as @_datefloor | sort @_datefloor asc ``` My understanding is that the number of unique values of a variable can never be more than the total number of values of a variable. When I ran this query I was concerned that I might be misunderstanding the correct usage of `datefloor`, so I tried this query: ```sql parse @message /(?<@unique_key>Processing key: \w+\/[\w=_-]+\/\w+\.\d{4}-\d{2}-\d{2}-\d{2}\.[\w-]+\.\w+\.\w+)/ | filter @message like /Processing key: \w+\/[\w=_-]+\/\w+\.\d{4}-\d{2}-\d{2}-\d{2}\.[\w-]+\.\w+\.\w+/ | stats count(@unique_key) - count_distinct(@unique_key) as @distinct_unique_keys_delta ``` The result of this query for the time range I chose (a whole day), was -20,347 for the `@distinct_unique_keys_delta` variable. To me this result seems completely nonsensical. Am I doing something wrong, interpreting the results wrong or is there a bug in the code running this log insights query?
1
answers
0
votes
473
views
asked 4 months ago

Convert log fields into table columns with aws cloudwatch log insights

i've a lambda function and i want to have a cloudwatch logs table with errors and warnings columns.Actually I was able with this query to get an error / warnings report per day: ``` parse "[E*]" as @error | parse "[W*]" as @warning | filter ispresent(@warning) or ispresent(@error) | stats count(@error) as error, count(@warning) as warning by bin(15m) ``` Here are two example messages of the lambda: WARNING: ``` Field Value @ingestionTime 1653987507053 @log XXXXXXX:/aws/lambda/lambda-name @logStream 2022/05/31/[$LATEST]059106a15343448486b43f8b1168ec64 @message 2022-05-31T08:58:18.293Z b1266ad9-95aa-4c4e-9416-e86409f6455e WARN error catched and errorHandler configured, handling the error: Error: Error while executing handler: TypeError: Cannot read property 'replace' of undefined @requestId b1266ad9-95aa-4c4e-9416-e86409f6455e @timestamp 1653987498296 ``` ERROR: ``` Field Value @ingestionTime 1653917638480 @log XXXXXXXX:/aws/lambda/lambda-name @logStream 2022/05/30/[$LATEST]bf8ba722ecd442dbafeaeeb3e7251024 @message 2022-05-30T13:33:57.406Z 8b5ec77c-fb30-4eb3-bd38-04a10abae403 ERROR Invoke Error {"errorType":"Error","errorMessage":"Error while executing configured error handler: Error: No body found in handler event","stack":["Error: Error while executing configured error handler: Error: No body found in handler event"," at Runtime.<anonymous> (/var/task/index.js:3180:15)"]} @requestId 8b5ec77c-fb30-4eb3-bd38-04a10abae403 @timestamp 1653917637407 errorMessage Error while executing configured error handler: Error: No body found in handler event errorType Error stack.0 Error: Error while executing configured error handler: Error: No body found in handler event stack.1 at Runtime.<anonymous> (/var/task/index.js:3180:15) ``` Can you help me understand how to set up the query in order to have a table with the following columns and their values: from @message extract timestamp, requestID, type (WARN or ERROR), errorMessage and if feasible also the name of the lambda from @log and the @logStream. Can you help me understand how such a query is produced?
2
answers
0
votes
37
views
asked 4 months ago
  • 1
  • 12 / page