By using AWS re:Post, you agree to the Terms of Use

Cloudwatch Log Insights - Line Chart Issues

0

Hello :wave:

We are trying to use Cloudwatch Logs together with Cloudwatch Log Insights to Query through application's logs and visualise the API usage by different consumers. A single log consists of fields such as:

  • consumer (api user)
  • controller (base endpoint on which the consumer made a request)
  • action (exact action on which someone sent a request)

Sample entry could look like this (obviously it truncated for the sake of explaining the problem)

{
  consumer: 'test-app-1',
  controller: 'api/test-controller',
  action: 'test-action
}

Now, I would like to find a way in which I can group this in a line chart, for example requests made to different controllers by the consumers over time or exact action requests by consumers over time.

I am not sure how to write the query, if I am to be honest, as

filter @logStream = 'someLogStream'
 | fields @timestamp, @message
 | stats count() by consumer, controller, action

does not generate a line chart over time, and adding

filter @logStream = 'someLogStream'
 | fields @timestamp, @message
 | stats count() by consumer, controller, action, bin(15m)

does not change the error message on visualisation screen, which states that:

The data is not suitable for a line chart. Try a bar chart, or group your result by bin function.

Can anyone advise on what should I do? For starters, requests over time per consumer would be a nice metric and I believe that once we get to that point, we will be able to modify it per action / per controller. I think that I lack proficiency in the Insights syntax to achieve what I want.

Thanks!

1 Answer
0

Thank you for this question. You are right about a lot of things here. Below I have explained a few pointers, that can help in writing your queries.

The following statement needs to be changed, for proper use of the bin() function:

stats count() by consumer, controller, action, bin(15m)
  • In the above statement, you might have observed an error (on hovering your mouse on the underlined red-lines in your query on console) stating that bin() is not valid in this position - because, EOF or other applicable keywords are expected after by in the query. For example stats count(action = 'test-action') by bin(15m) would be valid.
  • count() expects parameters i.e. either count() or count(fieldName: LogField). You can also use * as your parameter like this - count(*). Read more here.

Coming to your use case to count number of requests for a specific consumer - for example: to count the number of log events for consumer "test-app-1", change your query to something like the following:

 filter @logStream = 'someLogStream' and consumer = 'test-app-1'
 | fields @timestamp, @message
 | stats count(*) by bin(15m)

Notice that I have used the filter statement in order to only fetch the results for a specific consumer, feel free to experiment with such options for other filtering that you might want to perform.

I recommend reading the AWS documentation titled "CloudWatch Logs Insights query syntax". Specifically, for this question - the sections "Matches and regular expressions in the filter command" [1] & "Supported operations and functions" [2] can be helpful in getting more clarity & for further learning.

I hope this answer helps in starting with your analysis using Log Insights queries. Feel free to comment on any clarification needed.

Happy learning!

profile picture
SUPPORT ENGINEER
answered 16 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions