By using AWS re:Post, you agree to the Terms of Use

Questions tagged with Amazon CloudWatch

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Lambda random long execution while running QLDB query

I have a lambda triggered by a SQS FIFO queue when there are messages on this queue. Basically this lambda is getting the message from the queue and connecting to QLDB through a VPC endpoint in order to run a simple SELECT query and a subsequent INSERT query. The table selected by the query has a index for the field used in the where condition. Flow (all the services are running "inside" a VPC): `SQS -> Lambda -> VPC interface endpoint -> QLDB` Query SELECT: `SELECT FIELD1, FIELD2 FROM TABLE1 WHERE FIELD3 = "ABCDE"` Query INSERT: `INSERT INTO TABLE1 .....` This lambda is using a shared connection/session on QLDB and this is how I'm connecting to it: ``` import { QldbDriver, RetryConfig } from 'amazon-qldb-driver-nodejs' let driverQldb: QldbDriver const ledgerName = 'MyLedger' export function connectQLDB(): QldbDriver { if ( !driverQldb ) { const retryLimit = 4 const retryConfig = new RetryConfig(retryLimit) const maxConcurrentTransactions = 1500 driverQldb = new QldbDriver(ledgerName, {}, maxConcurrentTransactions, retryConfig) } return driverQldb } ``` When I run a load test that simulates around 200 requests/messages per second to that lambda in a time interval of 15 minutes, I'm starting facing a random long execution for that lambda while running the queries on QLDB (mainly the SELECT query). Sometimes the same query retrieves data around 100ms and sometimes it takes more than 40 seconds which results in lambda timeouts. I have changed lambda timeout to 1 minute but this is not the best approch and sometimes it is not enough too. The VPC endpoint metrics are showing around 250 active connections and 1000 new connections during this load test execution. Is there any QLDB metric that could help to identify the root cause of this behavior? Could it be related to some QLDB limitation (like the 1500 active sessions described here: https://docs.aws.amazon.com/qldb/latest/developerguide/limits.html#limits.default) or something related to concurrency read/write iops?
1
answers
0
votes
52
views
asked 10 days ago

Cloudwatch Log Insights - Line Chart Issues

Hello :wave: We are trying to use Cloudwatch Logs together with Cloudwatch Log Insights to Query through application's logs and visualise the API usage by different consumers. A single log consists of fields such as: - consumer (api user) - controller (base endpoint on which the consumer made a request) - action (exact action on which someone sent a request) Sample entry could look like this (obviously it truncated for the sake of explaining the problem) ``` { consumer: 'test-app-1', controller: 'api/test-controller', action: 'test-action } ``` Now, I would like to find a way in which I can group this in a line chart, for example requests made to different controllers by the consumers over time or exact action requests by consumers over time. I am not sure how to write the query, if I am to be honest, as ``` filter @logStream = 'someLogStream' | fields @timestamp, @message | stats count() by consumer, controller, action ``` does not generate a line chart over time, and adding ``` filter @logStream = 'someLogStream' | fields @timestamp, @message | stats count() by consumer, controller, action, bin(15m) ``` does not change the error message on visualisation screen, which states that: >The data is not suitable for a line chart. Try a bar chart, or group your result by bin function. Can anyone advise on what should I do? For starters, requests over time per consumer would be a nice metric and I believe that once we get to that point, we will be able to modify it per action / per controller. I think that I lack proficiency in the Insights syntax to achieve what I want. Thanks!
1
answers
0
votes
27
views
asked 10 days ago