Large number of unaccounted for SQS requests

Question

Hi, I just started experimenting with SQS a couple days ago - subscribing a queue to an SNS topic that receives notifications from SES, and I can't understand the enormous amount of SQS requests that I see in my billing dashboard.  
  
For example, when I send an email through SES, two notifications get sent to the SQS queue ( via the SNS topic ) - a "send", and a "delviery" notification. I have set up a lambda trigger on the receiving queue, that simply sends the message to another queue and deletes it from the original one.  
  
I would expect to see a handful of SQS requests per sent email, however it looks like each email accounts for about a **thousand** SQS requests in the billing dashboard. In my latest experiment, a single email resulted in exactly 2 messages in the queue, and when I checked the billing dashboard a few hours later, the "Current usage" of SQS requests had jumped from 49878 to 54809. Overall these 54000+  requests have resulted from fewer than 50 SES emails (~100 notification messages) over the last couple days.  
  
At the same time all my CloudWatch metrics for SQS do show the low numbers that I expect to see.  
  
What could possibly be making all these requests?   
  
Any pointers on resolving this mistery will be much appreciated.  
  
Edited by: rebor691 on Sep 26, 2019 7:22 PM

Accepted Answer

(moving to AWS Lambda forum)  
  
According to the feature announcement, calls to SQS made by the Lambda SQS poller will be billed to your account by SQS.  
  
https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/

Answer

> _Kuba@AWS wrote:_    
>   
> According to the feature announcement, calls to SQS made by the Lambda SQS poller will be billed to your account by SQS.  
>   
> https://aws.amazon.com/blogs/aws/aws-lambda-adds-amazon-simple-queue-service-to-supported-event-sources/  
  
From the linked blog post: _"... the Lambda service will begin polling the SQS queue using five parallel long-polling connections."_  
  
Didn't realize a lambda trigger meant continuously long-polling the queue using at least five threads.  
  
All is accounted for now: 5 connections, 20 sec per long poll => 900 req / hour => ~54k in ~60 hours.  
  
Thanks a lot.

Answer

Hi,  
It sounds like you are possibly seeing a higher than expected bill with SQS because you are using the default "Short Polling" instead of "Long Polling". If you have a single thread that is polling a single queue, you will definitely want to set the timeout to 20s (long polling), so that you are NOT making continuous requests to the SQS queue to see if there is data available. The default, 'Short polling' would ONLY make sense, if you have a single thread that has to poll multiple queues (thus, checking each queue for data quickly rather than spending 20s waiting for a message to arrive on a queue before moving on to check on the next queue for a message).

```
Long polling offers the following benefits:
- Eliminate empty responses by allowing Amazon SQS to wait until a message is available in a queue before sending a response. Unless the connection times out, the response to the ReceiveMessage request contains at least one of the available messages, up to the maximum number of messages specified in the ReceiveMessage action.
- Eliminate false empty responses by querying all—rather than a subset of—Amazon SQS servers.
- Return messages as soon as they become available.
```

This link has directions on how to setup Long Polling  
  
{code}  
Hope this helps,  
-randy

Large number of unaccounted for SQS requests

Relevant content