SQS Cost Calculation For Batch Requests

0

I am using SQS Batching using a java client(AmazonSQSBufferedAsyncClient) for sending messages to SQS. I am sending 10 messages in one batch, But SQS monitoring is showing the Number Of Messages Sent is 10. Same for Number Of Messages Received and Number Of Messages Deleted Metric(Showing 10 count, expecting 1).

Below is the code to create buffered client

final AmazonSQSAsync sqsAsync = new AmazonSQSAsyncClient(SQSConfig.getCreds());

        final QueueBufferConfig config = new QueueBufferConfig()
                .withMaxBatchOpenMs(10000)
                .withMaxBatchSize(10)
                .withMaxInflightReceiveBatches(2)
                .withMaxDoneReceiveBatches(15);

        config.setLongPoll(true);

        bufferedSqs = new AmazonSQSBufferedAsyncClient(sqsAsync, config);

Below is the code to send a message.

public void sendBufferMessageInBatch(String message, String messageGroupId) throws ExecutionException, InterruptedException {

        SendMessageBatchRequest sqsBatchRequest = new SendMessageBatchRequest();

        sqsBatchRequest.setQueueUrl(SQSConfig.endpoint);
        List<SendMessageBatchRequestEntry> entryList = Stream.iterate(1, i -> i < 11, i -> i + 1)
                .map(numb -> {
                    String sendMessage = message;
                    sendMessage = sendMessage + "_" + numb;
                    sendMessage = sendMessage + "_" + messageGroupId;
                    SendMessageBatchRequestEntry entry = new SendMessageBatchRequestEntry();
                    entry.setMessageBody(sendMessage);
                    entry.setId("id_" + numb);
                    printSendMessage(sendMessage);
                    return entry;
                })
                .collect(Collectors.toList());

        sqsBatchRequest.setEntries(entryList);
        Future<SendMessageBatchResult> sendBatchMessageFuture = SQSConfig.bufferedSqs.sendMessageBatchAsync(sqsBatchRequest);
//        sendBatchMessageFuture.get();
    }

Questions:

  1. Why are Number Of Messages Sent metrics showing count 10 instead of count 1?
  2. What is SQS database or storage to store messages?
asked a year ago360 views
1 Answer
0

The reason why the "Number Of Messages Sent" metric is showing a count of 10 instead of 1 is because you are sending 10 messages in 1 batch using the AmazonSQSBufferedAsyncClient. From the perspective of the SQS service, this is considered as 10 separate messages being sent, even though they are being sent together in a batch.

Amazon SQS offers a reliable, highly-scalable hosted queue for storing messages as they travel between applications or microservices. When you send a message to an SQS queue, the message is stored redundantly across multiple servers in multiple data centers to ensure high availability and durability. The messages are not stored in a traditional database, but rather in a distributed storage service that is optimized for storing large numbers of messages with high availability and durability. Note that SQS messages have a maximum storage size of 256 kb and a maximum storage time of 14 days.

AWS
David C
answered a year ago
  • How AWS SQS does charge/bill for a batch request? How can I monitor the number of API requests on a daily basis(on that bill generated)? As per this link(https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-client-side-buffering-request-batching.html), Batch request cost is lower than single-single messages.

  • Every xxx KB segment incurs a single charge. Therefore, we can accumulate messages and send them out as a single request to SQS when we reach the xxx KB limit. This means we send, receive, or delete multiple messages in a single request, reducing the total number of requests and therefore lowering costs. Batches: Send, receive, or delete messages in batches of up to 10 messages or 256KB. Batches cost the same amount as single messages, meaning SQS can be even more cost effective for customers that use batching.

    NumberOfMessagesSent is the metric that measures the number of messages added to a queue.

  • I am sending messages in batches and the total payload of messages(all messages in batch) is less than 256kb. But still, I am seeing the "Number Of Messages Sent" metric is showing 10. How can I verify AWS is charging on 1 request, not on 10 requests? I wanted to see how many API requests happening for one batch request. "Number of Messages sent" metric means the number of API calls that happen to send messages. If "Number Of Messages Sent" metric is showing 10, that mean AWS will charge me for 10 API calls?

  • The metric "Number of Messages sent" is exactly what you described: the number of messages published to the queue, regardless of whether they were sent individually or in batches. If your goal is to monitor the number of API operations in an SQS queue and therefore understand how much you will be billed, you can access the Cost Explorer and filter by "Group by": "API operation" and select "Service": "SQS". This will allow you to determine the number of API operations in your SQS queue and its associated cost.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions