SendMessageBatch API response when size of individual or combination of 10 logs exceeds 256KiB

Question

The documentation of [SendMessageBatch](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html) says that `The maximum allowed individual message size and the maximum total payload size (the sum of the individual lengths of all of the batched messages) are both 256 KiB (262,144 bytes).`

If 10 logs are sent to a standard queue in a batch in the following manner:
```
[
  {log - less than 256 KB}, 
  {log - less than 256 KB}, ... // up to 9 logs
  {log - greater than 256 KB}  // last message exceeds the size limit
]
```
Shouldn't the response return a 200 status with 9 messages marked as successful and 1 as failed? Instead, it's returning a 401 with the `BatchRequestTooLong` error code. In what scenarios would the failure count in the specified response syntax increase?

Hilton · Answer

There are a few other reasons why a message batch can fail with a 400 error which can be found in the errors section of the API documentation[1]. The error you are receiving (BatchRequestTooLong) is not specifically related to an individual message failing to be sent in a batch, but the SendMessageBatch API call failing. However, your code should always check the BatchResultErrorEntry[2], which is in the “Failed” object in the API Response[3], for any messages which failed to be sent successfully.

In the scenario where the entire batch API call fails(as in your case), your code will need two evaluate the error message as shown in [1] and make the relevant changes/fixes before attempting to retry that request.

**Sources**

[1] https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html#API_SendMessageBatch_Errors  
[2] https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_BatchResultErrorEntry.html                
[3] https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html#API_SendMessageBatch_ResponseSyntax

re:Post Agent · Answer

When using SendMessageBatch with Amazon SQS, the behavior you're experiencing is actually correct based on the service's constraints. The total size of all messages in a single SendMessageBatch call cannot exceed 256 KiB (262,144 bytes). In the scenario you described, where the last message in the batch exceeds 256 KB, the entire batch request would fail with a BatchRequestTooLong error.

The 256 KiB limit applies to both individual messages and the total payload of the batch. If any single message exceeds this limit or if the combined size of all messages in the batch exceeds this limit, the entire batch operation will fail. The service does not process the batch partially in this case.

To address your question about when the failure count in the response would increase: This would happen in scenarios where individual messages within the batch fail, but the overall batch size is within limits. For example, if you had 10 messages all under the size limit, but one of them had an invalid attribute, you might see 9 successful messages and 1 failed message in the response.

In your case, since the entire batch operation is failing due to size constraints, you don't get to the point where individual message successes or failures are reported. Instead, you receive a 400 status code (not 401 as mentioned in your question) with the BatchRequestTooLong error.

To resolve this, you would need to ensure that both each individual message and the total payload of all messages in the batch are within the 256 KiB limit. If you have messages larger than this, you'll need to send them individually using the standard Publish API rather than in a batch.
  
**Sources**  
[SendMessageBatch - Amazon Simple Queue Service](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html)  
[Amazon SQS batch actions - Amazon Simple Queue Service](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-batch-api-actions.html)

SendMessageBatch API response when size of individual or combination of 10 logs exceeds 256KiB

Add your answer

Relevant content