Skip to content

SNS PublishBatch API returns generic SnsException for rate/quota exceeded errors

0

I'm encountering an issue with the AWS SDK for Java v2 where the SNS publishBatch method returns a generic SnsException when hitting rate limits or quota exceeded errors, rather than a specific exception type.

During a recent high-traffic event, our application encountered rate limit errors with SNS. While our quota has since been increased and this is not a frequent occurrence, the impact was significant when it happened.

software.amazon.awssdk.services.sns.model.SnsException: Rate exceeded (Service: Sns, Status Code: 400

The issue is that when exceeding SNS publishing rate limits, the SDK throws a generic SnsException with a message containing "Rate exceeded" rather than a specific exception type. This makes it difficult to implement proper error handling without relying on error message parsing.

We already have exponential backoff with jitter implemented, but need to distinguish between different types of errors for proper handling and disable retries when service qouta or rate exceeds.

https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-sns/src/main/java/com/amazonaws/services/sns/AmazonSNS.java#L1241

What's the recommended approach for handling these errors without relying on error message parsing?

2 Answers
0

When handling rate limit or quota exceeded errors with the SNS PublishBatch API in AWS SDK for Java v2, you're correct that these come through as generic SnsException instances rather than specific exception types.

The recommended approach for handling these errors without relying on error message parsing is to examine the error details within the SnsException. Specifically, you can check the error code and HTTP status code from the AWS error details:

try {
PublishBatchResponse response = snsClient.publishBatch(request);
// Process successful messages
response.successful().forEach(success -> {
// Handle successful messages
});

// Check for failed messages within the batch
response.failed().forEach(failure -> {
// Handle individual message failures
});
} catch (SnsException e) {
// Check for throttling/rate limit errors
if (e.awsErrorDetails().sdkHttpResponse().statusCode() == 429 || 
"Throttling".equals(e.awsErrorDetails().errorCode())) {
// Handle throttling with exponential backoff
} else if (e.awsErrorDetails().errorCode().contains("Limit") || 
e.awsErrorDetails().errorMessage().contains("Rate exceeded")) {
// Handle quota/limit exceeded errors
} else {
// Handle other types of errors
}
}

Amazon SNS considers all 5XX errors and 429 (too many requests) errors as retryable. For these errors, implementing exponential backoff with jitter is appropriate. When you encounter throttling errors, the SDK internally categorizes them as SdkErrorType.THROTTLING.

For batch operations, it's also important to check the response object for partial failures. The PublishBatch operation can result in a combination of successful and unsuccessful actions, so you should check for batch errors even when the call returns an HTTP status code of 200.

Remember that Amazon SNS makes a total of 50 attempts to deliver a message before discarding it. If you need to retain messages that can't be delivered after all retries, consider configuring a dead-letter queue (DLQ) for your subscription.
Sources
SnsClient (AWS SDK for Java - 2.21.28)
SdkErrorType (AWS SDK for Java - 2.21.29)
Amazon SNS message batching - Amazon Simple Notification Service
Amazon SNS message delivery retries - Amazon Simple Notification Service

answered a year ago

0

Keyword Limit also exists in Throttling error codes, also Rate exceeded is checking the error message which I want to avoid as the error message is not future-proof.

else if (e.awsErrorDetails().errorCode().contains("Limit") ||  e.awsErrorDetails().errorMessage().contains("Rate exceeded"))

        THROTTLING_ERROR_CODES.add("Throttling");
        THROTTLING_ERROR_CODES.add("ThrottlingException");
        THROTTLING_ERROR_CODES.add("ThrottledException");
        THROTTLING_ERROR_CODES.add("ProvisionedThroughputExceededException");
        THROTTLING_ERROR_CODES.add("SlowDown");
        THROTTLING_ERROR_CODES.add("TooManyRequestsException");
        THROTTLING_ERROR_CODES.add("RequestLimitExceeded");
        THROTTLING_ERROR_CODES.add("BandwidthLimitExceeded");
        THROTTLING_ERROR_CODES.add("RequestThrottled");
        THROTTLING_ERROR_CODES.add("RequestThrottledException");
        THROTTLING_ERROR_CODES.add("EC2ThrottledException");
        THROTTLING_ERROR_CODES.add("PriorRequestNotComplete");

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.