Skip to content

Sqs send message retry

0

I have a service running on Node.js 20 with the following workflow: S3 Event → Lambda → SQS Question 1: Occasionally, the SQS client throws a ServiceUnavailable error when calling sendMessage. Even though you have configured maxRetries to 4, the client does not retry sending the message. You are looking for a way to ensure the SQS client retries the sendMessage operation without relying on the Lambda retry mechanism.

Question 2: When the service throws an error, a JSON.parse error occurs due to deserialization of a hidden field {error}.$response. The error message is:

Unexpected token '<', "<ServiceUn"... is not valid JSON

Is the secondary error (happening when an exception is thrown )causing my function to not retry?. TIA

asked a year ago657 views
1 Answer
1

To address your concerns about SQS message sending and retries, let's tackle each question separately:

Question 1: If the SQS client is not retrying the sendMessage operation despite configuring maxRetries, you can implement a custom retry mechanism in your Lambda function. Here's an approach you can take:

  1. Create a wrapper function for the sendMessage operation that includes retry logic.
  2. Use an exponential backoff strategy to space out retry attempts.
  3. Implement a counter to track the number of attempts.

Here's a basic example of how you might implement this:

const { SQSClient, SendMessageCommand } = require("@aws-sdk/client-sqs");
const sqs = new SQSClient();

async function sendMessageWithRetry(params, maxRetries = 4) {
  let retries = 0;
  while (retries <= maxRetries) {
    try {
      const command = new SendMessageCommand(params);
      return await sqs.send(command);
    } catch (error) {
      if (error.name === 'ServiceUnavailable' && retries < maxRetries) {
        retries++;
        await new Promise(resolve => setTimeout(resolve, 2 ** retries * 100)); // Exponential backoff
      } else {
        throw error;
      }
    }
  }
}

This approach ensures that your function will attempt to retry the sendMessage operation independently of the Lambda retry mechanism.

Question 2: The JSON.parse error you're encountering is likely not directly related to the SQS client's retry mechanism. This error suggests that when an exception is thrown (such as the ServiceUnavailable error), there's an attempt to parse an error response that isn't in valid JSON format.

This secondary error is not likely to be the cause of your function not retrying. The retry behavior should be determined by the primary error (ServiceUnavailable in this case) rather than any subsequent parsing errors.

To address this issue:

  1. Ensure you're using the latest version of the AWS SDK for JavaScript (v3).
  2. When catching errors, avoid directly stringifying or parsing the entire error object. Instead, access specific properties of the error.

For example:

try {
  // Your SQS send operation
} catch (error) {
  console.error('Error name:', error.name);
  console.error('Error message:', error.message);
  // Handle the error or rethrow as needed
}

By focusing on specific error properties, you can avoid the JSON parsing issue while still getting the necessary error information.

Remember, for both issues, proper error handling and logging are crucial. Make sure to implement comprehensive logging to track these issues in production and to help with debugging.

Sources
SendMessage - Amazon Simple Queue Service
Community | Point-to-Point Messaging with Amazon SQS

answered a year ago
EXPERT
reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.