Skip to content

SQS messages getting skipped

0

Hi,

We are using sqs to process request. This sqs messages are consumed by Lambda function. However some messages from sqs are not getting processed and it is getting skipped. Below are some sqs message id 5bdefac5-de6f-41bf-8ae8-f5dfcdcfaaa7. 599bb266-9221-454d-b77b-aa812f9d9b88.

Please check and revert us on what exactly with this sqs messages id has happened, need logs for the same as well.

asked a year ago535 views
3 Answers
1

Is it possible that your Lambda function code isn't written to handle batches of messages? I've see before where folks write code that expects only a single message in the Records array (instead of iterating over all records). With the result being that when the volume of messages increases, they see more than one message per batch, but only process the first one.

https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#example-standard-queue-message-event

{
    "Records": [
        {
            "messageId": "059f36b4-87a3-44ab-83d2-661975830a7d",
            "receiptHandle": "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a...",
            "body": "Test message.",
            "attributes": {
                "ApproximateReceiveCount": "1",
                "SentTimestamp": "1545082649183",
                "SenderId": "AIDAIENQZJOLO23YVJ4VO",
                "ApproximateFirstReceiveTimestamp": "1545082649185"
            },
            "messageAttributes": {},
            "md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
            "eventSource": "aws:sqs",
            "eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:my-queue",
            "awsRegion": "us-east-2"
        },
        {
            "messageId": "2e1424d4-f796-459a-8184-9c92662be6da",
            "receiptHandle": "AQEBzWwaftRI0KuVm4tP+/7q1rGgNqicHq...",
            "body": "Test message.",
            "attributes": {
                "ApproximateReceiveCount": "1",
                "SentTimestamp": "1545082650636",
                "SenderId": "AIDAIENQZJOLO23YVJ4VO",
                "ApproximateFirstReceiveTimestamp": "1545082650649"
            },
            "messageAttributes": {},
            "md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
            "eventSource": "aws:sqs",
            "eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:my-queue",
            "awsRegion": "us-east-2"
        }
    ]
}
AWS
answered a year ago
0

Afraid this is a User forum and not AWS support so no one here will be able to look into the issue. However, what I would recommend is to make sure you have DLQ's setup on the queue so that failed messages will be sent to the DLQ for you to review. It sounds like they are not being skipped but perhaps they are failing to be processed.

Failed messages will be sent to the DLQ for you to review.

EXPERT
answered a year ago
  • I have already setup dead letter queue as well, but no messages is received in that. Neither any logs in lambda which is processing it.

0

Sure, here’s a concise way to diagnose why some SQS messages are skipped and not processed by your Lambda function:

=>Check CloudWatch Logs: Go to CloudWatch console. Navigate to /aws/lambda/<Lambda-function-name> log group. Search for the specific message IDs (5bdefac5-de6f-41bf-8ae8-f5dfcdcfaaa7 and 599bb266-9221-454d-b77b-aa812f9d9b88).

=>Verify Dead Letter Queue (DLQ): Check if your SQS queue has a DLQ configured. Look in the DLQ for the skipped messages and analyze why they were moved there.

=>Review Lambda Configuration: Ensure the Lambda execution role has permissions for SQS and CloudWatch Logs. Verify Lambda timeout settings and error handling.

=>Check SQS Queue Settings: Ensure visibility timeout is greater than Lambda function’s timeout. Check for any throttling issues. IAM Policy for Lambda

=>Make sure your Lambda has the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sqs:ReceiveMessage",
                "sqs:DeleteMessage",
                "sqs:GetQueueAttributes"
            ],
            "Resource": "arn:aws:sqs:us-east-2:123456789012:my-sqs-queue"
        },
        {
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "arn:aws:logs:us-east-2:123456789012:log-group:/aws/lambda/my-lambda-function:*"
        }
    ]
}

=>By following these steps, you can diagnose why specific SQS messages were skipped and gather the necessary logs to understand the issue

EXPERT
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.