- Newest
- Most votes
- Most comments
Is it possible that your Lambda function code isn't written to handle batches of messages? I've see before where folks write code that expects only a single message in the Records array (instead of iterating over all records). With the result being that when the volume of messages increases, they see more than one message per batch, but only process the first one.
https://docs.aws.amazon.com/lambda/latest/dg/with-sqs.html#example-standard-queue-message-event
{
"Records": [
{
"messageId": "059f36b4-87a3-44ab-83d2-661975830a7d",
"receiptHandle": "AQEBwJnKyrHigUMZj6rYigCgxlaS3SLy0a...",
"body": "Test message.",
"attributes": {
"ApproximateReceiveCount": "1",
"SentTimestamp": "1545082649183",
"SenderId": "AIDAIENQZJOLO23YVJ4VO",
"ApproximateFirstReceiveTimestamp": "1545082649185"
},
"messageAttributes": {},
"md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
"eventSource": "aws:sqs",
"eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:my-queue",
"awsRegion": "us-east-2"
},
{
"messageId": "2e1424d4-f796-459a-8184-9c92662be6da",
"receiptHandle": "AQEBzWwaftRI0KuVm4tP+/7q1rGgNqicHq...",
"body": "Test message.",
"attributes": {
"ApproximateReceiveCount": "1",
"SentTimestamp": "1545082650636",
"SenderId": "AIDAIENQZJOLO23YVJ4VO",
"ApproximateFirstReceiveTimestamp": "1545082650649"
},
"messageAttributes": {},
"md5OfBody": "e4e68fb7bd0e697a0ae8f1bb342846b3",
"eventSource": "aws:sqs",
"eventSourceARN": "arn:aws:sqs:us-east-2:123456789012:my-queue",
"awsRegion": "us-east-2"
}
]
}
Afraid this is a User forum and not AWS support so no one here will be able to look into the issue. However, what I would recommend is to make sure you have DLQ's setup on the queue so that failed messages will be sent to the DLQ for you to review. It sounds like they are not being skipped but perhaps they are failing to be processed.
Failed messages will be sent to the DLQ for you to review.
Sure, here’s a concise way to diagnose why some SQS messages are skipped and not processed by your Lambda function:
=>Check CloudWatch Logs: Go to CloudWatch console. Navigate to /aws/lambda/<Lambda-function-name> log group. Search for the specific message IDs (5bdefac5-de6f-41bf-8ae8-f5dfcdcfaaa7 and 599bb266-9221-454d-b77b-aa812f9d9b88).
=>Verify Dead Letter Queue (DLQ): Check if your SQS queue has a DLQ configured. Look in the DLQ for the skipped messages and analyze why they were moved there.
=>Review Lambda Configuration: Ensure the Lambda execution role has permissions for SQS and CloudWatch Logs. Verify Lambda timeout settings and error handling.
=>Check SQS Queue Settings: Ensure visibility timeout is greater than Lambda function’s timeout. Check for any throttling issues. IAM Policy for Lambda
=>Make sure your Lambda has the following permissions:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sqs:ReceiveMessage",
"sqs:DeleteMessage",
"sqs:GetQueueAttributes"
],
"Resource": "arn:aws:sqs:us-east-2:123456789012:my-sqs-queue"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "arn:aws:logs:us-east-2:123456789012:log-group:/aws/lambda/my-lambda-function:*"
}
]
}
=>By following these steps, you can diagnose why specific SQS messages were skipped and gather the necessary logs to understand the issue
Relevant content
- AWS OFFICIALUpdated 2 years ago
I have already setup dead letter queue as well, but no messages is received in that. Neither any logs in lambda which is processing it.