- Newest
- Most votes
- Most comments
The most probable reason for this behavior is that your visibility timeout is too short. The visibility timeout controls how long after the message is retrieved by one consumer it will be visible to other consumers. In other words, it controls how long you have to process the message and delete it before it will be delivered to other consumers. You can find more information about the visibility timeout here.
You should not delete the message from the queue immediately after reading it but rather after processing it. If you delete it immediately and the consumer fails before being able to process the message, you will loose that message. For this reason you should set a visibility timeout which is longer than the time it takes to process the message. If it takes very long to process a message and you fo not want to wait the entire time in case of consumer failure you should set a shorter timeout and let the consumer extend it while processing the message.
It is probably because you receive message multiple times
The ReceiptHandle is associated with a specific instance of receiving a message. If you receive a message more than once, the ReceiptHandle is different each time you receive a message. When you use the DeleteMessage action, you must provide the most recently received ReceiptHandle for the message (otherwise, the request succeeds, but the message might not be deleted).
I have to delete the message immediately after reading so that the same message should not be processed again by the client
For this you can setup a proper visibility timeout - its will prevent other receives during the timeout. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
To avoid all these issues I would recommend to consume SQS using AWS Lambda (if your setup allows this of course) - when consuming with Lambda you don't need to worry about deletions and many other things.
Hi!
A couple things to try that will help with the handle expiration.
-
Extend the visibility timeout for the message (VisibilityTimeout in the request): https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#SQS.Client.receive_message.
-
On the queue itself, you can modify the value of Visibility Timeout (for console, check here): https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-configure-queue-parameters.html
-
If you'd like to change the visibility of a message via API, you can do so with the Change Message Visibility call: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ChangeMessageVisibility.html
-
There is also a ChangeMessageVisibilityBatch (For multiple messages). Boto3: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#SQS.Client.change_message_visibility
Changing those settings may be the most straightforward way to ensure that your deletion requests can find the message by the handle prior to the handle expiring.
Keep in mind that for FIFO queues, there can be a maximum of 20,000 inflight messages. If this limit is reached, SQS may not return error messages.
Relevant content
- asked a year ago
- asked 5 years ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated a year ago
I have left the visibility time out at default 30 seconds as it is.
Now I changed it to 5 minutes. Now I am able to delete it.
I think there may be a bug with 30 seconds setting. Even when I am deleting immediately it fails.
I noticed that the sample code to receive message has visibility time out set to 0. I removed the parameter and it is able to delete.