- Newest
- Most votes
- Most comments
I have resolved the issue.
I was using the Topic Replay function to handle the functionality to resend messages. I did not realize that setting a replay paused all messages in the subscription, so I either needed to create a new replay policy without an ending DateTime, or I needed to clear out the replay policy. I did not think it was worth the hassle of trying to confirm if the Replay was completed or not and ended up reworking the resend logic to work in a different manner.
This needs to be addressed by SQS service team. Team will need ARN of the SNS an SQS so that they can check the CloudWatch metrics of the resources. Also, some message IDs with timestamps corresponding to the problematic SNS messages. Alternatively you can share the SNS request IDs with approximate timestamps with the support so that a thorough investigation on the message life cycle can be kicked off.
Some common causes and troubleshooting steps to identify and resolve the issue:
1. Permission Issues
Even though replaying messages seems to work, there could still be intermittent permission issues or misconfigurations that cause temporary disruptions.
Check IAM Policies: Ensure that the IAM policies attached to both the SNS topic and SQS queue allow the necessary actions (sns:Publish, sqs:SendMessage, etc.) and include the correct ARNs. Check SNS Topic Policies: Ensure the SNS topic policy allows the SNS service to publish to your SQS queue. Verify that there are no condition statements that might cause intermittent failures.
2. Throttling
Both SNS and SQS have rate limits, and hitting these limits can cause messages to be dropped or delayed.
Monitor CloudWatch Metrics: Check CloudWatch metrics for both SNS and SQS to see if there are any indications of throttling.
Rate Limiting: Ensure that the message throughput does not exceed the limits set for your SQS queue or SNS topic.
3. Queue Configuration Issues
The configuration of the SQS queue might cause issues with message delivery.
Dead-Letter Queue (DLQ): If your SQS queue is configured with a dead-letter queue, check if messages are being sent there. This could indicate processing issues or message retention problems.
Visibility Timeout: Ensure the visibility timeout is correctly set to avoid messages being considered undeliverable.
4. Message Deduplication ID Issues
FIFO queues require a MessageDeduplicationId and a MessageGroupId. Issues with these IDs can cause messages to be ignored or dropped.
Check Deduplication IDs: Ensure that the MessageDeduplicationId is unique for each message unless you intend to deduplicate them.
Check Group IDs: Ensure MessageGroupId is set correctly and consistently for messages meant to be in the same group.
5. Network and Connectivity Issues
Temporary network issues between the SNS service and the SQS queue might cause message delivery failures.
VPC Endpoints: If you're using VPC endpoints, ensure they are properly configured and not causing connectivity issues.
6. SNS to SQS Subscription Issues
Sometimes, the subscription itself may have issues.
Resubscribe: As a last resort, try deleting the existing subscription and creating a new one to ensure there are no underlying issues with the subscription itself.
please accept the answer if it was helpful