An SNS message with a double quote in ANY attribute is filtered out. Is this a bug?
The documentation https://docs.aws.amazon.com/sns/latest/dg/sns-subscription-filter-policies.html says
When Amazon SNS evaluates message attributes against the subscription filter policy, it ignores message attributes that aren't specified in the policy.
Through some experimentation I have determined that SNS messages with a double quote in one of their attribute values will always be filtered out for a subscription with a filter policy, even if the filter policy doesn't mention the attribute. It does NOT get filtered out if you put a backslash character before the double quote. I can't find any reference to this in the documentation so I don't know if it's expected behavior. It seems like a bug.
Example:
SNS subscription filter policy:
{
"action": ["purchase"]
}
Message Attributes (PASSES filter):
- action - DataType: String, StringValue: purchase
- metadata - DataType: String, StringValue: abc
Message Attributes (DOES NOT PASS filter):
- action - DataType: String, StringValue: purchase
- metadata - DataType: String, StringValue: "abc
Message Attributes (PASSES filter):
- action - DataType: String, StringValue: purchase
- metadata - DataType: String, StringValue: "abc
This problem occurred because we have a String type attribute with a JSON blob in the value. I eventually narrowed the problem down to just the presence of the double quote character.
I have found a few examples of other people encountering this issue:
- https://forums.aws.amazon.com/thread.jspa?threadID=300797
- https://forums.aws.amazon.com/thread.jspa?threadID=318176
- https://forums.aws.amazon.com/message.jspa?messageID=841824
Is this a bug?
Hello,
Thank you for reaching out to us. SNS service team is aware of the issue with few characters that might cause an issue with filter policy. When there is filter policy associated with a subscription and the published messages contains attributes with escapable characters such as " and , SNS expects the characters to be double-escaped otherwise the message filtering is affected thereby causing the message delivery failure to the endpoint.
Please find the list of such special characters:
====================== Escape Characters List
- "
- \
- /
- b
- f
- n
- r
- t
- uXXXX (where X is hex eg. u123a)
For example:
Character in escape list:
\” bad
\” ok
\\” bad
Character not in escape list:
\p ok
\p bad
\\p ok
Internal service team is working towards a fix and I am unable to provide an ETA for this , updates to the services are announced at AWS What's New [1] and RSS feed [2].
References:
Relevant questions
Is there a possibility SNS topic subscription can filter messages based on custom message header ? And Does SNS supports message grouping ?
asked 5 months agoConnection is filtered when using Elastic IP for EC2 Linux 2
asked 2 years agoHow can resolve "This is a bug in the provider, which should be reported in the provider's own │ issue tracker." problem in terraform apply to iam role creation?
asked 4 months agoWhat is the maximum email sending rate when using Amazon SES out of the sandbox?
Accepted Answerasked 3 months agoCan lambda know the specific attribute which changed in the dynamo db table update, when a lambda is triggered by dynamo db table update item??
asked 3 months agoBuild a turn-based game with Amazon DynamoDB and Amazon SNS - UK telephone number problem.
asked a month agoWhy is my SMS received from SNS having a different font?
asked 5 months agoCreate a notification once CloudFormation StackSet is finished
asked 5 months agoAn SNS message with a double quote in ANY attribute is filtered out. Is this a bug?
asked 4 months agoNumber messageAttribute delivered as String in Lambda subscriber to SNS Topic
Accepted Answerasked 2 months ago