DynamoDB streams filter with nested fields not working

Question

I have a Lambda hooked up to my DynamoDB stream. It is configured to trigger if both criteria are met:
* eventName = "MODIFY"
* status > 10

My filter looks as follows:

`{"eventName": ["MODIFY"], "dynamodb": {"NewImage": {"status": [{"numeric": [">", 10]}]}}}`

If the filter is configured to only trigger if the event name is MODIFY it works, however anything more complicated than that does not trigger my Lambda. The event looks as follows:

```
{
    "eventID": "ba1cff0bb53fbd7605b7773fdb4320a8",
    "eventName": "MODIFY",
    "eventVersion": "1.1",
    "eventSource": "aws:dynamodb",
    "awsRegion": "us-east-1",
    "dynamodb":
    {
        "ApproximateCreationDateTime": 1643637766,
        "Keys":
        {
            "org":
            {
                "S": "test"
            },
            "id":
            {
                "S": "61f7ebff17afad170f98e046"
            }
        },
        "NewImage":
        {
            "status":
            {
                "N": "20"
            }
        }
    }
}
```
When using the test_event_pattern endpoint it confirms the filter is valid:

```
filter = {
    "eventName":  ["MODIFY"],
    "dynamodb": {
        "NewImage": {
            "status":  [ { "numeric": [ ">", 10 ] } ]
        }
    }
}

response = client.test_event_pattern(
    EventPattern=json.dumps(filter),
    Event="{\"id\": \"e00c66cb-fe7a-4fcc-81ad-58eb60f5d96b\", \"eventName\": \"MODIFY\", \"dynamodb\": {\"NewImage\":{\"status\": 20}}, \"detail-type\": \"myDetailType\", \"source\": \"com.mycompany.myapp\", \"account\": \"123456789012\", \"time\": \"2016-01-10T01:29:23Z\", \"region\": \"us-east-1\"}"
)
print(response) >> {'Result': True, 'ResponseMetadata': {'RequestId':...}
```
Is there something that I'm overlooking? Do DynamoDB filters not work on the actual new image?

**---UPDATE---**

Still haven't made much progress. I can confirm that the following _works_:
* `{"dynamodb":{"NewImage":{"status":{"N":["11","12","13","14","15"]}}}}`
* `{"dynamodb":{"NewImage":{"status":{"N":["11","12","13","14","15"]},"progress":{"S":["not_done","done"]}}}}`

proving that you can filter on on nested and multiple fields. However I can't for the life of me get the more advanced filters to work (in any variation from documentation: https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html#filtering-syntax):
* `{"eventName":["MODIFY"],"dynamodb":{"NewImage":{"status":{"N":[{"numeric":[">",10]}]}}}}`
* `{"eventName":["MODIFY"],"dynamodb":{"NewImage":{"status":{"N":{"numeric":[">",10]}}}}}`
* `{"eventName":["MODIFY"],"dynamodb":{"NewImage":{"status":{"N":["numeric",">",10]}}}}`

These do **not** _work_.

Answer

Before digging into this further, let me confirm one thing:
- How is your DynamoDB Stream configured, ensure it produces ```NEW_AND_OLD_IMAGES```. [Docs](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html#Streams.Enabling)

Answer

Your pattern should look like this:
```
{"eventName": ["MODIFY"], "dynamodb": {"NewImage": {"status": { "N": [{"numeric": [">", 10]}]}}}}
```
Note the "N" which is part of the object.

Answer

I have had the same problem using dynamodb kinesis stream.
Also I had removed the filterCriteria after dealing with this issue, and the events were still filtering. Confirmed there was no filter in place with the cli, and in the console. Seems like some weird behavior from the filterCriteria. I had to delete the entire stack to remove the filter. Interested to see what your outcome is.

DynamoDB streams filter with nested fields not working

Relevant content