DynamoDB streams filter with nested fields not working

0

I have a Lambda hooked up to my DynamoDB stream. It is configured to trigger if both criteria are met:

  • eventName = "MODIFY"
  • status > 10

My filter looks as follows:

{"eventName": ["MODIFY"], "dynamodb": {"NewImage": {"status": [{"numeric": [">", 10]}]}}}

If the filter is configured to only trigger if the event name is MODIFY it works, however anything more complicated than that does not trigger my Lambda. The event looks as follows:

{
    "eventID": "ba1cff0bb53fbd7605b7773fdb4320a8",
    "eventName": "MODIFY",
    "eventVersion": "1.1",
    "eventSource": "aws:dynamodb",
    "awsRegion": "us-east-1",
    "dynamodb":
    {
        "ApproximateCreationDateTime": 1643637766,
        "Keys":
        {
            "org":
            {
                "S": "test"
            },
            "id":
            {
                "S": "61f7ebff17afad170f98e046"
            }
        },
        "NewImage":
        {
            "status":
            {
                "N": "20"
            }
        }
    }
}

When using the test_event_pattern endpoint it confirms the filter is valid:

filter = {
    "eventName":  ["MODIFY"],
    "dynamodb": {
        "NewImage": {
            "status":  [ { "numeric": [ ">", 10 ] } ]
        }
    }
}

response = client.test_event_pattern(
    EventPattern=json.dumps(filter),
    Event="{\"id\": \"e00c66cb-fe7a-4fcc-81ad-58eb60f5d96b\", \"eventName\": \"MODIFY\", \"dynamodb\": {\"NewImage\":{\"status\": 20}}, \"detail-type\": \"myDetailType\", \"source\": \"com.mycompany.myapp\", \"account\": \"123456789012\", \"time\": \"2016-01-10T01:29:23Z\", \"region\": \"us-east-1\"}"
)
print(response) >> {'Result': True, 'ResponseMetadata': {'RequestId':...}

Is there something that I'm overlooking? Do DynamoDB filters not work on the actual new image?

---UPDATE---

Still haven't made much progress. I can confirm that the following works:

  • {"dynamodb":{"NewImage":{"status":{"N":["11","12","13","14","15"]}}}}
  • {"dynamodb":{"NewImage":{"status":{"N":["11","12","13","14","15"]},"progress":{"S":["not_done","done"]}}}}

proving that you can filter on on nested and multiple fields. However I can't for the life of me get the more advanced filters to work (in any variation from documentation: https://docs.aws.amazon.com/lambda/latest/dg/invocation-eventfiltering.html#filtering-syntax):

  • {"eventName":["MODIFY"],"dynamodb":{"NewImage":{"status":{"N":[{"numeric":[">",10]}]}}}}
  • {"eventName":["MODIFY"],"dynamodb":{"NewImage":{"status":{"N":{"numeric":[">",10]}}}}}
  • {"eventName":["MODIFY"],"dynamodb":{"NewImage":{"status":{"N":["numeric",">",10]}}}}

These do not work.

NicoDL
질문됨 2년 전1373회 조회
3개 답변
1

Before digging into this further, let me confirm one thing:

  • How is your DynamoDB Stream configured, ensure it produces NEW_AND_OLD_IMAGES. Docs
profile pictureAWS
전문가
답변함 2년 전
  • It is configured to produce the new image: stream: StreamViewType.NEW_IMAGE. In the meantime I made some progress. I was able to get an exact match on the status field working by using the following filter: {"dynamodb":{"NewImage":{"status":{"N":["20"]}}}}. It seems that the data type needs to be included in the filter and the value needs to be quoted (even though the documentation specifies otherwise). However I can't get more advanced filters to work for the life of me, e.g. (from the documentation):

    "Price": [ { "numeric": [ ">", 10, "<=", 20 ] } ]

0

Your pattern should look like this:

{"eventName": ["MODIFY"], "dynamodb": {"NewImage": {"status": { "N": [{"numeric": [">", 10]}]}}}}

Note the "N" which is part of the object.

profile pictureAWS
전문가
Uri
답변함 2년 전
  • That doesn't work sadly so and is one of the countless possibilities I tried. The Lambda shows the following configured pattern:

    {
      "filters": [
        {
          "pattern": "{\"eventName\": [\"MODIFY\"], \"dynamodb\": {\"NewImage\": {\"status\": { \"N\": [{\"numeric\": [\">\", 10]}]}}}}"
        }
      ]
    }
    

    but after modifying a record in the database (both by editing inline using the console and updating the record's status by code) the last processing result on the table still states "No records processed", and I can confirm that the Lambda does not trigger.

0

I have had the same problem using dynamodb kinesis stream. Also I had removed the filterCriteria after dealing with this issue, and the events were still filtering. Confirmed there was no filter in place with the cli, and in the console. Seems like some weird behavior from the filterCriteria. I had to delete the entire stack to remove the filter. Interested to see what your outcome is.

답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠