Step functions Filter support for jsonpath - feature request
AWS Step function's Jsonpath implementation has some known limitations compared to the original jsonpath implementation:
A reference path is a path whose syntax is limited in such a way that it can identify only a single node in a JSON structure:
You can access object fields using only dot (.) and square bracket ( ) notation.
The operators @ .. , : ? * aren't supported.
Functions such as length() aren't supported.
I'd like request to implement these unsupported features.
Sample use-case:
Cognito User Pools: AdminGetUser
step-function action returns UserAttrubutes
in an array of json nodes:
"UserAttributes": [
{
"Name": "sub",
"Value": "f3cab39d-bfe5-4b1a-a8fc-e1a97ec0d18b"
},
{
"Name": "email_verified",
"Value": "true"
},
{
"Name": "custom:name",
"Value": "My Name"
}
{
"Name": "email",
"Value": "myname@mycompany.com"
}
]
Jsonpath filters would provide a very simple way of extracting data from this payload based on "Name" field of the nodes. For example getting the email for a user:
$.UserAttributes[?(@.Name=='email')].Value
Without this feature I needed to implement a lambda for this purpose, because I can't rely on the order of the response array.
Would jsonpath logic applied to "interests" payload in the example below achieve what you are after? https://aws.amazon.com/blogs/compute/using-jsonpath-effectively-in-aws-step-functions/
Adding this blog about the Data Flow Simulator that you might find useful as well for your testing. https://aws.amazon.com/blogs/compute/modeling-workflow-input-output-path-processing-with-data-flow-simulator/
Relevant questions
Step Function Retry Metrics
asked 3 months agoLimitations on Step Functions SDK Integration for Elasticsearch
Accepted Answerasked 7 months agoHow do I use Step Functions to create EMR clusters with different specifications?
Accepted Answerasked 2 years agoDo Lambda functions called inside Step Functions still have a 250MB package limit?
Accepted AnswerStep functions pass input into Fargate instance :
asked 3 months agoState Language Specification
asked 2 months agodesign suggestions to modernize .Net application
asked 3 months agoAWS StepFunctions: sum() function returns error about not finding path
asked 2 months agoStep functions Filter support for jsonpath - feature request
asked 2 months agoRetrieve or store AWS Step function Execution history older than 90 days.
asked 2 years ago
Thank you for your answer. This is exactly what I need. I was trying the filter expression within dataflow-simulator previously, but I failed to produce the result, than I red the ResourcePath limitation, and thought it is global for all AWS json path implementation. But fortunately not. So the real issue with my filter exrpression was about quoting:
$.UserAttributes[?(@.Name=='email')].Value
$.UserAttributes[?(@.Name==email)].Value
In the original jsonpath filter operations string literals must be enclosed by single or double quotes. In AWS you can't use quotes. This is a bit confusing, and would be nice to have highlighted in the AWS documentations.