- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
Hi there, please try as per above user suggestion in the comments and let us know the results. jqplay is currently blocked on our end so couldn't check the validity. I believe having brackets for each expression should fix the issue. However if that doesn't work, I would recommend you to resort to using a transformation Lambda function like mentioned here: https://docs.aws.amazon.com/firehose/latest/dev/dynamic-partitioning.html in the case where field is not present in the incoming data. That way you would have a lot more control on the fields.
AWS-User-4356880 is correct! Many thanks. I had used a transformation lambda function as you also suggest and this works too but the former was a much simpler, quicker way to go.
I can confirm the example provided by AWS-User-4356880
works! Using an if
statement in a Kinesis Data Firehose works. For exmaple:
Source data might looks like this:
{
"playerName": "Alex",
"score": 1200,
"playtype": "single",
"level": 5,
"duration": "15 minutes"
}
Dynamic partitioning keys can look like:
- Key:
playtype_val
- JQ expression:
(.playtype| if . == null or . == "" then "hardcodedefault" else . end)
And S3 bucket prefix as:
intel/history/!{partitionKeyFromQuery:interaction_type}/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/
What this does is:
- If the field is empty or missing, it returns the hardcodeddefault,
- Else it returns the value inside the field
- And the
/timestamp=
is a built in feature that allows you to create S3 partitions from the timestamp of the event (no need to define an expression).
Relevanter Inhalt
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor 2 Jahren
- AWS OFFICIALAktualisiert vor einem Jahr
- AWS OFFICIALAktualisiert vor 2 Jahren
(.playtype | if . == null or . == "" then "some-hardcoded-default" else . end)
put it inside the brackets , it will work .