API Gateway body template mapping to retain valid json data-structure of data-element, while still allow for base64-encoding

Question

A customer in need of configuring an API Gateway fronting a Kinesis Stream. We're currently working through the docs to [Create an API Gateway API as an Amazon Kinesis Proxy][1].

For the `PutRecord` and `PutRecords` APIs, customer wants to support the following payload structure (note the data-elements are passed in as json):

{
        "records": [
            {
                "data": ,
                "PartitionKey": 
            },
            {
                "data": ,
                "PartitionKey": 
            }
        ]
    }

e.g. like the following:

{
        "records": [
            {
                "data": {
                    "Events": [
                        {"DSType":"RTDS","DSInstance":"Unit 1","DSPoint":"Tag 1","TimeStamp":"2017-10-10T11:10:00.0000000+02:00","Value":8.0,"Quality":192},
                        {"DSType":"RTDS","DSInstance":"Unit 2","DSPoint":"Tag 2","TimeStamp":"2017-10-10T11:10:00.0000000+02:00","Value":7.0,"Quality":193}
                        ],
                        "Project":"EventAcquisitionStream",
                        "Plant":"Plant1"
                },
                "partition-key": "some key"
            },
            {
                "data": {
                    "Events": [
                        {"DSType":"RTDS","DSInstance":"Unit 3","DSPoint":"Tag 3","TimeStamp":"2017-10-11T11:10:00.0000000+02:00","Value":8.0,"Quality":192},
                        {"DSType":"RTDS","DSInstance":"Unit 4","DSPoint":"Tag 4","TimeStamp":"2017-10-10T11:10:00.0000000+02:00","Value":7.0,"Quality":193}
                        ],
                        "Project":"EventAcquisitionStream",
                        "Plant":"Plant2"
                },
                "partition-key": "some key"
            }
        ]
    }

Per AWS documentation referenced above, we have configured the following model and body template mapping for the `PutRecords` API:

Model:

{
      "$schema": "http://json-schema.org/draft-04/schema#",
      "title": "PutRecords proxy payload data model",
      "type": "object",
      "properties": {
        "records": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "data": { 
                  "type": "object",
                  "properties": {
                    "events": { 
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "dstype": { "type": "string" },
                                "dsinstance": { "type": "string" },
                                "dspoint": { "type": "string" },
                                "timeStamp": { "type": "string" },
                                "value": { "type": "number" },
                                "quality": { "type": "integer" }
                            }
                        }
                    },
                    "project": { "type": "string" },
                    "plant": { "type": "string" }
                  }
                },
              "partition-key": { "type": "string" }
            }
          }
        }
      }
    }

Body template mapping:

{
        "StreamName": "$input.params('stream-name')",
        "Records": [
           #foreach($elem in $input.path('$.records'))
              {
                "Data": "$util.base64Encode($elem.data)",
                "PartitionKey": "$elem.partition-key"
              }#if($foreach.hasNext),#end
            #end
        ]
    }

Problem is, when this is decoded on the Kinesis consumer-side, the contents of `$elem.data` has been transformed into invalid json, e.g. replacing `:` with `=` (note for `Events`, `Project` and `Plant` elements):

Decoded payload: {Events=[{"DSType":"RTDS","DSInstance":"Unit 1","DSPoint":"Tag 1","TimeStamp":"2017-10-10T11:10:00.0000000+02:00","Value":8.0,"Quality":192},{"DSType":"RTDS","DSInstance":"Unit 2","DSPoint":"Tag 2","TimeStamp":"2017-10-10T11:10:00.0000000+02:00","Value":7.0,"Quality":193}], Project=EventAcquisitionStream, Plant=Plant1}

What is going on here, how can I configure my body template mapping to retain the json data-structure within the data elements of the request, while still allowing for base64-encoding?

[1]: http://docs.aws.amazon.com/apigateway/latest/developerguide/integrating-api-with-aws-services-kinesis.html#api-gateway-get-and-add-records-to-stream

Accepted Answer

Recreating the JSON and double-quoting the strings. Solution referenced from [https://forums.aws.amazon.com/thread.jspa?threadID=233060](https://forums.aws.amazon.com/thread.jspa?threadID=233060).

Answer

Since the link above no longer works posting my somewhat hacky solution.. 
``` 
{ "StreamName": "event-stream-qa", "Records": [
#set($datakey = ".data")
#set($partitionkey = ".key")
#foreach($elem in $input.path('$.records'))
    #set($pathBegin = "$.records[")
    #set($pathEnd = "]")
    #set($currentIndex = $foreach.index)
    #set($dataPath = "$pathBegin$currentIndex$pathEnd$datakey")
    #set($partitionPath = "$pathBegin$currentIndex$pathEnd$partitionkey")
    #set($dataJson = $input.json($dataPath)) 
    #set($partitionJson = $input.json($partitionPath)) 
  { 
    "Data": "$util.base64Encode($dataJson)",
    "PartitionKey": $partitionJson
  }
    #if($foreach.hasNext),#end
#end  
  ]
}
```

API Gateway body template mapping to retain valid json data-structure of data-element, while still allow for base64-encoding

関連するコンテンツ