I am using a lambda to perform some transformation on records buffered into a kinesis stream.
This is my lambda (data is ingested in Gzip):
ef lambda_handler(event, context):
output = []
for record in event['records']:
print(record['recordId'])
data = base64.b64decode(record['data'])
content = gzip.decompress(data).decode('utf-8')
payload = json.loads(content)
message = payload["logEvents"][0]["message"]
print(message)
output_record = {
'recordId': record['recordId'],
'result': 'Ok',
'data': base64.b64encode(json.dumps(message).encode('utf-8')).decode('utf-8')
}
output.append(output_record)
print('Successfully processed {} records.'.format(len(event['records'])))
return {'records': output}
The issue is that Kinesis does not understand that the data sent back is in json and output as escaped strings:
"{\"id\": 123, \"name\" : \"Andrea\"}"
Any suggestion? I need the data back in json format to query that from Athena