By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

importing form encoded/csv data from data logger using aws api gateway,lamda function and dynamodb

0

greetings, for past few weeks i have been trying to fetch data using a solar data logger using an api gateway that sends data to my api gateway which is integrated to my lamda function but the problem is that api gateway even after adding binary support for text/csv and application/x-www-form-urlencoded and i have been trying to parse the data into json in lamda function and api request using mapping template for data transformation but i've been largely unsuccessful as the content that sent to the lambda is empty also trying to send same data from postman just shows successfully and my table is empty to be frank im new to cloud and i am still learning i will be providing you with all necessary information with screenshots and code

import boto3
import csv
import decimal
import datetime

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('S####')

def lambda_handler(event, context):
    print('Event:', event)
    if 'body' in event:
        try:
            with open(event['body'], 'r') as csvfile:
                reader = csv.reader(csvfile, delimiter=',')
                for row in reader:
                    datetime_now = datetime.datetime.now()
                    datetime_string = datetime_now.strftime('%Y-%m-%d %H:%M:%S')
                    datetime_timestamp = datetime.datetime.timestamp(datetime_now)
                    id = row[0]
                    # Split the row into two lists
                    data_list1 = row[0:183]
                    data_list2 = [float(v) for v in row[183:]]

                    # Update the DynamoDB table with the data from the CSV file
                    table.put_item(Item={
                        'Date_Time': datetime_timestamp,
                        **{k: (decimal.Decimal(str(v)) if isinstance(v, float) else v) for k, v in zip(data_list1, data_list2)}
                    },
                    ReturnValues='ALL_OLD'
                    )

                    # Check if the data was added to the table
                    response = table.get_item(Key={'Date_Time': datetime_timestamp})
                    if 'Item' in response:
                        print('Data added successfully')
                    else:
                        print('Data was not added to the DynamoDB table')
        except FileNotFoundError:
            print('The CSV file was not found')
        except:
            print('An error occurred:', sys.exc_info()[0])

    return {
        'statusCode': 200,
        'message': 'Data added successfully'
    }

here is the mapping template for x-www-form-ur

#set($allParams = $input.params())

#set($dataParams = $allParams.get("data"))
#set($data = $util.urlDecode($dataParams))

#set($keyParam = $allParams.get("key"))
#set($key = $util.urlDecode($keyParam))

#set($event.body = "$data")

{
    "key": "$key",
    "data": "$data",
    "checksum": "$util.urlDecode($allParams.get("md5sum"))"
}

here's the log from data logger and same is the data i am using to test api in postman

3330 ms : [send body] key=9C-95-6E-7B-C9-11&md5sum=b156d0c8ada364f1a7141eac396510d4&data=ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Power_2,inv_2_DC_Voltage_3,inv_2_DC_Current_3,inv_2_DC_Power_3,inv_2_DC_Voltage_4,inv_2_DC_Current_4,inv_2_DC_Power_4,inv_2_DC_Voltage_5,inv_2_DC_Current_5,inv_2_DC_Power_5,inv_2_DC_Voltage_6,inv_2_DC_Current_6,inv_2_DC_Power_6,inv_2_DC_Voltage_7,inv_2_DC_Current_7,inv_2_DC_Power_7,inv_2_DC_Voltage_8,inv_2_DC_Current_8,inv_2_DC_Power_8,inv_2_DC_Voltage_9,inv_2_DC_Current_9,inv_2_DC_Power_9,inv_2_DC_Voltage_10,inv_2_DC_Current_10,inv_2_DC_Power_10,inv_2_DC_Voltage_11,inv_2_DC_Current_11,inv_2_DC_Power_11,inv_2_DC_Voltage_12,inv_2_DC_Current_12,inv_2_DC_Power_12,inv_2_kWh_Day_Active,inv_2_kWh_Total_Active,inv_2_Status_Code,inv_2_Event_Code,inv_2_percent,inv_1_AC_Active_Power,inv_1_AC_Reactive_Power,inv_1_AC_Frequency,inv_1_kWh_Total_Active,inv_1_kWh_Day_Active,inv_1_Status_Code,inv_1_Event_Code,inv_1_Event_Code_2,inv_1_Event_Code_3,inv_1_Event_Code_4,inv_1_Event_Code_5,inv_1_DC_Voltage_1,inv_1_DC_Current_1,inv_1_DC_Voltage_2,inv_1_DC_Current_2,inv_1_DC_Voltage_3,inv_1_DC_Current_3,inv_1_DC_Power_1,inv_1_DC_Power_2,inv_1_DC_Power_3,inv_1_AC_Voltage_AN,inv_1_AC_Current_A,inv_1_AC_Voltage_BN,inv_1_AC_Current_B,inv_1_AC_Voltage_CN,inv_1_AC_Current_C,inv_1_Temperature_Internal,inv_1_Current_String_1,inv_1_Current_String_2,inv_1_Current_String_3,inv_1_Current_String_4,inv_1_Current_String_5,inv_1_Current_String_6,inv_1_Current_String_7,inv_1_Current_String_8,inv_1_Current_String_9,inv_1_Current_String_10,inv_1_Current_String_11,inv_1_Current_String_12,inv_1_percent,meter_grid_AC_Active_Power,meter_grid_AC_Reactive_Power,meter_grid_AC_Apparent_Power,meter_grid_kWh_Total_Import,meter_grid_AC_Voltage_AN,meter_grid_AC_Voltage_BN,meter_grid_AC_Voltage_CN,meter_grid_AC_Voltage_AB,meter_grid_AC_Voltage_BC,meter_grid_AC_Voltage_CA,meter_grid_AC_Current_A,meter_grid_AC_Current_B,meter_grid_AC_Current_C,meter_grid_AC_Active_Power_A,meter_grid_AC_Active_Power_B,meter_grid_AC_Active_Power_C,meter_grid_AC_Apparent_Power_A,meter_grid_AC_Apparent_Power_B,meter_grid_AC_Apparent_Power_C,meter_grid_AC_Reactive_Power_A,meter_grid_AC_Reactive_Power_B,meter_grid_AC_Reactive_Power_C,meter_grid_AC_PF,meter_grid_AC_Frequency,meter_grid_kVAh_Total_Import,meter_grid_kVARh_Total_Import,meter_grid_kVARh_Total_Export,meter_DG125_AC_Active_Power,meter_DG125_AC_Reactive_Power,meter_DG125_AC_Apparent_Power,meter_DG125_kWh_Total_Import,meter_DG125_AC_Voltage_AN,meter_DG125_AC_Voltage_BN,meter_DG125_AC_Voltage_CN,meter_DG125_AC_Voltage_AB,meter_DG125_AC_Voltage_BC,meter_DG125_AC_Voltage_CA,meter_DG125_AC_Current_A,meter_DG125_AC_Current_B,meter_DG125_AC_Current_C,meter_DG125_AC_Active_Power_A,meter_DG125_AC_Active_Power_B,meter_DG125_AC_Active_Power_C,meter_DG125_AC_Apparent_Power_A,meter_DG125_AC_Apparent_Power_B,meter_DG125_AC_Apparent_Power_C,meter_DG125_AC_Reactive_Power_A,meter_DG125_AC_Reactive_Power_B,meter_DG125_AC_Reactive_Power_C,meter_DG125_AC_PF,meter_DG125_AC_Frequency,meter_DG125_kVAh_Total_Import,meter_DG125_kVARh_Total_Import,meter_DG125_kVARh_Total_Export,meter_DG250_AC_Active_Power,meter_DG250_AC_Reactive_Power,meter_DG250_AC_Apparent_Power,meter_DG250_kWh_Total_Import,meter_DG250_AC_Voltage_AN,meter_DG250_AC_Voltage_BN,meter_DG250_AC_Voltage_CN,meter_DG250_AC_Voltage_AB,meter_DG250_AC_Voltage_BC,meter_DG250_AC_Voltage_CA,meter_DG250_AC_Current_A,meter_DG250_AC_Current_B,meter_DG250_AC_Current_C,meter_DG250_AC_Active_Power_A,meter_DG250_AC_Active_Power_B,meter_DG250_AC_Active_Power_C,meter_DG250_AC_Apparent_Power_A,meter_DG250_AC_Apparent_Power_B,meter_DG250_AC_Apparent_Power_C,meter_DG250_AC_Reactive_Power_A,meter_DG250_AC_Reactive_Power_B,meter_DG250_AC_Reactive_Power_C,meter_DG250_AC_PF,meter_DG250_AC_Frequency,meter_DG250_kVAh_Total_Import,meter_DG250_kVARh_Total_Import,meter_DG250_kVARh_Total_Export,eb_state,kWh_Curtailment_Day,kWh_Curtailment_Lifetime,kWh_Curtailment_DG_Day,kWh_Curtailment_Grid_Day
1691214121,0,0,49.940,232.100,0.120,0,0,233.100,0.100,0,0,235.400,0.120,0,0,490.100,0,0,490.200,0,0,276.200,0,0,490,0,0,490,0,0,490.100,0,0,490.400,0,0,490.100,0,0,489.900,0,0,489.600,0,0,96.600,0,0,96,0,0,0,26852.400,0,0,0,0,0,0,16095,170.800,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,60185.984,-14305.585,61893.344,11868.400,234.840,233.440,232.690,405.390,403.900,404.800,87.664,103.856,72.824,20249.582,23392.458,16985.238,20642.376,24265.726,16985.238,-4007.763,-6451.228,-3846.593,-0.972,49.940,12596.900,645.800,2867,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,883.300,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,918.800,122.800,94.300,0,0,0,0,0
1691214158,0,0,49.920,232.300,0.110,0,0,232.800,0.100,0,0,235,0.130,0,0,493.400,0,0,493.600,0,0,277.200,0,0,493.300,0,0,493.300,0,0,493.400,0,0,493.800,0,0,493.300,0,0,493.300,0,0,493,0,0,97.200,0,0,96.700,0,0,0,26852.400,0,0,0,0,0,0,16095,170.800,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,61599.024,-14434.688,63296.924,11869,234.630,233.250,232.460,405.020,403.540,404.420,89.472,105.704,75.400,20582.072,23784.232,17527.482,20992.814,24655.458,17527.482,-4132.382,-6496.302,-3915.754,-0.973,49.923,12597.600,645.800,2867.200,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,883.300,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,918.800,122.800,94.300,0,0,0,0,0


0 ms : [ip export custom] Checking... cnt=24
10 ms : [headers] Found static csv_headers
35 ms : [file] 1. last_export_file=20230811_0.csv
40 ms : [file] 2. last_export_file=20230811_0.csv
55 ms : 1208115 689973
55 ms : [file] Comparison : last_size <= last_export_filepos + 10 = 0
65 ms : [file] 3. last_export_file=20230811_0.csv
100 ms : [csv] Line: 1691761261,38890,-370,50.060,234.200,55.070,0,0,235.300,55.240,0,0,237.500,55.220,0,0,644.800,6.650,4280,691.500,6.270,4330,344,6.180,2120,749.600,6.130,4600,681,6.530,4450,686.100,6.350,4350,625.700,6.180,3860,607.900,6.350,3860,600,6.460,3870,653.100,6.330,4130,136.600,0.080,10,136.100,0.080,10,210.760,29447,2,0,0,22280,-260,50.040,17584,121.600,2,0,0,0,0,0,606.800,12.440,591.400,12.790,587.500,12.760,7550,7560,7500,235,31.590,237.700,31.650,235.500,31.720,51,6.650,-0.040,6.680,0.030,7.130,-0.030,6.660,-0.010,6.890,0.050,6.640,0.080,0,60358.524,-24576.964,65185.660,21675.500,234.980,233.710,233.270,404.920,405.280,405.610,94.128,88.088,97,20504.932,18869.464,22627.190,22118.198,20587.044,22627.190,-8292.313,-8232.236,-7959.583,-0.927,50.051,23035.400,1575.900,5011.600,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,1390.600,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1453.600,255.700,98.800,0,0,0,0,0
100 ms : [csv] 2023 OK
120 ms : [csv] Line: 1691761321,42220,-290,50.060,233.500,59.730,0,0,235.700,59.970,0,0,237.600,59.960,0,0,644.800,7.270,4690,695.600,6.760,4700,344.100,6.740,2320,749.700,6.710,5030,685,7.070,4840,690.100,6.860,4730,633.700,6.540,4140,611.900,6.840,4190,600,7.060,4240,653.200,6.950,4540,136.900,0.090,10,136.400,0.080,10,211.550,29447.800,2,0,0,24160,-960,50.060,17584,122,2,0,0,0,0,0,611.100,13.500,578.900,14.200,570.100,14.160,8240,8210,8050,234.500,34.290,237.500,34.410,235.900,34.380,51,7.210,-0.060,7.260,0.030,7.920,-0.030,7.410,-0.030,7.670,0.030,7.420,0.060,0,49252.548,-22679.726,54304.400,21676.400,234.630,233.940,232.680,404.610,405.170,404.800,76.560,69.432,87.648,16095.598,14133.363,20390.320,17796.936,16117.139,20390.320,-7593.593,-7746.624,-7340.842,-0.908,50.068,23036.400,1575.900,5012,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,1390.600,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1453.600,255.700,98.800,0,0,0,0,0
125 ms : [csv] 2023 OK
140 ms : [csv] Line: 1691761381,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,211.610,29447.900,0,0,0,27400,-1120,50.040,17585,122.400,2,0,0,0,0,0,611.900,15.300,579.300,15.910,564.700,16.100,9320,9220,9110,236,38.330,238.900,38.400,237.400,38.400,51,8.180,-0.040,8.130,0.070,8.840,-0.040,8.320,-0.030,8.710,0.040,8.450,0.060,0,34351.536,-23857.242,41902.484,21677.100,235.550,234.850,233.830,406.450,406.680,406.610,59.344,54.256,61.512,11608.176,9722.452,14383.350,13978.479,12742.021,14383.350,-7787.692,-8236.083,-7579.152,-0.817,50.045,23037.200,1575.900,5012.400,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,1390.600,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1453.600,255.700,98.800,0,0,0,0,0
155 ms : [csv] 2023 OK
170 ms : [csv] Line: 1691761441,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,212.630,29448.900,0,0,0,30150,-360,50.050,17585,122.900,2,0,0,0,0,0,615.200,16.680,598.500,16.890,581.600,17.490,10280,10150,10170,235.200,42.380,239.300,42.430,236.800,42.500,51,9.010,-0.080,8.870,0.060,9.470,-0.030,8.800,-0.040,9.510,0.030,9.160,0.070,0,29887.372,-47430.432,56132.824,21677.700,235.470,234.070,232.850,405.610,405.160,405.780,80.472,78.472,80.808,10574.492,8648.894,18816.142,18948.740,18367.940,18816.142,-15723.704,-16204.255,-15502.471,-0.543,50.062,23038,1576,5012.800,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,1390.600,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1453.600,255.700,98.800,0,0,0,0,0
180 ms : [csv] 2023 OK
190 ms : [csv] Line: 1691761501,53650,-390,50.090,235.700,75.400,0,0,236.900,75.470,0,0,239.700,75.530,0,0,648.800,8.930,5790,699.500,8.690,6070,364.100,7.650,2780,749.700,8.510,6380,685.100,9.090,6230,694.200,8.770,6090,601.800,8.770,5270,615.900,8.690,5350,600.100,8.770,5260,653.200,8.750,5720,137,0.110,10,136.500,0.100,10,213.850,29450.100,2,0,0,31230,-1430,50.070,17586,123.400,2,0,0,0,0,0,610.400,17.350,596.200,17.680,580.300,18.130,10600,10550,10530,235.200,44.020,238.500,44.120,237.500,44.070,51,9.270,-0.070,9.340,0.070,9.900,-0.050,9.240,-0.030,9.830,0.020,9.540,0.040,0,28175.388,22360.068,36025.396,21678.200,234.710,234.670,232.430,405.500,405.520,404.930,55.280,45.880,52.800,10566.872,7855.781,12292.895,12968.134,10764.364,12292.895,7517.562,7359.229,7483.276,0.783,50.080,23038.800,1576.200,5013.100,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,1390.600,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1453.600,255.700,98.800,0,0,0,0,0
200 ms : [csv] 2023 OK
200 ms : [csv] EXPORT_SIZE=4403
210 ms : [csv] EXPORT_SIZE2=4404
220 ms : [csv] 
Found 5 lines
225 ms : [csv] Export_file =20230811_0.csv
235 ms : [csv] File_size =1208115
240 ms : [csv] Export_size =4404
245 ms : url=6lgxld85fh.execute-api.ap-south-1.amazonaws.com/Dev/*****
245 ms : 47
260 ms : 6lgxld85fh.execute-api.ap-south-1.amazonaws.com port ***
2280 ms : [send header] POST /Dev/**** HTTP/1.0
Host: *********.execute-api.ap-south-1.amazonaws.com 
Content-Type: application/x-www-form-urlencoded
Connection: Keep-Alive
Content-Length: 8724
Topics
ServerlessComputeFront-End Web & MobileNetworking & Content DeliveryDatabase
Tags
AWS LambdaAmazon API GatewayAmazon DynamoDBExtract Transform & Load Data
Language
English
lambda
asked 8 months ago396 views
4 Answers
1

In AWS API Gateway REST API with a single POST method, no other changes (no Content Type headers), I added the following in Integration Request Mapping Template (application/json):

{
#set ($body = $input.body.toString())
#set ($myArray1 = $body.split("&"))
#set ($key = $myArray1[0])
#set ($key = $key.replace("key=", ""))
  "key": "$key",
#set ($checksum =  $myArray1[1])
#set ($checksum = $checksum.replace("md5sum=", ""))
  "checksum": "$checksum",
#set ($data = $myArray1[2])
#set ($data = $data.replace("data=", ""))
  "data": "$data"
}

In the API POST method Test utility, I added the following Request Body (copied from your log above):

key=9C-95-6E-7B-C9&md5sum=b156d0c8ada364f1a7141eac396510d4&data=ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_V

This produced the following JSON document to be passed to your Lambda function:

{
  "key": "9C-95-6E-7B-C9",
  "checksum": "b156d0c8ada364f1a7141eac396510d4",
  "data": "ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_V"
}

I hope this helps get you closer to your solution. If this answered your question please accept the Answer in re:Post.

profile picture
cyrk-aws
answered 8 months ago
  • lambda
    8 months ago

    firstly i would like to thank you for helping me out i have been working on this quite project a while now i am new to cloud and i have limited understanding as to how aws api behave what does mapping template do and what are models for and specifically how what content type does api support natively and why does my lambda function showed up empty event body in cloud watch could you suggest some resources please and now lets talk about the testing yes it does work even in testing end point but i do not understand why does my table not update if message although you've done a lot already to help me also i am updating the previous log and providing latest logs if it would help :)

  • lambda
    8 months ago

    my cloud watch does show this but again on testing with postman

        "statusCode": 200,
    "body": "Data added successfully"
}

    START RequestId: 97f94289-9b9c-4b4b-9456-ba9ce9a0d33c Version: $LATEST
The CSV file was not found
END RequestId: 97f94289-9b9c-4b4b-9456-ba9ce9a0d33c
REPORT RequestId: 97f94289-9b9c-4b4b-9456-ba9ce9a0d33c	Duration: 1.04 ms	Billed Duration: 2 ms	Memory Size: 128 MB	Max Memory Used: 75 MB	
XRAY TraceId: 1-64dbc577-19ea647e0e7338562658bc0b	SegmentId: 1f36a72864fdb44b	Sampled: true	```
  • cyrk-aws
    8 months ago

    The next step, once you have a sample json document from API Gateway, use the Lambda test function to test your code with that json. My guess is the testing you are doing with Postman is not the same way Lambda is being called from API Gateway. This is why it is key first make sure the JSON body being sent from API Gateway is correct, then capture an example document and make sure your Lambda function can process it.

  • lambda
    8 months ago

    i have added more information below as an answer since these forums do not allow images to be added as comments also can we discuss this over email that would seem better

1

Start troubleshooting in API Gateway, select the POST or PUT method and use the TEST feature to simulate creating an API request. Plug in body data, headers, params you used with Postman and hit the Test button.. Look at the response on the Test page, the logs section will show incoming data, the results of any transformation in the mapping template and then the results of trying to call your Lambda.

Once you have the mapping template producing the data in the format your Lambda can handle you can then test/troubleshoot your Lambda.

Hope this helps, not much more information to go on.

profile picture
cyrk-aws
answered 8 months ago
  • lambda
    8 months ago

    Execution log for request fa3818dd-8e3f-459b-b239-49caae9ca0f2 Tue Aug 15 04:11:59 UTC 2023 : Starting execution for request: fa3818dd-8e3f-459b-b239-49caae9ca0f2 Tue Aug 15 04:11:59 UTC 2023 : HTTP Method: POST, Resource Path: /solar Tue Aug 15 04:11:59 UTC 2023 : Method request path: {} Tue Aug 15 04:11:59 UTC 2023 : Method request query string: {} Tue Aug 15 04:11:59 UTC 2023 : Method request headers: {Content-Type=application/x-www-form-urlencoded}Tue Aug 15 04:11:59 UTC 2023 : Method request body before transformations: key=9C-95-6E-7B-C9&md5sum=b156d0c8ada364f1a7141eac396510d4&data=ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_V,[TRUNCATED] Tue Aug 15 04:11:59 UTC 2023 : Request validation succeeded for content type application/json Tue Aug 15 04:11:59 UTC 2023 : Execution failed: null Tue Aug 15 04:11:59 UTC 2023 : Method completed with status: 415 here's a log for api gateway request body unfortunately i can't post ss here Request: /solar Status: 415 Latency: 14 ms Response Body {"message": "Unsupported Media Type"} Response Headers {"x-amzn-ErrorType":["BadRequestException"]} here's my mapping template #set($allParams = $input.params())

    #set($dataParams = $allParams.get("data")) #set($data = $util.urlDecode($dataParams))

    #set($keyParam = $allParams.get("key")) #set($key = $util.urlDecode($keyParam))

    #set($event.body = "$data")

    { "key": "$key", "data": "$data", "checksum": "$util.urlDecode($allParams.get("md5sum"))" }

  • lambda
    8 months ago

    i am getting separate results like this i do not know why,in my api gateway console i have added the mapping template and selected the option to passthrough request body only when there are no templates defined and also i have added following templates and similar binary data types support for api Content-Type:application/x-www-form-urlencoded,application/xml,text/csv,text/plain

0
Accepted Answer

Here's the modified mapping template that is able to completely transform the data into json

{
#set ($body = $input.body.toString())
#set ($myArray1 = $body.split("&"))
#set ($key = $myArray1[0])
#set ($key = $key.replace("key=", ""))
  "key": "$key",
#set ($checksum =  $myArray1[1])
#set ($checksum = $checksum.replace("md5sum=", ""))
  "checksum": "$checksum",
#set ($data = $myArray1[2])
#set ($data = $data.replace("data=", ""))
#set ($data = $util.escapeJavaScript($data))
  "data": "$data"
}

special thanks to cyrk-aws for his mapping template that helped me figure out a way although there's still work needed to be done but as far as mapping template is concerned this is fully functional when tested via aws api gateway with request payload key=9C-95-6E-7B-C9-11&md5sum=b156d0c8ada364f1a7141eac396510d4&data=ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Power_2,inv_2_DC_Voltage_3,inv_2_DC_Current_3,inv_2_DC_Power_3,inv_2_DC_Voltage_4,inv_2_DC_Current_4,inv_2_DC_Power_4,inv_2_DC_Voltage_5,inv_2_DC_Current_5,inv_2_DC_Power_5,inv_2_DC_Voltage_6,inv_2_DC_Current_6,inv_2_DC_Power_6,inv_2_DC_Voltage_7,inv_2_DC_Current_7,inv_2_DC_Power_7,inv_2_DC_Voltage_8,inv_2_DC_Current_8,inv_2_DC_Power_8,inv_2_DC_Voltage_9,inv_2_DC_Current_9,inv_2_DC_Power_9,inv_2_DC_Voltage_10,inv_2_DC_Current_10,inv_2_DC_Power_10,inv_2_DC_Voltage_11,inv_2_DC_Current_11,inv_2_DC_Power_11,inv_2_DC_Voltage_12,inv_2_DC_Current_12,inv_2_DC_Power_12,inv_2_kWh_Day_Active,inv_2_kWh_Total_Active,inv_2_Status_Code,inv_2_Event_Code,inv_2_percent,inv_1_AC_Active_Power,inv_1_AC_Reactive_Power,inv_1_AC_Frequency,inv_1_kWh_Total_Active,inv_1_kWh_Day_Active,inv_1_Status_Code,inv_1_Event_Code,inv_1_Event_Code_2,inv_1_Event_Code_3,inv_1_Event_Code_4,inv_1_Event_Code_5,inv_1_DC_Voltage_1,inv_1_DC_Current_1,inv_1_DC_Voltage_2,inv_1_DC_Current_2,inv_1_DC_Voltage_3,inv_1_DC_Current_3,inv_1_DC_Power_1,inv_1_DC_Power_2,inv_1_DC_Power_3,inv_1_AC_Voltage_AN,inv_1_AC_Current_A,inv_1_AC_Voltage_BN,inv_1_AC_Current_B,inv_1_AC_Voltage_CN,inv_1_AC_Current_C,inv_1_Temperature_Internal,inv_1_Current_String_1,inv_1_Current_String_2,inv_1_Current_String_3,inv_1_Current_String_4,inv_1_Current_String_5,inv_1_Current_String_6,inv_1_Current_String_7,inv_1_Current_String_8,inv_1_Current_String_9,inv_1_Current_String_10,inv_1_Current_String_11,inv_1_Current_String_12,inv_1_percent,meter_grid_AC_Active_Power,meter_grid_AC_Reactive_Power,meter_grid_AC_Apparent_Power,meter_grid_kWh_Total_Import,meter_grid_AC_Voltage_AN,meter_grid_AC_Voltage_BN,meter_grid_AC_Voltage_CN,meter_grid_AC_Voltage_AB,meter_grid_AC_Voltage_BC,meter_grid_AC_Voltage_CA,meter_grid_AC_Current_A,meter_grid_AC_Current_B,meter_grid_AC_Current_C,meter_grid_AC_Active_Power_A,meter_grid_AC_Active_Power_B,meter_grid_AC_Active_Power_C,meter_grid_AC_Apparent_Power_A,meter_grid_AC_Apparent_Power_B,meter_grid_AC_Apparent_Power_C,meter_grid_AC_Reactive_Power_A,meter_grid_AC_Reactive_Power_B,meter_grid_AC_Reactive_Power_C,meter_grid_AC_PF,meter_grid_AC_Frequency,meter_grid_kVAh_Total_Import,meter_grid_kVARh_Total_Import,meter_grid_kVARh_Total_Export,meter_DG125_AC_Active_Power,meter_DG125_AC_Reactive_Power,meter_DG125_AC_Apparent_Power,meter_DG125_kWh_Total_Import,meter_DG125_AC_Voltage_AN,meter_DG125_AC_Voltage_BN,meter_DG125_AC_Voltage_CN,meter_DG125_AC_Voltage_AB,meter_DG125_AC_Voltage_BC,meter_DG125_AC_Voltage_CA,meter_DG125_AC_Current_A,meter_DG125_AC_Current_B,meter_DG125_AC_Current_C,meter_DG125_AC_Active_Power_A,meter_DG125_AC_Active_Power_B,meter_DG125_AC_Active_Power_C,meter_DG125_AC_Apparent_Power_A,meter_DG125_AC_Apparent_Power_B,meter_DG125_AC_Apparent_Power_C,meter_DG125_AC_Reactive_Power_A,meter_DG125_AC_Reactive_Power_B,meter_DG125_AC_Reactive_Power_C,meter_DG125_AC_PF,meter_DG125_AC_Frequency,meter_DG125_kVAh_Total_Import,meter_DG125_kVARh_Total_Import,meter_DG125_kVARh_Total_Export,meter_DG250_AC_Active_Power,meter_DG250_AC_Reactive_Power,meter_DG250_AC_Apparent_Power,meter_DG250_kWh_Total_Import,meter_DG250_AC_Voltage_AN,meter_DG250_AC_Voltage_BN,meter_DG250_AC_Voltage_CN,meter_DG250_AC_Voltage_AB,meter_DG250_AC_Voltage_BC,meter_DG250_AC_Voltage_CA,meter_DG250_AC_Current_A,meter_DG250_AC_Current_B,meter_DG250_AC_Current_C,meter_DG250_AC_Active_Power_A,meter_DG250_AC_Active_Power_B,meter_DG250_AC_Active_Power_C,meter_DG250_AC_Apparent_Power_A,meter_DG250_AC_Apparent_Power_B,meter_DG250_AC_Apparent_Power_C,meter_DG250_AC_Reactive_Power_A,meter_DG250_AC_Reactive_Power_B,meter_DG250_AC_Reactive_Power_C,meter_DG250_AC_PF,meter_DG250_AC_Frequency,meter_DG250_kVAh_Total_Import,meter_DG250_kVARh_Total_Import,meter_DG250_kVARh_Total_Export,eb_state,kWh_Curtailment_Day,kWh_Curtailment_Lifetime,kWh_Curtailment_DG_Day,kWh_Curtailment_Grid_Day 1691214121,0,0,49.940,232.100,0.120,0,0,233.100,0.100,0,0,235.400,0.120,0,0,490.100,0,0,490.200,0,0,276.200,0,0,490,0,0,490,0,0,490.100,0,0,490.400,0,0,490.100,0,0,489.900,0,0,489.600,0,0,96.600,0,0,96,0,0,0,26852.400,0,0,0,0,0,0,16095,170.800,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,60185.984,-14305.585,61893.344,11868.400,234.840,233.440,232.690,405.390,403.900,404.800,87.664,103.856,72.824,20249.582,23392.458,16985.238,20642.376,24265.726,16985.238,-4007.763,-6451.228,-3846.593,-0.972,49.940,12596.900,645.800,2867,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,883.300,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,918.800,122.800,94.300,0,0,0,0,0 1691214158,0,0,49.920,232.300,0.110,0,0,232.800,0.100,0,0,235,0.130,0,0,493.400,0,0,493.600,0,0,277.200,0,0,493.300,0,0,493.300,0,0,493.400,0,0,493.800,0,0,493.300,0,0,493.300,0,0,493,0,0,97.200,0,0,96.700,0,0,0,26852.400,0,0,0,0,0,0,16095,170.800,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,61599.024,-14434.688,63296.924,11869,234.630,233.250,232.460,405.020,403.540,404.420,89.472,105.704,75.400,20582.072,23784.232,17527.482,20992.814,24655.458,17527.482,-4132.382,-6496.302,-3915.754,-0.973,49.923,12597.600,645.800,2867.200,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,883.300,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,918.800,122.800,94.300,0,0,0,0,0

lambda
answered 8 months ago
profile picture
EXPERT
Giovanni Lauria
reviewed a month ago
0

here's my testing method in API gateway since this forum doesn't allow replying with images Enter image description here here's the log message

Request: /*****
Status: 400
Latency: 17 ms
Response Body
{"message": "Could not parse request body into json: Could not parse payload into json: Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: (byte[])\"{\r\n  \"key\": \"9C-95-6E-7B-C9-11\",\r\n  \"checksum\": \"b156d0c8ada364f1a7141eac396510d4\",\r\n  \"data\": \"ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Pow\"[truncated 5288 bytes]; line: 4, column: 4265]"}
Response Headers
{"x-amzn-ErrorType":["BadRequestException"]}
Logs
Execution log for request f47fcfb4-c6b7-40ff-940e-c93b45f2cb24
Thu Aug 17 14:32:38 UTC 2023 : Starting execution for request: f47fcfb4-c6b7-40ff-940e-c93b45f2cb24
Thu Aug 17 14:32:38 UTC 2023 : HTTP Method: POST, Resource Path: /*****
Thu Aug 17 14:32:38 UTC 2023 : Method request path: {}
Thu Aug 17 14:32:38 UTC 2023 : Method request query string: {}
Thu Aug 17 14:32:38 UTC 2023 : Method request headers: {Content-Type=application/json}
Thu Aug 17 14:32:38 UTC 2023 : Method request body before transformations: key=9C-95-6E-7B-C9-11&md5sum=b156d0c8ada364f1a7141eac396510d4&data=ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Power_2,inv_2_DC_Voltage_3,inv_2_DC_Current_3,inv_2_DC_Power_3,inv_2_DC_Voltage_4,inv_2_DC_Current_4,inv_2_DC_Power_4,inv_2_DC_Voltage_5,inv_2_DC_Current_5,inv_2_DC_Power_5,inv_2_DC_Voltage_6,inv_2_DC_Current_6,inv_2_DC_Power_6,inv_2_DC_Voltage_7,inv_2_DC_Current_7,inv_2_DC_Power_7,inv_2_DC_Voltage_8,inv_2_DC_Current_8,inv_2_DC_Power_8,inv_2_DC_Voltage_9,inv_2_DC_Current_9,inv_2_DC_Power_9,inv_2_DC_Voltage_10,inv_2_DC_Current_10,inv_2_DC_Power_10,inv_2_DC_Voltage_11,inv_2_DC_Current_11,inv_2_DC_Power_11,inv [TRUNCATED]
Thu Aug 17 14:32:38 UTC 2023 : Request validation succeeded for content type application/json
Thu Aug 17 14:32:38 UTC 2023 : Endpoint request URI: https://lambda.ap-south-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:ap-south-1:962835665798:function:importfromwattmon/invocations
Thu Aug 17 14:32:38 UTC 2023 : Endpoint request headers: {X-Amz-Date=20230817T143238Z, x-amzn-apigateway-api-id=6lgxld85fh, Accept=application/json, User-Agent=AmazonAPIGateway_6lgxld85fh, Host=lambda.ap-south-1.amazonaws.com, X-Amz-Content-Sha256=d586e560694e1d808aee55be53579c7396103121be42f37fb595bc1df5ddccf5, X-Amzn-Trace-Id=Root=1-64de2f86-3882899036cb4ed4e9f6557d, x-amzn-lambda-integration-tag=f47fcfb4-c6b7-40ff-940e-c93b45f2cb24, Authorization=**********************************************************************************************************************************************************************************************************************************************************************************************************************************************adc955, X-Amz-Source-Arn=arn:aws:execute-api:ap-south-1:962835665798:*****d85fh/test-invoke-stage/POST/****, X-Amz-Security-Token=I*******Z2luX2VjEL7//////////wEaCmFwLXNvdXRoLTEiRjBEAiBcBqiLTO1up8WnjQphPq4/Sx74FClDaH870KpqM81D2gIgAZ0E76M41KEMM6H8tyDkEAGkH5GXsL0qZc/3 [TRUNCATED]
Thu Aug 17 14:32:38 UTC 2023 : Endpoint request body after transformations: {
  "key": "9C-95-6E-7B-C9-11",
  "checksum": "b156d0c8ada364f1a7141eac396510d4",
  "data": "ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Power_2,inv_2_DC_Voltage_3,inv_2_DC_Current_3,inv_2_DC_Power_3,inv_2_DC_Voltage_4,inv_2_DC_Current_4,inv_2_DC_Power_4,inv_2_DC_Voltage_5,inv_2_DC_Current_5,inv_2_DC_Power_5,inv_2_DC_Voltage_6,inv_2_DC_Current_6,inv_2_DC_Power_6,inv_2_DC_Voltage_7,inv_2_DC_Current_7,inv_2_DC_Power_7,inv_2_DC_Voltage_8,inv_2_DC_Current_8,inv_2_DC_Power_8,inv_2_DC_Voltage_9,inv_2_DC_Current_9,inv_2_DC_Power_9,inv_2_DC_Voltage_10,inv_2_DC_Current_10,inv_2_DC_Power_10,inv_2_DC_Voltage_11,inv_2_DC_Cu [TRUNCATED]
Thu Aug 17 14:32:38 UTC 2023 : Sending request to https://lambda.ap-south-1.amazonaws.com/2015-03-31/functions/arn:aws:lambda:ap-south-1:962835665798:function:importfromwattmon/invocations
Thu Aug 17 14:32:38 UTC 2023 : Received response. Status: 400, Integration latency: 11 ms
Thu Aug 17 14:32:38 UTC 2023 : Endpoint response headers: {Date=Thu, 17 Aug 2023 14:32:38 GMT, Content-Type=application/json, Content-Length=808, Connection=keep-alive, x-amzn-RequestId=cd65f8b3-1bb0-4f83-a369-078faf2e30f6, x-amzn-ErrorType=InvalidRequestContentException}
Thu Aug 17 14:32:38 UTC 2023 : Endpoint response body before transformations: {"Type":"User","message":"Could not parse request body into json: Could not parse payload into json: Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value\n at [Source: (byte[])\"{\r\n  \"key\": \"9C-95-6E-7B-C9-11\",\r\n  \"checksum\": \"b156d0c8ada364f1a7141eac396510d4\",\r\n  \"data\": \"ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Pow\"[truncated 5288 bytes]; line: 4, column: 4265]"}
Thu Aug 17 14:32:38 UTC 2023 : Lambda invocation failed with status: 400. Lambda request id: cd65f8b3-1bb0-4f83-a369-078faf2e30f6
Thu Aug 17 14:32:38 UTC 2023 : Execution failed: Could not parse request body into json: Could not parse payload into json: Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value
 at [Source: (byte[])"{
  "key": "9C-95-6E-7B-C9-11",
  "checksum": "b156d0c8ada364f1a7141eac396510d4",
  "data": "ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Pow"[truncated 5288 bytes]; line: 4, column: 4265]
Thu Aug 17 14:32:38 UTC 2023 : Method completed with status: 400

Enter image description here the test data i am using for both is

key=9C-95-6E-7B-C9-11&md5sum=b156d0c8ada364f1a7141eac396510d4&data=ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,inv_2_AC_Voltage_BN,inv_2_AC_Current_B,inv_2_AC_Active_Power_B,inv_2_AC_PF_B,inv_2_AC_Voltage_CN,inv_2_AC_Current_C,inv_2_AC_Active_Power_C,inv_2_AC_PF_C,inv_2_DC_Voltage_1,inv_2_DC_Current_1,inv_2_DC_Power_1,inv_2_DC_Voltage_2,inv_2_DC_Current_2,inv_2_DC_Power_2,inv_2_DC_Voltage_3,inv_2_DC_Current_3,inv_2_DC_Power_3,inv_2_DC_Voltage_4,inv_2_DC_Current_4,inv_2_DC_Power_4,inv_2_DC_Voltage_5,inv_2_DC_Current_5,inv_2_DC_Power_5,inv_2_DC_Voltage_6,inv_2_DC_Current_6,inv_2_DC_Power_6,inv_2_DC_Voltage_7,inv_2_DC_Current_7,inv_2_DC_Power_7,inv_2_DC_Voltage_8,inv_2_DC_Current_8,inv_2_DC_Power_8,inv_2_DC_Voltage_9,inv_2_DC_Current_9,inv_2_DC_Power_9,inv_2_DC_Voltage_10,inv_2_DC_Current_10,inv_2_DC_Power_10,inv_2_DC_Voltage_11,inv_2_DC_Current_11,inv_2_DC_Power_11,inv_2_DC_Voltage_12,inv_2_DC_Current_12,inv_2_DC_Power_12,inv_2_kWh_Day_Active,inv_2_kWh_Total_Active,inv_2_Status_Code,inv_2_Event_Code,inv_2_percent,inv_1_AC_Active_Power,inv_1_AC_Reactive_Power,inv_1_AC_Frequency,inv_1_kWh_Total_Active,inv_1_kWh_Day_Active,inv_1_Status_Code,inv_1_Event_Code,inv_1_Event_Code_2,inv_1_Event_Code_3,inv_1_Event_Code_4,inv_1_Event_Code_5,inv_1_DC_Voltage_1,inv_1_DC_Current_1,inv_1_DC_Voltage_2,inv_1_DC_Current_2,inv_1_DC_Voltage_3,inv_1_DC_Current_3,inv_1_DC_Power_1,inv_1_DC_Power_2,inv_1_DC_Power_3,inv_1_AC_Voltage_AN,inv_1_AC_Current_A,inv_1_AC_Voltage_BN,inv_1_AC_Current_B,inv_1_AC_Voltage_CN,inv_1_AC_Current_C,inv_1_Temperature_Internal,inv_1_Current_String_1,inv_1_Current_String_2,inv_1_Current_String_3,inv_1_Current_String_4,inv_1_Current_String_5,inv_1_Current_String_6,inv_1_Current_String_7,inv_1_Current_String_8,inv_1_Current_String_9,inv_1_Current_String_10,inv_1_Current_String_11,inv_1_Current_String_12,inv_1_percent,meter_grid_AC_Active_Power,meter_grid_AC_Reactive_Power,meter_grid_AC_Apparent_Power,meter_grid_kWh_Total_Import,meter_grid_AC_Voltage_AN,meter_grid_AC_Voltage_BN,meter_grid_AC_Voltage_CN,meter_grid_AC_Voltage_AB,meter_grid_AC_Voltage_BC,meter_grid_AC_Voltage_CA,meter_grid_AC_Current_A,meter_grid_AC_Current_B,meter_grid_AC_Current_C,meter_grid_AC_Active_Power_A,meter_grid_AC_Active_Power_B,meter_grid_AC_Active_Power_C,meter_grid_AC_Apparent_Power_A,meter_grid_AC_Apparent_Power_B,meter_grid_AC_Apparent_Power_C,meter_grid_AC_Reactive_Power_A,meter_grid_AC_Reactive_Power_B,meter_grid_AC_Reactive_Power_C,meter_grid_AC_PF,meter_grid_AC_Frequency,meter_grid_kVAh_Total_Import,meter_grid_kVARh_Total_Import,meter_grid_kVARh_Total_Export,meter_DG125_AC_Active_Power,meter_DG125_AC_Reactive_Power,meter_DG125_AC_Apparent_Power,meter_DG125_kWh_Total_Import,meter_DG125_AC_Voltage_AN,meter_DG125_AC_Voltage_BN,meter_DG125_AC_Voltage_CN,meter_DG125_AC_Voltage_AB,meter_DG125_AC_Voltage_BC,meter_DG125_AC_Voltage_CA,meter_DG125_AC_Current_A,meter_DG125_AC_Current_B,meter_DG125_AC_Current_C,meter_DG125_AC_Active_Power_A,meter_DG125_AC_Active_Power_B,meter_DG125_AC_Active_Power_C,meter_DG125_AC_Apparent_Power_A,meter_DG125_AC_Apparent_Power_B,meter_DG125_AC_Apparent_Power_C,meter_DG125_AC_Reactive_Power_A,meter_DG125_AC_Reactive_Power_B,meter_DG125_AC_Reactive_Power_C,meter_DG125_AC_PF,meter_DG125_AC_Frequency,meter_DG125_kVAh_Total_Import,meter_DG125_kVARh_Total_Import,meter_DG125_kVARh_Total_Export,meter_DG250_AC_Active_Power,meter_DG250_AC_Reactive_Power,meter_DG250_AC_Apparent_Power,meter_DG250_kWh_Total_Import,meter_DG250_AC_Voltage_AN,meter_DG250_AC_Voltage_BN,meter_DG250_AC_Voltage_CN,meter_DG250_AC_Voltage_AB,meter_DG250_AC_Voltage_BC,meter_DG250_AC_Voltage_CA,meter_DG250_AC_Current_A,meter_DG250_AC_Current_B,meter_DG250_AC_Current_C,meter_DG250_AC_Active_Power_A,meter_DG250_AC_Active_Power_B,meter_DG250_AC_Active_Power_C,meter_DG250_AC_Apparent_Power_A,meter_DG250_AC_Apparent_Power_B,meter_DG250_AC_Apparent_Power_C,meter_DG250_AC_Reactive_Power_A,meter_DG250_AC_Reactive_Power_B,meter_DG250_AC_Reactive_Power_C,meter_DG250_AC_PF,meter_DG250_AC_Frequency,meter_DG250_kVAh_Total_Import,meter_DG250_kVARh_Total_Import,meter_DG250_kVARh_Total_Export,eb_state,kWh_Curtailment_Day,kWh_Curtailment_Lifetime,kWh_Curtailment_DG_Day,kWh_Curtailment_Grid_Day
1691214121,0,0,49.940,232.100,0.120,0,0,233.100,0.100,0,0,235.400,0.120,0,0,490.100,0,0,490.200,0,0,276.200,0,0,490,0,0,490,0,0,490.100,0,0,490.400,0,0,490.100,0,0,489.900,0,0,489.600,0,0,96.600,0,0,96,0,0,0,26852.400,0,0,0,0,0,0,16095,170.800,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,60185.984,-14305.585,61893.344,11868.400,234.840,233.440,232.690,405.390,403.900,404.800,87.664,103.856,72.824,20249.582,23392.458,16985.238,20642.376,24265.726,16985.238,-4007.763,-6451.228,-3846.593,-0.972,49.940,12596.900,645.800,2867,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,883.300,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,918.800,122.800,94.300,0,0,0,0,0
1691214158,0,0,49.920,232.300,0.110,0,0,232.800,0.100,0,0,235,0.130,0,0,493.400,0,0,493.600,0,0,277.200,0,0,493.300,0,0,493.300,0,0,493.400,0,0,493.800,0,0,493.300,0,0,493.300,0,0,493,0,0,97.200,0,0,96.700,0,0,0,26852.400,0,0,0,0,0,0,16095,170.800,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,61599.024,-14434.688,63296.924,11869,234.630,233.250,232.460,405.020,403.540,404.420,89.472,105.704,75.400,20582.072,23784.232,17527.482,20992.814,24655.458,17527.482,-4132.382,-6496.302,-3915.754,-0.973,49.923,12597.600,645.800,2867.200,0,0,0,350.200,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,357.800,1.900,66.700,0,0,0,883.300,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,918.800,122.800,94.300,0,0,0,0,0

btw i took this data from this log

Found 2 lines
135 ms : [csv] Export_file =20230805_0.csv
140 ms : [csv] File_size =220296
145 ms : [csv] Export_size =1434
150 ms : url=6lgxld85fh.execute-api.ap-south-1.amazonaws.com/Dev/*****
150 ms : 47
160 ms : 6lgxld85fh.execute-api.ap-south-1.amazonaws.com port 443
3210 ms : [send header] POST /Dev/***** HTTP/1.0
Host: ******85fh.execute-api.ap-south-1.amazonaws.com 
Content-Type: application/x-www-form-urlencoded
Connection: Keep-Alive
Content-Length: 5755
3325 ms : [progress] sent=0
3330 ms : [send body]**the data i provided above **SIZE_SENT=57553475 ms : [receive header] HTTP/1.1 400 Bad Request
3530 ms : [receive header] Date: Sat, 05 Aug 2023 00:12:50 GMT
3540 ms : [receive header] Content-Type: application/json
3550 ms : [receive header] Content-Length: 779
3560 ms : [receive header] Connection: keep-alive
3570 ms : [receive header] x-amzn-RequestId: a7501fc0-7aa4-4f6a-9862-7832ab615891
3595 ms : [receive header] x-amzn-ErrorType: BadRequestException
3615 ms : [receive header] x-amz-apigw-id: JKQQXH3vhcwFVQA=
3625 ms : [receive header] 
3625 ms : [receive] Waiting for body...
3640 ms : [receive body] {"message": "Could not parse request body into json: Could not parse payload into json: Unrecognized token \'key\': was expecting (JSON String, Number, Array, Object or token \'null\', \'true\' or \'false\')\n at [Source: (byte[])\"key=9C-95-6E-7B-C9-11&md5sum=b156d0c8ada364f1a7141eac396510d4&data=ts,inv_2_AC_Active_Power,inv_2_AC_Reactive_Power,inv_2_AC_Frequency,inv_2_AC_Voltage_AN,inv_2_AC_Current_A,inv_2_AC_Active_Power_A,inv_2_AC_PF_A,i
lambda
answered 8 months ago
  • cyrk-aws
    8 months ago

    Using that test API Request body data above (key=9C-95-6E-7B-C9-11&md5sum=...) to transform this data to JSON use a Request Mapping Template like I posted above. Can you share what you want the JSON data to be? This is the JSON you will pass to your Lambda function.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content