Hello!
We have an IoT Core Rule which is fired when a MQTT message is published to certain topic. The message structure is:
{
"triggers": ["door"],
"datetime": "2022-06-01T00:00:00Z",
"errCodes": [100],
"strength": 107,
"net": "GMS",
"eco": 0,
"light": 0,
"def": 0,
"fan": 0,
"defrost": 1,
"Mdef": 0,
"comp": 0,
"door": 0,
"Tcond": 38.1,
"Tevap": 1,
"Tint": 3.8,
"topic": "abc/ar/data/NVC1/test-vscode-3"
}
We have a requirement where data buffered in remote devices could be sent to IoT Core, so we need to send a "datetime" field (the second one) in the payload. The Iot Core Rule fires a AWS Lambda function, which generates a multi-measure record to be finally sent to Timestream:
{
"Dimensions":[
{
"Name":"hw_model",
"Value":"NVC1"
},
{
"Name":"serial_device",
"Value":"test-vscode-3"
}
],
"MeasureName":"multimeasuredata",
"MeasureValueType":"MULTI",
"MeasureValues":[
{
"Name":"Tint",
"Value":"3.8",
"Type":"DOUBLE"
},
{
"Name":"Tevap",
"Value":"1",
"Type":"DOUBLE"
}
],
"Time":"1654041600000"
}
The Timestream table retention periods are:
Memory store retention: 45 days
Magnetic store retention: 180 days
Magnetic store writes: ENABLED
The exception thrown is:
{
"errorType":"RejectedRecordsException",
"errorMessage":"One or more records have been rejected. See RejectedRecords for details.",
"name":"RejectedRecordsException",
"$fault":"client",
"$metadata":{
"httpStatusCode":419,
"requestId":"VKL72WIIMCBGQNWMMSQLK7CAAQ",
"attempts":1,
"totalRetryDelay":0
},
"RejectedRecords":[
{
"Reason":"The record timestamp is outside the time range [2022-06-17T15:21:13.756Z, 2022-06-27T22:51:04.174Z) of the data ingestion window.",
"RecordIndex":0
}
],
"__type":"com.amazonaws.timestream.v20181101#RejectedRecordsException",
"message":"One or more records have been rejected. See RejectedRecords for details.",
"stack":[
"RejectedRecordsException: One or more records have been rejected. See RejectedRecords for details.",
" at deserializeAws_json1_0RejectedRecordsExceptionResponse (/var/task/node_modules/@aws-sdk/client-timestream-write/dist-cjs/protocols/Aws_json1_0.js:947:23)",
" at deserializeAws_json1_0WriteRecordsCommandError (/var/task/node_modules/@aws-sdk/client-timestream-write/dist-cjs/protocols/Aws_json1_0.js:888:25)",
" at processTicksAndRejections (node:internal/process/task_queues:96:5)",
" at async /var/task/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24",
" at async /var/task/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:11:20",
" at async StandardRetryStrategy.retry (/var/task/node_modules/@aws-sdk/middleware-retry/dist-cjs/StandardRetryStrategy.js:51:46)",
" at async /var/task/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:6:22",
" at async Runtime.exports.handler (/var/task/lambda.js:58:20)"
]
}
We are not falling out the memory retention period (45 days) in this example (27 days), but the range returned in the exception is (IDK why) [2022-06-17T15:21:13.756Z, 2022-06-27T22:51:04.174Z)
Do you have any ideas of why this is the range and hence why the record cannot be inserted?
Thanks !
I'll assume this is caused by what you're describing, since the retention periods were modified a couple of times until finding a good value. Thanks for your support!