- 最新
- 最多得票
- 最多評論
Hi NadirS
Looks like you have a late data arrival issue, when query finished data does not occur after several hours?
"The write occur without error" --> Do you mean query finished fast in a minute or two?
This does feel need more investigation, I would suggest open a support case with us to investigate further.
We opened a case with AWS Support and they pointed us to the relevant documentation.
The maximum latency for reads after magnetic writes is 6 hours. You can find that number in the documentation here, https://docs.aws.amazon.com/timestream/latest/developerguide/data-ingest.html, emphasis mine:
If you expect late arriving data, turn on magnetic store writes for your table. This will allow ingestion for data with timestamps that fall outside your memory store retention period but still within your magnetic store retention period. You can set this by updating the EnableMagneticStoreWrites flag in the MagneticStoreWritesProperties for your table. This property is false by default. Please note that writes to the magnetic store will not be immediately available to query. They will be available within 6 hours.
There is also a useful metric in CloudWatch for tracking this latency. The MagneticStorePendingRecordsLatency provides the oldest magnetic write not available for querying. https://docs.aws.amazon.com/timestream/latest/developerguide/metrics-dimensions.html#mcs-metrics
In my case, the MagneticStorePendingRecordsLatency was about 4.5 hours for a recent batch write, which is within the documented 6 hour limit.
相關內容
- 已提問 6 個月前
Thanks for your response, Kevin. I mean that the query finishes fast (sub 10 seconds), but the data isn't there. If I re-run the same query hours or a day after I do the batch, magnetic write, the data is there.
I have opened a support case, but I haven't heard back. The documentation for Timestream does say it is eventually consistent, so I don't expect the data to return immediately post-write, but the documentation says a "short time". I would like further clarification on the definition of "short time". I also expect that the magnetic write consistency latency will be higher than the memory write consistency latency, but the amount of latency I'm currently seeing seems high.
https://docs.aws.amazon.com/timestream/latest/developerguide/writes.html#writes.eventual-consistency