How to export timestream table for modification and reingesting of data?

0

I have a Timestream table with about 500,000 records. There are some of those that I want to delete, clean up and rearrange. What is the best method for this task? I understand that Timestream does not allow for deletions (currently anyway). I am thinking about exporting, cleaning the data, deleting the table, creating a new table and then inserting the cleaned up data into the newly created table.

I don't see anything out there that might fit the bill except writing some pathon or something. Anybody have input for this?

flycast
已提問 2 年前檢視次數 297 次
2 個答案
0

Currently Timestream does not support CTAS statements, there is no mechanism to directly export data from Timestream to a S3 as well. As it stand now, the service has been released only recently and as time goes by it will mature with more features being added.

The simplest method would be to issue a SELECT * query using the required time-range and re-ingest back into Timestream in a new table.

In your case you would have to build a small application to pull relevant data that you require from Timestream using one of our SDKs and then depending on the language, use that languages built in libraries to write that data to the new table. Currently AWS Timestream supports the following languages --> Java, Go, Python, Javascript and .Net. For more information on using the SDKs please refer to article [1].

I also found this 3rd party article that will help you to a certain extent [2] [3].

Following best practices will help you keep the costs down [4].

There is no simple method of extracting certain data from the Timestream table onto another table that I am aware of, you can also consider exporting the data to a Json file using Lambda and use Lambda to export this data to the other Timestream table but this will also require some Lambda for the same.

References: [1] Getting started with SDK's: https://docs.aws.amazon.com/timestream/latest/developerguide/getting-started-sdks.html

[2] Export table to Json: https://stackoverflow.com/questions/69943145/export-aws-timestream-table-to-json-csv

[3] Timestream to Json: https://www.anycodings.com/questions/export-aws-timestream-table-to-jsoncsv

[4] Cost Optimization: https://docs.aws.amazon.com/timestream/latest/developerguide/metering-and-pricing.cost-optimization.html

AWS
支援工程師
已回答 2 年前
0

if all the fixed you want to do fit in a single statement you could write a scheduled query that you manually execute once and target it to a new table

已回答 2 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南