How to export timestream table for modification and reingesting of data?

0

I have a Timestream table with about 500,000 records. There are some of those that I want to delete, clean up and rearrange. What is the best method for this task? I understand that Timestream does not allow for deletions (currently anyway). I am thinking about exporting, cleaning the data, deleting the table, creating a new table and then inserting the cleaned up data into the newly created table.

I don't see anything out there that might fit the bill except writing some pathon or something. Anybody have input for this?

flycast
質問済み 2年前283ビュー
2回答
0

Currently Timestream does not support CTAS statements, there is no mechanism to directly export data from Timestream to a S3 as well. As it stand now, the service has been released only recently and as time goes by it will mature with more features being added.

The simplest method would be to issue a SELECT * query using the required time-range and re-ingest back into Timestream in a new table.

In your case you would have to build a small application to pull relevant data that you require from Timestream using one of our SDKs and then depending on the language, use that languages built in libraries to write that data to the new table. Currently AWS Timestream supports the following languages --> Java, Go, Python, Javascript and .Net. For more information on using the SDKs please refer to article [1].

I also found this 3rd party article that will help you to a certain extent [2] [3].

Following best practices will help you keep the costs down [4].

There is no simple method of extracting certain data from the Timestream table onto another table that I am aware of, you can also consider exporting the data to a Json file using Lambda and use Lambda to export this data to the other Timestream table but this will also require some Lambda for the same.

References: [1] Getting started with SDK's: https://docs.aws.amazon.com/timestream/latest/developerguide/getting-started-sdks.html

[2] Export table to Json: https://stackoverflow.com/questions/69943145/export-aws-timestream-table-to-json-csv

[3] Timestream to Json: https://www.anycodings.com/questions/export-aws-timestream-table-to-jsoncsv

[4] Cost Optimization: https://docs.aws.amazon.com/timestream/latest/developerguide/metering-and-pricing.cost-optimization.html

AWS
サポートエンジニア
回答済み 2年前
0

if all the fixed you want to do fit in a single statement you could write a scheduled query that you manually execute once and target it to a new table

回答済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ