How to export timestream table for modification and reingesting of data?

0

I have a Timestream table with about 500,000 records. There are some of those that I want to delete, clean up and rearrange. What is the best method for this task? I understand that Timestream does not allow for deletions (currently anyway). I am thinking about exporting, cleaning the data, deleting the table, creating a new table and then inserting the cleaned up data into the newly created table.

I don't see anything out there that might fit the bill except writing some pathon or something. Anybody have input for this?

flycast
질문됨 2년 전298회 조회
2개 답변
0

Currently Timestream does not support CTAS statements, there is no mechanism to directly export data from Timestream to a S3 as well. As it stand now, the service has been released only recently and as time goes by it will mature with more features being added.

The simplest method would be to issue a SELECT * query using the required time-range and re-ingest back into Timestream in a new table.

In your case you would have to build a small application to pull relevant data that you require from Timestream using one of our SDKs and then depending on the language, use that languages built in libraries to write that data to the new table. Currently AWS Timestream supports the following languages --> Java, Go, Python, Javascript and .Net. For more information on using the SDKs please refer to article [1].

I also found this 3rd party article that will help you to a certain extent [2] [3].

Following best practices will help you keep the costs down [4].

There is no simple method of extracting certain data from the Timestream table onto another table that I am aware of, you can also consider exporting the data to a Json file using Lambda and use Lambda to export this data to the other Timestream table but this will also require some Lambda for the same.

References: [1] Getting started with SDK's: https://docs.aws.amazon.com/timestream/latest/developerguide/getting-started-sdks.html

[2] Export table to Json: https://stackoverflow.com/questions/69943145/export-aws-timestream-table-to-json-csv

[3] Timestream to Json: https://www.anycodings.com/questions/export-aws-timestream-table-to-jsoncsv

[4] Cost Optimization: https://docs.aws.amazon.com/timestream/latest/developerguide/metering-and-pricing.cost-optimization.html

AWS
지원 엔지니어
답변함 2년 전
0

if all the fixed you want to do fit in a single statement you could write a scheduled query that you manually execute once and target it to a new table

답변함 2년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠