How to export timestream table for modification and reingesting of data?

0

I have a Timestream table with about 500,000 records. There are some of those that I want to delete, clean up and rearrange. What is the best method for this task? I understand that Timestream does not allow for deletions (currently anyway). I am thinking about exporting, cleaning the data, deleting the table, creating a new table and then inserting the cleaned up data into the newly created table.

I don't see anything out there that might fit the bill except writing some pathon or something. Anybody have input for this?

flycast
gefragt vor 2 Jahren299 Aufrufe
2 Antworten
0

Currently Timestream does not support CTAS statements, there is no mechanism to directly export data from Timestream to a S3 as well. As it stand now, the service has been released only recently and as time goes by it will mature with more features being added.

The simplest method would be to issue a SELECT * query using the required time-range and re-ingest back into Timestream in a new table.

In your case you would have to build a small application to pull relevant data that you require from Timestream using one of our SDKs and then depending on the language, use that languages built in libraries to write that data to the new table. Currently AWS Timestream supports the following languages --> Java, Go, Python, Javascript and .Net. For more information on using the SDKs please refer to article [1].

I also found this 3rd party article that will help you to a certain extent [2] [3].

Following best practices will help you keep the costs down [4].

There is no simple method of extracting certain data from the Timestream table onto another table that I am aware of, you can also consider exporting the data to a Json file using Lambda and use Lambda to export this data to the other Timestream table but this will also require some Lambda for the same.

References: [1] Getting started with SDK's: https://docs.aws.amazon.com/timestream/latest/developerguide/getting-started-sdks.html

[2] Export table to Json: https://stackoverflow.com/questions/69943145/export-aws-timestream-table-to-json-csv

[3] Timestream to Json: https://www.anycodings.com/questions/export-aws-timestream-table-to-jsoncsv

[4] Cost Optimization: https://docs.aws.amazon.com/timestream/latest/developerguide/metering-and-pricing.cost-optimization.html

AWS
SUPPORT-TECHNIKER
beantwortet vor 2 Jahren
0

if all the fixed you want to do fit in a single statement you could write a scheduled query that you manually execute once and target it to a new table

beantwortet vor 2 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen