Did AWS Redshift support Apache Hudi Time Travel Query?

0

Recently, I started using AWS Redshift to query my Apache Hudi format data. And Hudi's time travel query fit my business well. So I wonder if AWS Redshift support Hudi time travel query?

Nep
질문됨 3달 전313회 조회
1개 답변
0
수락된 답변

Hello,

Please note that currently, Redshift does not Support the 'Time Travel' feature that exists in the Delta Tables.

Further, as you are aware, every time changes are made in the Delta Tables, they create new parquet files which contain the latest data. Thus, we are required to generate a new manifest file in order to get a consistent snapshot of the data stored in the table. That is how the Time Travel functionality works, i.e. it is able to look back through these older parquet files and see the previous values.

Considering the above, we can use a workaround where you can create a new table each time the data changes, that is, once the new parquet files are created and the manifest is updated and you can create an external table at the time using the specific manifest file. This would require that you maintain the copies of all the manifest files as well as all of the parquet files, doing so would allow you to do something similar to Delta Tables time travel but in Redshift

I do hope, that the above workaround can point you in the right direction or provide more information as to how you could possibly implement something similar as well as determine if it would be suitable for your use case.

profile pictureAWS
지원 엔지니어
답변함 3달 전
profile pictureAWS
전문가
검토됨 2달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인