How to update parque format_version locally?

So, we are having an issue with one table in our ETL process stacking parque files.

The question is, at DMS we can set parquet format_version 1_0 or 2_0, How this change can impact production and there is a form that I can update the version locally? I've already downloaded the files but can't find a way to update the version

Thanks.

主題: 遷移與現代化
標籤: AWS 資料庫移轉服務
語言: English

Marcelo

已提問 2 年前檢視次數 269 次

1 個回答

最新
最多得票
最多評論

you can try using pyarrow which could change the version

import pyarrow as pa
import pyarrow.parquet as pq

# Read Parquet file (version 2.0)
table = pq.read_table('input.parquet')

# Write Parquet file (version 1.0)
pq.write_table(table, 'output.parquet', version='1.0')

Subhash Raut

已回答 2 年前

專家

Osvaldo Marte

已審閱 2 年前

How to update parque format_version locally?

新增您的答案

相關內容