Redshift Streaming Ingestion From Kinesis: New Column in Materialized View?

0

I recently added a new key/value pair in the JSON I'm sending to my Kinesis stream that I want to propagate to my Redshift "table" (materialized view). I understand that DDL is not possible in this scenario, so I believe I am stuck with creating a new materialized view entirely?

What is the best approach for preserving the data I have collected in the current materialized view thus far (~150M records over a few months) in order to INSERT/UNION into my new materialized view (with the additional column) once it is up and running?

已提问 1 年前473 查看次数
1 回答
1
已接受的回答

You're correct that you can't directly alter the schema of the existing materialized view. Instead, you'll need to create a new materialized view that includes the additional column.

The most straight-forward approach would be to create a new table in Redshift with the same structure as the materialized view. Then, copy data from the materialized view into the newly created table. You may want to do this in batches depending on the size so you don't run into resource constraints. Once the data are set up, keep you data in sync with automatic inserts or unions from the Kinesis stream. You should then be able to run queries on the table with the updated column.

profile pictureAWS
专家
pechung
已回答 1 年前
profile picture
专家
已审核 1 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则