Redshift Streaming Ingestion From Kinesis: New Column in Materialized View?

0

I recently added a new key/value pair in the JSON I'm sending to my Kinesis stream that I want to propagate to my Redshift "table" (materialized view). I understand that DDL is not possible in this scenario, so I believe I am stuck with creating a new materialized view entirely?

What is the best approach for preserving the data I have collected in the current materialized view thus far (~150M records over a few months) in order to INSERT/UNION into my new materialized view (with the additional column) once it is up and running?

asked a year ago454 views
1 Answer
1
Accepted Answer

You're correct that you can't directly alter the schema of the existing materialized view. Instead, you'll need to create a new materialized view that includes the additional column.

The most straight-forward approach would be to create a new table in Redshift with the same structure as the materialized view. Then, copy data from the materialized view into the newly created table. You may want to do this in batches depending on the size so you don't run into resource constraints. Once the data are set up, keep you data in sync with automatic inserts or unions from the Kinesis stream. You should then be able to run queries on the table with the updated column.

profile pictureAWS
EXPERT
pechung
answered a year ago
profile picture
EXPERT
reviewed 23 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions