Redshift Streaming Ingestion From Kinesis: New Column in Materialized View?

0

I recently added a new key/value pair in the JSON I'm sending to my Kinesis stream that I want to propagate to my Redshift "table" (materialized view). I understand that DDL is not possible in this scenario, so I believe I am stuck with creating a new materialized view entirely?

What is the best approach for preserving the data I have collected in the current materialized view thus far (~150M records over a few months) in order to INSERT/UNION into my new materialized view (with the additional column) once it is up and running?

feita há um ano473 visualizações
1 Resposta
1
Resposta aceita

You're correct that you can't directly alter the schema of the existing materialized view. Instead, you'll need to create a new materialized view that includes the additional column.

The most straight-forward approach would be to create a new table in Redshift with the same structure as the materialized view. Then, copy data from the materialized view into the newly created table. You may want to do this in batches depending on the size so you don't run into resource constraints. Once the data are set up, keep you data in sync with automatic inserts or unions from the Kinesis stream. You should then be able to run queries on the table with the updated column.

profile pictureAWS
ESPECIALISTA
pechung
respondido há um ano
profile picture
ESPECIALISTA
avaliado há um mês

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas