SSE error occurs with Glue 3.0 but not 2.0

0

An identical job that is writing a dynamic frame via write_dynamic_frame.from_options is succeeding in Glue 2.0 (as usual) and fails with the below error message when changing the version to Glue 3.0. Anyone run into this and found the cause or workaround?

An error occurred while calling o90.pyWriteDynamicFrame. java.io.IOException: Unknown Server Side Encryption algorithm aws:kms

tjtoll
feita há 3 anos317 visualizações
1 Resposta
0
Resposta aceita

I ran into the same issue. For me, the solution was to switch from using the "s3a" URI schema to just "s3" when reading files.
I went from this
df = spark.read.json('s3a://...')
to this
df = spark.read.json('s3://...')

Hope this helps

CoryMcC
respondido há 3 anos

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas