SSE error occurs with Glue 3.0 but not 2.0

0

An identical job that is writing a dynamic frame via write_dynamic_frame.from_options is succeeding in Glue 2.0 (as usual) and fails with the below error message when changing the version to Glue 3.0. Anyone run into this and found the cause or workaround?

An error occurred while calling o90.pyWriteDynamicFrame. java.io.IOException: Unknown Server Side Encryption algorithm aws:kms

tjtoll
asked a year ago95 views
1 Answer
0
Accepted Answer

I ran into the same issue. For me, the solution was to switch from using the "s3a" URI schema to just "s3" when reading files.
I went from this
df = spark.read.json('s3a://...')
to this
df = spark.read.json('s3://...')

Hope this helps

CoryMcC
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions