SSE error occurs with Glue 3.0 but not 2.0

Question

An identical job that is writing a dynamic frame via write_dynamic_frame.from_options is succeeding in Glue 2.0 (as usual) and fails with the below error message when changing the version to Glue 3.0. Anyone run into this and found the cause or workaround?  
  
An error occurred while calling o90.pyWriteDynamicFrame. java.io.IOException: Unknown Server Side Encryption algorithm aws:kms

Accepted Answer

I ran into the same issue. For me, the solution was to switch from using the "s3a" URI schema to just "s3" when reading files.  
I went from this   
df = spark.read.json('s3a://...')  
to this  
df = spark.read.json('s3://...')  
  
Hope this helps

SSE error occurs with Glue 3.0 but not 2.0

相关内容