AWS Glue Notebook issue when running SQL script

Question

I am following the steps outlined in the link below:

https://aws.amazon.com/blogs/big-data/introducing-native-delta-lake-table-support-with-aws-glue-crawlers/

(1) No issue with Query Delta Lake tables using Amazon Athena, able to query out the data.

(2) Issue AWS Glue Notebook:
     
 (a) created IAM role named "AWSGlueServiceRoleDefault" which includes
          
  (i) AmazonS3FullAccess, which is AWS managed

(ii) AWSGlueServiceRole, which is AWS managed

(iii) PassRolePolicy, which is Customer Inline
![Enter image description here](/media/postImages/original/IMJg7oAm3fR3OkaIpVMcdwow)

PassRolePolicy as below:
![Enter image description here](/media/postImages/original/IM8dcWkRD_ToO9eBwrc2TK4Q)

Following the instructions from the link, which I used the IAM role ("AWSGlueServiceRoleDefault") I created above, the first part of the Python code is able to run as shown below:

![Enter image description here](/media/postImages/original/IMeM7YT71ATb6IPltYvZDCAQ)

But the SQL portion throws me a list of Py4JJavaError:

![Enter image description here](/media/postImages/original/IMx0UyBLnATX-5_AwUXV_yuQ)

Even the below Python code which tries to retrieve information about the table generates the same error as below:

![Enter image description here](/media/postImages/original/IMWVoBJf0NSmWjvCY3DVETog)

The above error is repeatable, which can be viewed here:
https://justpaste.it/1zxzz

Would appreciate if you could help me to see which portion is creating the error, and how to remedy it. Thanks.

Accepted Answer

The libraries for the Delta format are not enabled by default.  
The job needs a parameter: --datalake-formats=delta

From a notebook you need to pass it as config before you run any other cell (once the session is started, it won't take effect)  
```
%%configure
{
    "--datalake-formats": "delta"
}
```

AWS Glue Notebook issue when running SQL script

Relevant content