- 최신
- 최다 투표
- 가장 많은 댓글
That is not supported, even if you solve the path issues, that is not a valid Delta/symlink table, it will create a generic Hive table with an array field.
The OSS Delta community is working to rectify that, in the meanwhile you can register Delta tables on the catalog using a Glue crawler or the DeltaTable "generate" API , but not directly via SparkSQL
Thanks for confirming it. If it is not supported the first example in this doc page https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-delta-lake.html should be corrected as it produces the exact same issue. It basically attempts to do the same thing using pyspark syntax.
Example: Write a Delta Lake table to Amazon S3 and register it to the AWS Glue Data Catalog The following AWS Glue ETL script demonstrates how to write a Delta Lake table to Amazon S3 and register the table to the AWS Glue Data Catalog.*
Thanks for reporting that
관련 콘텐츠
- AWS 공식업데이트됨 2년 전
- AWS 공식업데이트됨 일 년 전
- AWS 공식업데이트됨 3년 전
I am able to use Spark SQL (CREATE OR REPLACE TABLE USING DELTA AS SELECT * FROM TempVIEW)
Setting database location to a appropriate s3 path will resolve issue. In glue catalog go to database and edit the database to have s3 path location. Use glue version 4.0 or latest.
We don't need to put location in spark code.