AWS Crawler to directly read Delta lake files from S3

0

Are there any ways to read delta lake files from s3 and create Data catalog on top of this to run Glue ETL job? When I crawl in delta folders it creates separate schema for log, manifest & parquets rather then each tables with all the delta log, manifest files and parquet files , https://docs.aws.amazon.com/glue/latest/dg/crawler-data-stores.html says Crawler now have native client then why it is not recognizing the path , Am I missing something.

已提问 2 年前452 查看次数
1 回答
0

Hi,

Please try installing the Delta Lake Connector for AWS Glue which can be found here.

https://aws.amazon.com/marketplace/pp/prodview-seypofzqhdueq

The Delta Lake Connector will allow you to connect to Delta Lake tables from your Glue jobs and it is offered at no additional cost.

Hope this helps.

profile pictureAWS
已回答 2 年前
profile picture
支持工程师
已审核 2 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则