1 Answer
- Newest
- Most votes
- Most comments
0
Hi KG,
It is in fact possible for AWS Glue to crawl only new partitions that are added in Amazon S3 folders since the last crawl. If the schemas are compatible, new partitions will be added to existing tables. To do so, click on the name of your crawler > edit. Under Step 2: Choose data sources and classifiers, click edit. Select the Data Source of interest and click edit. From there, you should be able to select the option Crawl new sub-folders only. Hope this helps!
answered 5 months ago
Relevant content
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 3 years ago
Hi I did the same steps you mentioned , that is when i get the error where it does not update the crawlering to new partitions only