1 Answer
- Newest
- Most votes
- Most comments
1
Custom classifiers cannot be used to create partitioned table if the data itself is not partitioned.
Crawler creates a partitioned table based on directory structure and schema similarity at each level. Suppose there are 2 sub-directories in a target path and data in these directories are at least 70% similar, then it creates a partitioned table, otherwise it creates two table one for each sub-directory.
answered a year ago
Relevant content
- Accepted Answerasked 4 years ago
- asked 2 years ago
- asked 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 8 months ago
Thanks Sachin , do you have any workflow recommendation besides repartition of data directories ?