1 Answer
- Newest
- Most votes
- Most comments
0
The exclude pattern could be of special help here:
try using the version*/_temporary**
as the exclude pattern.
This would exclude all the unwanted files other than the parquet files.
For the include pattern, use s3://a/b/c/products/'
you would not need to provide a level for this case.
Check "Create single schema for each S3 path"
This would create one table with "version*" as partitions.
Reference: https://docs.aws.amazon.com/glue/latest/dg/crawler-s3-folder-table-partition.html
answered 2 years ago
Relevant content
- Accepted Answerasked 2 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated 5 months ago