Glue Crawler - skip adding partitions
Partitioned data in S3 as below -
some-dataset/2019-12-28/
some-dataset/2019-12-29/
some-dataset/2019-12-30/
some-dataset/2019-12-31/
some-dataset/2020-01-01/
some-dataset/2020-01-02/
some-dataset/2020-01-03/
When running a Glue crawler against the above dataset to create the schema in the catalog, how can we configure the crawler to avoid adding partitions for year 2019 into the table?
If that is not possible, is there a way to totally avoid registering ANY partitions and only create the table schema?
- 新しい順
- 投票が多い順
- コメントが多い順
You can add excludes to the crawler, e.g. for anything 2019 "**/2019-**"
The second question doesn't make any sense to me, if you have a partitioned table you need at least one partition. You can disable the crawler once the table is created with at least one.
関連するコンテンツ
- 質問済み 9ヶ月前
- 質問済み 6ヶ月前
AWS公式更新しました 3年前- AWS公式更新しました 1年前
- AWS公式更新しました 3年前
- AWS公式更新しました 1年前
