Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Glue Crawler , negative exclusion pattern

0

How can I apply a negative exclusion pattern on the configuration of my Crawler.

I would like to negate every folder that does not match the following

!prd**/queries/**

I want to exclude everything that does not match with this

  • Gonzalo Herreros EXPERT
    3 years ago

    I don't think that's possible. What you do is multiple inputs to include each of those prd**/queries paths (maybe automating it with a small script)

Topics
Analytics
Tags
AWS Glue
Language
English
asked 3 years ago606 views
1 Answer
0

We cannot negate an exclusion pattern to exclude all folders except those that matches the pattern. Negate (!) is used within brackets [ ] as noted in the doc below - https://docs.aws.amazon.com/glue/latest/dg/define-crawler.html#crawler-data-stores-exclude

If there are too many folders, programmatically filtering and adding them in include path should help in this case.

AWS
answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content