Glue crawler: exclude field or predefine it's type

0

I am getting json files to my s3. For example:

{
    "name" : "John",
    "lastname": "Doe",
    "meta" : {
        "x": "a",
        "y": "b",
        "unwanted_field": {
            "some": "stuff",
            "to": "ignore"
        }
    }
}

Is there a way exclude meta.unwanted_field key or predefine it's type as STRING during crawling process? Maybe somebody has a solution how I can achieve this without file reprocessing?

Thank you for any help.

asked 21 days ago84 views
1 Answer
0

Yes, you can prevent a document or several from being read using glob pattern, in the field where I indicate with an arrow

Enter image description here

profile picture
EXPERT
answered 21 days ago
profile picture
EXPERT
reviewed 13 days ago
  • this functionality let's you to exclude whole file based on name pattern. i want to exclude key within the file.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions