Glue crawler: exclude field or predefine it's type

0

I am getting json files to my s3. For example:

{
    "name" : "John",
    "lastname": "Doe",
    "meta" : {
        "x": "a",
        "y": "b",
        "unwanted_field": {
            "some": "stuff",
            "to": "ignore"
        }
    }
}

Is there a way exclude meta.unwanted_field key or predefine it's type as STRING during crawling process? Maybe somebody has a solution how I can achieve this without file reprocessing?

Thank you for any help.

已提問 2 個月前檢視次數 115 次
1 個回答
0

Yes, you can prevent a document or several from being read using glob pattern, in the field where I indicate with an arrow

Enter image description here

profile picture
專家
已回答 2 個月前
profile picture
專家
已審閱 2 個月前
  • this functionality let's you to exclude whole file based on name pattern. i want to exclude key within the file.

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南