Glue crawler: exclude field or predefine it's type


I am getting json files to my s3. For example:

    "name" : "John",
    "lastname": "Doe",
    "meta" : {
        "x": "a",
        "y": "b",
        "unwanted_field": {
            "some": "stuff",
            "to": "ignore"

Is there a way exclude meta.unwanted_field key or predefine it's type as STRING during crawling process? Maybe somebody has a solution how I can achieve this without file reprocessing?

Thank you for any help.

feita há 2 meses112 visualizações
1 Resposta

Yes, you can prevent a document or several from being read using glob pattern, in the field where I indicate with an arrow

Enter image description here

profile picture
respondido há 2 meses
profile picture
avaliado há 2 meses
  • this functionality let's you to exclude whole file based on name pattern. i want to exclude key within the file.

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas