flatten deeply nested JSON with Crawler

0

My customer wants to flatten deeply nested JSON object. They used Glue Crawler Classifier with $[*] (lift the array elements up one level, so that each JSON record gets loaded into its own row). When they ran Crawler and view results he saw some array type instead of struct.

I saw a previous response to similar but need to understand in more details how to fix that

AWS
질문됨 4년 전2468회 조회
1개 답변
0
수락된 답변

They are seeing Arrays and Structs based on the schema of the JSON document.

{
   "event_params":[ {"key":"Value"}, {"Key","value"}],
   "geolocation": { "lat": 56.333333, "lng": 57.44333 }
}

If this is my JSON document, when I run the crawler it will result in this schema

event_params: Array
geolocation: Struct

The customer can use Custom Classifiers in Glue to extract the data further. Or, they can keep it as is, and run a glue job to format the data into a new table in columnar format.

답변함 4년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠