I have the following JSON data, which I would like to parse as individual records:
[{
"Application": 0,
"BaselineSetName": "",
"CompareToSetName": "",
"Date": 1658500320,
"EnableOptimizedVisualization": 1,
"EnableRuntimeChunkMerging": 1,
"IsHostBasedRunForRapidAccelSim": 0,
"MarkedForDelete": 0,
"OutputMetaData": {
"DatasetSignalFormat": 0
},
"id": 4
}, {
"Application": 0,
"BaselineSetName": "",
"CompareToSetName": "",
"Date": 1658504698,
"EnableOptimizedVisualization": 1,
"EnableRuntimeChunkMerging": 0,
"IsHostBasedRunForRapidAccelSim": 0,
"MarkedForDelete": 0,
"OutputMetaData": {
"DatasetSignalFormat": 0
},
"id": 65
}]
I tried several combinations of JSON classifiers like:
- $[*].
- $[*].OutputMetaData
My data is always parsed as an array object. I have deleted the table and run the modified crawler several times but the output is always an array oject. Is there a trick to editing classifiers, or checking the output without running the crawler?
AWS OFFICIALUpdated 2 months ago
I am seeing the same issue. It keeps creating a table with the entire record as a struct even though I am specifying one column:
eg. [ {"A":1,"B":"test1"}, {"A":2,"B":"test2"} ]
With either $[*] OR and with classifier $[*].A I get the following structure:
CREATE EXTERNAL TABLE
testing(arrayarray<struct<A:int,B:string>> COMMENT 'from deserializer') ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES ( 'paths'='array') ....