By using AWS re:Post, you agree to the Terms of Use

JSON Custom Classifier not parsing data

0

I have the following JSON data, which I would like to parse as individual records:

[{
  "Application": 0,
  "BaselineSetName": "",
  "CompareToSetName": "",
  "Date": 1658500320,
  "EnableOptimizedVisualization": 1,
  "EnableRuntimeChunkMerging": 1,
  "IsHostBasedRunForRapidAccelSim": 0,
  "MarkedForDelete": 0,
  "OutputMetaData": {
    "DatasetSignalFormat": 0
  },
  "id": 4
}, {
  "Application": 0,
  "BaselineSetName": "",
  "CompareToSetName": "",
  "Date": 1658504698,
  "EnableOptimizedVisualization": 1,
  "EnableRuntimeChunkMerging": 0,
  "IsHostBasedRunForRapidAccelSim": 0,
  "MarkedForDelete": 0,
  "OutputMetaData": {
    "DatasetSignalFormat": 0
  },
  "id": 65
}]

I tried several combinations of JSON classifiers like:

  1. $[*].
  2. $[*].OutputMetaData

My data is always parsed as an array object. I have deleted the table and run the modified crawler several times but the output is always an array oject. Is there a trick to editing classifiers, or checking the output without running the crawler?

  • I am seeing the same issue. It keeps creating a table with the entire record as a struct even though I am specifying one column:

    eg. [ {"A":1,"B":"test1"}, {"A":2,"B":"test2"} ]

    With either $[*] OR and with classifier $[*].A I get the following structure:

    CREATE EXTERNAL TABLE testing( array array<struct<A:int,B:string>> COMMENT 'from deserializer') ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe' WITH SERDEPROPERTIES ( 'paths'='array') ....

asked 2 months ago25 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions