How to create simple Athena table from simple JSON data?


I have simple JSON log data like:

  "data": {
    "datum1": "value1",

How do I get that stuff easily in the Athena so I can type:
select * from mytable where data.ts > '2022-06-01 and datum1 = 'filter'

The documention is really comprehensive (spent a couple of hours browsing it), but I am really not that interesting about knowing all the SerDe details, I just need to get this simple thing going and preferrably fast. Like in Snowflake (took a minute or two to do it).

BR, Joni

asked 2 months ago29 views
1 Answer

I was able to get a crawler to crawl the data and write a quick query in Athena to be able to accomplish your use case effectively. Please take a look and try it out. If you have any questions, please let me know.

Step 1: I mocked up data for 10 different dates and moved the data in S3 Enter image description here Step 2: I created a crawler to crawl the S3 location and ran the crawler. It created a table with 2 columns referer and data. Step 3: I ran the athena query below to get the dates > 9/4/2022

SELECT referer, data.datum2, data.ts, data.datum1, date_format(from_unixtime(data.ts),'%Y-%m-%d') FROM "raw_db"."nested_json" 
where date_format(from_unixtime(data.ts),'%Y-%m-%d') > '2022-09-04'
limit 10;

Enter image description here

profile picture
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions