AWS S3 select_object_content TIMESTAMP column

0

Hi I got a parquet file in my bucket i am trying to read using the select_object_content API.

client.select_object_content(
            Bucket="bucket-name",
            Key=file, #"path/file.parquet",
            ExpressionType='SQL',
            Expression=f'SELECT "Time" as t1, "col2"  FROM s3object',
            InputSerialization=input_spec,
            OutputSerialization={'JSON': {}}
            #ScanRange=scan_range
        )

Here the column "Time" is TimestampType column. When i make the call what i get back in the response is some long number that looks like this "45357758218963327341449984"

I am trying to understand what this number represent ? Is there way i can get back the column in iso-format. I initially thought this number is the byte representation that happens when the timestamp value is converted to a binary encoding represenation. But when i applied the logic mentioned https://stackoverflow.com/questions/54657496/how-to-write-timestamp-logical-type-int96-to-parquet-using-parquetwriter It didnt reproduce the value i expected.

Any help is appreciated.

gitfy
asked 2 years ago300 views
1 Answer
0

Timestamp sometimes stores as Epoch.

https://www.epochconverter.com/

  • For example, the current time I'm drafting this response in Epoch is 1653878630
  • The number you've given as an example using Epoch woul be GMT: Saturday, May 2, 3407 1:16:58.963 AM
AWS
Rachel
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions