Query data stored in S3 using Athena

Question

Hi AWS, I have a folder inside s3 bucket where the cost and usage report data is stored both in .csv and .csv.gz format. When I am creating the TABLE using that LOCATION the records are not printed in proper format. Also, I need to segregate both type of files so that I not need to spend my time in data munging, cleaning etc.

Can you please help in knowing how to query .csv.gz file formats using Athena and print the proper formatted records.

Thanks

Can you please help in knowing how to query .csv.gz file formats using Athena and print the proper formatted records.

Thanks

Answer

CUR supports Apache Parquet compression format for Athena.

While creating CUR, select whether you want to enable your Cost and Usage Reports to integrate with Amazon Athena, Amazon Redshift, or Amazon QuickSight. The report is compressed in the following formats:
Athena: parquet format
Amazon Redshift or Amazon QuickSight: .gz compression

Refer to this for more details: https://docs.aws.amazon.com/cur/latest/userguide/cur-query-athena

Query data stored in S3 using Athena

Contenido relevante