Read VPC flow logs on S3 bucket with Athena
I'm trying to use Athena to generate queries from VPC flow logs stored in an S3 bucket. I followed the guideline of "Querying Amazon VPC flow logs"[1], and I tried to access by both folders and files.
However, the generated queries were empty, with only title and no contents. I double checked that the files are not empty, and that the path is correct. I also tried to create partitions, but the results were unreadable.
Is there a way to access the contents of the S3 bucket? Thank you!
- Newest
- Most votes
- Most comments
This article has generic info applicable to VPC Flow Logs, especially about partiioning and querying - https://www.linkedin.com/pulse/using-athena-partition-projection-query-cloudtrail-other-kinsman/.
It's hard to be sure from the image you pasted, but it looks like you've both set LOCATION to a specific date "2023/03/01/" and partitioned by date? That doesn't look right.
What query did you try?
Relevant content
- Accepted Answerasked 2 years ago
- asked 6 months ago
AWS OFFICIALUpdated 4 months ago- AWS OFFICIALUpdated 6 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 9 months ago
I tired both "s3://.../2023/" and "s3://.../2023/03/01/". They return the same results. I just did it again and I'll add the screenshot in the original post.
I tried the "ALTER TABLE vpc_flow_logs ADD PARTITION (
date='YYYY-MM-dd') LOCATION ..." query but the results were unreadable.I'm trying out the LinkedIn article right now.
Thanks!
Should have used s3://.../{location}/