It's intriguing that Athena is returning the COLUMN_NOT_FOUND error even if your Glue jobs and data previews are functioning normally.
In order to find the problem's cause and fix it, let's try the following steps:
To access the databases and tables, AWS Glue and Amazon Athena must be in the same AWS region. Verify that the region you selected for both services is the same.
Check one of the Parquet files in the S3 bucket manually, since S3 is the store you're using. To check if the columns are constructed properly, you can download a file and examine it with a Parquet viewer.
This may sound redundant, but occasionally a straightforward restart of the Glue crawler will assist in detecting the new schemas.
When using Athena with Glue, it's crucial to have compatible versions.
If it doesn't work, no worries, we'll sort it out together! Stay cool and keep clouding!
- Accepted Answerasked a year ago
- AWS OFFICIALUpdated 2 years ago
- Why does my AWS Glue job fail with lost nodes when I migrate a large data set from Amazon RDS to Amazon S3?AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 8 months ago
- EXPERTpublished a year ago
- EXPERTpublished 4 months ago