- Newest
- Most votes
- Most comments
I have had this issue. It seems like an acess issue. Are you using Lake Formation? If so, please check if SELECT access has been granted on the table level and 'Describe' has been granted at the database level.
Hello there,
It's intriguing that Athena is returning the COLUMN_NOT_FOUND error even if your Glue jobs and data previews are functioning normally.
In order to find the problem's cause and fix it, let's try the following steps:
-
To access the databases and tables, AWS Glue and Amazon Athena must be in the same AWS region. Verify that the region you selected for both services is the same.
-
Check one of the Parquet files in the S3 bucket manually, since S3 is the store you're using. To check if the columns are constructed properly, you can download a file and examine it with a Parquet viewer.
-
This may sound redundant, but occasionally a straightforward restart of the Glue crawler will assist in detecting the new schemas.
-
When using Athena with Glue, it's crucial to have compatible versions.
If it doesn't work, no worries, we'll sort it out together! Stay cool and keep clouding!
Thank you for your answer; but I am afraid I am still stuck. I have tried using the same version and double-checked the region (points 1 and 4). Point 2: I am sure the Parquet files are OK because when I use Athena straight via S3 (I bypass the catalog) then the query works. And as for point 3: I have not used a Glue crawler in my setup.
Relevant content
- asked 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 4 years ago
The compression should be inside the parquet files, so the reader figures out how the row groups are compressed. It's not relevant to the table since each file (actually each row group inside) could use a different compression. Is Athena able to list the columns in the table list on the left?
@Gonzalo. Yes, the columns are shown correctly in the table list on the left-hand side. If I bypass the Glue Catalog -- i.e. add a table from Athena via S3 directly -- it works as expected.