- Newest
- Most votes
- Most comments
The "TABLE_NOT_FOUND" error you're experiencing when running Athena queries from Lambda is a common issue that typically stems from permissions or configuration problems. Based on your description, there are several potential causes to investigate:
-
IAM Role Permissions: The most likely cause is that your Lambda execution role is missing the necessary Glue permissions. Specifically, your Lambda role needs permissions like
glue:GetTableandglue:GetDatabaseto access the table metadata in the Glue Data Catalog. This is a common oversight - the role might have Athena permissions but not the required Glue permissions. -
Catalog Configuration: When using a custom catalog (not the default AwsDataCatalog), ensure that your Lambda role has the appropriate permissions to access that specific catalog. Your code shows you're using "my_custom_catalog" - verify that this catalog name is exactly correct, including case sensitivity.
-
Query Execution Context: While your code appears to set the QueryExecutionContext correctly, the error message showing it ran against an empty database ("") suggests the context isn't being applied properly. Double-check that there are no typos in your catalog or database names.
-
Cross-Account Access: If your Glue catalog and Lambda function are in different AWS accounts, you'll need to set up cross-account permissions.
-
Workgroup Configuration: Verify that the "primary" workgroup doesn't override your query execution context settings.
To resolve this issue:
- Add the missing Glue permissions to your Lambda execution role (
glue:GetTable,glue:GetDatabase) - Verify the exact spelling and case of your catalog and database names
- Implement more detailed error logging in your Lambda function to capture additional error details
- Try running a simple SELECT query first to verify connectivity before attempting INSERT operations
This issue is particularly common when the Lambda role has permissions to execute Athena queries but lacks the necessary permissions to access the Glue Data Catalog metadata that Athena needs to locate your tables.
Sources
Athena Query Error using Python | AWS re:Post
AWS HealthLake: TABLE_NOT_FOUND Error in Lambda When Querying Athena | AWS re:Post
TABLE_NOT_FOUND when Running from Lambda but query runs successfully when running from editor | AWS re:Post
Relevant content
asked a year ago
asked 10 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 3 years ago
