AWS Athena Query is Failing Randomly

Question

Hi,

My Athena Query to transform the data is failing randomly with the error:

Exception: Final state of Athena job is FAILED, query_execution_id is 97b49928-68f8-44e0-af7e-1fc7ec7cefcb. Error: [ErrorCode: INTERNAL_ERROR_QUERY_ENGINE] Amazon Athena experienced an internal error while executing this query. Please contact AWS support for further assistance. You will not be charged for this query. We apologize for the inconvenience. You may need to manually clean the data at location 's3:///airflow_query_output/tables/97b49928-68f8-44e0-af7e-1fc7ec7cefcb' before retrying. Athena will not delete data in your account.

When I rerun this query, it gets succeeded in the second or third attempt. Since we have hourly pipelines scheduled in MWAA, this is breaking our production data platform.

We had the same issue last month also, but it got resolved automatically after 3 days and its happening again. Can any one provide any resolution for this.

Didier_Durand · Answer

Hi,

Such intermittent issues as the one you experience may be caused by improperly escaped Athena-reserved words in your queries.

See https://docs.aws.amazon.com/athena/latest/ug/reserved-words.html for reserved words and how to escape them properly in case you use some.

Quite a few of those reserved words are frequent names for columns in user data

Best,

Didier

AWS Athena Query is Failing Randomly

Relevant content