Athena returns "FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null"
Following the well architected labs 200: Cost and usage analysis I get the following error when adding partitions in Athena Query Editor:
MSCK REPAIR TABLE `cost_optimization_10XXXXXXXX321`;
and it returned the following error:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null This query ran against the "costfubar" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 856e146a-8b13-4175-8cd8-692eef6d3fa5
The table was created correctly in Glue with
Name cost_optimization_10XXXXXXXXX21
Description
Database costfubar
Classification parquet
Location
s3://cost-optimization-10XXXXXXX321//
Connection
Deprecated No
Last updated Wed Apr 20 16:46:28 GMT-500 2022
Input format org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat
Output format org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat
Serde serialization lib org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe
Serde parameters
serialization.format 1
Table properties
sizeKey 4223322objectCount 4UPDATED_BY_CRAWLER costfubarCrawlerSchemaSerializerVersion 1.0recordCount 335239averageRecordSize 27exclusions ["s3://cost-optimization-107457606321/**.json","s3://cost-optimization-1XXXXXXXX21/**.csv","s3://cost-optimization-107457606321/**.sql","s3://cost-optimization-1XXXXXXXX321/**.gz","s3://cost-optimization-107457606321/**.zip","s3://cost-optimization-107457606321/**/cost_and_usage_data_status/*","s3://cost-optimization-107457606321/**.yml"]CrawlerSchemaDeserializerVersion 1.0compressionType nonetypeOfData file
and has the following partitions shown in Glue:
partition_0
partition_1
year
month
detailed-cur-1XXXXXXXX57
detailed-cur-1XXXXXXXX57
2018
12
View files
View properties
detailed-cur-1XXXXXXXXX57
detailed-cur-1XXXXXXXXX57
2022
4
View files
View properties
detailed-cur-1XXXXXXXXX57
detailed-cur-1XXXXXXXXX57
2018
11
View files
View properties
detailed-cur-1XXXXXXXX57
detailed-cur-1XXXXXXXX57
2018
10
View files
View properties
Can you please check the troubleshooting section here - https://docs.aws.amazon.com/athena/latest/ug/msckrepair-table.html#msck-repair-table-troubleshooting. Its mostly due to permission issues like missing glue:BatchCreatePartition or KMS permissions or s3:GetObject.
Relevant questions
FAILED: Execution Error, return code 1
asked 3 years agoFirst run. free tier. Build failed to start. The following error occurred: Cannot have more than 0 builds in queue for the account
Accepted Answerasked 3 months agoAthena permission error running a query
asked 3 months agoMSCK REPAIR TABLE returns FAILED org.apache.hadoop.hive.ql.exec.DDLTask. nu
asked 5 years agoAthena returns "FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. null"
asked a month agoFailed when run code deploy
asked a month agoBuild failed to start. The following error occurred: Cannot have more than 0 builds in queue for the account
Accepted Answerasked 4 months ago[AthenaJDBC](100131) An error has been thrown from the AWS SDK client. Connecting to Amazon Athena Data Source
Accepted Answerasked 2 months agoBuild failed to start. The following error occurred: Cannot have more than 0 builds in queue for the account
asked a month agoCodeDeploy/CodePipeline Action execution failed InternalError. Error reference code
asked 19 days ago
Running as Admin Role.