Unable to read Hive Acid tables in Athena using Athena Hive data connector
Hi, We are trying to use Athena as our consumption service. We have migrated most of the hive databases/tables from external Hive meta store to AWS Glue except those database that has Hive ACID tables because Glue don't support Hive ACID tables. To read Hive ACID tables from Athena, we have configured Athena connector for Hive based this article https://docs.aws.amazon.com/athena/latest/ug/connect-to-data-source-hive.html and used AthenaHiveMetastoreFunctionWithLayer jar.
When try to query Hive ACID table (based on ORC file format ) from Athena using newly created custom catalog for Hive, I get below error.
"HIVE_CURSOR_ERROR: Failed to read ORC file: s3://my-datalake-bkt-dev/test/acid/ug/base_0000002/bucket_00000"
It looks like Athena not able to read the hive ACID file format. Can some one please help me?
HIVE_INVALID_BUCKET_FILES: Hive table 'default.acid_tbl' is corrupt. Found sub-directory in bucket directory for partition:
Presto appears to only supports reading ACID tables starting from Presto 331
However as per this doc ,Athena do support ACID transactions via AWS Lakeformation Governed tables or Icerberg. If you are looking to move your Hive ACID tables to AWS, then I would suggest you to check on the AWS LakeFormation governed tables feature which uses the same Glue catalog.
Ref: AWS lakeformation governed tables blog series
Unable to query dynamodb table in Athena using DynamoDB ConnectorAccepted Answerasked 2 years ago
DMS service for hive table migrationAccepted Answerasked 2 years ago
Best way to structure data for fast access via Athena?asked 4 months ago
Redshift Spectrum and tables with nested columns in Hive Metastoreasked 2 years ago
Athena : HIVE_BAD_DATAasked 6 months ago
Quicksight - Create data set (Athena) - Table query times out when trying to create data setasked a month ago
HIVE_CANNOT_OPEN_SPLIT Error from Athena queryasked 3 months ago
Unable to read Hive Acid tables in Athena using Athena Hive data connectorasked 24 days ago
HIVE_INVALID_METADATA: Hive metadata for table region_us_east_2 is invalid: Table descriptor contains duplicate columnasked a month ago
Aws athena- query both s3 and rdsasked 3 years ago