Unable to run iceberg insert in hive deployed on EMR

Question

Hello,

I configured iceberg formatted table with transaction in hive on EMR 6.4.1. When I insert data into the table, the operation get stuck, without any error.

Any insights are highly appreciated. Thanks
 
```
hive> insert into iceberg_test1 values(1);
Automatically selecting local only mode for query
Query ID = hadoop_20240318143559_03034f6a-a277-4e77-80e5-c19f68b41cbb
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Job running in-process (local Hadoop)
2024-03-18 14:36:01,110 Stage-2 map = 100%, reduce = 0%
```

Accepted Answer

Hello,

I presume your issue might relies on any of the below considerations,

1. Only the MR (MapReduce) execution engine is supported for DML (data manipulation language) operations, and MR is deprecated in Hive 3.1.3.

2. AWS Glue Data Catalog is not currently supported for Iceberg with Hive.

Please check your hive engine as Tez is the default engine in EMR 6.4.1 and make sure it is mr. Also check if you are not using glue as metastore. It requires external metastore to interact with iceberg and transaction tables.

References:

[1] - https://docs.aws.amazon.com/emr/latest/ReleaseGuide/hive-considerations.html

Unable to run iceberg insert in hive deployed on EMR

관련 콘텐츠