- EMR Version: 6.15.0
- Spark Conf
- "spark.sql.catalog.spark_catalog": "org.apache.iceberg.spark.SparkSessionCatalog"
- "spark.sql.catalog.spark_catalog.catalog-impl": "org.apache.iceberg.aws.glue.GlueCatalog"
- "spark.sql.catalog.spark_catalog.io-impl": "org.apache.iceberg.aws.s3.S3FileIO"
- "spark.sql.extensions": "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions"
Create Table, write, read is OK.
But I get the following error when I drop table:
spark.sql("drop table spark_catalog.$db.test_iceberg_0 purge")
pyspark.errors.exceptions.captured.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to fetch table test_iceberg_0. StorageDescriptor#InputFormat cannot be null for table: test_iceberg_0 (Service: null; Status Code: 0; Error Code: null; Request ID: null; Proxy: null)
Thank you for your help.