Skip to content

Redshift - Iceberg tables - Query View with Where clause

0
  • I have a Iceberg table in s3 and I have created the external schema and external table for the same in redshift.
  • I am able to query the table with a where clause and it works.
  • I am able to query the view with a select * from view
  • But the view is not allowing to query with a where clause. Getting the below error.
  • The table has no partitions.

ERROR: Assert Detail: ----------------------------------------------- error: Assert code: 1000 context: nested_rte->nested_path != nullptr - nested_rte->nested_path = nullptr. query: 0 location: iceberg_partition.cpp:264 process: padbmaster [pid=1073947154] ----------------------------------------------- [ErrorId: 1-687e4cd5-52ab06f242d3725e2bf24fe8]

asked 10 months ago143 views
1 Answer
0

The error you're encountering when trying to query an Iceberg view with a WHERE clause in Amazon Redshift appears to be related to how Redshift processes queries against Iceberg tables, particularly when views are involved.

Amazon Redshift supports read-only access to Apache Iceberg tables with transactionally consistent SELECT queries. While direct queries with WHERE clauses on Iceberg tables work fine, there seems to be a limitation when applying WHERE clauses to views created on top of Iceberg tables.

This could be related to how Redshift handles the partition information or metadata when working with views on external Iceberg tables. Even though you mentioned your table has no partitions, Redshift might still be trying to process partition-related information for the query optimization.

Here are some potential workarounds:

  1. Instead of using a view, query the Iceberg table directly with your WHERE clause.

  2. Create a materialized view instead of a regular view. Materialized views on Apache Iceberg tables are supported in Amazon Redshift, though they have some limitations (automatic refreshes and query rewriting are not supported).

  3. Import the data from your Iceberg table into a local Amazon Redshift table using INSERT INTO or CREATE TABLE AS commands, and then create your view on the local table.

  4. If you need to use a view, consider creating a stored procedure that dynamically builds and executes the query with the WHERE clause against the base Iceberg table.

The error message indicates an internal assertion failure in Redshift's query processing for Iceberg tables, specifically in the iceberg_partition.cpp file, which suggests this is related to how Redshift handles partition information for Iceberg tables when accessed through views.
Sources
Using Apache Iceberg tables with Amazon Redshift - Amazon Redshift

answered 10 months ago
  • But its working for some views. all the views are simple

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.