Redshift Specturm: Create table with sortkey and distkey from a union all query

0

Hey there,

I have a create table query with sortkey and distkey defined, the source data comes from s3. The query looks like this

create  table "database_name"."schema_name"."new_table_name"
    diststyle key distkey (id)
    compound sortkey(event_at)
as (
     SELECT
        id
        , event_at
        , ...
    FROM "database_name"."schema_name"."source_table_a"
    WHERE ....
    
    UNION ALL
    
     SELECT
        id
        , event_at
        , ...
    FROM "database_name"."schema_name"."source_table_a"
    WHERE ....
  )

When I run the query, I always receive database error: Table database_name_schema_name_source_table_a_60bb138c76a18 not found. This 60bb138c76a18 suffix is always different when I run the query.

It works when:

  • remove sort key and dist key from table creation
  • remove union all

Does anyone experience the same issue? Thanks in advacne.

  • I'm not sure why this particular error is happening but I'm curious as to why you are using UNION ALL. Scanning that external table twice will take take twice as long and cost twice as much. Can you remove the UNION ALL and use CASE statements instead? But if you can't resolve this with enhancing the SQL, you should open a support case.

  • Hi,

    Have you tried to execute the SELECT query with UNION ALL without the CREATE TABLE?

    Regards,

  • Hey @Ziad, Yes, with table creation, query works fine.

  • Hey Jon, This is a simplified query, I remove some complexity to better focus on the issue. Thanks for your input.

asked 5 months ago199 views
1 Answer
1

Hi,

I have tested multiple similar use cases (CTAS + sortkey + distkey + union all + Spectrum tables). They executed successfully without any error message. I recommend in this case to open a support ticket for further analysis so you can share with the support team the real query.

Regards,

AWS
EXPERT
Ziad
answered 5 months ago
  • Hey Ziad, thanks a lot for testing out. Seems like something specific only for my certain table. I will reach out to support team.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions