Crawler pulled Snowflake table, but Athena failed to query it.

0

Using Glue we can crawl snowflake table properly in catalog but Athena failed to query table data: HIVE_UNSUPPORTED_FORMAT: Unable to create input format

Googling results suggested it's because of table created by crawler has empty "input format", "Output format"... yes they are empty for this table crawled from Snowflake. So the question is 1 why didn't crawler set them? (crawler can classify the table is snowflake correctly) 2 what the values should be if manual edit is needed?

Is Snowflake table able to be queried by Athena?

Any idea? Thanks,

  • That error means Athena doesn't recognize it's a special table and it's trying to use it like a standard Hive table. I don't think it's possible for Athena to query a JDBC table at this time

profile picture
asked a year ago642 views
1 Answer
1

Hi,

You can query Snowflake using Athena via the Federated query feature of Athena. Crawling the Snowflake tables will add them to your data catalog, but you can't point Athena directly to these tables and query them. I will drop the blog on how to set up the Athena connector for Snowflake, and also a blog on crawling your Snowflake tables and the use cases for that.

  1. Athena Federated query to Snowflake
  2. Crawling Snowflake tables with AWS Glue
hamltm
answered a year ago
  • Thanks for the links! It seems "crawling Snowflake table schema to catalog" is NOT a way to go. Would you please confirm that?

  • In general crawled tables are meant to be used by Glue jobs, there is no guarantee that Athena or other tools outside of Glue will be able to read it. But any day Athena can add the support for that kind of table

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions