Skip to content

AWS Glue crawler on Oracle views

0

I am using AWS Glue crawler with a JDBC connection to an Oracle database. The crawler successfully discovers oracle tables, but it does not discover Oracle views.

Can you confirm whether AWS Glue crawlers support discovering Oracle database views through JDBC connections, or if crawlers are limited to tables only?

Thanks.

asked 3 months ago59 views
2 Answers
1

Agree with the re:post agent. AWS Glue crawlers using JDBC connections to Oracle databases are limited to discovering tables and views are not supported.

This is documented in the Supported data sources for crawling page which states that for relational databases like Oracle, crawlers can discover "databases, schemas, and tables." Views are not listed as a object type for JDBC sources. (Note: Snowflake is the only JDBC source where crawlers explicitly support views.)

As a workaround, you can read the view directly in a Glue ETL job and use a JDBC connection with the dbtable parameter set to your Oracle view name (e.g., MY_SCHEMA.MY_VIEW). Glue Spark jobs can read from views via JDBC even though crawlers cannot discover them.

AWS
answered 3 months ago
0

Based on the available information about AWS Glue crawlers and JDBC connections, I can confirm that AWS Glue crawlers do support discovering database views through JDBC connections for certain databases. For example, when crawling Snowflake databases via JDBC, the crawler explicitly supports discovering Views (along with Tables, External Tables, and Materialized Views).

However, the specific behavior for Oracle database views is not explicitly documented in the available sources. While AWS Glue does support JDBC connections to Oracle databases and crawlers can discover Oracle tables, the sources don't provide detailed information about whether Oracle views are included in the discovery process.

If your crawler is successfully discovering Oracle tables but not views, this could indicate a limitation specific to Oracle, or it might be related to your crawler configuration, connection permissions, or the IAM role permissions being used. For relational databases accessed through JDBC, you can typically choose which objects are crawled, so you may want to verify your crawler's include/exclude path settings and ensure the database user has appropriate permissions to access the views.

I'd recommend checking your crawler configuration settings and the permissions granted to your database connection user to ensure views are included in the scope of what the crawler is attempting to discover.
Sources
Supported data sources for crawling - AWS Glue
Using custom connectors and connections with AWS Glue Studio - AWS Glue

answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.