AWS Glue 5.0 - JDBC URL issue connecting to Databricks

0

Hi team, we are trying to upgrade from Glue 5.0 from Glue 4.0. However, we are getting errors and the message is that there is something wrong with our JDBC string. The current JDBC string we use is:

"jdbc:databricks://our-host.cloud.databricks.com:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/abc123;UseNativeQuery=0;StripCatalogName=0;"

This works when we use Glue 4.0, but upon updating we get the following error:

LAUNCH ERROR | Glue bootstrap failed. Please refer logs for details. caused by LAUNCH ERROR | JDBC connection URL jdbc:databricks://our-host.cloud.databricks.com:443/default;transportMode=http;ssl=1;AuthMech=3;httpPath=/sql/1.0/warehouses/abc123;UseNativeQuery=0;StripCatalogName=0; is not supported. Check the Developer Guide for the list of supported data stores / URL formatting. Please refer logs for details.

Any suggestions on the issue or a link to any docs related to connecting to Databricks?

Thanks!

asked a month ago111 views
1 Answer
0

The error you're encountering suggests that there might be changes in how AWS Glue 5.0 handles JDBC connections, particularly for Databricks. While the JDBC URL format you're using worked in Glue 4.0, it appears to be incompatible with Glue 5.0.

Here are some general steps you can try:

  1. Check if there's an updated JDBC driver for Databricks that's compatible with Glue 5.0. You might need to download and include a newer version of the Databricks JDBC driver in your Glue job.
  2. Ensure that all the parameters in your JDBC URL are still supported and necessary in Glue 5.0.

You may need to contact AWS Support for more specific guidance on connecting to Databricks using Glue 5.0. They might have more up-to-date information on any changes or requirements for JDBC connections in the new version.
Sources
Upgrading from AWS Glue 4.0 to 5.0: PROXY Error and GlueBootstrap Bootstrapping Failure | AWS re:Post
Troublehsoot AWS Glue connection to a JDBC data source | AWS re:Post

profile picture
answered a month ago
profile pictureAWS
SUPPORT ENGINEER
reviewed a month ago
profile pictureAWS
SUPPORT ENGINEER
revised a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions