AWS Glue: Data Catalog with JDBC Snowflake connection: IllegalArgumentException: No group with name <host>

0

Hi,

  1. I have prepared the correct connection to Snowflake in AWS Glue. According to the instructions from here. The URL looks like this: jdbc:snowflake://account_name.snowflakecomputing.com/?user=user_name&db=sample&role=role_name&warehouse=warehouse_name, exactly like in the documentation
  2. I created a crawler using a prepared connection for a selected table from Snowflake.
  3. The crawler started successfully, and all table elements were loaded correctly. The schema was loaded correctly
  4. In job Glue (Job is created by the script, not visual builder, etc.), using the create_dynamic_frame_from_catalog method, I get the following error:
23/06/22 10:58:18 ERROR ProcessLauncher: Error from Python:Traceback (most recent call last):
  File "/tmp/sample.py", line 94, in <module>
    GluePythonSampleJob().run()
  File "/tmp/sample.py", line 52, in run
    dyf = self.read_data_from_catalog(self.context)
  File "/tmp/sample.py", line 63, in read_data_from_s3
    table_name='test-table',
  File "/opt/amazon/lib/python3.6/site-packages/awsglue/context.py", line 186, in create_dynamic_frame_from_catalog
    makeOptions(self._sc, additional_options), catalog_id),
  File "/opt/amazon/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 1305, in __call__
    answer, self.gateway_client, self.target_id, self.name)
  File "/opt/amazon/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 117, in deco
    raise converted from None
pyspark.sql.utils.IllegalArgumentException: No group with name <host>

Method body:

        dyf = glue_context.create_dynamic_frame_from_catalog(
            database='db-snowflake',
            table_name='test-table',
            transformation_ctx="datasource0")

When the table is prepared on the basis of CSV from S3, using a crawler - everything works fine. This looks like a connection issue to Snowflake to me. However, I do not understand why this problem occurs, since the crawler correctly read all the data.

UPDATE Exactly the same happens for the virtual job, based on this catalog.

What can I do?

  • sounds like a bug in the url handling since snowflake works with account instead of host, do you have the rest of the exception stackrtrace to see what is trying to get the host?

  • @twood Did you solve this issue. Im getting the same error. Can you post your answer here?

twood
질문됨 일 년 전157회 조회
답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠