AWS Glue Visual Studio - Redshift Target Node

0

Hey all, I have been trying to perform a simple S3 to redshift data push using an S3 source node and a Amazon Redshift target node. I have been getting errors such as 'Failed to connect to IP Address', 'py4j.protocol.Py4JJavaError: An error occurred while calling o101.getSink. : com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

Have checked permissions, policies, etc with AWS Admin. Any tips to fix this? Note- I am attempting this in a glue job which also has several other nodes for transforms, etc.

sg03
質問済み 5ヶ月前324ビュー
4回答
0
承認された回答

In my case, I had 2 connections on my glue job and I only required the JDBC connection to redshift. I had an additional network connection which wasn't required since everything was on the same network

sg03
回答済み 5ヶ月前
0

Notice that Glue is trying to use the MySQL driver driver, which will use a different port and connectivity.
Doublecheck you are using the right connection and that is well defined (or point the target directly to Redshift without a connection/table)

profile pictureAWS
エキスパート
回答済み 5ヶ月前
  • Hey Gonzalo thanks very much for your response. Currently, I have picked the appropriate connection to my database in redshift, and also picked the schema and the table to ingest into in redshift. What do you mean by point to redshift without the connection/table? Where would the data end up going then?

0

Double check your Glue connection was set up for Redshift and not generic JDBC (as was pointed out, your connection thinks it is MySQL). Also as a reminder, when using Glue with Redshift we also strongly recommend using Glue 4.0 as the newest connectors are exponentially better than the old.

For running queries before or after a data load I recommend using the redshift_connector python library on pypi via --additional-python-modules.

AWS
Zach
回答済み 5ヶ月前
0

'Communications link failure' typically indicates that the Glue job is unable to reach the source and/or destination target.

Check the vpc setup, like the subnets and the security groups, refer the below documentation for reference

[+] Redshift connections - Set up Amazon VPC - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect-redshift-home.html#aws-glue-programming-etl-redshift-config-vpc

AWS
サポートエンジニア
回答済み 5ヶ月前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ