AWS Glue Visual Studio - Redshift Target Node

0

Hey all, I have been trying to perform a simple S3 to redshift data push using an S3 source node and a Amazon Redshift target node. I have been getting errors such as 'Failed to connect to IP Address', 'py4j.protocol.Py4JJavaError: An error occurred while calling o101.getSink. : com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

Have checked permissions, policies, etc with AWS Admin. Any tips to fix this? Note- I am attempting this in a glue job which also has several other nodes for transforms, etc.

4개 답변
0
수락된 답변

In my case, I had 2 connections on my glue job and I only required the JDBC connection to redshift. I had an additional network connection which wasn't required since everything was on the same network

sg03
답변함 5달 전
0

Notice that Glue is trying to use the MySQL driver driver, which will use a different port and connectivity.
Doublecheck you are using the right connection and that is well defined (or point the target directly to Redshift without a connection/table)

profile pictureAWS
전문가
답변함 5달 전
  • Hey Gonzalo thanks very much for your response. Currently, I have picked the appropriate connection to my database in redshift, and also picked the schema and the table to ingest into in redshift. What do you mean by point to redshift without the connection/table? Where would the data end up going then?

0

Double check your Glue connection was set up for Redshift and not generic JDBC (as was pointed out, your connection thinks it is MySQL). Also as a reminder, when using Glue with Redshift we also strongly recommend using Glue 4.0 as the newest connectors are exponentially better than the old.

For running queries before or after a data load I recommend using the redshift_connector python library on pypi via --additional-python-modules.

AWS
Zach
답변함 5달 전
0

'Communications link failure' typically indicates that the Glue job is unable to reach the source and/or destination target.

Check the vpc setup, like the subnets and the security groups, refer the below documentation for reference

[+] Redshift connections - Set up Amazon VPC - https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect-redshift-home.html#aws-glue-programming-etl-redshift-config-vpc

AWS
지원 엔지니어
답변함 5달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠