An error occurred while calling o93.getDynamicFrame. The connection attempt failed. I copied the exact same code from AWS Glue Visual editor and created new ETL script.

0

An error occurred while calling o93.getDynamicFrame. The connection attempt failed.

The AWS Glue Visual Editor script is not working when I use same script to create a new job. The AWS Glue Visual Editor job working successfully. I am getting a below error. An error occurred while calling o93.getDynamicFrame. The connection attempt failed.

Here is my script:

import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

args = getResolvedOptions(sys.argv, ["JOB_NAME"])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args["JOB_NAME"], args)

# Script generated for node PostgreSQL
PostgreSQL_node1699996107716 = glueContext.create_dynamic_frame.from_options(
    connection_type="postgresql",
    connection_options={
        "useConnectionProperties": "true",
        "dbtable": "dc.action",
        "connectionName": "DCvNEXT_POSTGRES",
    },
    transformation_ctx="PostgreSQL_node1699996107716",
)

job.commit()

```
Bhavesh
asked 8 months ago491 views
3 Answers
1
Accepted Answer

That connection very likely needs a VPC, when you use it in a visual job it gets automatically added to the job (in the Job Details tab) and your script job doesn't have that.

profile pictureAWS
EXPERT
answered 8 months ago
0

So where and how can I add the VPC , subnet and SG group info in above script?

Bhavesh
answered 8 months ago
  • notice I said you have to add it in the Job Details tab, not the script

0

I found that option in Job details. I see the PostgreSQL connection part is working fine. I am still getting connection timeout error for snowflake. I am using snowflake connection type in destination. I have SF credentials in AWS secret manager. Do I need to change anything in script for Snowflake part. I already added the VPC info for snowflake connection and postgresql connection in job details.

An error occurred while calling o101.pyWriteDynamicFrame. JDBC driver encountered communication error. Message: Exception encountered for HTTP request: Connect to qha38537-etlworks-us-prod.snowflakecomputing.com:443 [qha38537-etlworks-us-prod.snowflakecomputing.com/52.27.88.193, qha38537-etlworks-us-prod.snowflakecomputing.com/44.241.78.214, qha38537-etlworks-us-prod.snowflakecomputing.com/34.215.202.59] failed: connect timed out.

Enter image description here

Bhavesh
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions