An error occurred while calling o93.getDynamicFrame. The connection attempt failed. I copied the exact same code from AWS Glue Visual editor and created new ETL script.

0

An error occurred while calling o93.getDynamicFrame. The connection attempt failed.

The AWS Glue Visual Editor script is not working when I use same script to create a new job. The AWS Glue Visual Editor job working successfully. I am getting a below error. An error occurred while calling o93.getDynamicFrame. The connection attempt failed.

Here is my script:

import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

args = getResolvedOptions(sys.argv, ["JOB_NAME"])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args["JOB_NAME"], args)

# Script generated for node PostgreSQL
PostgreSQL_node1699996107716 = glueContext.create_dynamic_frame.from_options(
    connection_type="postgresql",
    connection_options={
        "useConnectionProperties": "true",
        "dbtable": "dc.action",
        "connectionName": "DCvNEXT_POSTGRES",
    },
    transformation_ctx="PostgreSQL_node1699996107716",
)

job.commit()

```
Bhavesh
gefragt vor 6 Monaten376 Aufrufe
3 Antworten
1
Akzeptierte Antwort

That connection very likely needs a VPC, when you use it in a visual job it gets automatically added to the job (in the Job Details tab) and your script job doesn't have that.

profile pictureAWS
EXPERTE
beantwortet vor 6 Monaten
0

So where and how can I add the VPC , subnet and SG group info in above script?

Bhavesh
beantwortet vor 6 Monaten
  • notice I said you have to add it in the Job Details tab, not the script

0

I found that option in Job details. I see the PostgreSQL connection part is working fine. I am still getting connection timeout error for snowflake. I am using snowflake connection type in destination. I have SF credentials in AWS secret manager. Do I need to change anything in script for Snowflake part. I already added the VPC info for snowflake connection and postgresql connection in job details.

An error occurred while calling o101.pyWriteDynamicFrame. JDBC driver encountered communication error. Message: Exception encountered for HTTP request: Connect to qha38537-etlworks-us-prod.snowflakecomputing.com:443 [qha38537-etlworks-us-prod.snowflakecomputing.com/52.27.88.193, qha38537-etlworks-us-prod.snowflakecomputing.com/44.241.78.214, qha38537-etlworks-us-prod.snowflakecomputing.com/34.215.202.59] failed: connect timed out.

Enter image description here

Bhavesh
beantwortet vor 6 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen