An error occurred while calling o93.getDynamicFrame. The connection attempt failed. I copied the exact same code from AWS Glue Visual editor and created new ETL script.

0

An error occurred while calling o93.getDynamicFrame. The connection attempt failed.

The AWS Glue Visual Editor script is not working when I use same script to create a new job. The AWS Glue Visual Editor job working successfully. I am getting a below error. An error occurred while calling o93.getDynamicFrame. The connection attempt failed.

Here is my script:

import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

args = getResolvedOptions(sys.argv, ["JOB_NAME"])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args["JOB_NAME"], args)

# Script generated for node PostgreSQL
PostgreSQL_node1699996107716 = glueContext.create_dynamic_frame.from_options(
    connection_type="postgresql",
    connection_options={
        "useConnectionProperties": "true",
        "dbtable": "dc.action",
        "connectionName": "DCvNEXT_POSTGRES",
    },
    transformation_ctx="PostgreSQL_node1699996107716",
)

job.commit()

```
Bhavesh
demandé il y a 6 mois376 vues
3 réponses
1
Réponse acceptée

That connection very likely needs a VPC, when you use it in a visual job it gets automatically added to the job (in the Job Details tab) and your script job doesn't have that.

profile pictureAWS
EXPERT
répondu il y a 6 mois
0

So where and how can I add the VPC , subnet and SG group info in above script?

Bhavesh
répondu il y a 6 mois
  • notice I said you have to add it in the Job Details tab, not the script

0

I found that option in Job details. I see the PostgreSQL connection part is working fine. I am still getting connection timeout error for snowflake. I am using snowflake connection type in destination. I have SF credentials in AWS secret manager. Do I need to change anything in script for Snowflake part. I already added the VPC info for snowflake connection and postgresql connection in job details.

An error occurred while calling o101.pyWriteDynamicFrame. JDBC driver encountered communication error. Message: Exception encountered for HTTP request: Connect to qha38537-etlworks-us-prod.snowflakecomputing.com:443 [qha38537-etlworks-us-prod.snowflakecomputing.com/52.27.88.193, qha38537-etlworks-us-prod.snowflakecomputing.com/44.241.78.214, qha38537-etlworks-us-prod.snowflakecomputing.com/34.215.202.59] failed: connect timed out.

Enter image description here

Bhavesh
répondu il y a 6 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions