AWS Glue - An error occurred while calling o95.getDynamicFrame. Communications link failure

Question

Hi all,

I'm trying to connect to an external MariaDB database instance using a AWS Glue Spark script and a JDBC Glue connection.

The code snippet from the Spark script is:

dyf = glueContext.create_dynamic_frame.from_options(
    connection_type = 'mysql',
    connection_options = { 
        "useConnectionProperties": "true", 
        "connectionName": "sample_con_name",
        "dbtable": "admins",
        "customJdbcDriverS3Path": "s3://path_to_connector/mysql-connector-java-8.0.17.jar",
        "customJdbcDriverClassName": "com.mysql.cj.jdbc.Driver"
    } 
)

The error that I'm getting after running the Glue Spark job is: **An error occurred while calling o95.getDynamicFrame. Communications link failure**

The Glue connection is based on a private subnet in my VPC. The private subnet has a route through a NAT gateway in my VPC and an elastic IP is attached to the NAT gateway - this elastic IP is already whitelisted by the external DB's admins.

Just for testing, I made a EC2 instance on the same private subnet, and wrote a small PHP script to connect to the same external database, and it was connecting successfully.

How do I fix this issue? Is it a driver/certificate issue?

Thanks!

Answer

Having exact same problem, have you found a solution?

Answer

* Note that Glue will only use private IP addresses from the subnet you have configured, so you will be required to make sure that these private IPs have access to the data store

``` So the EC2 instance you used to test the connection, make sure you were able to connect from its private IP address to the external database```
* Additionally, the security group used in the connection should have a self referencing rule to allow connectivity between the Glue workers

AWS Glue - An error occurred while calling o95.getDynamicFrame. Communications link failure

관련 콘텐츠