CloudFormation creates invalid Glue Connection if JDBC_ENFORCE_SSL is not specified

0

Possibly related to https://repost.aws/questions/QUqYIZ6_LdQomBCbJz0_63Uw/jdbc-enforce-ssl-doesnt-work-for-cloudformation-type-aws-glue-connection

As described here, JDBC_ENFORCE_SSL is an optional property when creating a Glue Connection. However, if this value is left unspecified the created connection does not receive a default value of 'false', and any attempts to use the connection result in the following error:

JobName:ExampleGlueJob and JobRunId:jr_12345 failed to execute with exception Unable to resolve any valid connection (Service: AWSGlueJobExecutor; Status Code: 400; Error Code: InvalidInputException; Request ID: abcde-12345; Proxy: null)

Editing the connection and saving it via the Web GUI results in the JDBC_ENFORCE_SSL: false property being set on the connection, and it can be used without further errors.

Example CFN Template:

rGlueConnection:
    Type: 'AWS::Glue::Connection'
    Properties:
      CatalogId: !Ref 'AWS::AccountId'
      ConnectionInput:
        ConnectionType: JDBC
        ConnectionProperties:
          JDBC_CONNECTION_URL: !Ref pJDBCConnectionURL
          USERNAME: !Sub '{{resolve:secretsmanager:${pSecretsManagerName}:SecretString:username}}'
          PASSWORD: !Sub '{{resolve:secretsmanager:${pSecretsManagerName}:SecretString:password}}'
        Name: !Ref pGlueConnectionName
        PhysicalConnectionRequirements:
          SecurityGroupIdList:  !Ref pSecurityGroupIds
          SubnetId: !Ref pSubnet

Connection after creation (no JDBC_ENFORCE_SSL specified, jobs with connection attached fail to run):

ConnectionProperties:
    JDBC_CONNECTION_URL: jdbc:redshift://example.com:5439/example
    PASSWORD: 123
    USERNAME: abc
  ConnectionType: JDBC
  CreationTime: '2023-03-23T13:40:36.839000-07:00'
  LastUpdatedTime: '2023-03-23T13:40:36.839000-07:00'
  Name: ExampleConnection
  PhysicalConnectionRequirements:
    SecurityGroupIdList:
    - sg-1234
    SubnetId: subnet-12345

Connection after opening and re-saving in Web Console (JDBC_ENFORCE_SSL:false specified, no error on job run):

ConnectionProperties:
    JDBC_CONNECTION_URL: jdbc:redshift://example.com:5439/example
    PASSWORD: 123
    USERNAME: abc
    JDBC_ENFORCE_SSL: 'false'
  ConnectionType: JDBC
  CreationTime: '2023-03-23T13:40:36.839000-07:00'
  LastUpdatedTime: '2023-03-23T13:40:36.839000-07:00'
  Name: ExampleConnection
  PhysicalConnectionRequirements:
    SecurityGroupIdList:
    - sg-1234
    SubnetId: subnet-12345
asked a year ago458 views
2 Answers
0

Hello,

Thank you for reporting this issue.

I tested on my end where I create a Glue connection from below CFN template and I was able to run the Glue job successfully without defining the JDBC_ENFORCE_SSL Property.

`

Resources:
  rGlueConnection:
    Type: AWS::Glue::Connection
    Properties:
      CatalogId: <account_name>
      ConnectionInput:
        ConnectionProperties:
          JDBC_CONNECTION_URL: jdbc:redshift://<URL >/database
          USERNAME: <username>
          PASSWORD: <passwprd>
          Description: "Connect to JDBC"
          Name: "Redshift_ssl_cfn_test"
        ConnectionType: JDBC
        PhysicalConnectionRequirements:
          AvailabilityZone: "<region>"
          SecurityGroupIdList:
           - "sg-xxxxx"
          SubnetId: "subnet-xxx`xxx"

`

To further understand the issue, I would like to suggest you to create support case with complete error messages and with your sample CFN template. This will helps us to get more logs to troubleshoot this issue further..

Thank you for using Amazon Glue.

AWS
SUPPORT ENGINEER
answered a year ago
  • I am confused by your template. According to the reference docs, 'Name' and 'Description' should be children of 'ConnectionInput', whereas in your example they are children of 'ConnectionProperties'. Could you re-create the connection and provide the output of aws glue get-connection --name Redsfhit_sll_cfn_test?

0

Hello,

I tested on my end with Name' and 'Description' under children of ConnectionInput and ConnectionProperties and only difference I found is on Glue connection naming convention.

i.e. With 'Name' and 'Description' defined as a children of ConnectionInput --> This created a Glue connection name i.e. Redshift_ssl_cfn_test

With 'Name' and 'Description' defined as a children of ConnectionProperties --> This created a Glue connection with random name such as: rGlueConnection-TyM6rvk6AjD8, rGlueConnection-Ab1ZlQtH2Aex

With both Glue connection name, I was able to run the Glue job successfully..

Here, is output of Cli command as you requested:

aws glue get-connection --name Redshift_ssl_cfn_test


`{
    "Connection": {
        "Name": "Redshift_ssl_cfn_test",
        "Description": "Connect to JDBC",
        "ConnectionType": "JDBC",
        "ConnectionProperties": {
            "JDBC_CONNECTION_URL": "jdbc:redshift://xxxxxxxxx.redshift.amazonaws.com:5439/xxx",
            "PASSWORD": "xxxxxxx",
            "USERNAME": "xxxxxxx"
        },
        "PhysicalConnectionRequirements": {
            "SubnetId": "subnet-xxxxxxxxx",
            "SecurityGroupIdList": [
                "sg-xxxxxxxxx"
            ],
            "AvailabilityZone": "us-east-1a"
        },

        "CreationTime": "2023-04-04T08:30:27.406000+10:00",
        "LastUpdatedTime": "2023-04-04T08:30:27.406000+10:00"
    }
}`

aws glue get-connection --name rGlueConnection-Ab1ZlQtH2Aex

`{
    "Connection": {
        "Name": "rGlueConnection-Ab1ZlQtH2Aex",
        "ConnectionType": "JDBC",
        "ConnectionProperties": {
            "JDBC_CONNECTION_URL": "jdbc:redshift://xxxxxxx.redshift.amazonaws.com:5439/xxxxx",
            "Description": "Connect to JDBC",
            "PASSWORD": "xxxxxxx",
            "USERNAME": "xxxxxxx",
            "Name": "Redshift_ssl_cfn_test"
        },
        "PhysicalConnectionRequirements": {
            "SubnetId": "subnet-xxxxxxxx",
            "SecurityGroupIdList": [
                "sg-xxxxxxxxxx"
            ],
            "AvailabilityZone": "us-east-1a"
        },
        "CreationTime": "2023-04-04T08:58:42.352000+10:00",
        "LastUpdatedTime": "2023-04-04T08:58:42.352000+10:00"
    }
}`
AWS
SUPPORT ENGINEER
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions