Creating AWS Glue Connection with AWS Redshift Serverless Instance - 2 Different AWS Accounts

Question

Hi all,

I have 2 AWS accounts (say staging ETL and staging redshift).
1. In the staging redshift account, I have set up a AWS Redshift Serverless workgroup+namespace
2. In the staging ETL account, I want to create a Data Connection in AWS Glue, which can identify the above serverless instance

How do I achieve this cross-account connection? I have already tried VPC peering but the AWS Glue service is still unable to identify the serverless instance in the other account.

Thanks.

Accepted Answer

In order to establish a Glue connection from Account A (Glue service) to Redshift Cluster in Account B, you can follow these steps:

1. In Account A, edit the rules of Security group "SG-A" to allow all traffic from itself and from the VPC-B (by providing its CDIR) as shown below:
```
 Type              | Protocol | Port Range | Source          | Description
 All Traffic      | ALL         | ALL             | 10.0.0.0/16  | Allow traffic from VPC-B 
 All Traffic      | ALL         | ALL             | SG-A             | Self referencing SG 2
```

2. In Acount B, edit the rules of Redshift security group "SG-B" to allow all traffic from VPC-A similar to below:
```
Type         | Protocol | Port Range | Source          | Description
All Traffic   | ALL         | ALL             | 172.31.0.0/16 | Allow traffic from VPC-A 
All Traffic   | ALL         | ALL             | SG-B              | Self referencing SG
```

3. Establish a cross-account VPC peering connection between VPC-A and VPC-B as per the instructions in this document [http://docs.aws.amazon.com/AmazonVPC/latest/PeeringGuide/create-vpc-peering-connection.html#create-vpc-peering-connection-remote]()

4. After peering connection is setup properly, please make sure that route tables on both VPC's "VPC-A" and VPC-B" have a route between through them via PCX. For example, on subnet "Subnet-A", your route should look similar to:
```
Destination | Target
10.0.0.0/16   | pcx-xxxxxxx (your Peering connection_-ID)
```

5. Similarly Redshift's Subnet route table should looks as:
```
Destination   | Target
172.31.0.0/16 | pcx-xxxxxxx (your Peering connection_-ID)
```
    
6. Make sure Redshift cluster in Account B "publicly accessible" field is set to "No" as VPC-A and VPC-B can only tal with private. [In case public ip connections are enabled for the Redshift cluster you need to edit the Peering connections of the account where your Redshift cluster is to Allow accepter VPC to resolve DNS of requester VPC hosts to private IP]

7. Now the networking setup is complete, please add a new connection in Glue in "Acc-A" with following properties: Connection Type: 
```
    JDBC JDBC URL:  
    Username:  
    Password:  
    VPC : VPC-A 
    Subnet: Subnet-A 
    Security Group: SG-A
```

8. Test the connection and verify if it is setup successfully

Creating AWS Glue Connection with AWS Redshift Serverless Instance - 2 Different AWS Accounts

Relevant content