cross-account Lambda function needs access Amazon Redshift serverless in a different account

To enable a Lambda function in Account A to connect to Amazon Redshift Serverless in Account B using psycopg2 and a Private Hosted Zone (PHZ), you must configure three distinct layers: Network Connectivity, DNS Resolution, and IAM Permissions.

1. Network Layer (VPC Connectivity)

Since your Lambda is VPC-resident, it needs a private path to Account B.

• VPC Peering or Transit Gateway: Establish a peering connection between the Lambda VPC (Account A) and the Redshift VPC (Account B).
• Routing: Update the Route Tables in both accounts. Account A needs a route for Account B’s CIDR pointing to the Peering Connection, and vice versa.
• Security Groups: The Redshift Serverless Security Group in Account B must allow inbound traffic on port 5439 from the CIDR block of the Lambda VPC in Account A.

2. DNS Layer (Private Hosted Zone)

To ensure the Lambda can resolve the Redshift endpoint via your Private Hosted Zone (PHZ):

Cross-Account PHZ Association: You must associate the PHZ in Account B with the VPC in Account A.

• Step 1 (Account B): Create a VPC Association Authorization using the AWS CLI:

aws route53 create-vpc-association-authorization --hosted-zone-id <HZ-ID> --vpc VPCRegion=<Region>,VPCId=<VPC-ID-A>

• Step 2 (Account A): Accept the association: aws route53 associate-vpc-with-hosted-zone --hosted-zone-id <HZ-ID> --vpc VPCRegion=<Region>,VPCId=<VPC-ID-A>

https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/hosted-zone-private-associate-vpcs-different-accounts.html

CDK Implementation: In your CDK code, you can use PrivateHostedZone.fromLookup or manage the association via a Custom Resource if you want it fully automated across accounts.

3. Identity Layer (IAM & Secrets)

Since psycopg2 requires database credentials:

• Secrets Manager: Store Redshift credentials in Account B.
• Resource-based Policy: Attach a policy to the Secret in Account B to allow the Lambda Execution Role from Account A to perform secretsmanager:GetSecretValue.
• KMS Decryption: If the secret is encrypted with a customer-managed key (CMK), ensure the Lambda role also has kms:Decrypt permissions for that specific key.

4. CDK Snippet (Conceptual)

In your Account A CDK stack, define the Lambda within the VPC and grant access to the cross-account secret:

const lambdaFn = new lambda.Function(this, 'CrossAccountRedshiftLambda', {
  runtime: lambda.Runtime.PYTHON_3_11,
  vpc: vpcA, // Ensure Lambda is in the VPC
  handler: 'index.handler',
  code: lambda.Code.fromAsset('lambda'),
  layers: [psycopg2Layer], // psycopg2 requires a native library layer
});

// Import the secret from Account B
const secret = secretsmanager.Secret.fromSecretAttributes(this, 'RedshiftSecret', {
  secretArn: 'arn:aws:secretsmanager:region:accountB:secret:redshift-creds-abcd',
});

secret.grantRead(lambdaFn);

https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html

You need to consider:

• Psycopg2 Compatibility: Standard psycopg2 often fails in Lambda due to missing PostgreSQL libraries. Use the aws-psycopg2 community layer or a Docker-bundled Lambda image to ensure all C-dependencies are included.
• Interface Endpoints: If you prefer not to use Peering, consider Redshift Managed VPC Endpoints, which allow you to grant access to a specific VPC in another account without full network peering.