How to connect a Lambda function in private subnet to a kafka service in another AWS account

https://aws.amazon.com/about-aws/whats-new/2023/04/amazon-msk-multi-vpc-private-connectivity-cross-account-access/

Amazon MSK now offers multi-VPC private connectivity and cross-account access
Posted On: Apr 28, 2023

Amazon Managed Streaming for Apache Kafka (MSK) now offers multi-VPC private connectivity that simplifies connectivity and access to your Amazon MSK clusters from your Apache Kafka clients hosted in any VPC or AWS account.

We are launching multi-VPC private connectivity (powered by AWS PrivateLink) that allows you to easily establish cross-VPC and cross-account connectivity between your Apache Kafka clients and your Amazon MSK cluster, while keeping all traffic within the AWS Network. With a few clicks, you can turn on multi-VPC private connectivity for one or more authentication modes on your cluster. This feature is supported for IAM, SASL SCRAM, and mutual TLS authentication modes. You can then create Amazon MSK managed VPC Connections and use them to allow your Apache Kafka clients to connect privately to the cluster.

You can also now provide access for Apache Kafka clients to connect privately from a different AWS account through a cluster policy. By combining this with MSK’s IAM functionality, you can also provide these clients fine grained access control to Apache Kafka resources on the cluster. Together, IAM based authentication and cluster policies simplify permission management for cross-account connectivity to an MSK cluster. Multi-VPC private connectivity is available with pay-as-you-go pricing and is supported in all AWS regions where MSK is available. You also pay standard AWS PrivateLink pricing for the Amazon MSK Managed VPC connections used by your Apache Kafka clients to connect privately to the cluster. To learn how to get started with multi-VPC private connectivity and cluster policies, visit our launch blog and the Amazon MSK Developer Guide.