Setup for Lambda to reach SQS via private subnet using VPC end point interface

Question

My question is regarding how Lambda functions access SQS to send messages when the Lambda function is associated with private VPC subnets.

I must associate my VPC private subnets 1, 2, and 3 with my Lambda function because it has to connect to MSK and MSK is running three brokers, one each in my private subnets 1, 2 and 3. My Lambda function is also configured to receives SQS events via an SQS event source trigger and must be able to publish messages to SQS.

I understand the following:
* When you associate a Lambda with a VPC private subnet, it looses access to the public internet and hence, looses access to all AWS services via their default public API end points
* Inbound event sources will still work the same regardless of whether or not you have a VPC associated with the Lambda function. Only outbound access to the internet if cut off, and hence AWS services is impacted

I am trying to understand why my Lambda function can send SQS messages after associating the function with my VPC private subnet when I DO NOT yet have an SQS VPC end point interface configured. I do have public subnets with NAT routers configured. I have a security group attached to my Lambda function with no inbound rules and one outbound rule for all-traffic 0.0.0.0/0.

**Question 1** - Is the outbound SQS traffic from my Lambda function able to reach the public SQS service because the traffic is flowing into my private subnet, then to the public subnet NAT router (allowed due to the 0.0.0.0/0 rule on the function SG), hitting the SQS public API end point, and returning the same way? That is the only thing I can think of that allows this to work.

**Question 2** - My Lambda function can publish messages to MSK. I assume that is because this outbound traffic flows to the private subnet and the MSK brokers are already running in that same subnet so the routing to those brokers seems obvious being all within the same subnet already.

**Question 3** - If question 1 is yes, I may want to eliminate that hop through the public internet for the SQS send message traffic. To do that, I believe I would need to do the following:
1. Create a new security group (A) that I will attach to a new SQS VPC end point interface that allows all inbound traffic from the SG (B) that is already attached to my Lambda function
2. Create a new SQS service VPC end point interface and attach the SG (A) to it

**Question 4** - If I do #3, do I need to configure my SQS client (i am using the Go SDK) with one of the alternate endpoint DNS names that are created when you create a VPC end point or does it some how automatically resolve to the VPC end point when the traffic originates from within AWS?

Thanks in advance,
Mark

Accepted Answer

Hello.

> Question 1 - Is the outbound SQS traffic from my Lambda function able to reach the public SQS service because the traffic is flowing into my private subnet, then to the public subnet NAT router (allowed due to the 0.0.0.0/0 rule on the function SG), hitting the SQS public API end point, and returning the same way? That is the only thing I can think of that allows this to work.

Yes, with NAT Gateway, you can access SQS from Lambda without having to set up a VPC endpoint.  
It is also mentioned in the documentation below.  
https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html#vpc-internet

> Question 2 - My Lambda function can publish messages to MSK. I assume that is because this outbound traffic flows to the private subnet and the MSK brokers are already running in that same subnet so the routing to those brokers seems obvious being all within the same subnet already.

I think it's a private connection if it's within the same VPC.  
https://docs.aws.amazon.com/msk/latest/developerguide/client-access.html

> Question 3 - If question 1 is yes, I may want to eliminate that hop through the public internet for the SQS send message traffic. To do that, I believe I would need to do the following:

Yes, by creating a VPC endpoint, you can access SQS privately without going through NAT Gateway.  
Also, I think you can connect using the settings you described.  
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-sending-messages-from-vpc.html#create-vpc-endpoint-for-sqs

> Question 4 - If I do #3, do I need to configure my SQS client (i am using the Go SDK) with one of the alternate endpoint DNS names that are created when you create a VPC end point or does it some how automatically resolve to the VPC end point when the traffic originates from within AWS?

As far as I can see here, it seems that you need to set endpoint_url.  
https://github.com/boto/boto3/issues/1900  
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/core/session.html  
```
import boto3
session = boto3.Session()
sqs_client = session.client(
    service_name='sqs',
    endpoint_url='https://sqs.ap-northeast-1.amazonaws.com',
)
sqs_client.send_message(...)
```

Setup for Lambda to reach SQS via private subnet using VPC end point interface

Relevant content