Questions tagged with AWS PrivateLink

Content language: English

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

ECR Cross Account Private Link

Hi All, Struggling for a couple of days already with the following: I've followed this guide: https://aws.amazon.com/blogs/networking-and-content-delivery/centralize-access-using-vpc-interface-endpoints/ I have setup AWS Organisations with all the separate Accounts like nonprd, prd, .... AND the Shared resources account.... CIDR for Shared: 10.40.0.0/16 CIDR for nonprd: 10.0.0.0/16 CIDR for prd: 10.1.0.0/16 In this shared resources account, I've created the 4 vpc endoints for ECR (Shared resources account holds our ecr docker repos for other accounts). logs,dkr,api and S3. I've setup VPC peering with my nonprd and prd account. I've created the route table entries so that all traffic is flowing from shared to the vpc-peering connections cidr and visa versa. The private dns option for the VPC Endpoints are disabled and manually created as private Route53 records. Exactly as the ecr domain. So I have 3 extra private records IN the SHARED resources account: * api.ecr.eu-west-1.amazonaws.com * dkr.ecr.eu-west-1.amazonaws.com * logs.eu-west-1.amazonaws.com I've create the alias record pointing to the private hosted zones. I've done the Associations for Route53 from all the VPC's in nonprd and prd. I CAN resolve the dns records. BUT... And now the problem arises... When I try to run the containers in my nonprd account in any of the vpc's there, my tasks are given one of the following errors: * ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 1 time(s): AccessDeniedException: User: arn:aws:sts::${AWS::AccountId}:assumed-rol... * ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post https://api.ecr.... The policy on the VPC endpoints (complete snippet from my cfn-template): ``` EcrApiEndpoint: Type: AWS::EC2::VPCEndpoint Properties: PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: AWS: - "arn:aws:iam::51*******0:root" # NonPrd - "arn:aws:iam::1******3:root" # Prd - !Sub arn:aws:iam::${AWS::AccountId}:root Action: - ecr:BatchGetImage - ecr:GetAuthorizationToken - ecr:GetDownloadUrlForLayer - ecr:BatchCheckLayerAvailability - ecr:PutImage - ecr:InitiateLayerUpload - ecr:UploadLayerPart - ecr:CompleteLayerUpload Resource: - !Sub "arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/*" VpcId: !FindInMap [Environments, !Ref Environment, VPC] VpcEndpointType: Interface PrivateDnsEnabled: false SecurityGroupIds: - !GetAtt VPCESecurityGroup.GroupId SubnetIds: - !Select [ 0, !FindInMap [Environments, !Ref Environment, PrivateSubnets], ] - !Select [ 1, !FindInMap [Environments, !Ref Environment, PrivateSubnets], ] - !Select [ 2, !FindInMap [Environments, !Ref Environment, PrivateSubnets], ] ServiceName: Fn::Join: - "" - - "com.amazonaws." - !Ref "AWS::Region" - ".ecr.api" ``` So the VPC endpoints run in the Private subnet of the SHARED resources account. The ecs fargate service/task also has the correct permissions (everything is working fine without the VPC endpoints). Can someone help... Please...
4
answers
0
votes
420
views
asked 8 months ago

How to invoke a private REST API (created with AWS Gateway) endpoint from an EventBusRule?

I have setup the following workflow: - private REST API with sources `/POST/event` and `/POST/process` - a `VPCLink` to an `NLB` (which points to an `ALB` pointing to a microservice running on `EKS`) - a `VPC endpoint` with DNS name `vpce-<id>-<id>.execute-api.eu-central-1.vpce.amazonaws.com` with `Private DNS enabled` - an EventBridge `EventBus` with a rule that has two targets: 1 `API Destination` for debugging/testing and 1 `AWS Service` which points to my private REST Api on the source `/POST/process` - all required `Resource Policies` and `Roles` - all resources are defined within the same AWS Account The **designed** worflow is as follows: - invoke `POST/event` on the VPC endpoint (any other invocation is prohibited by the `Resource Policy`) with an `event` payload - the API puts the `event` payload to the `EventBus` - the `EventBusRule` is triggered and sends the `event` payload to the `POST/process` endpoint on the private REST API - the `POST/process` endpoint proxies the payload to a microservice running on EKS (via `VPCLink` > `NLB` > `ALB`> `k8s Service`) **What does work** so far: - invoking `POST/event` on the VPC endpoint - putting the `event` payload to the `EventBus` - forwarding the `event` payload to the `API Destination` set up for testing/debugging (it's a temporary endpoint on https://webhook.site) - testing the `POST/event` and `POST/process` integration in the AWS Console (the latter is verified by checking that the `event` payload reaches the microservice on EKS successfully) That is all single steps in the workflow seem to work, and all permissions seem to be set properly. **Whad does not work **is invoking the `POST/process` endpoint from the `EventBusRule`, i.e. invoking `POST/event` does not invoke `POST/process` via the `EventBus`, _although_ the `EventBusRule` was triggered. So my **question** is: **How to invoke a private REST API endpoint from an EventBusRule?** **What I have already tried:** - change the order of the `EventBusRule targets` - create a Route 53 record pointing to the `VPC endpoint` and treat it as an (external) `API Destination` - allow access from _anywhere_ by _anyone_ to the REST API (temporarily only, of course) **Remark on the design:** I create _two_ endpoints (one for receiving an `event`, one for processing it) with an EventBus in between because - I have to expect a delay of several minutes between the `Event Creation/Notification` and the successful `Event Processing` - I expect several hundred `event sources`, which are different AWS and Azure accounts - I want to keep track of all events that _reach_ our API and of their successful _processing_ in one central EventBus and _not_ inside each AWS account where the event stems from - I want to keep track each _failed_ event processing in the same central EventBus with only one central DeadLetterQueue
1
answers
0
votes
309
views
asked 8 months ago

Benefits to S3 cross-region access with VPC peered interface endpoints vs. public internet using NAT gateways?

My team is looking to setup EMR clusters in private VPCs in all regions while having our main storage as S3 buckets in us-east-1. We will need cross-region access to S3 and have been looking at different ways of accomplishing it. We have considered two approaches: 1. Setting up isolated VPCs with no internet access, one in us-east-1 for the S3 bucket access and one in every region to launch our EMR clusters in. We will pair each of the VPCs with the one in us-east-1 and then setup an interface endpoint in the us-east-1 VPC to allow S3 access through the interface endpoint with VPC peering. This utilizes AWS PrivateLink. 2. Setting up a private VPC with internet gateway and NAT gateways in public subnets while launching EMR clusters in the private subnets. We will access S3 across regions through public internet. For both solutions, we will utilize gateway endpoints when the compute and storage is in the same region as we found this should yield the same benefits as interface endpoints but with no additional cost. Through my research, I have found that AWS PrivateLink is more secure due to no public internet usage and has a significant latency advantage of up to 70% according to this experiment: https://blogs.vmware.com/security/2020/03/performance-testing-justifying-cost-and-performance-improvements-part-2.html I am wondering if we will still see this latency benefit if we are using VPC peering or if it would be better to go with the internet route.
2
answers
1
votes
445
views
asked 9 months ago

Unexpected URI while testing API gateway to NLB

I have a Private Link setup that points at an NLB that further has routes setup to an EC2 instance that provides my HTTP end point. I am trying to setup an API gateway to use the private link. I tried setting this up as proxy. This allows me to test an endpoint without authorization, and provide both path and query parameters. This is the log of the output I am seeing. ``` Execution log for request e114f111-f6b5-413d-9592-ec1ecb72d848 Mon Mar 07 19:38:02 UTC 2022 : Starting execution for request: e114f111-f6b5-413d-9592-ec1ecb72d848 Mon Mar 07 19:38:02 UTC 2022 : HTTP Method: GET, Resource Path: /api/falcfunnelsession/getInvestorResponse Mon Mar 07 19:38:02 UTC 2022 : Method request path: {proxy=api/falcfunnelsession/getInvestorResponse} Mon Mar 07 19:38:02 UTC 2022 : Method request query string: {sessionUUID=7a0e0113-05ab-4382-bed8-f91e9bc3ef3c} Mon Mar 07 19:38:02 UTC 2022 : Method request headers: {} Mon Mar 07 19:38:02 UTC 2022 : Method request body before transformations: Mon Mar 07 19:38:02 UTC 2022 : Endpoint request URI: https://funnel-backend-103-nlb-bc34d52397cf6529.elb.us-east-2.amazonaws.com?sessionUUID=7a0e0113-05ab-4382-bed8-f91e9bc3ef3c Mon Mar 07 19:38:02 UTC 2022 : Endpoint request headers: {x-amzn-apigateway-api-id=8v1qbgdlb8, User-Agent=AmazonAPIGateway_8v1qbgdlb8, Host=funnel-backend-103-nlb-bc34d52397cf6529.elb.us-east-2.amazonaws.com} Mon Mar 07 19:38:02 UTC 2022 : Endpoint request body after transformations: Mon Mar 07 19:38:02 UTC 2022 : Sending request to https://funnel-backend-103-nlb-bc34d52397cf6529.elb.us-east-2.amazonaws.com?sessionUUID=7a0e0113-05ab-4382-bed8-f91e9bc3ef3c Mon Mar 07 19:38:07 UTC 2022 : Execution failed due to configuration error: There was an internal error while executing your request Mon Mar 07 19:38:07 UTC 2022 : Method completed with status: 500 ``` The following path and query are as expected ``` Mon Mar 07 19:38:02 UTC 2022 : HTTP Method: GET, Resource Path: /api/falcfunnelsession/getInvestorResponse Mon Mar 07 19:38:02 UTC 2022 : Method request path: {proxy=api/falcfunnelsession/getInvestorResponse} Mon Mar 07 19:38:02 UTC 2022 : Method request query string: {sessionUUID=7a0e0113-05ab-4382-bed8-f91e9bc3ef3c} ``` However the URI request seems to leave out the path. ``` Mon Mar 07 19:38:02 UTC 2022 : Endpoint request URI: https://funnel-backend-103-nlb-bc34d52397cf6529.elb.us-east-2.amazonaws.com?sessionUUID=7a0e0113-05ab-4382-bed8-f91e9bc3ef3c ``` Wondering is there is something I am doing wrong here. thanks!
2
answers
0
votes
599
views
asked 9 months ago