By using AWS re:Post, you agree to the Terms of Use

Questions tagged with AWS PrivateLink

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Unable to run kubectl & eks commands in a fully private cluster

I have created a VPC fully private (no direct internet access), let's call it VPC-A. This vpc is peer connected to another VPC, let's call it VPC-B. This VPC-B has internet connection and is being used as a gateway for VPC-A. I have deployed a fully private cluster noly (not any node) in the private subnet of the VPC-A using the [guide](https://eksctl.io/usage/eks-private-cluster/). The problem is I am not able to run any kubectl and eks command just like mentioned in the [guide](https://eksctl.io/usage/eks-private-cluster/). After digging a lot on the internet and I found few things to access the cluster. One thing is that I must create a machine in that private VPC and try to access the cluster from there. I also created many issues on github but did not get proper answer. Below are some experts' answers > You can communicate with the K8s API by deploying EC2 instance inside that VPC and defining the EKS K8s API to your kubectl. Well, I have deployed an instance within the vpc of my cluster but whenever I run the kubectl command from the instance inside the private vpc, I get the following error message `Unable to connect to the server: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)` Also in the [EKS fully private cluster guide](https://eksctl.io/usage/eks-private-cluster/) it is mentioned that > If your setup can reach the EKS API server endpoint via its private address, and has outbound internet access (for EKS:DescribeCluster), all eksctl commands should work. Can please someone guide me properly that how can I create such setup? I ran a number of commands to check if anything is wrong with accessing the server address. ``` nmap -p 443 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com Starting Nmap 7.80 ( https://nmap.org ) at 2022-09-09 11:11 UTC Nmap scan report for 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com (192.168.*.*) Host is up (0.00031s latency). Other addresses for 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com (not scanned): 192.168.*.* rDNS record for 192.168.*.*: ip-192-168-*-*.eu-west-*.compute.internal PORT STATE SERVICE 443/tcp open https Nmap done: 1 IP address (1 host up) scanned in 0.04 seconds ``` Another command is ``` nslookup 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com Server: 127.0.0.53 Address: 127.0.0.53#53 Non-authoritative answer: Name: 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com Address: 192.168.*.* Name: 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com Address: 192.168.*.* ``` And another is ``` telnet 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com 443 Trying 192.168.*.*... Connected to 1E9057EC8C316E£D"@JY$J&G%1C94A.gr7.eu-west-*.eks.amazonaws.com Escape character is '^]'. ^CConnection closed by foreign hos ``` It is clear that I can access the api server endpoints from my machine which is in the same vpc as the api server. But still when I run the kubectl command I am getting this output `Unable to connect to the server: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)` When I ran the below command `kubectl cluster-info dump` I got the following error message `Unable to connect to the server: proxyconnect tcp: dial tcp: lookup socks5h on 127.0.0.53:53: server misbehaving` Thanks
1
answers
0
votes
47
views
asked 18 days ago

ECR Cross Account Private Link

Hi All, Struggling for a couple of days already with the following: I've followed this guide: https://aws.amazon.com/blogs/networking-and-content-delivery/centralize-access-using-vpc-interface-endpoints/ I have setup AWS Organisations with all the separate Accounts like nonprd, prd, .... AND the Shared resources account.... CIDR for Shared: 10.40.0.0/16 CIDR for nonprd: 10.0.0.0/16 CIDR for prd: 10.1.0.0/16 In this shared resources account, I've created the 4 vpc endoints for ECR (Shared resources account holds our ecr docker repos for other accounts). logs,dkr,api and S3. I've setup VPC peering with my nonprd and prd account. I've created the route table entries so that all traffic is flowing from shared to the vpc-peering connections cidr and visa versa. The private dns option for the VPC Endpoints are disabled and manually created as private Route53 records. Exactly as the ecr domain. So I have 3 extra private records IN the SHARED resources account: * api.ecr.eu-west-1.amazonaws.com * dkr.ecr.eu-west-1.amazonaws.com * logs.eu-west-1.amazonaws.com I've create the alias record pointing to the private hosted zones. I've done the Associations for Route53 from all the VPC's in nonprd and prd. I CAN resolve the dns records. BUT... And now the problem arises... When I try to run the containers in my nonprd account in any of the vpc's there, my tasks are given one of the following errors: * ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 1 time(s): AccessDeniedException: User: arn:aws:sts::${AWS::AccountId}:assumed-rol... * ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post https://api.ecr.... The policy on the VPC endpoints (complete snippet from my cfn-template): ``` EcrApiEndpoint: Type: AWS::EC2::VPCEndpoint Properties: PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: AWS: - "arn:aws:iam::51*******0:root" # NonPrd - "arn:aws:iam::1******3:root" # Prd - !Sub arn:aws:iam::${AWS::AccountId}:root Action: - ecr:BatchGetImage - ecr:GetAuthorizationToken - ecr:GetDownloadUrlForLayer - ecr:BatchCheckLayerAvailability - ecr:PutImage - ecr:InitiateLayerUpload - ecr:UploadLayerPart - ecr:CompleteLayerUpload Resource: - !Sub "arn:aws:ecr:${AWS::Region}:${AWS::AccountId}:repository/*" VpcId: !FindInMap [Environments, !Ref Environment, VPC] VpcEndpointType: Interface PrivateDnsEnabled: false SecurityGroupIds: - !GetAtt VPCESecurityGroup.GroupId SubnetIds: - !Select [ 0, !FindInMap [Environments, !Ref Environment, PrivateSubnets], ] - !Select [ 1, !FindInMap [Environments, !Ref Environment, PrivateSubnets], ] - !Select [ 2, !FindInMap [Environments, !Ref Environment, PrivateSubnets], ] ServiceName: Fn::Join: - "" - - "com.amazonaws." - !Ref "AWS::Region" - ".ecr.api" ``` So the VPC endpoints run in the Private subnet of the SHARED resources account. The ecs fargate service/task also has the correct permissions (everything is working fine without the VPC endpoints). Can someone help... Please...
4
answers
0
votes
297
views
asked 6 months ago