Questions tagged with Amazon Elastic Kubernetes Service
Content language: English
Sort by most recent
So I have created a EKS Cluster with 1 EKS managed node group. I also created a in-region Self-managed node group.
I was trying to install some workloads into the cluster using Helm, by simply invoking `helm install` method.
The first chart is installed into self-managed node group and everything works fine. But when I tried to install the second helm chart, it went to managed node group. However, the second helm chart has some dependency on the first helm chart and right now second helm chart is stuck trying to find the first helm chart. But it is in the other node group.
I am wondering what kind of Security Group rule changes I need to make so that I could make this cross node group communication work ?
Currently these two node groups are within the same VPC and Subnet
Thanks
Is it possible to extend an EKS cluster (on EC2) with on-prem nodes?
The on-prem nodes would ideally be connected securely to the VPC to avoid going over public internet.
The motivation behind this is to utilize existing servers on-prem for some of the workload, and during peak hours extend the capabilities of the cluster via autoscaling EKS on-demand.
Ideally everything would be centrally managed under AWS, therefore some EKS nodes would always be active for the control plane, data redundancy, etc.
In researching this topic so far I've only found resources on EKS via AWS Outposts, EKS Anywhere, joining federated clusters, etc. -- but it seems these solutions involve managing our own infrastructure, losing the benefits of fully-managed EKS on AWS. I can't find any information about extending AWS-managed EKS clusters with on-prem hardware (effectively allowing AWS to take ownership of the node/system and integrate it into the cluster). Has anyone accomplished this, or is not viable/supported? I appreciate any feedback, thanks!
Hi. Is it possible to set up routing rules for pods in EKS using standard mesh plugins? I’m not able to install plugins like Calico.
Hi,
I am using nlb for serving rtmp connections. Targets of nlb are multiple nodes in eks cluster and on nodes there are nginx-rtmp pods. When i stream multiple streams, i am getting connection dropped at client side, and getting "drop idle stream" log on nginx-rtmp. my idle timeout configuration on nginx-rtmp is 30 sec. I am using ec2 instances to generating load which have 5GB bandwidth.
I am not able to found why this is happening. Multiple connections dropping in a single second. and sometimes all of them are on same node.
Also when i am checking NLB access logs i found only two ips in target ip and i am not able to found both ip on any pod or node.
Hi Team,
Currently we have a customer running application in C program compiled as dll's and hosted the apps into Windows 2016 server in their on-prem data center.
Now we have to migrate the apps into AWS cloud. Customer prefer to deploy this apps into container solution without code change.
Is it possible to run C program dlls into EKS with small code change ? If NOT then what is the best possible treatment we should offer to customers for this application seamlessly deployed into AWS cloud.
Thanks.
I have a code build leading to EKS. When it calls this particular command "CREDENTIALS=$(aws sts assume-role --role-arn arn:aws:iam::3318******:role/EksWorkshopCodeBuildKubectlRole --role-session-name code-build --duration-seconds 900)" I get an error
"An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::331*****:assumed-role/codebuild-kubernetes-eks-service-role/AWSCodeBuild-31746234-c1a9-4fe9-9cbc-b0d54264613e is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::331879450537:role/EksWorkshopCodeBuildKubectlRole"
My code build trusted relationship looks like
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "codebuild.amazonaws.com"
},
"Action": "sts:AssumeRole",
"Condition": {}
}
]
}
I have an STS policy attached to my user, group, codebuils service role and eksworkshopkubectlrole.
What could I be doing wrong
Getting error while connecting EKS cluster error: You must be logged in to the server (Unauthorized)
Hi All,
I have create a EKS cluster . and i am trying to connect this cluster from my local machine .I am getting this error while executing kubectl command .
Required your support .
]# kubectl describe -n kube-system configmap/aws-auth
error: You must be logged in to the server (Unauthorized)
I am Deploying EKS cluster using CDK pipeline in Typescript
This is May Cluster Stack,
import { PhysicalName, Stack, StackProps } from "aws-cdk-lib";
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import { Vpc } from "aws-cdk-lib/aws-ec2";
import * as eks from 'aws-cdk-lib/aws-eks';
import { Cluster } from "aws-cdk-lib/aws-eks/lib/cluster";
import { AccountRootPrincipal,Role } from "aws-cdk-lib/aws-iam";
import { Construct } from "constructs";
export interface DevOpsClusterStackProps extends StackProps {
cluster:Cluster,
vpc:Vpc,
}
export class DevOpsClusterStack extends Stack {
public readonly cluster: eks.Cluster;
accountId = this.account;
clusterName = "DevOpsCluster"
Role: Role;
/* Cluster Role Defined */
constructor(scope: Construct, id: string, props: DevOpsClusterStackProps) {
super(scope, id, props);
this.accountId = this.account;
this.clusterName = "DevOpsCluster";
const clusterAdmin = new Role(this, 'clusterAdmin', {
assumedBy: new AccountRootPrincipal(),
roleName: "clusterAdmin",
});
/* Cluster Configuration */
const cluster = new eks.Cluster(this, 'DevOpsCluster', {
clusterName: "DevOpsCluster",
version: eks.KubernetesVersion.V1_23,
defaultCapacity: 3,
mastersRole: clusterAdmin,
defaultCapacityInstance: ec2.InstanceType.of(ec2.InstanceClass.M5, ec2.InstanceSize.LARGE),
vpc:props.vpc,
vpcSubnets: [{
subnetType: ec2.SubnetType.PUBLIC
}],
});
cluster.addAutoScalingGroupCapacity('spot-group', {
instanceType: new ec2.InstanceType('m5.xlarge'),
});
if (Stack.of(this).region==this.region)
this.Role = createDeployRole(this, `for-1st-region`, cluster);
this.cluster = cluster;
}
}
function createDeployRole(scope: Construct, id: string, cluster: eks.Cluster): Role {
const role = new Role(scope, id, {
roleName: PhysicalName.GENERATE_IF_NEEDED,
assumedBy: new AccountRootPrincipal()
});
cluster.awsAuth.addMastersRole(role);
return role;
}
export interface PipelineStack extends StackProps {
Cluster: eks.Cluster,
Role: Role,
}
and
This is My Pipeline Stack to Deploy this cluster using Pipeline
import { Stack, StackProps, Stage } from 'aws-cdk-lib';
import * as codecommit from 'aws-cdk-lib/aws-codecommit';
import { CodePipeline, CodePipelineSource } from 'aws-cdk-lib/pipelines';
import * as pipelines from 'aws-cdk-lib/pipelines';
import { Construct } from 'constructs';
import { VpcStack } from './vpc-stack';
import { Cluster } from 'aws-cdk-lib/aws-eks/lib/cluster';
import { DevOpsClusterStack } from '../lib/devops-cluster-stack';
class DevelopmentStage extends Stage {
cluster: Cluster;
constructor(scope: Construct, id: string, props: StackProps) {
super(scope, id, props);
const vpcStack = new VpcStack(this, "VpcStack", {});
const ClusterStack = new DevOpsClusterStack (this, 'DevOpsCluster',{vpc:vpcStack.vpc , cluster:this.cluster});
}
}
/**
* Create a CI/CD pipelines for cluster deployment
*/
export class PipelineStack extends Stack {
cluster: Cluster;
static cluster: Cluster;
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
/**
* Here we provide pipeline start point as a Codecommit Soursecode to Create a CI/CD pipelines for cluster deployment
*/
const repository = codecommit.Repository.fromRepositoryName(this, 'Repository', 'CDK-Typescript-Project');
const source = CodePipelineSource.codeCommit(repository, "feature/create-eks-cluster")
const pipeline = new CodePipeline(this, 'Pipeline', {
pipelineName: 'EKS-CICD-Pipeline',
synth: new pipelines.ShellStep('Synth', {
input: source,
installCommands: ['npm i -g npm@latest',"npm install -g typescript"],
commands: [
'npm ci',
'npm run build',
'npx cdk synth',
]
})
});
// Developemnt stage This could include things like EC2 instances and more, depending on the needs of the application being developed.
const devStage = new DevelopmentStage(this, "Development", {
});
pipeline.addStage(devStage);
}
}
Also I have Created Separate VPC Stack
import { App, Stack, StackProps } from "aws-cdk-lib";
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import { IpAddresses } from "aws-cdk-lib/aws-ec2";
import { Construct } from "constructs";
/**
* Create a VPC with one Public and one Private Subnet
*/
export class VpcStack extends Stack {
public readonly vpc: ec2.Vpc;
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const vpc = new ec2.Vpc(this, 'vpc', {
natGateways: 1,
ipAddresses: IpAddresses.cidr("10.1.0.0/16"),
subnetConfiguration: [
{
name: 'Public',
subnetType: ec2.SubnetType.PUBLIC,
},
{
name: 'Private',
subnetType: ec2.SubnetType.PRIVATE_WITH_EGRESS,
}
],
maxAzs: 2
});
this.vpc = vpc;
}
}
/*I am receiving following error while deploying the Cluster Stack*/
ERROR is like: instance Fails to Join Kubernetes Cluster
DevOpsClusterNodegroupDefaultCapacity90B6204B
CREATE_FAILED
Resource handler returned message: "[Issue(Code=NodeCreationFailure, Message=Instances failed to join the kubernetes cluster, ResourceIds=[i-02c060ccb6d6e8c6f, i-048feaa20bfdca377, i-0a7a4184599e60cd2])] (Service: null, Status Code: 0, Request ID: null)" (RequestToken: e94890a6-5074-b4a3-a4e3-916cf510ef8a, HandlerErrorCode: GeneralServiceException)
How do I delete a subnet in an eks cluster using the command:
aws eks.......
I need to delete the cluster subnet:
subnet-0b0c38aacba4f47d2
And add the subnet to the cluster:
subnet-093c61d66a8293e85
Warning FailedDeployModel 8m36s (x20 over 47m) ingress Failed deploy model due to Internal error occurred: failed calling webhook "mtargetgroupbinding.elbv2.k8s.aws": failed to call webhook: Post "https://aws-load-balancer-webhook-service.kube-system.svc:443/mutate-elbv2-k8s-aws-v1beta1-targetgroupbinding?timeout=10s": no endpoints available for service "aws-load-balancer-webhook-service"
I installed aws-ebs-csi-driver
```
eksctl get addon --name aws-ebs-csi-driver --cluster <cluster-name>
```
```
NAME VERSION STATUS ISSUES IAMROLE UPDATE AVAILABLE
aws-ebs-csi-driver v1.16.0-eksbuild.1 ACTIVE 0 arn:aws:iam::<numbers>:role/AmazonEKS_EBS_CSI_DriverRole
```
When I look at the objects it created, the container *node-driver-registrar* under the pod *ebs-csi-node* keeps on crashing.
I'am unable to get the logs because I was having another issue below when running kubectl logs command.
> Internal error occurred: Authorization error (user=kube-apiserver-kubelet-client, verb=get, resource=nodes, subresource=proxy)
We previously had pod scheduling issue with this cluster ,and we gave on solving the issue , so we removed nodes and kept the control plane for troubleshooting.
Trying to add worker nodes , I followed AWS documents but I made a mistake applying the following configmap :
```
apiVersion: v1
kind: ConfigMap
metadata:
name: aws-auth
namespace: kube-system
data:
mapRoles: |
- rolearn: 'arn:aws:iam::account-number:role/testyy-NodeInstanceRole-1FQVVVZPS0TDP'
username: system:node:{{EC2PrivateDNSName}}
groups:
- system:bootstrappers
- system:nodes
```
That caused me to lose the cluster access , as it seem to have replaced the existing one , not sure why that happened , eks 1.21 so it might be the API version ?
We cannot delete the cluster now , is there a way to regain access to the cluster ?
I can provide the cluster arn if anyone can help us regain access , thanks