How do I create an ALB Ingress in Amazon EKS and associate the ALB Ingress with AWS WAF?

Short description

The AWS Load Balancer Controller creates an Application Load Balancer when an Ingress object is created using the kubernetes.io/ingress.class: alb annotation. The Ingress resource configures the Application Load Balancer to route HTTP or HTTPS traffic to different pods within your Amazon EKS cluster. You can use AWS WAF to monitor the HTTP or HTTPS requests that are forwarded to the Application Load Balancer.

Resolution

Create an OIDC provider and IAM role for the AWS Load Balancer Controller

1.    Create an AWS Identity and Access Management (IAM) OIDC provider and associate the OIDC provider with your cluster:

eksctl utils associate-iam-oidc-provider --region region-code --cluster your-cluster-name --approve

2.    Download an IAM policy for the AWS Load Balancer Controller:

curl -o iam_policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.2.0/docs/install/iam_policy.json

Note: The IAM policy allows the AWS Load Balancer Controller to call AWS APIs on your behalf.

3.    Create an IAM policy using the policy that you downloaded from step 2:

aws iam create-policy --policy-name AWSLoadBalancerControllerIAMPolicy --policy-document file://iam_policy.json

Note: Copy the name of the policy's Amazon Resource Name (ARN) that's returned in Step 3.

4.    Create an IAM role for the AWS Load Balancer Controller and attach the role to the service account created in step 2:

eksctl create iamserviceaccount --cluster=your-cluster --namespace=kube-system --name=aws-load-balancer-controller --attach-policy-arn=arn:aws:iam::YOUR_AWS_ACCOUNT_ID:policy/AWSLoadBalancerControllerIAMPolicy --override-existing-serviceaccounts --approve

Uninstall the AWS ALB Ingress Controller for Kubernetes

The AWS Load Balancer Controller replaces the functionality of the AWS ALB Ingress Controller for Kubernetes.

Check if the AWS ALB Ingress Controller is installed:

kubectl get deployment -n kube-system alb-ingress-controller

If the AWS ALB Ingress Controller isn't installed, then you receive the following output:

Error from server (NotFound): deployments.apps "alb-ingress-controller" not found

Skip to the Install the AWS Load Balancer Controller using Helm 3.0.0 section.

-or-

If the AWS ALB Ingress Controller is installed, then you receive the following output:

NAME                   READY UP-TO-DATE AVAILABLE AGE
alb-ingress-controller 1/1   1          1         122d

Complete the following steps:

1.    Remove the AWS ALB Ingress Controller:

kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.8/docs/examples/alb-ingress-controller.yaml
kubectl delete -f https://raw.githubusercontent.com/kubernetes-sigs/aws-alb-ingress-controller/v1.1.8/docs/examples/rbac-role.yaml

Now, complete the following steps to add an IAM policy to your IAM role. The IAM policy gives the AWS Load Balancer Controller access to the resources created by the AWS ALB Ingress Controller for Kubernetes.

1.    Download the IAM policy:

curl -o iam_policy_v1_to_v2_additional.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.2.0/docs/install/iam_policy_v1_to_v2_additional.json

2.    Create the IAM policy, and then note the returned ARN:

aws iam create-policy --policy-name AWSLoadBalancerControllerAdditionalIAMPolicy --policy-document file://iam_policy_v1_to_v2_additional.json

3.    Attach the IAM policy to the IAM role that you created earlier:

aws iam attach-role-policy --role-name eksctl-your-role name  --policy-arn arn:aws:iam::111122223333:policy/AWSLoadBalancerControllerAdditionalIAMPolicy

Note: If you created the role using eksctl, then use the AWS CloudFormation console to find the role. Choose the eksctl-your-cluster-name-addon-iamserviceaccount-kube-system-aws-load-balancer-controller stack. Then, choose the Resources tab. The role name is in the Physical ID column. If you created the role using the AWS Management Console, then the role name is whatever you named it. For example: AmazonEKSLoadBalancerControllerRole.

Install the AWS Load Balancer Controller using Helm 3.0.0

1.    Install the TargetGroupBinding custom resource definitions:

kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller//crds?ref=master"

2.    Add the eks-charts repository:

helm repo add eks https://aws.github.io/eks-charts

3.    Install the AWS Load Balancer Controller using the command that corresponds to the AWS Region for your cluster:

helm upgrade -i aws-load-balancer-controller eks/aws-load-balancer-controller --set clusterName=your-cluster-name --set serviceAccount.create=false --set serviceAccount.name=aws-load-balancer-controller -n kube-system

Note: If you're deploying on Amazon EKS for Fargate, then you must add the following flags to your command:

* --set region=your-region-code
* --set vpcId=your-vpc-xxxxxxxx

4.    Verify that the AWS Load Balancer Controller is installed:

kubectl get deployment -n kube-system aws-load-balancer-controller

5.    Create a Kubernetes Ingress resource on your cluster with the following annotation:

annotations:
    kubernetes.io/ingress.class: alb

Note: The AWS Load Balancer Controller creates load balancers. The Ingress resource configures the Application Load Balancer to route HTTP or HTTPs traffic to different pods within your cluster.

6.    Add either an internal or internet-facing annotation to specify where you want the Ingress to create your load balancer:

alb.ingress.kubernetes.io/scheme: internal

-or-

alb.ingress.kubernetes.io/scheme: internet-facing

Note: Choose internal to create an internal load balancer, or internet-facing to create a public-facing load balancer.

Deploy an example application

1.    Deploy an example application to verify that the ALB Ingress Controller creates an Application Load Balancer as a result of the Ingress object:

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.2.0/docs/examples/2048/2048_full.yaml

2.    Verify that the Ingress resource is created and is associated with an Application Load Balancer:

kubectl get ingress ingress-2048 -n game-2048

Example output:

NAME           HOSTS   ADDRESS                                                                 PORTS      AGE
ingress-2048   *       example-2048game-2048ingr-xxxxxxx.region-code.elb.amazonaws.com         80          2h

Validate the Application Load Balancer address

1.    Open the Amazon Elastic Compute Cloud (Amazon EC2) console.

2.    On the navigation bar, choose Region. Be sure to select the same Region that you selected for your Amazon EKS cluster.

3.    On the navigation pane, in the LOAD BALANCING section, choose Load Balancers.

4.    Validate the address of your Application Load Balancer to see if the Ingress object has created the Application Load Balancer resource.

Create a web ACL

When you create a web ACL, choose the same Region that you're using for your Amazon EKS cluster and do the following:

1.    Associate your web ACL with the Application Load Balancer.

2.    Choose an Application Load Balancer as your resource type.

3.    Add the Amazon CloudWatch metric name for your web ACL.

4.    Add any desired conditions and rules to your web ACL.

5.    Copy the AWS WAF ARN from the AWS WAF console, or download the AWS WAF web ACL JSON file.

Example output of an AWS WAF web ACL in JSON:

{
  "Name": "",
  "Id": "ea199076-xxx-xxxx-xxxx-xxxxxxxxxxx",
  "ARN": "arn:aws:wafv2:us-east-2:<Account_Number>:regional/webacl/deployment/41b495a2-xxxx-xxxx-xxxx-xxxx16619441",
  "DefaultAction": {
    "Allow": {}
  },
  "Description": "",
  "Rules": [],
  "VisibilityConfig": {
    "SampledRequestsEnabled": ,
    "CloudWatchMetricsEnabled": ,
    "MetricName": ""
  },
  "Capacity": ,
  "ManagedByFirewallManager": 
}

Add the AWS WAF web ACL ARN annotation to your ALB Ingress

1.    Edit the ALB Ingress and add the alb.ingress.kubernetes.io/wafv2-acl-arn: annotation with the AWS WAF ID that you copied earlier:

kubectl edit ingress/ingress-2048 -n game-2048

Example output of the Ingress manifest:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/wafv2-acl-arn:  arn:aws:wafv2:us-east-2:<Account_Number>:regional/webacl/deployment/41b495a2-xxxx-xxxx-xxxx-xxxx16619441
    alb.ingress.kubernetes.io/scheme: internet-facing
    kubernetes.io/ingress.class: alb
  labels:
    app: 2048-ingress
  name: ingress-2048
  namespace: game-2048
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: service-2048
          servicePort: 80
        path: /*
status:
  loadBalancer: {}