Ingress annotations only for a specific path

Question

Hi,

I have this ingress configuration:

```
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "oidc-ingress"
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=300
    external-dns.alpha.kubernetes.io/hostname: example.com
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    alb.ingress.kubernetes.io/auth-type: oidc
    alb.ingress.kubernetes.io/auth-on-unauthenticated-request: authenticate
    alb.ingress.kubernetes.io/auth-idp-oidc: '{"issuer":"https://login.microsoftonline.com/some-id/v2.0","authorizationEndpoint":"https://login.microsoftonline.com/some-id/oauth2/v2.0/authorize","tokenEndpoint":"https://login.microsoftonline.com/some-id/oauth2/v2.0/token","userInfoEndpoint":"https://graph.microsoft.com/oidc/userinfo","secretName":"aws-alb-secret"}'
    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ssl-redirect
            port: 
              name: use-annotation
      - pathType: Prefix
        path: /jenkins
        backend:
          service:
            name: jenkins
            port: 
              number: 8080
      - pathType: Prefix
        path: /
        backend:
          service:
            name: apache
            port: 
              number: 80
```
If I `kubectl appy` this `Ingress` config it will apply `annotations` to all routing rules, which means:

```
/*
/jenkins
/jenkins/*
```

I would like to apply `OIDC annotations` only for the `Jenkins rules`, it means:

1. If I open `https://example.com` it will be available to everyone.
2. If I open `https://example.com/jenkins`, it will redirect me to `OIDC auth` page.

I can do this manually through `AWS console` when I remove `authenticate rule` from `/*` and leave it for `/jenkins/*` only.

However I would like to achieve this through `Ingress annotations` to be able to automate this process.

Please how can I do this?

Thanks for your help.

Accepted Answer

Hi, you should divide into several `Ingress`with `group` annotation. You may want to refer [this link](https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.1/guide/ingress/annotations/#group.order)

Test it like below!

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "base"
  annotations:
    alb.ingress.kubernetes.io/group.name: example
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
    alb.ingress.kubernetes.io/load-balancer-attributes: idle_timeout.timeout_seconds=300
    external-dns.alpha.kubernetes.io/hostname: example.com
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: ssl-redirect
            port: 
              name: use-annotation
```

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "jenkins"
  annotations:
    alb.ingress.kubernetes.io/group.name: example
    alb.ingress.kubernetes.io/group.order: 10
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/auth-type: oidc
    alb.ingress.kubernetes.io/auth-on-unauthenticated-request: authenticate
    alb.ingress.kubernetes.io/auth-idp-oidc: '{"issuer":"https://login.microsoftonline.com/some-id/v2.0","authorizationEndpoint":"https://login.microsoftonline.com/some-id/oauth2/v2.0/authorize","tokenEndpoint":"https://login.microsoftonline.com/some-id/oauth2/v2.0/token","userInfoEndpoint":"https://graph.microsoft.com/oidc/userinfo","secretName":"aws-alb-secret"}'
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /jenkins
        backend:
          service:
            name: jenkins
            port: 
              number: 8080
```

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: "default"
  annotations:
    alb.ingress.kubernetes.io/group.name: example
    alb.ingress.kubernetes.io/group.order: 20
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
spec:
  rules:
  - http:
      paths:
      - pathType: Prefix
        path: /
        backend:
          service:
            name: apache
            port: 
              number: 80
```

Answer

I would like to ask you, what's your opinion on this **OIDC solution** in terms of the security? Do you think **it's secure** to have such an **ALB** with inbound rules: `0.0.0.0/0` and restrict the `paths`, which I want to have private with **OIDC auth** only?

I and my colleagues work from different places, so it would be **NOT** possible to restrict the `inbound rules` with some specific IP addresses. We usually don't have a **public static IP** and we don't like an approach to connect to **VPN** which could provide us a **public static IP address**, that we could add to the `inbound rules` of the `ALB`.

I know there's an option to use this `annotation` : `alb.ingress.kubernetes.io/scheme: internal`, instead of `internet-facing`, but I'm not sure whether I can use this option for my use case and without a VPN access.

Thanks for your opinions.

Ingress annotations only for a specific path

Relevant questions

Validating annotations in AWS ground truth

Is it possible to assign an existing Network Load Balancer to an AWS EKS resource via annotations?

how to create multiple ingress port on Amazon EKS using single Application Load Balancer

Ground truth labeling job - unable to submit annotations

How do I set up Amazon VPC ingress routing with a stateless network appliance?

configure SSL in cluster of kubernetes

Service LoadBalancer Preserve DNS

Ingress annotations only for a specific path

ALB Ingress controller status is always showing pending.

Overlapping annotations AWS Comprehend