Deploy NGINX Ingress Controller

In this section you will configure and deploy the NGINX Ingress Controller and expose it using an Application Load Balancer that will terminate TLS with an ACM certificate.

Check Your Environment

Before you create an Application Load Balancer, your VPC must meet the following requirements:

  • It should have at least two subnets in different Availability Zones, both of which are either public or private. These subnets should have the following tag:

    • Key: kubernetes.io/cluster/<CLUSTERNAME>
    • Value: shared
  • The private subnets should have the following tag so that Kubernetes knows what subnets to use for internal load balancers:

    • Key: kubernetes.io/role/internal-elb
    • Value: 1
  • The public subnets should have the following tag so that Kubernetes knows what subnets to use for external load balancers:

    • Key: kubernetes.io/role/elb
    • Value: 1

Procedure

  1. Go to your GitOps repository, inside your rok-tools management environment:

    root@rok-tools:~# cd ~/ops/deployments
    
  2. Edit rok/nginx-ingress-controller/overlays/deploy/kustomization.yaml and use ingress-alb as base:

    bases:
    - ../ingress-alb
    #- ../service-elb
    #- ../service-azurelb
    #- ../service-gclb
    
  3. Edit rok/nginx-ingress-controller/overlays/deploy/kustomization.yaml and enable only the ingress-alb and ingress-alb patches:

    patches:
    - path: patches/ingress-alb.yaml
    - path: patches/service-alb.yaml
    #- path: patches/service-elb.yaml
    #- path: patches/service-azurelb.yaml
    #- path: patches/service-gclb.yaml
    
  4. Obtain the ARN of your ACM certificate and copy it to your clipboard, as you are going to use this value in a later step:

    root@rok-tools:~/ops/deployments# export CERT=$(aws acm list-certificates | \
    >     jq -r '.CertificateSummaryList[] | select(.DomainName == "'${SUBDOMAIN?}'") | .CertificateArn') && \
    >     echo ${CERT?}
    arn:aws:acm:us-east-1:123456789012:certificate/3727755d-ce67-42ab-b364-560f73a34e35
    
  5. Edit rok/nginx-ingress-controller/overlays/deploy/patches/ingress-alb.yaml and specify the ARN of your ACM certificate:

    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789012:certificate/9b414703-707a-4589-a0ef-86b3d38df62f  # <-- Update this line with the ARN of your ACM Certificate
    
  6. If your VPC configuration has only private subnets you have to use an internal ALB instead of a internet facing one. Edit rok/nginx-ingress-controller/overlays/deploy/patches/ingress-alb.yaml and set the corresponding annotation:

    alb.ingress.kubernetes.io/scheme: internal   # <-- Update this line.
    
  7. Enable the firewall in your Application Load Balancer and allow access only to specific CIDRs. Edit rok/nginx-ingress-controller/overlays/deploy/patches/ingress-alb.yaml and set the desired trusted CIDRs in the corresponding annotation. Leave the default value of 0.0.0.0/0 if you want to allow access for everyone:

    alb.ingress.kubernetes.io/inbound-cidrs: 1.2.3.4/32
    

    Note

    You can set multiple trusted inbound CIDRs by specifying them as a string list (comma separated list). For more information see the official AWS Load Balancer Controller docs.

  8. Commit your changes:

    root@rok-tools:~/ops/deployments# git commit -am "Expose NGINX Ingress Controller with an ALB"
    
  9. Deploy NGINX Ingress Controller:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/nginx-ingress-controller/overlays/deploy
    
  10. Wait until the AWS Load Balancer Controller provisions the necessary AWS resources:

    $ kubectl get ingress -n ingress-nginx
    NAME            HOSTS   ADDRESS                                                                    PORTS   AGE
    ingress-nginx   *       e53a524a-ingressnginx-ingr-1234-592794601.eu-central-1.elb.amazonaws.com   80      64d
    
    Troubleshooting
    The Ingress object does not get an ADDRESS.
    1. Inspect the logs of aws-load-balancert-controller deployment in the kube-system namespace:

      root@rok-tools:~/ops/deployments# kubectl logs -n kube-system deploy/aws-load-balancer-controller
      

      If you see a message like the following:

      "msg"="Reconciler error" "error"="failed to build LoadBalancer configuration due to retrieval of subnets failed to resolve 2 qualified subnets.
      

      it means that your subnets are misconfigured.

    2. Check your environment and also verify your VPC configuration.

  11. Obtain the address of the ALB and copy it to your clipboard:

    root@rok-tools:~/ops/deployments# kubectl get ingress -n ingress-nginx ingress-nginx -o json | \
    >     jq -r '.status.loadBalancer.ingress[].hostname'
    e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com
    
  12. Edit rok/nginx-ingress-controller/overlays/deploy/patches/service-alb.yaml and set externalName to the name of the ingress-nginx service:

    externalName: e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com  # <-- Update this line with your ALB name.
    
  13. Commit your changes:

    root@rok-tools:~/ops/deployments# git commit -am "Set externalName of ingress-nginx service"
    
  14. Re-apply the manifests:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/nginx-ingress-controller/overlays/deploy
    

Verify

  1. Verify that NGINX Ingress Controller is up-and-running. Check pod status and verify field STATUS is Running and field READY is 1/1:

    root@rok-tools:~/ops/deployments# kubectl -n ingress-nginx get pods
    NAME                                        READY   STATUS    RESTARTS AGE
    nginx-ingress-controller-7f74f657bd-ln59l   1/1     Running   0        1m
    
  2. Verify that the ALB Ingress has obtained an address:

    root@rok-tools:~/ops/deployments# kubectl -n ingress-nginx get ingress
    NAME            CLASS    HOSTS   ADDRESS                                                                 PORT(S)   AGE
    ingress-nginx   <none>   *       e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com   80        1m
    
  3. Verify that the ingress-nginx service has an external IP equal to the address of the ALB:

    root@rok-tools:~/ops/deployments# kubectl -n ingress-nginx get service ingress-nginx
    NAME            TYPE           CLUSTER-IP    EXTERNAL-IP                                                             PORT(S)   AGE
    ingress-nginx   ExternalName   <none>        e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com   <none>    1m
    

Summary

You have successfully deployed the NGINX Ingress Controller and exposed it using an Application Load Balancer

What’s Next

The next step is to expose Istio.