Deploy NGINX Ingress Controller¶
In this section you will configure and deploy the NGINX Ingress Controller and expose it using an Application Load Balancer that will terminate TLS with an ACM certificate.
Fast Forward
If you have already deployed NGINX Ingress Controller, expand this box to fast-forward.
Obtain the list of trusted inbound CIDRs of your ALB:
root@rok-tools:~# export EKS_ALB_NGINX_TRUSTED_CIDRS=$(kubectl get ingress \ > -n ingress-nginx ingress-nginx -o json \ > | jq -r '.metadata.annotations["alb.ingress.kubernetes.io/inbound-cidrs"] // "0.0.0.0/0"')
Obtain the address of the ALB:
root@rok-tools:~# export EKS_ALB_NGINX_HOSTNAME=$(kubectl get ingress \ > -n ingress-nginx ingress-nginx -o json \ > | jq -r '.status.loadBalancer.ingress[].hostname')
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deployments
Save your state:
root@rok-tools:~/ops/deployments# j2 deploy/env.eks-alb-nginx.j2 \ > -o deploy/env.eks-alb-nginx
Commit your changes:
root@rok-tools:~/ops/deployments# git commit -am "Deploy NGINX Ingress Controller"
Proceed to the Verify section.
See also
Choose one of the following options to deploy NGINX Ingress Controller:
- Option 1: Deploy NGINX Ingress Controller Automatically (preferred).
- Option 2: Deploy NGINX Ingress Controller Manually.
Overview
What You'll Need¶
- A configured management environment.
- Your clone of the Arrikto GitOps repository.
- An existing EKS cluster.
- An existing ACM certificate.
- A working AWS Load Balancer Controller.
- A set of configured ALB subnets.
Option 1: Deploy NGINX Ingress Controller Automatically (preferred)¶
Deploy NGINX Ingress Controller by following the on-screen instructions on
the rok-deploy
user interface.
If rok-deploy
is not already running, start it with:
root@rok-tools:~# rok-deploy --run-from eks-alb-nginx
Proceed to the Summary section.
Option 2: Deploy NGINX Ingress Controller Manually¶
If you want to deploy NGINX Ingress Controller manually, follow the instructions below.
Procedure¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deployments
Restore the required context from previous sections:
root@rok-tools:~/ops/deployments# source <(cat deploy/env.{eks-alb-subnets,eks-alb-acm})
root@rok-tools:~/ops/deployments# export EKS_ALB_SCHEME EKS_ALB_ACM_SUBDOMAIN \ > EKS_ALB_ACM_CERT
Edit
rok/nginx-ingress-controller/overlays/deploy/kustomization.yaml
and useingress-alb
as base:bases: - ../ingress-alb #- ../service-elb #- ../service-azurelb #- ../service-gclb
Edit
rok/nginx-ingress-controller/overlays/deploy/kustomization.yaml
and enable only theingress-alb
andservice-alb
patches:patches: - path: patches/ingress-alb.yaml - path: patches/service-alb.yaml #- path: patches/service-elb.yaml #- path: patches/service-azurelb.yaml #- path: patches/service-gclb.yaml
Obtain the ARN of your ACM certificate and copy it to your clipboard, as you are going to use it in the next step:
root@rok-tools:~/ops/deployments# echo ${EKS_ALB_ACM_CERT?} arn:aws:acm:us-east-1:123456789012:certificate/9b414703-707a-4589-a0ef-86b3d38df62f
Edit
rok/nginx-ingress-controller/overlays/deploy/patches/ingress-alb.yaml
and specify the ARN of your ACM certificate that you obtained in the previous step:alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789012:certificate/9b414703-707a-4589-a0ef-86b3d38df62f # <-- Update this line with the value of EKS_ALB_ACM_CERT
Edit
rok/nginx-ingress-controller/overlays/deploy/patches/ingress-alb.yaml
and set the corresponding annotation. Choose one of the following options, based on your ALB scheme:alb.ingress.kubernetes.io/scheme: internet-facing # <-- Update this line.
alb.ingress.kubernetes.io/scheme: internal # <-- Update this line.
Enable the firewall in your Application Load Balancer and allow access only to specific CIDRs. Choose one of the following options, based on your ALB scheme:
Specify the list of trusted inbound CIDRs for the ALB. Replace
<CIDRS>
with the desired value. Leave the default value of0.0.0.0/0
if you want to allow access for everyone:root@rok-tools:~/ops/deployments# export EKS_ALB_NGINX_TRUSTED_CIDRS=<CIDRS>
See also
Specify the list of trusted inbound CIDRs for the ALB. Replace
<CIDRS>
with the desired value. Leave the default value of0.0.0.0/0
if you want to allow access for everyone within the VPC:root@rok-tools:~/ops/deployments# export EKS_ALB_NGINX_TRUSTED_CIDRS=<CIDRS>
Note
Note that, since the Application Load Balancer will be an internal one, it already accepts connections only from within the VPC. Use the trusted CIDRs as an extra access control mechanism for controlling connections from within the VPC.
Note
You can set multiple trusted inbound CIDRs by specifying them as a string list (comma separated list). For more information, see the official AWS Load Balancer Controller docs.
Edit
rok/nginx-ingress-controller/overlays/deploy/patches/ingress-alb.yaml
and set the desired trusted CIDRs in the corresponding annotation:alb.ingress.kubernetes.io/inbound-cidrs: 0.0.0.0/0 # <-- Update this line with EKS_ALB_NGINX_TRUSTED_CIDRS
Deploy NGINX Ingress Controller:
root@rok-tools:~/ops/deployments# rok-deploy --apply rok/nginx-ingress-controller/overlays/deploy
Wait until the AWS Load Balancer Controller provisions the necessary AWS resources:
$ kubectl get ingress -n ingress-nginx NAME HOSTS ADDRESS PORTS AGE ingress-nginx * e53a524a-ingressnginx-ingr-1234-592794601.eu-central-1.elb.amazonaws.com 80 64d
Troubleshooting
The Ingress object does not get an ADDRESS.
Inspect the logs of
aws-load-balancert-controller
deployment in thekube-system
namespace:root@rok-tools:~/ops/deployments# kubectl logs -n kube-system deploy/aws-load-balancer-controller
If you see a message like the following:
"msg"="Reconciler error" "error"="failed to build LoadBalancer configuration due to retrieval of subnets failed to resolve 2 qualified subnets.
it means that your subnets are misconfigured.
Verify your subnets configuration.
Obtain the address of the ALB and copy it to your clipboard as you are going to use it in the next step:
root@rok-tools:~/ops/deployments# export EKS_ALB_NGINX_HOSTNAME=$(kubectl get ingress \ > -n ingress-nginx ingress-nginx -o json \ > | jq -r '.status.loadBalancer.ingress[].hostname') \ > && echo ${EKS_ALB_NGINX_HOSTNAME?} e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com
Edit
rok/nginx-ingress-controller/overlays/deploy/patches/service-alb.yaml
and setexternalName
to the name of theingress-nginx
service:externalName: e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com # <-- Update this line with EKS_ALB_NGINX_HOSTNAME
Save your state:
root@rok-tools:~/ops/deployments# j2 deploy/env.eks-alb-nginx.j2 \ > -o deploy/env.eks-alb-nginx
Commit your changes:
root@rok-tools:~/ops/deployments# git commit -am "Deploy NGINX Ingress Controller"
Re-apply the manifests:
root@rok-tools:~/ops/deployments# rok-deploy --apply rok/nginx-ingress-controller/overlays/deploy
Verify¶
Verify that NGINX Ingress Controller is up-and-running. Check pod status and verify field STATUS is Running and field READY is 1/1:
root@rok-tools:~# kubectl -n ingress-nginx get pods NAME READY STATUS RESTARTS AGE nginx-ingress-controller-7f74f657bd-ln59l 1/1 Running 0 1m
Verify that the ALB Ingress has obtained an address:
root@rok-tools:~# kubectl -n ingress-nginx get ingress NAME CLASS HOSTS ADDRESS PORT(S) AGE ingress-nginx <none> * e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com 80 1m
Verify that the
ingress-nginx
service has an external IP equal to the address of the ALB:root@rok-tools:~# kubectl -n ingress-nginx get service ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx ExternalName <none> e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com <none> 1m