Upgrade Istio

EKF 2.0 uses Istio 1.14.3. This guide will walk you through upgrading Istio to version 1.14.3 from Istio 1.9.6 that the previous version of EKF was using.

What You’ll Need

Procedure

  1. Go to your GitOps repository inside your rok-tools management environment:

    root@rok-tools:~# cd ~/ops/deployments
  2. Delete the previous Istio control plane installation:

    root@rok-tools:~/ops/deployments# rok-deploy --delete \ > rok/rok-external-services/istio/istio-1-9/istio-install/overlays/deploy \ > rok/rok-external-services/istio/istio-1-9/knative-serving/overlays/deploy
  3. Apply the new Istio control plane:

    root@rok-tools:~/ops/deployments# rok-deploy --apply \ > rok/rok-external-services/istio/istio-1-14/istio-crds/overlays/deploy \ > rok/rok-external-services/istio/istio-1-14/istio-namespace/overlays/deploy \ > rok/rok-external-services/istio/istio-1-14/istio-install/overlays/deploy \ > rok/rok-external-services/istio/istio-1-14/knative-serving/overlays/deploy
  4. Remove the deprecated Istio-related resources left by the previous version of EKF:

    root@rok-tools:~/ops/deployments# rok-kf-prune --app istio
  5. Confirm that the knative-serving and kubeflow namespaces, as well as all of the kubeflow user namespaces (namespaces that start with kubeflow-) have Istio sidecar injection enabled. Ensure that these namespaces show up in the following command’s output:

    root@rok-tools:~/ops/deployments# kubectl get ns -l istio-injection=enabled NAME STATUS AGE knative-serving Active 5d16h kubeflow Active 5d16h kubeflow-user Active 5d16h ...
  6. Upgrade the Istio sidecars, by deleting all Pods in the namespaces you found above. Istio will inject the new version sidecar once the owning controllers recreate the deleted Pods:

    root@rok-tools:~/ops/deployments# kubectl get ns -l istio-injection=enabled --no-headers \ > | awk '{print $1}' \ > | xargs -n1 -I {} kubectl delete pod --all -n {}
  7. Copy the patches from the old kustomization to the new one. Specifically:

    1. Copy the ingress-host patch:

      root@rok-tools:~/ops/deployments# cp -av \ > rok/rok-external-services/istio/istio-1-9/istio-install/overlays/deploy/patches/ingress-host.yaml \ > rok/expose-ekf/overlays/deploy/patches/ingress-host.yaml
    2. Copy the ingress-tls patch:

      root@rok-tools:~/ops/deployments# cp -av \ > rok/rok-external-services/istio/istio-1-9/istio-install/overlays/deploy/patches/ingress-tls.yaml \ > rok/expose-ekf/overlays/deploy/patches/ingress-tls.yaml
    3. Update the copied ingress-tls patch to use the new apiVersion for Ingress resources:

      root@rok-tools:~/ops/deployments# sed -i \ > -e 's|extensions/v1beta1|networking.k8s.io/v1|' \ > rok/expose-ekf/overlays/deploy/patches/ingress-tls.yaml
    4. Copy the certificate patch:

      root@rok-tools:~/ops/deployments# cp -av \ > rok/rok-external-services/istio/istio-1-9/istio-install/overlays/deploy/patches/certificate.yaml \ > rok/expose-ekf/overlays/deploy/patches/certificate.yaml
  8. Copy the secrets from the old kustomization to the new one:

    root@rok-tools:~/ops/deployments# cp -av \ > rok/rok-external-services/istio/istio-1-9/istio-install/overlays/deploy/secrets/ \ > rok/expose-ekf/overlays/deploy/
  9. Edit the rok/expose-ekf/overlays/deploy/kustomization.yaml file and enable the necessary resources and patches, as you see them in the snippets below. Choose one of the following options based on your cloud provider, the type of the load balancer you are using, and who manages your SSL certificates:

    resources: - ../../base - ../../base/ingress.yaml patches: - path: patches/ingress-host.yaml target: kind: Ingress name: istio-ingress - path: patches/xff.yaml target: kind: EnvoyFilter name: xff-trust-hops
    resources: - ../../base - ../../base/ingress.yaml - ../../base/certificate.yaml patches: - path: patches/ingress-host.yaml target: kind: Ingress name: istio-ingress - path: patches/ingress-tls.yaml - path: patches/certificate.yaml - path: patches/xff.yaml target: kind: EnvoyFilter name: xff-trust-hops
    resources: - ../../base - ../../base/ingress.yaml patches: - path: patches/ingress-host.yaml target: kind: Ingress name: istio-ingress - path: patches/ingress-tls.yaml - path: patches/xff.yaml target: kind: EnvoyFilter name: xff-trust-hops secretGenerator: - name: istio-ingress-tls-secret files: - secrets/tls.crt - secrets/tls.key type: "kubernetes.io/tls"
    resources: - ../../base - ../../base/ingress.yaml - ../../base/certificate.yaml patches: - path: patches/ingress-host.yaml target: kind: Ingress name: istio-ingress - path: patches/ingress-tls.yaml - path: patches/certificate.yaml - path: patches/xff.yaml target: kind: EnvoyFilter name: xff-trust-hops
    resources: - ../../base - ../../base/ingress.yaml - ../../base/certificate.yaml patches: - path: patches/ingress-host.yaml target: kind: Ingress name: istio-ingress - path: patches/ingress-tls.yaml - path: patches/certificate.yaml - path: patches/xff.yaml target: kind: EnvoyFilter name: xff-trust-hops
    resources: - ../../base - ../../base/ingress.yaml patches: - path: patches/ingress-host.yaml target: kind: Ingress name: istio-ingress - path: patches/ingress-tls.yaml - path: patches/xff.yaml target: kind: EnvoyFilter name: xff-trust-hops secretGenerator: - name: istio-ingress-tls-secret files: - secrets/tls.crt - secrets/tls.key type: "kubernetes.io/tls"
  10. Edit the rok/expose-ekf/overlays/deploy/patches/xff.yaml file.

    1. Set the number of trusted proxies in front of the Istio Gateway. Choose one of the following options based on your cloud provider and the type of load balancer your are using:

      root@rok-tools:~/ops/deployments# export TRUSTED_FRONT_PROXIES=2
      root@rok-tools:~/ops/deployments# export TRUSTED_FRONT_PROXIES=1
      root@rok-tools:~/ops/deployments# export TRUSTED_FRONT_PROXIES=1
      root@rok-tools:~/ops/deployments# export TRUSTED_FRONT_PROXIES=1
    2. Render the Istio envoy filter patch template with the values you have specified:

      root@rok-tools:~/ops/deployments# j2 \ > rok/expose-ekf/overlays/deploy/patches/xff.yaml.j2 \ > -o rok/expose-ekf/overlays/deploy/patches/xff.yaml
  11. Commit your changes:

    root@rok-tools:~/ops/deployments# git commit -am "Expose Istio"
  12. Apply the kustomization:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/expose-ekf/overlays/deploy

Verify

  1. Verify that the Pods in the istio-system namespace are up and running. Check the Pod status, and verify field STATUS is Running and field READY is 1/1 for all Pods:

    root@rok-tools:~# kubectl -n istio-system get pods NAME READY STATUS RESTARTS AGE authservice-0 1/1 Running 0 1m istio-ingressgateway-57f58bf544-x45kw 1/1 Running 0 1m istiod-68f6c899f5-wzjfm 1/1 Running 0 1m
  2. Verify that you have successfully created the Ingress object for Istio. The HOSTS field should match your SUBDOMAIN and the ADDRESS field shows the hostname of your Load Balancer:

    root@rok-tools:~/ops/deployments# kubectl -n istio-system get ingress NAME CLASS HOSTS ADDRESS PORTS AGE istio-ingress nginx arrikto-cluster.apps.example.com e53a524a-ingressnginx-ingr-8872-592794601.us-east-1.elb.amazonaws.com 80 1m
  3. Open your browser, and go to the EKF UI at

    https://<YOUR_SUBDOMAIN>/

    Replace <YOUR_SUBDOMAIN> with your the value shown in HOSTS above. For example:

    https://arrikto-cluster.apps.example.com/

Summary

You have successfully upgraded Istio.

What’s Next

The next step is to apply the latest manifest changes to Rok.