Switch Release Channel¶
This section describes how to switch from release-1.5
to
release-2.0
release channel for EKF.
What You’ll Need¶
- An upgraded management environment.
- Your local clone of the Arrikto GitOps repository with manifests for EKF 1.5.3 or later.
Check Your Environment¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deploymentsCheck your current release channel:
root@rok-tools:~/ops/deployments# git rev-parse --abbrev-ref HEAD release-1.5Fast Forward
If you have already switched to the release-2.0 release channel, expand this box to fast-forward:
Run
keychain
to start the SSH agent and use your SSH identity:root@rok-tools:~/ops/deployments# eval "$(keychain --eval id_rsa)" * keychain 2.8.2 ~ http://www.funtoo.org * Starting ssh-agent... * Adding 1 ssh key(s): /root/.ssh/id_rsa * ssh-add: Identities added: /root/.ssh/id_rsaTroubleshooting
can’t find id_rsa
If the above command shows the following warning message:
Warning: can't find id_rsa; skippingit means that keychain could not find the file
~/.ssh/id_rsa
, which normally contains your private SSH key.To proceed, follow the Configure Git guide to configure your private SSH key.
Cannot find public key for id_rsa
If the above command shows the following warning message:
Warning: Cannot find public key for id_rsait means that keychain clould not find the file
~/.ssh/id_rsa.pub
, which normally contains your public SSH key.To proceed, generate the public SSH key based on your private key:
root@rok-tools:~# ssh-keygen -yf /root/.ssh/id_rsa > /root/.ssh/id_rsa.pubProceed to the Verify section.
Get the current status of your GitOps repository and verify that your local repository is clean:
root@rok-tools:~/ops/deployments# git status On branch release-1.5 nothing to commit, working tree cleanEnsure that you are currently following the
release-1.5
release channel:root@rok-tools:~/ops/deployments# git rev-parse --abbrev-ref --symbolic-full-name @{u} origin/release-1.5
Procedure¶
Important
Mirror the GitOps repo to a private remote to be able to recover it in case of failure.
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deploymentsRun
keychain
to start the SSH agent and use your SSH identity:root@rok-tools:~/ops/deployments# eval "$(keychain --eval id_rsa)" * keychain 2.8.2 ~ http://www.funtoo.org * Starting ssh-agent... * Adding 1 ssh key(s): /root/.ssh/id_rsa * ssh-add: Identities added: /root/.ssh/id_rsaTroubleshooting
can’t find id_rsa
If the above command shows the following warning message:
Warning: can't find id_rsa; skippingit means that keychain could not find the file
~/.ssh/id_rsa
, which normally contains your private SSH key.To proceed, follow the Configure Git guide to configure your private SSH key.
Cannot find public key for id_rsa
If the above command shows the following warning message:
Warning: Cannot find public key for id_rsait means that keychain clould not find the file
~/.ssh/id_rsa.pub
, which normally contains your public SSH key.To proceed, generate the public SSH key based on your private key:
root@rok-tools:~# ssh-keygen -yf /root/.ssh/id_rsa > /root/.ssh/id_rsa.pubFetch latest upstream changes:
root@rok-tools:~/ops/deployments# git fetch --all -p Fetching originTroubleshooting
Permission denied
If the above command fails with the following error message:
Permission denied (publickey)ensure that you have run
keychain
as specified in the Procedure or the Fast-Forward section of this guide.If the error persists, it means that you do not have access to the Arrikto GitOps repository. To proceed, follow the Configure Git guide to configure your SSH identity.
Prepare your branch for the upcoming rebase, with the
rok-kf-rebase
tool:root@rok-tools:~/ops/deployments# rok-kf-rebaseAfter
rok-kf-rebase
completes, verify that the intermediate rebase branch was created:root@rok-tools:~/ops/deployments# git show --shortstat release-2.0-base commit 7ac4720df93325399aea44519bf69b55fec83e03 (release-2.0-base) Author: Rok Kubeflow Rebase <no-reply@arrikto.com> Date: Wed May 12 10:49:52 2021 +0000 Renaming commitNote
This may be an empty commit if no renames are required.
Switch to the
release-2.0
release channel and rebase your work on top of it:root@rok-tools:~/ops/deployments# git \ > -c merge.directoryRenames=false \ > -c merge.renameLimit=99999 \ > rebase -Xtheirs --onto origin/release-2.0 release-2.0-baseTroubleshooting
CONFLICT (modify/delete)
Rebasing your work may cause conflicts when you have modified a file that has been removed from the latest version of Arrikto manifests. In such a case the rebase process will fail with:
CONFLICT (modify/delete): path/to/file deleted in origin/release-2.0 and modified in HEAD~X. Version HEAD~X of path/to/file left in tree.Delete those files:
root@rok-tools:~/ops/deployments# git status \ > --porcelain \ > | awk '{if ($1=="DU") print $2}' \ > | xargs git rmContinue the rebase process:
root@rok-tools:~/ops/deployments# git rebase --continue
Configure your local branch to track the new release channel:
root@rok-tools:~/ops/deployments# git branch --set-upstream-to=origin/release-2.0Delete the intermediate rebase branch:
root@rok-tools:~/ops/deployments# git branch -D release-2.0-base
Air Gapped
If you are in an air gapped environment, expand this box.
- Follow the Mirror Images to Internal Registry guide to mirror all referenced images to an internal registry that your EKS cluster will have access to.
- Follow the Patch All Images for Your Deployment guide to patch all kustomizations to use the mirrored images from your internal Docker registry.
rok-kf-rebase
has reverted your custom changes in the JWA ConfigMap patch. Follow the Use Mirrored Kale Python Image guide again for HP tunning trials to work as expected.
Important
rok-kf-rebase
has reverted custom changes in the JWA ConfigMap patch. If
you had added custom JWA images you have to readd them by following the next
steps:
See the reverted changes:
root@rok-tools:~/ops/deployments# git diff release-1.5 -- \ > kubeflow/manifests/apps/jupyter/jupyter-web-app/upstream/overlays/deploy/patches/config-map.yamlFollow the Extend List of JWA Default Images guide to readd the desired changes.
Verify¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deploymentsEnsure that you are currently on the
release-2.0
branch:root@rok-tools:~/ops/deployments# git rev-parse --abbrev-ref HEAD release-2.0Get the current status of your GitOps repository and verify that your local repository is clean:
root@rok-tools:~/ops/deployments# git status On branch release-2.0 nothing to commit, working tree cleanEnsure that you are currently following the
release-2.0
release channel:root@rok-tools:~/ops/deployments# git rev-parse --abbrev-ref --symbolic-full-name @{u} origin/release-2.0