Configure Default Snapshot Policy for Notebooks¶
This guide will walk you through configuring the default snapshot policy for Jupyter Notebooks in an Arrikto EKF installation. By default, the policy will snapshot all notebooks in all user namespaces once per day.
What You'll Need¶
- A configured management environment.
- Your clone of the Arrikto GitOps repository.
- An existing Kubeflow deployment.
Procedure¶
Go to the
skel-resources
deploy overlay of your GitOps repository, inside yourrok-tools
management environment:root@rok-tools:~# cd ~/ops/deployments/kubeflow/manifests/common/skel-resources/overlays/deploy
Include the notebook snapshot policy patch in the kustomization resources of the overlay:
root@rok-tools:~/ops/deployments/kubeflow/manifests/common/skel-resources/overlays/deploy# kustomize \ > edit add patch patches/notebook-backup.yaml
If you wish to modify the frequency with which the policy will run, edit the
patches/notebook-backup.yaml
file and set your desired time interval. By default, the policy will run once per day:schedule: - interval: 1 day # <-- Set this line to your desired interval #filters: #- key: name # operator: StartsWith # value: prefix- #paused: false
Note
Rok policy intervals support the following units:
seconds
,minutes
,hours
,days
,weeks
,months
,years
.If you wish the policy to only snapshot a subset of the notebooks, follow these steps to configure a policy filter. If you wish to snapshot all notebooks, you may skip this step.
Edit the
patches/notebook-backup.yaml
file and uncomment the filters:schedule: - interval: 1 day filters: # <-- Uncomment this line - key: name # <-- Uncomment this line operator: StartsWith # <-- Uncomment this line value: prefix- # <-- Uncomment this line #paused: false
Set the
operator
of the filter to the comparison operator you wish to use to determine which notebooks to snapshot. For example, if the operator is equal toStartsWith
, the policy will only snapshot notebooks whose name starts with the provided value:filters: - key: name operator: StartsWith # <-- Update this line with your desired operator value: prefix-
Note
Rok policy filters support the following operators:
Exists
,Equal
,NotEqual
,Contains
,NotContains
,LessThan
,LessThanOrEqual
,GreaterThan
,GreaterThanOrEqual
,StartsWith
,NotStartsWith
,EndsWith
,NotEndsWith
.Set the
value
of the filter to the value you wish to compare against.filters: - key: name operator: StartsWith value: prefix- # <-- Update this line with your value
If you wish to temporarily disable the policy in all user namespaces, edit the
patches/notebook-backup.yaml
file, uncomment thepaused
field and set it totrue
:#filters: #- key: name # operator: StartsWith # value: prefix- paused: true # <-- Uncomment this line and set it to true
Note
Paused policies do not schedule any new runs. You can restore the value of this field at any point to resume the policies.
Return to the base directory of your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~/ops/deployments/kubeflow/manifests/common/skel-resources/overlays/deploy# cd ~/ops/deployments
Commit your changes:
root@rok-tools:~/ops/deployments# git commit \ > -am "Configure default snapshot policy for notebooks"
Apply the kustomization:
root@rok-tools:~/ops/deployments# rok-deploy \ > --apply kubeflow/manifests/common/skel-resources/overlays/deploy
Verify¶
Retrieve the policy in the
kubeflow-skel
namespace:root@rok-tools:~# kubectl get -n kubeflow-skel roksnapshotpolicy -o wide NAME KIND BUCKET POLICY ID PAUSED AGE notebook-backup Notebook auto-backups 010660d8-6e7d-4afc-ac51-ad165876b7a4 10m
Verify that the policy
KIND
is equal toNotebook
.Verify that the
POLICY ID
is not empty.Note
Repeat the following steps for every namespace you wish to verify.
Specify the namespace you wish to verify:
root@rok-tools:~# export NAMESPACE=<NAMESPACE>
Replace
NAMESPACE
with the name of the namespace that you wish to verify, for example:root@rok-tools:~# export NAMESPACE=kubeflow-user
Retrieve the policy:
root@rok-tools:~# kubectl get -n ${NAMESPACE?} roksnapshotpolicy -o wide NAME KIND BUCKET POLICY ID PAUSED AGE notebook-backup Notebook auto-backups 43f35af5-5827-4731-b020-6cc5bac38a30 5m
Verify that the policy
KIND
is equal toNotebook
.Verify that the
POLICY ID
is not empty.
Summary¶
You have successfully created a snapshot policy in all user namespaces.
What's Next¶
Check out the rest of the operations you can perform on your Kubeflow deployment.