Kubeflow Notebooks

This section will discuss the work done and steps necessary to support notebook servers from within Kubeflow.

Kubernetes Pods and Kubeflow

Kubeflow runs in Kubernetes clusters as a collection of Pods. Pods are the smallest deployable units of computing that you can create and manage in Kubernetes. A Pod is a group of one or more containers, with shared storage and network resources, and a specification for how to run the containers.

Kubeflow Docker Images and Notebooks

Kubeflow provides Docker images to run JupyterLab Notebooks (and other applications) in Kubernetes Pods. The open-source Docker images are layered from generic application support to more specific application and package deployments.

../_images/docker-image-hierarchy.png

Kale Docker Images and Notebooks

The notebook server runs as a Pod in the Kubernetes user’s namespace. By default Kale is not available on a JupyterLab or VSCode notebook server running on a Kubernetes cluster. Arrikto makes this possible with additional Docker image modifications.

When creating a new notebook server, select the default Kale Docker image, based on the notebook type you want to create. Arrikto ships Kale Docker images for JupyterLab and Visual Studio Code (VSCode) notebook servers.

../_images/docker-image-jupyterlab.png

The image has the following naming scheme:

gcr.io/arrikto/jupyter-kale-py38:<IMAGE_TAG>

The JupyterLab image offers:

  • the Kale UI-based extension for JupyterLab
  • the Kale SDK
../_images/docker-image-vscode.png

The image has the following naming scheme:

gcr.io/arrikto/vscode-kale-py38:<IMAGE_TAG>

The VSCode image offers:

  • the Kale SDK

Note

The <IMAGE_TAG> varies based on the MiniKF or EKF release.

Rok and Notebook Volumes

During notebook server creation and during execution of Kubeflow pipelines facilitated by Kale the Kubernetes Pods will request storage with a PersistentVolumeClaim and will host PersistentVolumes that are provisioned in response.

Every notebook server comes by default with at least one volume, called the workspace volume. The Rok driver, rok-csi, prepares this volume and mounts it under the user’s home directory by default. This volume hosts and stores user added code and libraries.

../_images/default-workspace-volume.png

The Rok driver (rok-csi) also provisions and mounts additional volumes, called data volumes, at locations of your choice during the notebook server creation.

../_images/default-data-volume.png