Add Extra Resources To All User Namespaces

In Arrikto EKF the Profile Controller takes care of deploying some Kubernetes resources under each user namespace by default. Those resources allow users to access Rok and Kubeflow Pipelines.

This section describes how to configure your Arrikto EKF installation to add extra, user-specified Kubernetes resources to all user namespaces. It makes use of the skel-resources deploy overlay.

What You'll Need

Procedure

  1. Switch to the kubeflow/manifests directory of your GitOps repository:

    root@rok-tools:~# cd ~/ops/deployments/kubeflow/manifests
    
  2. Add any additional files with your resources under common/skel-resources/overlays/deploy. For instance, to add an example PodDefault resource, save the following file as common/skel-resources/overlays/deploy/poddefault-example.yaml:

    apiVersion: kubeflow.org/v1alpha1
    kind: PodDefault
    metadata:
      name: example-extra-resource
    spec:
      desc: Example extra PodDefault resource
      env:
      - name: EXAMPLE_ENVVAR
        value: example
      selector:
        matchLabels:
          example-extra-resource: "true"
    
  3. Modify the common/skel-resources/overlays/deploy/kustomization.yaml file to include the files with your resources. For example:

    resources:
    - ../../base
    - poddefault-example.yaml  # <-- Add this line with your actual file name
    

    Note

    Add as many extra lines as needed to include all the files you have created.

  4. Detect the resource type of your extra resources, in the <name>.<api-group> format, where:

    • <name>: The lowercase, plural version of the kind field.
    • <api-group>: The part of the apiVersion field before the first slash (/).

    For instance, for the following example PodDefault resource, the desired resource type is poddefaults.kubeflow.org:

    apiVersion: kubeflow.org/v1alpha1
    kind: PodDefault
    metadata:
      name: example-extra-resource
    ...
    
  5. Edit apps/profiles/upstream/overlays/deploy/patches/configmap.yaml to update the list of resource types the Profile Controller manages:

    data:
      ...
      # <-- Update following line with the types of resources to manage -->
      SKEL_RESOURCES: secrets,configmaps,serviceaccounts,rolebindings.rbac.authorization.k8s.io,poddefaults.kubeflow.org,roksnapshotpolicies.crd.arrikto.com,rokpresentationpolicies.crd.arrikto.com
      ...
    

    Note

    The Profile Controller manages resources of the following types by default:

    • secrets
    • configmaps
    • serviceaccounts
    • rolebindings.rbac.authorization.k8s.io
    • poddefaults.kubeflow.org
    • roksnapshotpolicies.crd.arrikto.com
    • rokpresentationpolicies.crd.arrikto.com

    Skip this step if the above list includes the types of all your extra resources.

  6. Make sure that the above patch is enabled. Edit apps/profiles/upstream/overlays/deploy/kustomization.yaml and add patches/configmap.yaml under patches, if it doesn't exist already:

    patches:
    - patches/configmap.yaml  # <-- Add this line
    
  7. Commit your changes:

    root@rok-tools:~/ops/deployments/kubeflow/manifests# git commit -am "Add extra skel resources"
    
  8. Apply the kustomization:

    root@rok-tools:~/ops/deployments/kubeflow/manifests# rok-deploy --apply \
    >   common/skel-resources/overlays/deploy \
    >   apps/profiles/upstream/overlays/deploy
    

Verify

  1. Check that the kubeflow-skel namespace has the extra resources:

    root@rok-tools:~# kubectl get -n kubeflow-skel <RESOURCE_TYPE>
    

    Replace RESOURCE_TYPE with the type of the extra resource you added, for example:

    root@rok-tools:~# kubectl get -n kubeflow-skel poddefaults.kubeflow.org
    NAME                             AGE
    kale-python-image                19d
    access-ml-pipeline               19d
    rok-auth                         19d
    example-extra-resource           10d
    

Note

Repeat the following steps for every namespace that you wish to verify.

  1. Specify the namespace that you want to verify:

    root@rok-tools:~# export NAMESPACE=<NAMESPACE>
    

    Replace NAMESPACE with the name of the namespace that you wish to verify, for example:

    root@rok-tools:~# export NAMESPACE=kubeflow-user
    
  2. Check that the namespace has the extra resources:

    root@rok-tools:~# kubectl get -n ${NAMESPACE?} <RESOURCE_TYPE>
    

    Replace RESOURCE_TYPE with the type of the extra resource you added, for example:

    root@rok-tools:~# kubectl get -n ${NAMESPACE?} poddefaults.kubeflow.org
    NAME                             AGE
    kale-python-image                19d
    access-ml-pipeline               19d
    rok-auth                         19d
    example-extra-resource           10d
    

Summary

You have successfully added extra resources to all user namespaces.

What's Next

Check out the rest of the operations you can perform on your cluster.