Add Extra Resources To All User Namespaces

In Arrikto EKF the Profile Controller takes care of deploying some Kubernetes resources under each user namespace by default. Those resources allow users to access Rok and Kubeflow Pipelines.

This section describes how to configure your Arrikto EKF installation to add extra, user-specified Kubernetes resources to all user namespaces. It makes use of the skel-resources deploy overlay.

What You’ll Need

Procedure

  1. Switch to the kubeflow/manifests directory of your GitOps repository:

    root@rok-tools:~# cd ~/ops/deployments/kubeflow/manifests

  2. Add any additional files with your resources under common/skel-resources/overlays/deploy. For instance, to add an example PodDefault resource, save the following file as common/skel-resources/overlays/deploy/poddefault-example.yaml:

    apiVersion: kubeflow.org/v1alpha1 kind: PodDefault metadata: name: example-extra-resource spec: desc: Example extra PodDefault resource env: - name: EXAMPLE_ENVVAR value: example selector: matchLabels: example-extra-resource: "true"

    Note

    The resources can include template variables in the Go template format. The Profile Controller renders the templated resources before applying them to user namespaces. For example, to add a resource that includes an annotation with the user namespace without the kubeflow- prefix, insert the following in your resource:

    annotations: example.annotation: '{{ .Namespace|trimPrefix "kubeflow-" }}'

    In the above example:

    • .Namespace is the template variable, and
    • trimPrefix is the function that modifies the value of the variable.

    You can check more supported functions at the Sprig Go library.

    Supported variables are:

    • .Namespace: The user namespace where the Profile Controller applies the resource.

  3. Modify the common/skel-resources/overlays/deploy/kustomization.yaml file to include the files with your resources. For example:

    resources: - ../../base - poddefault-example.yaml # <-- Add this line with your actual file name

    Note

    Add as many extra lines as needed to include all the files you have created.

  4. Detect the resource type of your extra resources, in the <name>.<api-group> format, where:

    • <name>: The lowercase, plural version of the kind field.
    • <api-group>: The part of the apiVersion field before the first slash (/).

    For instance, for the following example PodDefault resource, the desired resource type is poddefaults.kubeflow.org:

    apiVersion: kubeflow.org/v1alpha1 kind: PodDefault metadata: name: example-extra-resource ...

  5. Edit apps/profiles/upstream/overlays/deploy/patches/configmap.yaml to update the list of resource types the Profile Controller manages:

    data: ... # <-- Update following line with the types of resources to manage --> SKEL_RESOURCES: secrets,configmaps,serviceaccounts,rolebindings.rbac.authorization.k8s.io,poddefaults.kubeflow.org,roksnapshotpolicies.crd.arrikto.com,rokpresentationpolicies.crd.arrikto.com,rokbucketconfigurations.crd.arrikto.com ...

    Note

    The Profile Controller manages resources of the following types by default:

    • secrets
    • configmaps
    • serviceaccounts
    • rolebindings.rbac.authorization.k8s.io
    • poddefaults.kubeflow.org
    • roksnapshotpolicies.crd.arrikto.com
    • rokpresentationpolicies.crd.arrikto.com
    • rokbucketconfigurations.crd.arrikto.com

    Skip this step if the above list includes the types of all your extra resources.

  6. Make sure that the above patch is enabled. Edit apps/profiles/upstream/overlays/deploy/kustomization.yaml and add patches/configmap.yaml under patches, if it doesn’t exist already:

    patches: - patches/configmap.yaml # <-- Add this line

  7. Commit your changes:

    root@rok-tools:~/ops/deployments/kubeflow/manifests# git commit -am "Add extra skel resources"

  8. Apply the kustomization:

    root@rok-tools:~/ops/deployments/kubeflow/manifests# rok-deploy --apply \ > common/skel-resources/overlays/deploy \ > apps/profiles/upstream/overlays/deploy

Verify

  1. Check that the kubeflow-skel namespace has the extra resources:

    root@rok-tools:~# kubectl get -n kubeflow-skel <RESOURCE_TYPE>

    Replace RESOURCE_TYPE with the type of the extra resource you added, for example:

    root@rok-tools:~# kubectl get -n kubeflow-skel poddefaults.kubeflow.org NAME AGE kale-python-image 19d access-ml-pipeline 19d rok-auth 19d example-extra-resource 10d

Note

Repeat the following steps for every namespace that you wish to verify.

  1. Specify the namespace that you want to verify:

    root@rok-tools:~# export NAMESPACE=<NAMESPACE>

    Replace NAMESPACE with the name of the namespace that you wish to verify, for example:

    root@rok-tools:~# export NAMESPACE=kubeflow-user

  2. Check that the namespace has the extra resources:

    root@rok-tools:~# kubectl get -n ${NAMESPACE?} <RESOURCE_TYPE>

    Replace RESOURCE_TYPE with the type of the extra resource you added, for example:

    root@rok-tools:~# kubectl get -n ${NAMESPACE?} poddefaults.kubeflow.org NAME AGE kale-python-image 19d access-ml-pipeline 19d rok-auth 19d example-extra-resource 10d

Summary

You have successfully added extra resources to all user namespaces.

What’s Next

Check out the rest of the operations you can perform on your cluster.

PreviousNext