Configure KServe¶
In this section you will configure KServe for exposing your served models to the outside world.
Fast Forward
If you have already configured KServe, expand this box to fast-forward.
- Proceed to the Verify section.
Overview
What You’ll Need¶
- A configured management environment.
- Your clone of the Arrikto GitOps repository.
- An existing Kubernetes cluster on your premises.
Procedure¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:/# cd ~/ops/deploymentsSpecify the type of serving. Choose one of the following options based on whether you want path-based or host-based serving.
root@rok-tools:~/ops/deployments# export SERVING_TYPE=path-basedHost-based serving is currently not supported.
Set the subdomain to use for Serving:
root@rok-tools:~/ops/deployments# export SERVING_SUBDOMAIN=${FQDN?}Note
This is the FQDN you assigned to your load balancer.
Specify the serving prefix. Choose one of the following options based on whether you use path-based or host-based serving.
root@rok-tools:~/ops/deployments# export SERVING_PREFIX=<PREFIX>Replace
<PREFIX>
with your desired prefix, for example:root@rok-tools:~/ops/deployments# export SERVING_PREFIX=/servingHost-based serving is currently not supported.
Set the KServe URL template. Choose one of the following options based on whether you use path-based or host-based serving.
root@rok-tools:~/ops/deployments# export SERVING_URL_TEMPLATE="https://${SERVING_SUBDOMAIN?}${SERVING_PREFIX?}/{{ .Namespace }}/{{ .Name }}"Host-based serving is currently not supported.
Render the patch for KServe:
root@rok-tools:~/ops/deployments# j2 kubeflow/manifests/contrib/kserve/kserve/overlays/deploy/patches/ingress.yaml.j2 \ > -o kubeflow/manifests/contrib/kserve/kserve/overlays/deploy/patches/ingress.yamlEdit
kubeflow/manifests/contrib/kserve/kserve/overlays/deploy/kustomization.yaml
and ensure that theingress
patch is enabled:patches: - path: patches/ingress.yamlCommit your changes:
root@rok-tools:~/ops/deployments# git commit -am "Configure KServe"Apply the manifests:
root@rok-tools:~/ops/deployments# rok-deploy --apply \ > kubeflow/manifests/contrib/kserve/kserve/overlays/deploy
Verify¶
Ensure that the
urlTemplate
for KServe is equal to the one you previously set. Choose one of the following options based on whether you use path-based or host-based serving.root@rok-tools:~# kubectl get configmap inferenceservice-config \ > -n kserve -o jsonpath={.data.ingress} \ > | jq .urlTemplate "https://serving.example.com/serving/{{ .Namespace }}/{{ .Name }}"Host-based serving is currently not supported.