Expose Serving

This section will walk you through the steps required to expose the served models of your EKS cluster to the outside world. You have two options:

  • Path-based serving: You will expose all models under a single subdomain but under a different prefix based on their namespace and name. Each model will be served under a URL with the following naming scheme: https://<domain>/<prefix>/<namespace>/<model-name>, for example, https://serving.example.com/serving/kubeflow-user/sklearn-iris.
  • Host-based serving: You will expose each model under a different subdomain. Each model will be served in a distinct subdomain that will have the following naming scheme: https://<model-name>-<namespace>.<domain>/, for example, https://sklearn-iris-kubeflow-user.serving.example.com/.

Note

EKF 1.5 supports only host-based serving that requires a wildcard certificate and a wildcard DNS entry. In EKF 2.0, we recommend you use path-based serving, since it is simpler and works on all platforms.

Important

Authentication will be enabled by default (similar to EKF), and users will be able to access their models using an external client. Currently all authenticated users will be able to access all models inside the cluster.

What You’ll Need

Procedure

Choose one of the following options, based on your cloud provider.

This section will guide you through exposing the served models of your EKS cluster using either Amazon Application Load Balancer or Amazon Classic Load Balancer.

This section will guide you through exposing the served models of your AKS cluster using Azure Load Balancer.

This section will guide you through exposing the served models of your GKE cluster using Google Cloud Load Balancer (GCLB).