This section explains the authentication and user-management integration of Kubeflow with GitLab.
OIDC, is a standardized OAuth2 flow that allows a Client (Kubeflow) to get a user’s identity from a trusted Provider (GitLab).
The Istio Gateway is an OIDC Client. Kubeflow’s applications do not contain any authentication code. Authentication is performed once at the Istio Gateway, when requests enter the Kubeflow cluster.
However, the Istio Gateway cannot act as an OIDC Client on its own. This is why we enhanced the Istio Gateway’s capabilities by creating an AuthService. which enables the Istio Gateway to act as an OIDC client. After the user is authenticated, the identity is placed in HTTP headers and the request is forwarded to Kubeflow’s applications. More details about this architecture can be found in this blog post.
Single Sign-On (SSO)¶
Single sign-on (SSO) enables users to authenticate with multiple applications by logging in only once. GitLab, as an Identity Provider, offers the following experience.
- User logs in to GitLab for the first time, typing their credentials.
- User navigates to Kubeflow, which is an OIDC Client to GitLab.
- Kubeflow initiates an OIDC flow to authenticate the user, redirecting them to GitLab to sign in.
- GitLab recognises the user is signed in, because it has stored a session cookie in the user’s browser.
- User is not asked to type their credentials. They may be asked to authorize the application to access their identity (configurable).
- After granting access to Kubeflow, the user is logged in and redirected to Kubeflow.
Single Logout (SLO)¶
Single logout (SLO) enables users to logout from all applications at once, without having to sign out from each one separately.
OIDC defines the following specs related to logout:
- OpenID Connect Session Management
- OpenID Connect Front-Channel Logout
- OpenID Connect Back-Channel Logout
Because Kubeflow is a confidential client that doesn’t live in the user’s browser, the Back-Channel logout is what applies here. The gist of the spec is:
- When a user logs out from the Provider (GitLab), the Provider sends requests to a logout endpoint at every Client (Kubeflow).
- When a user logs out from a Client, they are redirected to a special page at the Provider where they are prompted to logout. Then, the Provider initiated procedure described above logs them out of all Clients.
However, GitLab doesn’t seem to either:
- Implement the OpenID Connect Back-Channel Logout spec.
- Provide an administrative endpoint to log out users from all Clients.
To work around those issues, we found the following solutions:
To logout a user from GitLab when they logout from Kubeflow, we use an HTTP POST request to GitLab’s
/users/sign_outendpoint. This is effectively using CSRF so it is not ideal, however a recent GitLab issue removed CSRF protection of the logout endpoint for exactly this use.
To logout a user from Kubeflow (or other applications) when they logout from GitLab, we will force the AuthService to effectively use GitLab for session management and check the access/refresh token’s validity for every request.
Currently, when a user logs out from GitLab, GitLab doesn’t revoke all the tokens issued to applications. Since OIDC tokens are used for authentication and we want users to logout from all applications when they logout from GitLab, GitLab has to be extended to allow revoking all of the user’s OIDC tokens on logout.
GitLab as OIDC provider for Kubeflow¶
In this section we will describe how to use GitLab as OIDC provider for Kubeflow.
With the setup below internal components will have to connect to user
facing URLs, and thus we have to disable firewall. To do so remove the
alb.ingress.kubernetes.io/inbound-cidrs annotation from ingress-nginx
GitLab + Authservice¶
Authservice is a OIDC client. By default it is configured to talk to dex. Here we will bypass Dex completely and setup authservice to talk to GitLab directly.
First create an application, i.e., OIDC client credentials, in your GitLab. Use the admin account, go to Admin area -> Applications -> New Application, with the following settings:
include the generated credentials:
CLIENT_ID=<Application ID> CLIENT_SECRET=<Secret>
USERID_HEADER=kubeflow-userid USERID_CLAIM=email OIDC_PROVIDER=https://gitlab.example.com OIDC_SCOPES=read_user,profile,email,openid AUTHSERVICE_URL_PREFIX=https://demo.example.com/authservice/ CLIENT_NAME=Kubeflow TEMPLATE_PATH=web/templates/gitlab/auto_logout AFTER_LOGIN_URL=/reception STRICT_SESSION_VALIDATION=true
Commit and apply changes:
$ git commit -am "Use GitLab OIDC provider instead of Dex" $ kfctl apply -V -f kfctl_config.yaml
For changes to take effect we have to restart the pods manually:
$ kubectl delete pods -n istio-system -l app=authservice