Set up cluster-wide authenticated access to a Docker Registry

This section describes a way to provide an existing Kubernetes cluster with authenticated access to a Docker Registry using a DaemonSet (private-registry-docker), thus avoiding using imagePullSecrets manually.

Reasons for authenticated access to a Docker Registry are:

  • The Docker Registry is private.
  • The Docker Registry has rate-limiting when not authenticated.

What You’ll Need

The requirements for this guide depend on the type of your Docker registry. Choose one of the following options:

To provide cluster-wide authenticated access to Docker Hub you will have to create a Docker JSON configuration file. For that you’ll need:

  • Your Docker ID and Password.

This section is a work in progress.

Note

When running on EKS, your IAM role already provides authenticated access to your ECR. In this case, you can skip this guide.

This section is a work in progress.

Note

When running on GKE, your GCP service account already provides authenticated access to your GCR. In this case, you can skip this guide.

This section is a work in progress.

To provide cluster-wide authenticated access to a Docker Registry you’ll need:

  • A Docker JSON configuration file that contains a key with proper rights and has a format like this:

    {
          "auths": {
                   "https://index.docker.io/v1/": {
                            "auth": "<BASE_64_STRING>"
                   }
          }
    }
    

    This key will be used to provide your Kubernetes cluster authenticated access to the Docker Registry.

  • Optionally a second Docker JSON configuration file with the key that will be used to access the Docker Registry that hosts the Docker image for the private-registry-docker DaemonSet. Note that you can use the same Docker JSON configuration file as above, or not use one at all.

Procedure

This section describes the steps you can follow to configure and deploy the private-registry-docker DaemonSet. It consists of 3 sub-sections:

Configure cluster-wide authenticated access to a Docker Registry

To configure an existing Kubernetes cluster for authenticated access to a Docker registry follow the steps below:

  1. Ensure you have a Docker JSON configuration file. If you don’t have a custom Docker JSON configuration file, create one. Choose one of the following options, based on the type of the Docker Registry.

    Follow the steps below to create a Docker JSON configuration file:

    1. Create a temporary directory to store the Docker JSON configuration file:

      root@rok-tools:/# export TMPDOCKER=$(mktemp -d)
      
    2. Create the Docker JSON configuration file, using your Docker Hub credentials:

      root@rok-tools:/# docker --config ${TMPDOCKER?} login
      
    3. Store the path to the Docker JSON configuration file as a variable for easy access to it:

      root@rok-tools:/# export DOCKERCONFIG=${TMPDOCKER?}/config.json
      

    This section is a work in progress.

    This section is a work in progress.

    This section is a work in progress.

    If you have a custom Docker JSON configuration file to use for cluster-wide Docker Registry access already, store the path to it as a variable for easy access to it later on:

    root@rok-tools:/# export DOCKERCONFIG=<path/to/dockerconfig.json>
    
  2. Go to the deployment repository:

    root@rok-tools:/# cd ~/ops/deployments
    
  3. Copy your Docker JSON configuration file under rok/private-registry-docker/overlays/deploy/secrets/dockerconfig.json. This will end up under each Kubernetes node as /var/lib/kubelet/config.json and will be used for cluster-wide access to the Docker Registry:

    root@rok-tools:~/ops/deployments# mv ${DOCKERCONFIG?} rok/private-registry-docker/overlays/deploy/secrets/dockerconfig.json
    

If you don’t want to add an imagePullSecret for the private-registry-docker DaemonSet you can skip the Add an imagePullSecret for the busybox image section and move forward to Deploy the DaemonSet.

Add an imagePullSecret for the busybox image

The DaemonSet resource uses the busybox image, which is publicly available. You generally do not need to set an imagePullSecret to pull it. You may need to set an imagePullSecret to work around Docker Hub’s rate limiting for unauthenticated users, or, alternatively, in airgapped environments. Optionally, you can use an imagePullSecret for the private-registry-docker DaemonSet itself. To do so, follow the steps below.

This section is optional.

  1. Ensure you have a Docker JSON configuration file. If you want, you can reuse the same Docker JSON configuration file that provides cluster-wide authenticated access to the Docker Registry. Choose one of the following options, based on the type of the Docker Registry that hosts the busybox Docker image.

    Follow the steps below to create a Docker JSON configuration file:

    1. Create a temporary directory to store the Docker JSON configuration file:

      root@rok-tools:/# export TMPDOCKER=$(mktemp -d)
      
    2. Create the Docker JSON configuration file, using your Docker Hub credentials:

      root@rok-tools:/# docker --config ${TMPDOCKER?} login
      
    3. Store the path to the Docker JSON configuration file as a variable for easy access to it:

      root@rok-tools:/# export IMAGEDOCKERCONFIG=${TMPDOCKER?}/config.json
      

    This section is a work in progress.

    This section is a work in progress.

    This section is a work in progress.

    If you have a custom Docker JSON configuration file to use to access the Docker Registry that hosts the Docker image for the private-registry-docker DaemonSet, store the path to it as a variable for easy access to it later on:

    root@rok-tools:/# export IMAGEDOCKERCONFIG=<path/to/dockerconfig.json>
    
  2. Go to the deployment repository:

    root@rok-tools:/# cd ~/ops/deployments
    
  3. Copy your IMAGEDOCKERCONFIG under rok/private-registry-docker/overlays/deploy/secrets/image-dockerconfig.json. This will end up as imagePullSecret for the DaemonSet itself:

    root@rok-tools:~/ops/deployments# mv ${IMAGEDOCKERCONFIG?} rok/private-registry-docker/overlays/deploy/secrets/image-dockerconfig.json
    
  4. Edit rok/private-registry-docker/overlays/deploy/kustomization.yaml and uncomment the following sections:

    ...
    secretGenerator:
    - name: private-registry-docker-image-secret
      files:
      - .dockerconfigjson=secrets/image-dockerconfig.json
      type: kubernetes.io/dockerconfigjson
    ...
    patches:
    - path: patches/daemonset.yaml
    

Deploy the DaemonSet

  1. Go to the deployment repository:

    root@rok-tools:/# cd ~/ops/deployments
    
  2. Stage the changes in the private-registry-docker manifests:

    root@rok-tools:~/ops/deployments# git add rok/private-registry-docker/overlays/deploy
    
  3. Commit the changes:

    root@rok-tools:~/ops/deployments# git commit -m "Set up cluster-wide authenticated access to our Docker Registry"
    
  4. Apply the kustomization:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/private-registry-docker/overlays/deploy
    

Verify

To verify that your Kubernetes cluster has authenticated access to the Docker Registry follow the steps below:

  1. Ensure that the private-registry-docker DaemonSet is running:

    root@rok-tools:/# kubectl get pods -l app=private-registry-docker
    NAME                            READY   STATUS    RESTARTS   AGE
    private-registry-docker-9tgj2   1/1     Running   0          6h12m
    

    Note

    The kubelet caches imagePullSecrets. It may take a few minutes for the new imagePullSecret to take effect.

  2. Ensure that Pods with images from a private Docker Registry don’t stuck at ImagePullBackOff. You can do so by describing the desired pod:

    root@rok-tools:/# kubectl describe pods <POD>
    

Summary

You have successfully configured your Kubernetes cluster to use authenticated access for the Docker Registry.