Set Up Cluster-Wide Authenticated Access to a Docker Registry

This section describes a way to provide an existing Kubernetes cluster with authenticated access to a Docker Registry using a DaemonSet (private-registry-docker), thus avoiding using imagePullSecrets manually. The DeamonSet overrides var/lib/kubelet/config.json under each Kubernetes node with a user-specified Docker JSON configuration file, thus providing cluster-wide authenticated access to the Docker Registry.

You may want authenticated access to a Docker Registry for the following reasons:

• The Docker Registry is private.
• The Docker Registry has rate-limiting when not authenticated.

What You’ll Need

The requirements to provide cluster-wide authenticated access to a Docker Registry depend on the type of your Docker Registry. Choose one of the following options:

Docker Hub

ECR

GCR

Custom (username/password)

Custom (dockerconfig.json)

• A configured management environment.
• An existing Kubernetes cluster.
• Your clone of the Arrikto GitOps repository.
• Your Docker ID and Password.

This section is a work in progress.

Note

When running on EKS, the IAM role of your EKS cluster nodes already provides authenticated access to your ECR. In this case, you can skip this guide.

This section is a work in progress.

Note

When running on GKE, the GCP service account of your GKE cluster nodes already provides authenticated access to your GCR. In this case, you can skip this guide.

This section is a work in progress.

• A configured management environment.

• An existing Kubernetes cluster.

• Your clone of the Arrikto GitOps repository.

• A Docker JSON configuration file that contains a key with proper rights and has a format like this:

  

  { "auths": { "https://index.docker.io/v1/": { "auth": "<BASE_64_STRING>" } } }

Procedure

Air Gapped

Patch the kustomization to use the mirrored images. To do so, run:

root@rok-tools:/# rok-image-patch \ > --kustomizations ~/ops/deployments/rok/private-registry-docker/overlays/deploy

follow the on-screen instructions and provide any necessary input.

1. Go to your GitOps repository, inside your rok-tools management environment:

   

   root@rok-tools:/# cd ~/ops/deployments

2. Copy your Docker JSON configuration file under your clone of the GitOps repository. Choose one of the following options, based on the type of the Docker Registry.

   Docker Hub

   ECR

   GCR

   Custom (username/password)

   Custom (dockerconfig.json)

   1. Create a temporary directory to store the Docker JSON configuration file:

      

      root@rok-tools:~/ops/deployments# export TMPDOCKER=$(mktemp -d)

   2. Create the Docker JSON configuration file. When prompted, provide your Docker Hub credentials:

      

      root@rok-tools:~/ops/deployments# docker --config ${TMPDOCKER?} login Username: Password:

   3. Store the path to the Docker JSON configuration file as a variable for easy access to it:

      

      root@rok-tools:~/ops/deployments# export DOCKERCONFIG=${TMPDOCKER?}/config.json

   4. Copy your Docker JSON configuration file under rok/private-registry-docker/overlays/deploy/secrets/dockerconfig.json:

      

      root@rok-tools:~/ops/deployments# cp ${DOCKERCONFIG?} \ > rok/private-registry-docker/overlays/deploy/secrets/dockerconfig.json

   5. Copy your Docker JSON configuration file under rok/private-registry-docker/overlays/deploy/secrets/image-dockerconfig.json. This will end up as an imagePullSecret for the DaemonSet itself:

      

      root@rok-tools:~/ops/deployments# cp ${DOCKERCONFIG?} \ > rok/private-registry-docker/overlays/deploy/secrets/image-dockerconfig.json

      Note

      You can optionally use a different Docker JSON configuration file for the DaemonSet image.

   6. Remove the temporary directory created to store the Docker JSON configuration:

      

      root@rok-tools:~/ops/deployments# rm -r ${TMPDOCKER?} && unset TMPDOCKER

   This section is a work in progress.

   This section is a work in progress.

   This section is a work in progress.

   1. Store the path to the Docker JSON configuration file:

      

      root@rok-tools:~/ops/deployments# export DOCKERCONFIG=<PATH_TO_DOCKERCONFIG>

      Replace <PATH_TO_DOCKERCONFIG> with the path to the dockerconfig.json file. For example:

      

      root@rok-tools:~/ops/deployments# export DOCKERCONFIG=/root/dockerconfig.json

   2. Copy your Docker JSON configuration file under rok/private-registry-docker/overlays/deploy/secrets/dockerconfig.json:

      

      root@rok-tools:~/ops/deployments# cp ${DOCKERCONFIG?} \ > rok/private-registry-docker/overlays/deploy/secrets/dockerconfig.json

   3. Copy your Docker JSON configuration file under rok/private-registry-docker/overlays/deploy/secrets/image-dockerconfig.json. This will end up as an imagePullSecret for the DaemonSet itself:

      

      root@rok-tools:~/ops/deployments# cp ${DOCKERCONFIG?} \ > rok/private-registry-docker/overlays/deploy/secrets/image-dockerconfig.json

      Note

      You can optionally use a different Docker JSON configuration file for the DaemonSet image.

3. Stage the changes in the private-registry-docker manifests:

   

   root@rok-tools:~/ops/deployments# git add rok/private-registry-docker/overlays/deploy

4. Commit the changes:

   

   root@rok-tools:~/ops/deployments# git commit -m \ > "Set up cluster-wide authenticated access to our Docker Registry"

5. Apply the kustomization:

   

   root@rok-tools:~/ops/deployments# rok-deploy --apply \ > rok/private-registry-docker/overlays/deploy

Verify

1. Ensure that the private-registry-docker DaemonSet is running. Verify that field STATUS is Running and field READY is 1/1:

   

   root@rok-tools:/# kubectl get pods -l app=private-registry-docker NAME READY STATUS RESTARTS AGE private-registry-docker-9tgj2 1/1 Running 0 6h12m

2. Ensure that the DaemonSet has properly mounted the Docker JSON configuration file to the host:

   

   root@rok-tools:/# kubectl exec ds/private-registry-docker \ > ls /host/var/lib/kubelet/config.json /host/var/lib/kubelet/config.json

3. Ensure there are no Pods with images from the Docker Registry that fail with ImagePullBackOff, that is, the following command produces no output:

   

   root@rok-tools:/# kubectl get pods -A | grep ImagePullBackOff

   Troubleshooting

   There are Pods in ImagePullBackOff state

   • Ensure these Pods use images from the Docker Registry you granted authenticated access to.
   • Wait a few minutes for kubelet to use the new credentials, since it caches them.
   • Try to start a container from an image in your Docker Registry using kubectl run.

Summary

You have successfully configured your Kubernetes cluster to have authenticated access to a Docker Registry.

What’s Next

Check out the rest of the maintenance operations that you can perform on your cluster.

• Operations Guide