Authorize Access to Object Storage on Google Cloud¶
This guide will walk you through configuring Rok to have access to object storage resources on Google Cloud.
Overview
What You’ll Need¶
- A configured management environment.
- Your clone of the Arrikto GitOps repository.
- An existing GKE cluster.
- A cloud identity with access to your cloud provider’s storage service.
Procedure¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deploymentsEdit
rok/rok-cluster/overlays/deploy/kustomization.yaml
to set the parent of thedeploy
kustomization overlay togke
:bases: - ../gke # <-- Edit this line to point to the gke overlayRetrieve your bucket prefix. Copy the output to your clipboard, as you are going to use this value in the next step:
root@rok-tools:~/ops/deployments# echo ${BUCKET_PREFIX?} rok-myproject-us-east1-b-arrikto-clusterEdit
rok/rok-cluster/overlays/deploy/patches/configvars.yaml
to setdaemons.s3d.bucket_prefix
to your bucket prefix.spec: configVars: daemons.s3d.bucket_prefix: <BUCKET_PREFIX> # <-- Update this line with your bucket prefixRetrieve the ID of your Google project. Copy the output to your clipboard, as you are going to use this value in the next step:
root@rok-tools:~/ops/deployments# echo ${PROJECT_ID?} myprojectEdit
rok/rok-cluster/overlays/deploy/patches/configvars.yaml
to setdaemons.s3d.gcp.project_id
to the ID of your Google project.spec: configVars: daemons.s3d.gcp.project_id: <PROJECT_ID> # <-- Update this line with your GCP project IDEdit
rok/rok-cluster/overlays/deploy/patches/storage.yaml
to set thespec.s3.endpoint
field tohttps://storage.googleapis.com
.spec: s3: endpoint: https://storage.googleapis.com # <-- Update this line with the Google Cloud Storage endpoint.Retrieve the GCP region. Copy the output to your clipboard, as you are going to use this value in the next step:
root@rok-tools:~/ops/deployments# echo ${REGION?} us-east1Edit
rok/rok-cluster/overlays/deploy/patches/storage.yaml
to set thespec.s3.region
field to your GCP region.spec: s3: region: <REGION> # <-- Update this line with your GCP regionRetrieve the email of the Google service account you created for Rok. Copy the output to your clipboard, as you are going to use this value in the next step:
root@rok-tools:~/ops/deployments# echo ${GCP_SERVICE_ACCOUNT_EMAIL?} rok-arrikto-cluster@myproject.iam.gserviceaccount.comEdit
rok/rok-cluster/overlays/deploy/patches/storage.yaml
to set thespec.s3.GCPServiceAccount
field to the email of the Google service account you created for Rok.spec: s3: GCPServiceAccount: <GCP_SERVICE_ACCOUNT_EMAIL> # <-- Update this line with your GCP service account emailTrack all changes in the git repository:
root@rok-tools:~/ops/deployments# git add rok/rok-clusterCommit the changes:
root@rok-tools:~/ops/deployments# git commit -m "Configure object storage access for Rok"
Verify¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deploymentsDeploy a Pod using the service account of Rok:
Create the Rok namespaces:
root@rok-tools:~/ops/deployments# rok-deploy --apply rok/rok-namespaces/overlays/deployRender the
rok-verify-service-account-gke
manifest template:root@rok-tools:~/ops/deployments# j2 rok/rok-test/rok-verify-service-account-gke.yaml.j2 \ > -o rok/rok-test/rok-verify-service-account-gke.yamlAlternatively, download the
rok-verify-service-account-gke
manifest template provided below and use it locally.rok-verify-service-account-gke.yaml.j21 # This file is part of Rok. 2 # 3 # Copyright © 2021 Arrikto Inc. All Rights Reserved. 4-10 4 5 apiVersion: v1 6 kind: ServiceAccount 7 metadata: 8 labels: 9 app.kubernetes.io/part-of: rok-verify 10 annotations: 11 iam.gke.io/gcp-service-account: {{GCP_SERVICE_ACCOUNT_EMAIL}} 12 name: {{ROK_CLUSTER_NAME}} 13 namespace: {{ROK_CLUSTER_NAMESPACE}} Render the
rok-verify-storage-access-gke
manifest template:root@rok-tools:~/ops/deployments# j2 rok/rok-test/rok-verify-storage-access-gke.yaml.j2 \ > -o rok/rok-test/rok-verify-storage-access-gke.yamlAlternatively, download the
rok-verify-storage-access-gke
manifest template provided below and use it locally.rok-verify-storage-access-gke.yaml.j21 # This file is part of Rok. 2 # 3 # Copyright © 2021 Arrikto Inc. All Rights Reserved. 4-20 4 5 apiVersion: v1 6 kind: Pod 7 metadata: 8 name: rok-verify-storage-access 9 namespace: {{ROK_CLUSTER_NAMESPACE}} 10 spec: 11 serviceAccountName: {{ROK_CLUSTER_NAME}} 12 containers: 13 - name: rok-verify-storage-access 14 image: gcr.io/arrikto/rok-tools:release-2.0-l0-release-2.0.1 15 imagePullPolicy: IfNotPresent 16 workingDir: /root 17 env: 18 - name: BUCKET_PREFIX 19 value: "{{BUCKET_PREFIX}}" 20 - name: GCP_SERVICE_ACCOUNT_EMAIL 21 value: "{{GCP_SERVICE_ACCOUNT_EMAIL}}" 22 - name: REGION 23 value: "{{REGION}}" Air Gapped
Edit the rendered manifest
rok/rok-test/rok-verify-storage-access-gke.yaml
to use the mirroredrok-tools
image from your internal registry:containers: - name: rok-verify-storage-access image: gcr.io/arrikto/rok-tools:release-2.0-l0-release-2.0.1 # <-- Edit this line to point to your mirrored rok-tools imageCommit the changes:
root@rok-tools:~/ops/deployments# git commit -am "Configure Pod and SA to verify storage access"Create the
rok
service account:root@rok-tools:~/ops/deployments# kubectl create -f rok/rok-test/rok-verify-service-account-gke.yaml serviceaccount/rok createdTroubleshooting
The service account already exists
If you have already deployed Rok and created its service account, then the above command will fail with the following error:
Error from server (AlreadyExists): error when creating "rok/rok-test/rok-verify-service-account-gke.yaml": serviceaccounts "rok" already existsIn this case, ignore this error and proceed with the verification steps below.
Deploy the rendered
rok-verify-storage-access-gke
manifest to your GKE cluster:root@rok-tools:~/ops/deployments# kubectl apply -f rok/rok-test/rok-verify-storage-access-gke.yaml pod/rok-verify-storage-access createdWait for the
rok-verify-storage-access
Pod to come up:root@rok-tools:~/ops/deployments# kubectl get pods rok-verify-storage-access \ > -n ${ROK_CLUSTER_NAMESPACE?} NAME READY STATUS RESTARTS AGE rok-verify-storage-access 1/1 Running 0 20s
Verify the Pod credentials:
Enter the Pod:
root@rok-tools:~/ops/deployments# kubectl exec rok-verify-storage-access \ > -itn ${ROK_CLUSTER_NAMESPACE?} \ > -- /bin/bashEnsure that the Rok Google service account is the active account, that is, the output of the following command is
OK
:root@rok-verify-storage-access:~# gcloud auth list \ > --filter="status=ACTIVE AND account=${GCP_SERVICE_ACCOUNT_EMAIL?}" \ > --format=json \ > | jq -e '.[]' >/dev/null \ > && echo OK \ > || echo FAIL OK
Perform a request towards Cloud Storage:
Check if the
chocks-ca
Rok bucket exists. If so, go to step 4c.root@rok-verify-storage-access:~# gsutil ls -b gs://${BUCKET_PREFIX?}-chocks-ca/ BucketNotFoundException: 404 gs://rok-myproject-us-east1-b-arrikto-cluster-chocks-ca bucket does not exist.Troubleshooting
Command failed with ‘403 Forbidden’
If the above command failed with the following Python errors:
urllib.error.HTTPError: HTTP Error 403: Forbidden apitools.base.py.exceptions.CommunicationError: Could not reach metadata service: Forbiddenthen this means that the Kubernetes service account cannot impersonate the Google service account. This can happen if the Kubernetes service account does not have a role binding for the
iam.workloadIdentityUser
role on the Google service account. Make sure you have followed the Create Cloud Identity section to set up the necessary role bindings.Command failed with ‘403 AccessDenied’
If the above command failed with the following error:
AccessDeniedException: 403 rok-arrikto-cluster@myproject.iam.gserviceaccount.com does not have storage.buckets.create access to the Google Cloud project.then this means that the Kubernetes service account can impersonate the Google service account, but the Google service account does not have the
storage.buckets.get
permission on this bucket. Make sure you have followed the Create Cloud Identity section to set up the necessary role bindings.Create the Rok bucket:
root@rok-verify-storage-access:~# gsutil mb -l ${REGION?} gs://${BUCKET_PREFIX?}-chocks-ca/ Creating gs://rok-myproject-us-east1-b-arrikto-cluster-chocks-ca/...Troubleshooting
Command failed with ‘403 AccessDenied’
If the above command failed with the following error:
AccessDeniedException: 403 rok-arrikto-cluster@myproject.iam.gserviceaccount.com does not have storage.buckets.create access to the Google Cloud project.then this means that the Kubernetes service account can impersonate the Google service account, but the Google service account does not have the
storage.buckets.create
permission. Make sure you have followed the Create Cloud Identity section to set up the necessary role bindings.List the objects in the Rok bucket:
root@rok-verify-storage-access:~# gsutil ls gs://${BUCKET_PREFIX?}-chocks-ca/Troubleshooting
Command failed with ‘403 AccessDenied’
If the above command failed with the following error:
AccessDeniedException: 403 rok-arrikto-cluster@myproject.iam.gserviceaccount.com does not have storage.buckets.create access to the Google Cloud project.then this means that the Kubernetes service account can impersonate the Google service account, but the Google service account does not have the
storage.objects.list
permission on this bucket. Make sure you have followed the Create Cloud Identity section to set up the necessary role bindings.Ensure that the output of the above command is empty, or it lists some objects. This means that you have successfully impersonated the Rok Google service account and authorized a request with it.
Clean up the Pod:
Return to your management environment:
root@rok-verify-storage-access:~# exitDelete the Pod:
root@rok-tools:~/ops/deployments# kubectl delete -f rok/rok-test/rok-verify-storage-access-gke.yaml pod "rok-verify-storage-access" deletedDelete the Rok service account, if you created it in this Verify section (step 2e):
root@rok-tools:~/ops/deployments# kubectl delete sa \ > -l "app.kubernetes.io/part-of=rok-verify" \ > -n ${ROK_CLUSTER_NAMESPACE?} serviceaccount "rok" deleted
Summary¶
You have successfully provided Rok with access to the object storage service of your cloud provider.
What’s Next¶
The next step is to grant Rok access to Arrikto’s private container registry, so that it can pull images from it.