Authorize Access to Object Storage

This section will guide you through giving Rok access to the cloud provider’s Object Storage service.

Important

The provided manifests contain a deploy overlay to use as a guide for building your overlay, so you can edit it in-place. Should conflicts arise in these files, always keep the local changes. deploy overlays are user-owned.

Choose one of the following options to give Rok access to the Object Storage service:

What You’ll Need

Option 1: Authorize Access to Object Storage Automatically (preferred)

In this section you will configure Rok to have access to object storage resources in an automated manner, using the rok-deploy CLI.

Procedure

Choose one of the following options, based on your cloud provider.

To configure AWS so that Rok has access to S3, follow the on-screen instructions.

You may now proceed to the Summary section.

Rok does not currently support automatic authorization to assume an Azure Managed Identity. Please follow the instructions in the Option 2: Authorize Access to Object Storage Manually section to authorize Rok to use the Azure Managed Identity manually.
Rok does not currently support automatic authorization to assume a Google Workload Identity. Please follow the instructions in the Option 2: Authorize Access to Object Storage Manually section to authorize Rok to use the Google Workload Identity manually.

Option 2: Authorize Access to Object Storage Manually

If you want to configure Rok so that it has access to object storage resources manually, follow this section.

Procedure

Choose one of the following options, based on your cloud provider.

  1. Go inside your clone of the GitOps repo:

    root@rok-tools:~# cd ~/ops/deployments
    
  2. Edit rok/rok-cluster/overlays/deploy/patches/storage.yaml to specify the region to use:

    s3:
      endpoint: https://s3.<REGION>.amazonaws.com
      region: <REGION>
    

    Replace <REGION> with your desired region.

  3. Select the namespace in which to deploy Rok:

    root@rok-tools:~/ops/deployments# export ROK_CLUSTER_NAMESPACE=rok
    
  4. Select the name of the Rok cluster:

    root@rok-tools:~/ops/deployments# export ROK_CLUSTER_NAME=rok
    
  5. Select the bucket prefix Rok will use to store its snapshots in Amazon S3:

    root@rok-tools:~/ops/deployments# export BUCKET_PREFIX=rok-${AWS_ACCOUNT_ID?}-${AWS_DEFAULT_REGION?}-${CLUSTERNAME?}-${ROK_CLUSTER_NAMESPACE?}-${ROK_CLUSTER_NAME?}
    
  6. Remove the Rok cluster namespace and name if they are both equal to rok and obtain the final value:

    root@rok-tools:~/ops/deployments# export BUCKET_PREFIX=${BUCKET_PREFIX%-rok-rok} && echo ${BUCKET_PREFIX?}
    
  7. Edit rok/rok-cluster/overlays/deploy/patches/configvars.yaml to specify the bucket prefix to use:

    spec:
      configVars:
        ...
        daemons.s3d.bucket_prefix: <BUCKET_PREFIX>
    

    Replace <BUCKET_PREFIX> with your bucket prefix. For example:

    spec:
      configVars:
        ...
        daemons.s3d.bucket_prefix: rok-123456789012-us-west-2-arrikto-cluster
    

    Important

    Rok will create a number of buckets with this specific prefix. Please note that Rok assumes it owns all buckets with names starting with this prefix, e.g., for Garbage Collection purposes, so this prefix must not be shared with any other application.

  8. Select the ARN of the IAM role for the service account Rok will run with:

    root@rok-tools:~/ops/deployments# export IAM_ROLE_ARN=arn:aws:iam::${AWS_ACCOUNT_ID?}:role/rok-${AWS_DEFAULT_REGION?}-${CLUSTERNAME?}-${ROK_CLUSTER_NAMESPACE?}-${ROK_CLUSTER_NAME?}
    
  9. Remove the Rok cluster namespace and name if they are both equal to rok and obtain the final value:

    root@rok-tools:~/ops/deployments# export IAM_ROLE_ARN=${IAM_ROLE_ARN%-rok-rok} && echo ${IAM_ROLE_ARN?}
    
  10. Edit rok/rok-cluster/overlays/deploy/patches/storage.yaml to specify the IAM role ARN:

    s3:
      ...
      AWSRoleARN: <IAM_ROLE_ARN>
    

    Replace <IAM_ROLE_ARN> with the ARN of your IAM role. For example:

    s3:
      ...
      AWSRoleARN: arn:aws:iam::123456789012:role/rok-us-west-2-arrikto-cluster
    
  11. Stage your changes:

    root@rok-tools:~/ops/deployments# git add rok/rok-cluster
    
  12. Commit your changes:

    root@rok-tools:~/ops/deployments# git commit -m "Configure Object Storage access for Rok"
    
  1. Go inside your clone of the GitOps repo:

    root@rok-tools:~# cd ~/ops/deployments
    
  2. Retrieve the Resource ID of the Managed Identity:

    root@rok-tools:~/ops/deployments# export IDENTITY_RESOURCE_ID="$(az identity show \
    > -g ${AZ_RESOURCE_GROUP?} -n ${AZ_MANAGED_IDENTITY?} --query id -otsv)"
    
  3. Create the namespaces used by Rok:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/rok-namespaces/overlays/deploy
    
  4. Select the name of the Pod Identity Rok will use to access Azure Blob Storage:

    root@rok-tools:~/ops/deployments# export POD_IDENTITY_NAME=rok-${ROK_CLUSTER_NAME?}
    
  5. Remove the Rok cluster name from the Pod Identity name if it is equal to rok:

    root@rok-tools:~/ops/deployments# export POD_IDENTITY_NAME=${POD_IDENTITY_NAME%-rok}
    
  6. Create a Pod Identity for S3Proxy in the rok namespace:

    root@rok-tools:~/ops/deployments# az aks pod-identity add \
    >  --resource-group ${AZ_RESOURCE_GROUP?} \
    >  --cluster-name ${CLUSTERNAME?} \
    >  --namespace ${ROK_CLUSTER_NAMESPACE?} \
    >  --name ${POD_IDENTITY_NAME?} \
    >  --identity-resource-id ${IDENTITY_RESOURCE_ID?}
    
    Troubleshooting
    The command failed with a ‘Bad Request’ error

    Azure may take some time to enable Pod Identities in a cluster. When trying to create a Pod Identity before the changes are fully propagated, the following error may occur:

    Operation failed with status: 'Bad Request'. Details: Cluster identity has no assignment permission over identity 'resourceId: /subscriptions/f7a20dff-0a55-42bd-bec6-a18c6c370d0e/resourcegroups/arr/providers/Microsoft.ManagedIdentity/userAssignedIdentities/rok - clientId: 6c5f59e0-e4f3-4ac8-939c-66cf87d8056d - objectId: 95520a25-3c7d-4b94-b516-cdc4cb2d881a'. Please grant at least 'Managed Identity Operator' permission before assigning pod identity
    

    If you see this error, wait for a few minutes for Azure to propagate permissions, and try creating the Pod Identity again.

  7. Configure S3Proxy to access the Azure Storage Account:

    root@rok-tools:~/ops/deployments# j2 rok/rok-external-services/s3proxy/overlays/deploy/config.env.j2 -o rok/rok-external-services/s3proxy/overlays/deploy/config.env
    
  8. Copy your Pod Identity name to your clipboard as you are going to use this value in the following step:

    root@rok-tools:~/ops/deployments# echo ${POD_IDENTITY_NAME?}
    rok
    
  9. Edit rok/rok-external-services/s3proxy/overlays/deploy/patches/daemonset.yaml to set the aadpodidbinding label to the name of the Pod Identity you created, so S3Proxy can use it to access Azure Blob Storage.

    spec:
      template:
        metadata:
          labels:
            aadpodidbinding: rok  # <-- Update this line with your Pod Identity name
    
  10. Generate random credentials for Rok to access S3Proxy:

    root@rok-tools:~/ops/deployments# export S3PROXY_IDENTITY="$(openssl rand -hex 16)"
    root@rok-tools:~/ops/deployments# export S3PROXY_CREDENTIAL="$(openssl rand -hex 32)"
    
  11. Provide the generated credentials to S3Proxy:

    root@rok-tools:~/ops/deployments# j2 rok/rok-external-services/s3proxy/overlays/deploy/secrets/credentials.env.j2 -o rok/rok-external-services/s3proxy/overlays/deploy/secrets/credentials.env
    
  12. Edit rok/rok-cluster/overlays/deploy/kustomization.yaml to set the parent of the deploy kustomization overlay to aks:

    bases:
    - ../aks
    
  13. Select the bucket prefix Rok will use to store its snapshots in Azure Blob Storage:

    root@rok-tools:~/ops/deployments# BUCKET_PREFIX="rok-${ROK_CLUSTER_NAMESPACE?}-${ROK_CLUSTER_NAME?}"
    
  14. Remove the Rok cluster namespace and name if they are both equal to rok:

    root@rok-tools:~/ops/deployments# export BUCKET_PREFIX=${BUCKET_PREFIX%-rok-rok}
    
  15. Copy your bucket prefix to your clipboard, as you are going to use this value in the next step:

    root@rok-tools:~/ops/deployments# echo $BUCKET_PREFIX
    rok
    
  16. Edit rok/rok-cluster/overlays/deploy/patches/configvars.yaml to set the daemons.s3d.aws.access_key_id and daemons.s3d.aws.secret_access_key Rok Cluster configuration variables to the credentials you generated above.

    spec:
      configVars:
        daemons.s3d.bucket_prefix: "<BUCKET_PREFIX>"  # <-- Update this line with your bucket prefix
        daemons.s3d.aws.access_key_id: "<S3PROXY_IDENTITY>"  # <-- Update this line with your S3Proxy Access Key ID
        daemons.s3d.aws.secret_access_key: "<S3PROXY_CREDENTIAL>"  # <-- Update this line with your S3Proxy Secret Access Key
    
  17. Track all changes in the git repository:

    root@rok-tools:~/ops/deployments# git add rok/rok-cluster rok/rok-external-services
    
  18. Commit the changes:

    root@rok-tools:~/ops/deployments# git commit -m "Configure Azure Blob Storage access for Rok"
    
  1. Go inside your clone of the GitOps repo:

    root@rok-tools:~# cd ~/ops/deployments
    
  2. Edit rok/rok-cluster/overlays/deploy/kustomization.yaml to set the parent of the deploy kustomization overlay to gke:

    bases:
    - ../gke  # <-- Edit this line to point to the gke overlay
    
  3. Retrieve your bucket prefix. Copy the output to your clipboard, as you are going to use this value in the next step:

    root@rok-tools:~/ops/deployments# echo ${BUCKET_PREFIX?}
    
  4. Edit rok/rok-cluster/overlays/deploy/patches/configvars.yaml to set daemons.s3d.bucket_prefix to your bucket prefix and daemons.s3d.gcp.project_id to the ID of your Google project.

    spec:
      configVars:
        daemons.s3d.bucket_prefix: "<BUCKET_PREFIX>"  # <-- Update this line with your bucket prefix
        daemons.s3d.gcp.project_id: "<PROJECT_ID>"  # <-- Update this line with your GCP project ID
    
  5. Retrieve the email of the Google service account you created for Rok. Copy the output to your clipboard, as you are going to use this value in the next step:

    root@rok-tools:~/ops/deployments# echo ${GCP_SERVICE_ACCOUNT_EMAIL?}
    
  6. Edit rok/rok-cluster/overlays/deploy/patches/storage.yaml to set the spec.s3.region field to your GCP region, the spec.s3.GCPServiceAccount field to the email of the Google service account you created for Rok, and the spec.s3.endpoint field to https://storage.googleapis.com.

    spec:
      s3:
        endpoint: "https://storage.googleapis.com"  # <-- Update this line with the Google Cloud Storage endpoint.
        region: "<REGION>"  # <-- Update this line with your GCP region
        GCPServiceAccount: "<GCP_SERVICE_ACCOUNT_EMAIL>"  # <-- Update this line with your GCP service account email
    
  7. Track all changes in the git repository:

    root@rok-tools:~/ops/deployments# git add rok/rok-cluster
    
  8. Commit the changes:

    root@rok-tools:~/ops/deployments# git commit -m "Configure Object Storage access for Rok"
    

Summary

You have successfully configured Rok so that it uses resources on the object storage service of your cloud provider.

What’s Next

The next step is to grant Rok access to Arrikto’s private container registry, so that it can pull images from it.