Expose Sync Daemon on AWS

This section will walk you through the steps required to make your Rok sync daemon accessible from other Rok clusters and Registries, using an Amazon Classic Load Balancer.

What You’ll Need

Procedure

  1. Go to your GitOps repository, inside your rok-tools management environment:

    root@rok-tools:~# cd ~/ops/deployments

  2. Edit rok/rok-sync/overlays/deploy/kustomization.yaml and uncomment the service-elb patch to enable it:

    patches: - patches/service-elb.yaml

  3. Edit rok/rok-sync/overlays/deploy/patches/service-elb.yaml and set the aws-load-balancer-internal annotation based on the type of Load Balancer you are going to create:

    External Load Balancer

    Internal Load Balancer

    annotations: service.beta.kubernetes.io/aws-load-balancer-internal: "false"
    annotations: service.beta.kubernetes.io/aws-load-balancer-internal: "true"

    Air Gapped

    Use an Internal Load Balancer.

  4. Edit rok/rok-sync/overlays/deploy/patches/service-elb.yaml and configure loadBalancerSourceRanges to allow access from the CIDRs of the Rok Registry and the rest of your Rok clusters. Leave the default value of 0.0.0.0/0 if you want to allow access for everyone:

    spec: loadBalancerSourceRanges: - "0.0.0.0/0"

    Air Gapped

    If you don’t have specific firewalling requirements, leave loadBalancerSourceRanges to the default value. The Load Balancer will be an internal one and as such not reachable outside your VPC.

    Note

    If you need to specify multiple CIDRs, format them as a YAML list. For example:

    loadBalancerSourceRanges: - "1.2.3.4/32" - "5.6.7.8/32"

  5. Commit the changes:

    root@rok-tools:~/ops/deployments# git commit -am "Expose sync daemon on AWS"

  6. Apply the kustomization:

    root@rok-tools:~/ops/deployments# rok-deploy --apply rok/rok-sync/overlays/deploy

Verify

  1. Verify that the Load Balancer Service gets an EXTERNAL-IP:

    root@rok-tools:# kubectl get service -n rok rok-sync NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE rok-sync LoadBalancer 10.32.1.249 a4d794bfa6d7e440facc4398bf96edde-992601283.us-east-1.elb.amazonaws.com 32123:31282/TCP 1m

    Troubleshooting

    The Service object does not get an EXTERNAL-IP.

    1. Describe the service:

      root@rok-tools:# kubectl describe service -n rok rok-sync

    2. If you see an event like the following:

      Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning SyncLoadBalancerFailed 4s service-controller Error syncing load balancer: failed to ensure load balancer: TooManyLoadBalancers: Exceeded quota of account 123456789 status code: 400, request id: 1234abcd-12ab-34cd-56ef-123456abcdef

      it means that you have reached a quota limit in the Classic Load Balancers you can create in your account and you need to increase it.

  2. Verify that the Load Balancer allows connections only from the CIDRs of the Rok Registry and the rest of your Rok clusters:

    root@rok-tools:# kubectl get service -n rok rok-sync -o json | \ > jq -r '.spec.loadBalancerSourceRanges[]' 0.0.0.0/0

  3. Obtain the address of the Load Balancer Service:

    root@rok-tools:# export SYNC_ADDRESS=$(kubectl get service -n rok rok-sync -o json | \ > jq -r '.status.loadBalancer.ingress[].hostname')

  4. Check that you can connect to the sync daemon from your rok-tools container:

    root@rok-tools:# timeout 5 curl -m 3 -v telnet://${SYNC_ADDRESS?}:32123/ * Trying 192.168.86.198... * TCP_NODELAY set * Connected to a4d794bfa6d7e440facc4398bf96edde-992601283.us-east-1.elb.amazonaws.com (192.168.86.198) port 32123 (#0)

    Troubleshooting

    Could not resolve host.

    You typically need to wait for a few minutes after the creation of the Load Balancer service, before you can resolve its address.

    Connection timed out.

    Make sure that the firewall of the Load Balancer service allows connections from the rok-tools container. If you don’t want to allow access, you can skip this step.

Summary

You have successfully exposed the sync daemon of your Rok cluster to other Rok clusters and Registries, using an Amazon Classic Load Balancer.

What’s Next

The next step is to configure the sync daemon to announce this Load Balancer address to other peers.

PreviousNext