Rok v0.11

OpenStore Endpoints

Introduction

The command line arguments for specifying OpenStore Endpoints to connect/listen to have changed in Rok v0.11. This change expresses more clearly the four combinations to choose from, when defining one end of a control connection:

master/connect, master/listen, slave/connect, slave/listen

We now use two command line arguments (one for connect, and the other for listen) and we specify the OpenStore role (master vs slave) explicitly:

-C role=master,http://1.2.3.4:8080
   Your role is OpenStore master, connect to http://1.2.3.4:8080

-C [role=slave,]http://1.2.3.4:9090
   Your role is OpenStore slave, connect to http://1.2.3.4:9090

-L [role=master,]http://1.2.3.4:9090
   Your role is OpenStore master, listen at 1.2.3.4:9090

-L role=slave,http://1.2.3.4:8080
   Your role is OpenStore slave, listen at 1.2.3.4:8080

Upgrade steps

Modify /etc/rok/daemons.conf as well as any environment variables (e.g., OPENSTORE_CONTROLLER) to use the new format OpenStore Endpoints. More specifically, the scheme should change from tcp:// to http:// and a role should be used when appropriate. In addition, the --bind-http command line argument for the External Controller has been replaced by a --controller-listen with slave role.

Rok Gateway

With the Gateway gunicorn process and daemons stopped, run the following command to apply migrations to the Gateway’s store:

$ rok-gw-manage migrate

For more information on the available options, you can use the rok-gw-manage migrate --help as well as the rok-gw-manage showmigrations commands.

Rok Appliances

The upgrade of Rok appliances is performed by replacing the BOOT disk with the new version of the appliance and then running some scripts that perform all the upgrade steps. The upgrade procedure consists of upgrading the master node first and then the remaining nodes, in a rolling upgrade approach.

Upgrade the master node

Find the master node of the cluster:

$ rok-cluster-ctl member-list

Ensure that it will remain the master node of the cluster by setting it as the only master capable node. This step is required to avoid automatic failover to nodes that are using older versions.

$ rok-config set cluster.master_candidates=$(hostname).${ROK_CLUSTERNAME}

Power the machine off, change the BOOT disk with the new version of the appliance, and power on the machine. Login to the node and run the following command that completes all required tasks to upgrade the shared state of the Rok cluster:

$ rok-cluster-upgrade-v0.11.0 upgrade-cluster

After upgrading the cluster state, you need to run the following command to upgrade the node:

$ rok-cluster-upgrade-v0.11.0 upgrade-node

At this point, the master node is successfully upgraded to v0.11.0. You can continue by upgrading the remaining nodes of the cluster.

Upgrade the slave nodes

Upgrade the code of every node by replacing each node’s BOOT disk. Then run the upgrade-node command to upgrade the node:

$ rok-cluster-upgrade-v0.11.0 upgrade-node

Note

Running the upgrade-cluster is not required, since this command must run only on the master node.

Enable etcd SSL authentication and RBAC

After successfully upgrading all nodes to v0.11.0 you must enable the SSL authentication and RBAC in etcd instances by running the following command on the master node of the cluster:

$ rok-cluster-upgrade-v0.11.0 enable-etcd-auth

At this point, the upgrade to v0.11.0 is successfully completed.

Ganeti

This section describes the additional upgrade steps that you need to perform if RokE is running on a Ganeti cluster:

  1. Find the current RokE master and obtain its IP address

  2. Run ./scripts/rok-cluster-upgrade-ganeti for the current master:

    $ env ROKE_PASSWORD=roke-root-pass RAPI_PASSWORD=gnt-rapi-pass \
          python scripts/rok-cluster-upgrade-ganeti \
             --rok-version=0.11~rc1-22-gfc01058-1~6ad14b3~1.gbp6ad14b
             --rapi-host $GANETI_CLUSTER_NAME \
             --rapi-user $GANETI_RAPI_USER \
             $ROKE_MASTER_IP
    

    This command will evacuate the physical host that the RokE master is running, and will re-install the image of the RokE master. At some point the script will ask confirmation about migrating back Ganeti instances to this physical host. Do not continue until the following steps are performed.

  3. Login on the RokE master node and run the upgrade steps that are described in the Upgrade the master node section.

  4. Upgrade the Rok packages on the physical host, so that the Ganeti instances can connect to the upgraded RokE appliance.

    • Upgrade the Debian packages:

      # apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install rok-common rok-base -y
      
    • Update the configuration files to use the proper OpenStore Endpoint:

      # sed -i 's/tcp:/http:/g' /usr/bin/*kvm* /etc/ganeti/rok/rok.conf /etc/default/snf-image
      
  5. At this point, the RokE master node and the physical host are successfully upgraded. You can let the rok-cluster-upgrade-ganeti script from step (2) to continue with the migration of the Ganeti instances.

  6. Continue with rest of appliances and physical hosts, simply by following the Upgrade the slave nodes instead.