This file describes important changes, new features, new dependencies and known issues for Rok releases starting with Rok 0.4.
Version 1.0 (Platinum)¶
- Extend Rok’s provisioning tool for K8s with the –apply mode to avoid questions, skip regeneration of manifests and only apply specified Kustomize packages.
- The home directory of user root inside the rok-tools container is now persisted to retain personal keys and settings across restarts or upgrades.
- Introduce the rok-cluster-admin ClusterRole for Rok cluster administrators on Kubernetes.
- Add support for building reproducible rok-kmod images with rok-do.
- Rok’s provisioning tool for K8s now installs Istio 1.5.7 instead of 1.3.1.
- Improve the style of all links in the Rok UI.
- Display the number of versions in the object list of Rok UI
- Introduce search support for buckets and objects in Rok UI.
- Display and edit bucket descriptions in Rok UI.
- do: Add support for task caching
- Introduce group delete for objects and versions in Rok UI.
- Add rok-do tasks that build bootstrap images for Ubuntu, CentOS and Amazon Linux.
- Allow full access to the Rok API from within the roke and rok-operator pods.
- Make Rok Python bindings compatible with Python 3.
- Add support for Jupyter notebook and dataset snapshot policies in the Rok API.
- Run local Garbage collection more frequently to avoid out of space errors in systems with lots of snapshots.
- Add support for garbage collecting stale resources for rok-csi/rok-lvmd.
- Remove the “escalate” permission from the Rok Operator/Cluster pods.
- Handle transient disconnections in a less intrusive way in Rok UI.
- Introduce a user guide for snapshot and retention policies.
- Fix a bug where the account selector in the Rok UI sometimes displayed the incorrect account
- Do not display a logout button when logging out is not possible in the Rok UI
- Fix a bug where Rok API drivers would use the account instead of the user to perform authorization checks for tasks.
- Fix a bug where the Rok UI would sometimes raise an undefined variable exception after logging in.
- Fix a bug where the Rok UI would ignore the namespace selected via the Kubeflow dashboard selector
- Fix a bug where the Rok UI would not render correctly in a Kubeflow environment.
- Fix a bug where the messages of Kubernetes errors would not be visible in Rok task logs.
- Fix a bug where the Rok client would fail to retrieve the user’s ID when using static authentication.
- Fix a bug where Rok CSI would fail to hydrate a PVC or auto-register a VolumeSnapshot when the Rok API was using AuthService authentication.
- Correctly display the account name instead of the user ID in Rok CLI.
- Fix a bug where users were able to access resources for non-existent Kubernetes namespaces.
- Fix a bug where users were able to access resources created for a no longer existing Kubernetes namespace after a new namespace with that name was created.
- Fix a bug where the Rok file chooser would be displayed under the Kubeflow dashboard, and as a result be unable to select files.
- Fix a bug where Rok CSI would freeze and stop serving requests during the initialization of the Rok client.
- Fix Chrome’s bouncy behaviour when using the copy button in docs.
- Fix a bug where Rok CSI would sometimes use the incorrect Rok API version when restoring a volume from the Rok URL of a group.
- Fixed an issue where Rok deployment would sometimes fail due to S3 temporarily reporting that a bucket does not exist shortly after its creation.
- Remove dependency on lib2to3 when importing rok_common.sysutils.
- Enable log output in the console of Rok and Rok Registry UIs.
- Fixed a bug where the auto-snapshot feature or Rok CSI failed with a 403 Forbidden error because Rok CSI nodes were not allowed to access Rok API.
- Fix a bug where the Rok Composer could deadlock while deleting a fisk when GC was running.
- Reduce the log output of the Rok election daemon while the cluster is idle.
- Preserve query parameters when the namespace changes in Rok UI.
- Fix a bug where the Rok Operator could not properly scale a Rok cluster licensed for N nodes, when the cluster was scaled down to N-1 nodes and then back to N, with the Nth node changing.
- Allow LVMd to recover from an interrupted snapshot.
- Fix a bug where the UI was showing the wrong object count when deleting objects.
- Fix a bug in Rok UI where the navigation to a version failed in Kubeflow environment, the first time.
- Fine-tune the update strategy for rok-disk-manager and rok-kmod DaemonSets so that they can be upgraded in parallel.
Version 0.15.1 (Onyx)¶
(Fri, 17 Jul 2020)
- Make the building of docs depend on version-specific manifests.
Version 0.15 (Onyx)¶
(Thu, 25 Jun 2020)
- Extend Rok Operator to support Rok updates in K8s, by handling mutations of the RokCluster CR
- Support syncing versions with custom chock size.
- Extend the Rok Operator to be able to deploy Rok Registry clusters.
- Create Python 3 wheels for several Rok components, mainly the Rok Gateway client.
- Create separate ConfigMaps for the cluster init Job and the cluster itself, when deploying on Kubernetes.
- Support restricting the deployment of Rok and Rok CSI Pods on specific K8s nodes, based on the new nodeLabels field of the RokCluster spec.
- Add support in the Rok Gateway client for JSON/CSV output.
- Add support in the Rok client to delete multiple fisks.
- Add a dialog to copy files in the Rok UI.
- Enable default connection of all PUs to the external OpenStore controller
- Revamp the icons of the Rok UI.
- Switch lvmd over to using transient DM snapshots to avoid the overhead of storing on-disk the snapshot metadata.
- Update Angular to version 8.2.14
- Simplify the initialization of the Rok Gateway Client.
- Accelerate the loading of Rok’s landing page.
- Extend Rok Operator to produce events on the cluster CR in various phases of its deployment.
- Allow users to deploy Rok Registry with Istio.
- Revamp the policy schedule and login components in the Rok UI.
- Change RDM’s operating method from parse-all then load-or-save, to parse and load-or-save line by line.
- lvmd: Avoid copying deleted files when taking a volume snapshot.
- Extend rok-csi to support the mountOptions field of a StorageClass.
- Support setting arbitrary device attributes for CSI volumes (e.g., readahead)
- Introduce a C Redis client library.
- Disengage Rok Operator from managing Rok’s Storage and VolumeSnapshot classes and delegate this task to kustomize.
- Support auto-recover from latest snapshot for rok-csi volumes for which the corresponding Kubernetes node has been removed.
- Add bucket’s icon in Rok UI’s breadcrumb trail.
- Extend the nexus thread so that it can defer the transmission of a CCB if the underlying transport has run out of resources.
- Introduce the shared memory transport. The shared memory transport is a SCSI transport that allows PUs to share common memory space and thus exchanging SCSI commands with less data copies. It is expected to offer better I/O performance in comparison to the TCP and iSCSI transports.
- Extend the Controller’s Static Policy to allow choosing transport for a nexus. The transport for a nexus can be either TCP or Shared Memory.
- Extend the Rok and Rok Registry cluster CRDs with the status subresource, to monitor their health/state at any time.
- Fix a bug where fisks with names between 126 and 128 characters failed to be created.
- Migrate libmap’s epoch cache from etcd to Redis.
- Introduce preliminary support for replicated volumes in lvmd.
- Extend rok-init’s installation modes with upgrade to coordinate Rok and Rok Registry cluster upgrades.
- Extends Rok Operator’s business logic to carry out software upgrades of Rok and Rok Registry clusters on Kubernetes.
- Implement a badge to display files in a uniform manner in the Rok UI.
- Users can now pass Rok configuration variables as objects, instead of strings, in the Rok and Rok Registry CRs.
- Allow running rok-do tasks over SSH.
- Add support for IAM Roles for Service Accounts to the Rok S3 daemon.
- Extended the Rok AWS library with scripts to purge S3 buckets, authorize a Rok EKS installation to access S3 and attach EBS volumes to EKS cluster nodes.
- Support upgrading kernel modules with rok-kmod.
- Introduce mechanism to mark sections of Sphinx documents as ignored for specific doc builds e.g., internal.
- Prefer custom built modules in case the kernel already supports them.
- Support the
file:argument prefix in Rok C daemons, to allow passing an argument value from a file.
- Use the common AWS_* environment variables to provide credentials to the Rok S3 daemon, and drop support for the old ROK_S3_* variables.
- Make rok-probes library Python3 compatible.
- Make rok-kmod Python3 compatible.
- Employ rok-probed to securely probe Rok and Rok Registry external services in Kubernetes initContainers.
- Support building reproducible rok-kmod images locally.
- Allow fine-grained control over the sanitization of logs, which enables accurately logging multi-line messages or messages that include otherwise unsafe characters if required.
- Extend Rok clusters with a UUID, that can be used to identify resources owned by the cluster.
- Perform fine-grained tests regarding the S3 storage of a Rok cluster during its initialization phase, to catch and report configuration errors as fast as possible.
- Add initial version of
rok-deploy, an interactive CLI tool to declaratively configure and install Rok on Kubernetes.
- Automatically allow access to Rok API resources to users that have access to Kubeflow resources in the same Kubernetes namespace.
- Enable the Rok API to run behind Istio using AuthService for authentication.
- Allow switching the displayed namespace in the Rok UI.
- Allow selecting the account displayed in the Rok UI via the Kubeflow namespace selector.
- Configure Sphinx to fail if warnings are emitted when building Rok docs.
- Extend Rok Operator’s business logic to conditionally update the cluster configuration on etcd based on the spec.configVars field of the cluster CR
- Do not store the Rok Cluster’s configuration on etcd, but instead only store them locally in the image.
- Add instruction to upgrade a Rok Cluster on Kubernetes.
- Extend the Rok API to allow its clients to interact with any desired account, instead of necessarily the one that matches the UUID of their user.
- Improve messaging in UI’s network errors.
- Improve the Rok Thrower’s security, by sanitizing certain messages of other peers.
- Fix a bug where Target PUs would access fields of a CDB before checking whether its length matches the expected length (inferred from the opcode).
- Speed up Rok cluster initialization when deploying Rok on kubernetes
- Fix race between dm-clone device removal and ongoing snapshot in LVMd.
- Remove size restrictions for LDAP IDs in Fort
- Check if a user’s LDAP attributes are UTF-8 encoded and warn appropriately
- Manually wipe filesystem signatures on new LVs in LVMd to workaround lvcreate hangs on Stretch caused by lvcreate asking for user confirmation before wiping any FS signatures found.
- Fix an issue where tooltips would sometimes appear outside the edge of the screen in the Rok UI.
- Fix a bug in the files list page of the Rok UI where some files were not being displayed correctly if the bucket contained groups and standalone files with the same name.
- Improve termination of Rok Operator on Kubernetes by properly handling the SIGTERM signal.
- Fix keeping logs in case a cron job fails (like rok-cluster-gc).
- Support retries in libredis.
- Fix creating RAID with 1 drive using RDM.
- Fix object protection, when multiple reclassing is involved.
- Fixed a bug where Rok cluster members would sometimes fail to detect the master has changed after the master reboots (perhaps with a changed IP), ending up with stale information about the master.
- Fix dm-clone bug which could lead to discarding the wrong blocks, causing data corruption.
- Add missing oveflow check for total number of regions in dm-clone.
- Add missing casts to dm-clone to prevent overflows and data corruption.
- Fix dm-clone’s status output in case the total number of regions is 2^31.
- Fix a bug where the Rok client failed to output errors in Python 3.
- Fix a bug where the Rok S3 daemon would retry retrieving security credentials using all supported methods instead of only the one that previously succeeded.
- Fix a bug where the s3 credentials provided via a Kubernetes secret would be stored unencrypted in the Cluster’s etcd.
- Fix a bug where the Rok S3 daemon would attempt to retrieve credentials from EC2 instance metadata if assuming a role with AWS Web Identity was requested but failed.
- Fix a bug where the Rok S3 daemon could not start if it had failed to create its S3 buckets in a previous execution.
- Fix a bug where requests towards the Rok API could fail when Kubernetes authorization was enabled, due to a 401 Unauthorized error when submitting the SubjectAccessReview.
- Always invalidate the cached cluster configuration manager that Rok Operator uses to interact with the cluster configuration stored in etcd.
- Fix a bug where the Rok S3 daemon would not log what request is being performed when performing retries.
- Fix a bug where the Rok cluster would not go into an error state if the security credentials of the Rok S3 daemon became invalid.
Version 0.14.1 (Nephrite)¶
(Released Fri, 19 Jun 2020)
- Enable rok-csi to identify the case where a Kubernetes node has been removed, and allow the volume deletion to succeed.
- Add readiness and liveness probes in RDM.
- Fix a bug where RDM would operate on all available devices, if devices passed to get_disks() did not exist.
- Fix a bug where the cluster configuration would get messed up when multiple members were joining the Rok cluster at the same time.
- Fix bug in the progress reporting of LVMd where some steps appeared as stuck.
- Handle Kubernetes node removals/renames, by breaking the corresponding Rok member locks.
- Fix a bug in the Rok Gateway Kubernetes driver causing StatefulSet presentations to fail.
- Fix rok-liod to work with ec2-utils.
- Fix a bug in the transport library which could result in a Nexus failing to
handle a request with
- Fix a bug where the Rok S3 daemon would sometimes fail with an InvalidToken error due to incorrect parsing of the session token from the environment.
Version 0.14 (Nephrite)¶
(Released Mon, 16 Dec 2019)
- Use etcd v3 as the default store of the Rok Gateway.
- dm-clone: Hydrate regions in batches to achieve better hydration throughput.
- Use latest Minikube v1.2.0 for MiniKF.
- Make rok-csi topology aware
- Support Kubernetes v1.14
- Escape non-ASCII, non-printable characters from logs. This should provide protection against terminal manipulation and homoglyph attacks by malicious user input.
- Support using expressions as values for cluster configuration variables.
- Implement a new design in the bucket info page of the Rok UI.
- Support deploying Rok with LDAP integration.
- Allow the K8s Rok operator to scale Rok clusters up and down.
- Support glob pattern for secrets in rok-csi
- Enable user impersonation in Rok Gateway Kubernetes service drivers.
- Support Debian Stretch.
- Enable user to configure HTTP for Rok Gateway on RokE
- LVMd now automatically removes the dm-clone device when hydration finishes
- Extend SCSI
INQUIRYcommand to support retrieving composer parameters.
- Record access events of Fort tokens against the Fort API.
- Support resizing LVMd/CSI volumes restored from snapshots
- Enable rok-s3d to detect invalid bucket prefix
- Support adjusting dm-clone’s hydration parameters in LVMd.
- Use etcd v3 as the store for libmap’s epochs.
- Add support for configuring the snapshot chunk size in LVMd.
- Improve LVMd performance for snapshots on EBS backed devices.
- Add support for configuring the size of the COW device for thick snapshots.
- Allow rok-csi to rewrite the URLs provided by the user, so that they work even if the user visits Rok behind a proxy.
- Add RAID support in RDM.
- Support exposing extra block device attributes (e.g., model or wwid) in RDM.
- The opcode for the
ROK_RESIZE_FISKSCSI command was changed. Since the resize command is currently used only by rok-client and rokfs, this should not cause any problems to existing deployments.
- The format of the
ROK_FISK_CREATESCSI command was changed
- The format of the
ROK_FISK_COMPOSESCSI command was changed
- Use direct I/O when accessing block devices with LVMd
- Fix a bug where hasherd would erroneously delete a not-completed task
- Gracefully handle the deletion of the last version of a file in the Rok UI
- Fix a bug where the etcd v2 emulation client would fail to create the client prefix
- Fix a bug where rok-csi failed to provision volumes where the provided size was not a multiple of blksize (512 bytes)
- Fix a bug where lvmd kept nodelocal references to a volume’s internal snapshot, that might get lost as nodelocal storage is ephemeral.
- Fix a bug where lvmd failed to update dm-clone’s origin device.
- Make logs from C daemons visible to Kubernetes
- Fix a bug where rok-operator would remove and then re-add a cluster member during after a node reboot
- Fix a bug where the ML/K8s Rok GW driver would sometimes fail to complete the snapshot operation
- Fix lvmd bug where the size of volume’s underlying LV and the size of its internal snapshot could be different, because LVM automatically rounds up the size of LVs to be a multiple of its physical extent size.
- Fix a race which could crash tcmu-handler.
- Fix a race where PVs provisioned by local-static-provisioner could be used before the underlying storage has been mounted.
- Fix a race where RDM would not call udevadm settle after calling parted, which opens devices for write and can cause partitions to briefly disappear.
- Fix a bug in RDM where loading a Partition when the partition table is on a device with a symlink as path failed.
- Fix a bug which could cause stale fisks not being deleted when using rok-csi.
- Fix a race which could cause pui_map_fisk_get_plu() to fail with EAGAIN
- Workaround an issue where the kernel fails to allocate a SCSI loopback device, causing liod to hang.
- Fix a bug which resulted in the Rok Gateway driver for K8s showing an empty YAML during presentation.
- Increase etcd’s maximum supported size of incoming and outgoing messages to workaround a bug where requests to Rok API buckets exceed the default limits.
- Increase csi-snapshotter’s timeout to workaround the fact that Rok cannot take concurrent snapshots of the same source volume.
- Fix a bug in csi-snapshotter sidecar which could cause a snapshot operation to fail with VolumeSnapshotContent is missing error.
- Fix the way Rok Gateway calculates the object stats when object groups are involved.
Version 0.13 (Marble)¶
(Released Mon, 01 Jul 2019)
- Introduce a thread pool in C.
- Redirect the user to the requested page after a successful login in the Gateway UI
- Support deploying Rok and Rok registry on Kubernetes with helm.
- Run RokE as an non-privileged container.
- Add support for account roles in Fort.
- Introduce Fort services, i.e., applications that can access privileged Fort endpoints, such as the Rok Registry.
- Add support for Ahead-of-Time (AoT) compilation in the Rok Registry UI and Rok Gateway UI.
- Rewrite Rok Operator in Python.
- The embedded Controller now honors the ROK_PHYSICAL_HOST environment variable and reports it upstream
- Introduce Rok licensing mechanism that constrains Rok cluster-size.
- Introduce one-time secrets in Fort, which can be used for various tasks, such as email confirmations and password resets.
- Add support in the Fort API for confirming an account’s email.
- Reduce poll interval for external Controller
- Add support in the Fort API for resetting a user’s password.
- Add support for email confirmations to the Rok Registry.
- Add support for resetting a user’s password in the Rok Registry API.
- Add support for VolumeSnapshot CRs on rok-csi.
- Do not allow multiple Fort confirmed accounts with the same email address.
- Require users to confirm their email addresses before allowing them full access to the Fort API.
- Use HTTP liveness and readiness probes that query a rok-probed server in the Rok Kubernetes pod.
- Improve discard performance for dm-clone.
- Disable by default the endpoint that unconditionally lists the swarm IDs that the Rok Tracker tracks. Also, add a setting in the Rok Tracker that allows enabling it.
- Add reCAPTCHA protection to some Rok Registry API endpoints (email confirmations and password resets).
- Improve dm-clone’s overall performance.
- Add etcd-specific SSL arguments and environment variables, namely
- Add support for multiple environment variables to Rok arguments in C.
- Add breadcrumbs to the Rok UI.
- Make prefix for S3 bucket names for rok-s3d configurable by the cluster configuration mechanism.
- Add script to gather logs in MiniKF.
rok-conftool has been renamed to
rok-config-*tools have been renamed to
rok-ssh-root-passwordtool has been removed. Users can use the
cluster.ssh.root_password_logincluster configuration variable instead.
rok-dlmtool has been refactored. Users should use the
rok-dlm client-breakcommand to break both DLM locks and clients. The
--skip-unknownarguments have been removed in favor of the
--forcearguments, whose semantics have been changed for this reason. The old
--forcebehavior is now achieved by the
- Fix typo that caused
rok-election-ctl member-listto fail.
Version 0.13-rc1 (Marble)¶
(Released Fri, 07 Jun 2019)
- Fix dm-clone compilation error when building dm-clone for Linux kernel versions >= 4.17.0.
- Add copyright notices to all of our .yaml/.yml files, which were erroneously omitted.
- Fix refcount leak for LU objects
- Break stale rok-csi locks
- Fix a bug that would erroneously make successful commands appear as failed, if they produced a lot of output or if the server was under load.
- Fix a buffer overflow bug in libdlm
- Improve the algorithm for detecting stale PID files
- Fix a bug where a DLM client could be deleted despite holding locks
- Use DT_RPATH instead DT_RUNPATH to work-around Debian bug #859732
- Fix a bug where rok-csi would not terminate when initialization failed
- Fixed a bug where tasks could fail to cleanup their temporary state after a restart of the Rok Gateway Task Daemon.
- Fix various memory errors discovered by AddressSanitizer
- Fix an issue where the Rok thrower was not accessible from the NodePort service.
- Fix the alignment of empty list messages in the Rok UI.
Version 0.12 (Lignite)¶
(Released Mon, 18 Mar 2019)
- Update the default values of some performance-related thrower options, in order to make the thrower faster by default.
- Support providing an existing CA certificate when creating a RokE cluster, rather than always creating a self-signed internal CA.
- Support deploying a RokE cluster which uses an external etcd.
- Port rok-csi to CSI spec v0.3.0
- Add support for Kubernetes >= v1.10, and remove the need for custom kubelet
- Extend the Rok Python etcd v3 client with the ability to emulate the etcd v2 API, to allow the easy transition of Rok components to etcd v3.
- Support transactions in the Rok Python etcd v3 client
- Allow users to add a new provider from the Rok UI.
- Add a new page in the Rok UI to display task details.
- Introduce settings section in the Gateway UI wich contains available providers and user tokens.
- Introduce a writeback cache for map updates in rok-composerd.
- Support running hooks when a RokE member is promoted to master.
- Allow rok-gc to run in parallel on all nodes when running in nodelocal mode.
- Make RokE appliances create a cluster-wide SSH keypair for the root user.
- Support setting the root password in RokE appliances by providing the hash of the password in the preseed file.
- Mak RokE disk setup script specify mounts instead of filesystem labels.
- [EXPERIMENTAL] Introduce hierarchical maps. Only basic operations and garbage collection are currently implemented.
- Display a footer in the Gateway UI with copyright information and Rok’s build ID.
- Allow parsing URLs of Rok Gateway object versions via the Rok Gateway API and client.
- Support gevent in the Rok Python etcd3 client.
- Strongly associate Rok Gateway buckets with Indexers. Previously, only the Indexer link was stored in the bucket.
- Introduce a management command in the Rok Gateway to allow the garbage collection of old tasks.
- Fix various memory leaks throughout the code base.
- Fix syncing of group versions with same members.
- Fix bulk deletion of Gateway versions.
Version 0.12-rc1 (Lignite)¶
(Released Thu, 01 Nov 2018)
- Fix a bug where secret service parameters were ignored when providing suggestions for a policy update.
- Fix handling of nodelocal mode in rok-composer-tool
- Support unregistering an Indexer from the Rok Gateway while the thrower is running.
- Fix a bug in the Rok Gateway where secret service parameters were ignored when providing suggestions for a policy update.
- Fix a bug in Gateway policies where policy run tasks would display outdated parameters if the policy was updated.
- Correctly display the filters, register name, retention rules and backup action parameters of policy tasks in the Gateway client.
- Fix a bug in the Rok Gateway where suggestions could not be provided for
variables in the
infonamespace, e.g., object and version name, if the service driver defined a parameter with the same name.
- Fix a bug where the Rok Gateway task daemon could leave behind stale tasks in the pending or running state if the connection to the Gateway store was lost during the task’s execution.
- Fix a bug in the Rok Gateway where the subtasks of failed or interrupted tasks would not be canceled even though the parent task had failed.
- Fix a bug where security sensitive fields could accidentally be logged by the Rok Gateway or by Rok Gateway service drivers.
- Fix a bug where the thrower did not update the error reason of a bucket, when a new error, with the same error code as the previous one but a different reason, was encountered.
- Make the Rok S3 daemon compatible with the IBM S3 service, by introducing the –assume-no-versioning command line option.
- Fix a bug in the S3 daemon where if a protocol was provided as part of the S3 endpoint it would be included twice in API calls towards the S3 service.
- Fix a bug where the Rok S3 daemon would not exit if its S3 endpoint and credentials were incorrect, causing the Rok composer to fail when attempting to create OSD partitions.
- Fix a bug where LIOd manager failed list_volumes() request if a loop device disappeared, because someone else un-looped the device.
Version 0.11.1 (Kryptonite)¶
(Released Fri, 19 Oct 2018)
- Install proper kubernetes version in Appliances
- Fix templating error in Gateway configuration file in Appliances
- Minor fixes regarding management tools for Rok clusters on AWS
Version 0.11 (Kryptonite)¶
(Released Mon, 15 Oct 2018)
- Rework tcmu-handler to process requests in parallel
- Support thick volume pools in lvmd
- Add support for customizing the appliance disk setup procedure using preseed file
- Use internal jessie-backports repo for all non jessie dependencies
- Support daemon reload
- Support mail notifications in appliances
- Use systemctl for managing Rok daemons
- Improve the efficiency of subtasks
- Support tasks with service-defined actions
- Support nested subtasks in the Gateway API and UI
- Revamp the Kubernetes Gateway driver to use subtasks
- Revamp task, event and policy icons in the Gateway UI
- Support subscribing existing bucket in the Gateway UI
- Invite or remove bucket collaborators in the Indexer UI
- Update collaborator’s permissions in the Indexer UI
- Display inactive Rok’s in the Indexer UI
- Implement new design for the OAuth page in the Indexer UI
- Introduce a Python Rok client for etcd v3
- Revamp the format of version retention policy rules
- Allow version retention policies to delete current object versions
- Display version retention information for all Gateway objects
- Support creating object groups via the Gateway client
- Add rok-cluster-aws script to manage a Rok cluster on AWS
- Allow tuning rok-gc with cluster configuration
- Discard zero CA chocks from fisks
- Rename rok-clusterd to rok-electiond
- Improve the execution logic of external commands
- Add support Subject Alternative Names (SAN) in X.509 certificates
- Add support for templated cluster configuration variables
- Support per-host cluster configuration variables
- Make rok-conf use the cluster configuration mechanism to configure the appliance
- Secure all etcd communications using SSL in appliances
- Secure the cluster join procedure using authentication
- Add widgets for multiline input
- Ensure users can SSH in appliances early to continue the appliance configuration with copy/paste ability
- Introduce the rok-cluster tool to manage the Rok cluster
- Introduce various config variables that help tweaking Rok clusters
- Fix various configuration rendering issues in appliances
- Fix backwards compatibility issue with the OpenStore controller
- Fix a bug that prevented the thrower from recovering a bucket from an error
- Fix a thrower issue, where it would not show the connected peers if the tracker was down
- Fix a thrower bug that randomly prevented two peers from connecting, if they were registered with two or more Indexers with the same common name for their CA
- Fix cluster GC to work with SSL
- Make etcd clients handle transient etcd errors (SSL connect)
Version 0.11-rc1 (Kryptonite)¶
(Released Tue, 25 Sep 2018)
- Fix dm-clone bug wrt overwrite BIOs, which could lead to data corruption.
- Include version hash in all JS and CSS artifacts into fix unwanted cache effects
- Fix false requests cancellation if responses are slower than the polling interval in the Gateway UI
- Fix the retry logic of the Gateway client and prevent timeouts in object uploads
- Fix a potential security issue that involved mutable values in Python function definitions
- The format of Rok-specific SCSI management commands has changed, so PUs of this version cannot connect with PUs of previous versions
- The format of OpenStore Controller endpoints has changed (see upgrade notes)
Version 0.10.3 (Jade)¶
(Released Tue, 17 Jul 2018)
- Adjust tracker announcement frequency in the thrower
- Visually distinguish inactive Roks in the Indexer UI
- Make rok-csi use heartbeat for its DLM locks
- Track progress of volume/snapshot creation in lvmd
- Report progress in Kubernetes Gateway driver
- Improve performance of Copy-on-write on rok-composerd
- Make embedded controller (libctrl) retry failed connections
- Fix rok-composerd to correctly handle namespace aliasing
- Fix a bug in the task daemon where policy tasks would sometimes not start at their scheduled timestamp
- Fix a GC bug that could result in data corruption
- Retry operations when S3 throws unknown errors
- Fix logrotate script that did not rotate all Rok daemons
- Fix a thrower bug that would result in rapidly opening/closing HTTP connections to the tracker
- Fix a thrower issue with regards to unfair job scheduling
- Fix a thrower bug where important PPSPP messages would get reordered, leading to connection issues
- Fix a bug where retention policies would not work as expected for versions created by the Rok Thrower
- Fix a number of UI problems in smaller screens
- Accelerate stats calculation for versions
- Handle transient Indexer related errors and update bucket state accordingly
- Fix CEF shutdown when running workstation with VBox
- Fix VMware Fusion port forwarding in workstation
- Display bucket errors properly in UI
Version 0.10.2 (Jade)¶
(Released Fri, 04 May 2018)
- Handle user provided CA certificates, SSH keys, APT keys in Appliances
- Use sparse disks for RokW helper VM
- Enable SSH on RokW
Version 0.10.1 (Jade)¶
(Released Thu, 19 Apr 2018)
- Support suggestions in the Rok Gateway client
- Add dm-messages to disable/enable hydration in dm-clone
- Support creation of dm-clone devices with hydration disabled
- Support RokE deployment on Oracle Cloud Infrastructure with automatic creation/expansion/upgrade of Rok cluster
- Introduce port-forwarding for RokW helper VM to enable thrower communication between Rok Workstations
- Use DKMS for our kernel modules
- Create rok-csi docker image using Debian packages
- Fix a race condition in Rok Gateway that could cause a policy task to be canceled just after it starts execution
- Fix bug in lvmd’s snapshot workflow that could cause the creation of corrupted volume snapshots
- Fix GC in RokE clusters
Version 0.10 (Jade)¶
(Released Tue, 27 Mar 2018)
- Introduce the Rok S3 daemon
- Support named OSD partitions
- Support storing metadata in persistent storage for dm-clone
- Support Rok snapshots in lvmd
- Support handling concurrent requests in lvmd
- Use Gunicorn as lvmd’s HTTP server
- Support ACL in Indexer buckets
- Implement CSI plugin for Rok
- Implement Rok GW driver to register/present Kubernetes StatefulSets
- Introduce clustered config in Rok Appliances
- Support Rok Workstation on Mac OS
- Use CEF browser for Gateway UI in Rok Workstation
- Introduce IFC library to support non-strict consistency object stores (S3)
- Support discard requests in dm-clone
- Support object groups in the Gateway
- Discard unallocated and zero chocks when creating a volume from a fisk in lvmd
- Add “Local Filesystem + Amazon S3” Rok storage backend in RokE appliances
- Support making RokE appliances part of an existing Kubernetes cluster
- Master failover in a RokE cluster is not working when running in node-local mode or when RokE is deployed on Amazon Web Services (AWS).
- The CSI Plugin for Rok (rok-csi) will fail to unpublish a volume that is being snapshotted at the same time, due to a bug in Kubernetes error handling.
- The LVMd is not upgradable from version 0.9 without deleting its database, since the etcd directory format has changed from the previous version.
Version 0.9 (Iron)¶
(Released Wed, 14 Feb 2018)
- Implement resizing fisks
- Support regular expressions for zoning on auto-exported LUs
- Implement Docker LVM Volume Plugin with support for changed block tracking
- Implement Rok GW driver to support smart backups of Docker Volumes
- Implement dm-clone, a Linux kernel module for live cloning of block devices
- Support maintenance mode in Rok appliances
- Use VASA events to discover new PEs
- The effect of resizing a fisk that is used by a VM will become visible only after restarting the VM.
- The VASA OpenStore policy does not support setups where the same ProtocolEndpoint is being exported by more than one PUs.
- If local affinity is not enforced in controller policies and an unclean shutdown of an Appliance takes place then stale locks should be manually broken otherwise the I/O of the VMs that get re-connected will be frozen.
Version 0.9-rc3 (Iron)¶
(Released Mon, 29 Jan 2018)
- Fix Rok Workstation issues with Windows Samba caching
- Cleanup SSH keys from appliance images and generate new during initial config
- Stop using dummy Django secrets
- Prevent master failover ping-pongs in case the new elected master fails to setup everything properly
- Fix sorting issues in UI
Version 0.9-rc2 (Iron)¶
(Released Thu, 11 Jan 2018)
- Fix RokW issue with VMDK snapshot chain in VMware Fusion
- Fix compatibility issue with Django 1.7
- Fix controller compatibility with PUs of older versions
- Enable static policy for the Root Controller in RokE appliances
- Fix RokE appliance to export multiple protocol endpoints
- Verify that all protocol endpoints are visible per ESXi host when connecting Rok
- Allow SSH password authentication for root in RokE appliances
- Fix locking when xstat()ing a map
- Allow the user to specify the physical host on which the RokE appliance is running
Version 0.9-rc1 (Iron)¶
(Released Wed, 20 Dec 2017)
- Restarting the Composer no longer needs a rescan in the ESXi host, and will not cause running VMs on the Rok VASA datastore to become invalid.
Version 0.8.1 (Hematite)¶
(Released Fri, 01 Dec 2017)
- Fix upgrade notes to run the required database migrations
- Fix controller issues when listening to 0.0.0.0
- Fix StaticPolicy to work with PUs running on older Rok versions
Version 0.8 (Hematite)¶
(Released Thu, 30 Nov 2017)
- Add scheduling to Gateway policies
- Implement version retention policies in the Gateway
- Enable Rok Controller to work in a clustered environment
- Add support for more than one Protocol Endpoints to the Rok VASA provider
- Implement Virtual Machine and Virtual Disk registration on Rok Workstation
- Support Gateway migrations
- Handle temporary etcd failures in Rok base
- Add support for appliance clustering with Rok Enterprise
- Address Samba client caching in Rok Workstation
- Use proper image path in Rok Workstation on Linux
- Add missing desktop entry on Linux
- Fix VirtualBox thread handling in Rok Workstation
- Make scheduling resilient to restarts in Gateway
- Handle early timeouts in policyd
Version 0.8-rc1 (Hematite)¶
(Released Mon, 27 Nov 2017)
- Fix VirtualBox DNS bug on Windows
- Fix liod crashing on signal reception (during logrotate)
Version 0.7.2 (Granite)¶
(Released Mon, 27 Nov 2017)
- Fix a bug in Thrower stats
- Fix Gateway client password authentication against Fort
- Make Gateway management commands compatible with Django 1.7
- Fix registration policy of Synnefo machines
Version 0.7.1 (Granite)¶
(Released Mon, 13 Nov 2017)
- Fix librok_trpt to handle real-time signals
- Fix description of intermediate snapshots in Synnefo Gateway driver
- Fix librok_sg to retry only transient request failures
Version 0.7 (Granite)¶
(Released Tue, 07 Nov 2017)
- Encrypt password hashes with secure key in Fort
- Support service tasks in Gateway Client
- Report login errors in UI
- Fix handling of chunked responses in Gateway Client
- Prevent unmounting the filesystem used by Filed
- Support Fort authentication in Gateway Client
- Support Keystone v2.0 authentication in Gateway Client
- Automatically refresh expiring tokens in Gateway Client
- Use PostgreSQL instead of SQLite as the proposed database
- Support mounting RokFS via a systemd mount unit
- Introduce subtasks for service drivers
- Run policies as tasks
- Support presenting VMDK in RokW
- Integrate UI with Fort
- Introduce management tool for VASA
- Update pagination logic in Django Apps
- Switch to Angular 4.3.6 in UI
- Introduce management tool for Gateway
- Introduce command to verify state of maps
- Split docs into public and internal
- Move common UI to separate/reusable Angular module
- Support GC when Ceph is full
- Add tool to integrate Rok platform with vSphere
- Generate MSI for RokW using WiX toolset
- Use multiple/separate disks in Rok Appliances
- Add support for overlayfs in Appliances
- Add initramfs scripts to grow partitions in Appliances
- Introduce migrations for RokVP
- Integrate Fort with LDAP
- Support fast recovery from a closed connection in Controller
- Support per-user auto-configuration during first run in RokW
- Configure Rok VASA Provider with Rok Enterprise
- Disallow passwords shorter than 8 characters in Fort
- Create and configure Rok Indexer appliance
- Use tasks instead of synchronous service events in Gateway
- Report peer connection status in stats
- Extend Virtualbox driver to register VMs and register/present VMDKs
- Hash the stored token in the Indexer
- Obtain the Rok VASA Provider Storage Container UUID automatically
- Implement a virtual scroll component to allow dynamic rendering of list elements
- Limit outstanding requests per Thrower peer to one
- Add application level keepalives in Thrower
- Add sorting arrows on table headers in UI
- Implement dummy Task Management on iSCSI
- Extend OpenStack Service Driver to be able to register Nova instances
- Various user-visible improvements in Rok Appliances
- Split etcd cluster in Rok Appliances
- Enhance appliances with configuration management tools
- OpenStack support works with multi-storage backends
- VASA Provider
- Upgrading from version 0.6 to 0.7 is supported using the provided migration
script, but with the following limitations:
- Any Virtual Machines that have one or more Virtual Disks located in
the Rok VVol datastore and these Virtual Disk VMDK files do not
reside inside the VM folder (along with the
.vmxfile), will fail to power on after the upgrade. The administrator should manually edit the above problematic VM files and replace the old Storage Container UUID with the new one.
- Any Virtual Machines that have one or more Virtual Disks located in the Rok VVol datastore and these Virtual Disk VMDK files do not reside inside the VM folder (along with the
- Upgrading from version 0.6 to 0.7 is supported using the provided migration script, but with the following limitations:
(Released Fri, 27 Oct 2017)
- Fix Thrower handling of forgotten chunks
- Fix Fort DB migration to handle existing user passwords
- Fix Controller to continue applying policies even after one fails
- Fix image handling in rok-data MSI upon upgrades
- OpenStack support does not require a patched version of nova any more
Version 0.7-rc1 (Granite)¶
(Released Thu, 05 Oct 2017)
- Suggest only the accessible datastores in VMware driver
- Fix concurrent presentations in VMware driver
- Don’t use smart copy if VM is on Rok when registering a VM in VMware driver
- Correctly handle timed out requests in Thrower
- Do not require passwords for any ESXi hosts in VMware VM driver
Version 0.6.2 (Flint)¶
(Released Thu, 05 Oct 2017)
- Fix a race in Thrower that resulted in high memory consumption
Version 0.6 (Flint)¶
(Released Mon, 04 Sep 2017)
- Introduce Fort user system, make Indexer depend on it and integrate Gateway with it
- Allow administrators to limit the number of parallel compositions in Rok Thrower
- Add support for multiple trackers and automatic discovery in Rok Thrower
- Show more peer info in the Gateway UI stats (name, country, connection status)
- Download chunks only once in Rok Thrower
- Reduce the memory usage of stream swarms in Rok Thrower
- Add support for LZ4 compression in Rok Thrower
- Implement signature verification of bucket updates in Rok Thrower
- Handle garbage collection while Rok Thrower runs
- Enhance Gateway client to support service actions
- Introduce Rok FUSE
- Improve SSL certificate handling in VASA provider
- Split Gateway API and backend configuration
- Support bucket/object names including Unicode characters
- Support booting instances on AWS
- Introduce Rok Workstation
- Support booting instances on Virtualbox/VMware Workstation
- Support TLS/SSL everywhere
- Support etcd authentication
- Introduce OpenStore VASA policy
- Make Rok transport more robust
- Rok VASA Provider spawns its own nginx instance
- Introduce map version v2 and support map migrations
- Add support for breaking DLM locks automatically
- Support logrotate in Python daemon, C daemons and gunicorn Apps
- Support accurate stats in the Gateway [via new “Stats” daemon]
- Support asynchronous tasks in the Gateway [via new “Task” daemon]
- Add suggestions in Gateway drivers
- Introduce suggested buckets in the Gateway
- Sign and verify swarms in Rok Thrower
- Support sessions in Rok VASA Provider
- Support on-demand auto-exported LUs and initiator zoning
- Introduce OpenStore iSCSI policy
- Ship Tracker within Indexer
- Use a more user friendly error handling in UI
- Let API serve UI settings
- Implement sign-up in Indexer
- Use deterministic PU names
- Improve performance of policy runs
- Extend the Gateway VMware drivers to register/present both VMDKs and VMs
- Implement bucket deletion in Indexer UI
- Use atomic requests in Fort and Indexer
- Add various management commands in Fort
- Implement password update in Fort
- Show Rok installations in Indexer UI
- Make OS suggestions case insensitive in Gateway Services
- The composer has a number of concurrency issues which may cause data loss when a fisk is being accessed by more than one composerd instances. This will most likely occur during VM live migrations.
- client: The Gateway client does not support asynchronous presentations and registrations via tasks
- VMware: Cannot take a quiesced snapshot using VSS of a Windows VM that is backed by a disk on the Rok VVol datastore
- User creation is not supported in Rok Gateway
- When user logs out, some requests might be sent after the deletion of the token resulting to error popups in the UI
- Failures due to composer errors or GC or not tested
- When the thrower restarts, it may not create a swarm for a bucket if it can’t connect to the Indexer
- Port checking is done every second, which adds a significant load to the tracker
- VMware: The Rok VASA provider does not report metrics properly, including Storage Container size and free space
- VMware: A VM snapshot operation may fail if the Rok backend fails to complete the fisk snapshot operation in time
- VMware: The Rok VASA Provider does not support SPBM (Storage Policy Based Management) and its derivatives, such as Storage Profiles, Capabilities and compliance checks
- VMware: The Rok VASA Provider supports only one Protocol Endpoint per Storage Container
- OpenStack: A patched version of Nova and libvirt is required
- The iSCSI policy does not have a state of its own, hence fisks exported over iSCSI will not persist over Composer restarts
- VASA Provider
- The RokVP is not upgradable from version 0.5 without deleting its database, since the etcd directory format has changed from the previous version.
Version 0.6-rc4 (Flint)¶
(Released Wed, 09 Aug 2017)
- Handle IO transport failures in SG
- Re-discover LUs on a re-established nexus
- Garbage collect stale ports in Controller’s context
- Fix a use-after-free in iSCSI when nexus closes before CCB completes
- Fix tasks polling in UI
- Kill Thrower greenlets in a pool properly
- Fix race in rokfs service unit
- Filter expired OAuth tokens in Gateway
Version 0.6-rc3 (Flint)¶
(Released Tue, 18 Jul 2017)
- No need to define ESXi initiators in Rok VASA Provider config
- Avoid executing tasks more than once in Gateway
- Handle errors on configuration parsing in RokW
- Report failed logins in Gateway UI
- Various fixes in VMware services
- Correctly negotiate metadata need info between throwers
- Do not fail when the thrower composes a version and the underlying fisk exists
Version 0.6-rc2 (Flint)¶
(Released Mon, 10 Jul 2017)
- Fix bug for Rok folder path on Windows in VMware Workstation service driver
- Fix typos in AWS service driver
- Various improvements in Rok VASA Provider (configuration files, snapshot handling)
- Various fixes in services configuration files
- Fix Rok GC error handling
- Handle old buckets with no tasks properly
- Don’t validate token scope’s ACL in Indexer’s UI
Version 0.6-rc1 (Flint)¶
(Released Tue, 04 Jul 2017)
- Handle etcd Raft errors properly
- Close old nexus when a new one has been created in Rok Transport
- Fix memory leak when joining nexus thread in Rok Transport
- Export LUs only on target nexuses
- Create and write OSD objects in filed atomically
- Fix epochs of COWed chocks
- Fix a memory leak error by freeing sg_bidi in liod
- Fix memory leak on ioctl() in Rok SG
- Make OpenStack Gateway driver thread safe
Version 0.5 (Emerald)¶
(Released Wed, 22 Feb 2017)
- Implement iSCSI transport for Rok
- Use LUN addressing methods defined in SAM5
- Support Logical Unit Conglomerate structures
- Extend OpenStore to export/unexport LUs
- Pass VASA 2.0 certification tests for Rok VASA provider (nonVVol)
- Add support for new FUSE-based filesystem (RokFS)
- Implement username/password authentication method for the Indexer
- Implement an OAuth flow for publishing buckets to the Indexer
- Make Garbage Collector multithreaded
- Add Synnefo Plankton Driver for Rok
- Add Gateway Service Driver for Synnefo Volumes
- Implement command-line client for the Rok Gateway
- Implement new designs in services forms of Gateway
- Implement OAuth 2.0
- Update Angular to version 2.1.0
- Ship UI’s static files under static folder
Version 0.4.5 (Diamond)¶
(Released Thu, 19 Jan 2017)
- Use access=userspace when creating Ganeti disks in the Gateway driver
- Check if the Ganeti instance already exists before creating it
- Fix single registration message in the UI
- Fix critical bug in libtrpt
Version 0.4.4 (Diamond)¶
(Released Mon, 09 Jan 2017)
- Support presenting Gateway versions as disks of existing Ganeti instances
- Introduce driver that creates Ganeti instances with a Gateway version as boot disk
- Add support for remote initiator PUs in controller
- Support different Identity API versions in our Django apps
Version 0.4.3 (Diamond)¶
(Released Fri, 16 Dec 2016)
- Remove unused dependency on django middleware
Version 0.4.2 (Diamond)¶
(Released Fri, 16 Dec 2016)
- Update Gateway logo
- Implement cookie authentication method
Version 0.4.1 (Diamond)¶
(Released Tue, 10 Oct 2016)
- Fix urllib3 compatibility issue in the controller
Version 0.4 (Diamond)¶
(Released Tue, 25 Oct 2016)
- The map format has changed from previous versions. You must recreate all fisks (clones, snapshots).
- This version of the thrower cannot connect to throwers of previous versions, due to changes in the on-the-write protocol.
- Rok GW
- Policies: The GW supports user-defined policies.
- Policies: The policy execution framework is now multithreaded, and can support registration/presentation of multiple objects concurrently.
- Thrower: The GW now exports thrower stats via the API and the UI.
- Drivers: Rok now comes with VMware, OpenStack, Ganeti, Synnefo drivers out of the box.
- The composer can now garbage-collect live (UUID) maps.
- VMware: Rok now includes a VASA provider, and can expose VVols directly to VMware ESXi over iSCSI.
- Python: PU bindings for Python are now green and can now be used in the context of a greenlet, without blocking the whole process.
There are too many dependencies to mention individually.
Please see the dependencies of individual Debian packages and the contents
setup.py files throughout the Rok repository for the initial list of
We will record individual changes in the set of dependencies in forthcoming versions.
- VMware: The semantics of the current implementation of
Snapshotdeviate from the VASA 2.0 specification and will be fixed in the next version.
- VMware: The VASA provider does not return proper SOAP Faults on error conditions.
- VMware: The VASA provider does not report metrics properly, including Container size, and free space.
- VMware: The semantics of the current implementation of
- The composer may leak memory after long periods of time, potentially due to allocating but not releasing memory in the case of missing fisks during map requests.
- There is currently no way to enforce a global limit on the memory usage or TCP connections of thrower.
- libpu: On the target side, libpu-managed objects are not garbage-collected when the corresponding nexuses disappear.
- libctrl: The embedded controller has a use-after-free bug for nexuses returned
- libmpath: Mpath will erroneously close a nexus when failing to submit a request
to it because it was full (returned
EAGAIN). It should check if its queue is full, and not consider it for submission instead.
- libmpath: libmpath will not release a nexus from an mpath if this nexus has pending I/O, even when this I/O is not related with said mpath.
- Python: Have the Python bindings return finer-grained errors as subclasses
PUErrorexceptions. Currently, the GW assumes specific type of errors when a generic
PUErrorexception is thrown.
- Python: Have the Python bindings return finer-grained errors as subclasses of