ScyllaDB 6.0: with Tablets & Strongly-Consistent Topology Updates

The ScyllaDB team is pleased to announce ScyllaDB Open Source 6.0, a production-ready major release. ScyllaDB 6.0 introduces two major features which change the way ScyllaDB works: Tablets, a dynamic way to distribute data across nodes that significantly improves scalability Strongly consistent topology, Auth, and Service Level updates Note: Join ScyllaDB co-founder Dor Laor on June 27 to explore what learn what this architectural shift means for elasticity and operational simplicity. Join the livestream In addition, ScyllaDB 6.0 includes many other improvements in functionality, stability, UX and performance. Only the latest two minor releases of the ScyllaDB Open Source project are supported. With this release, only ScyllaDB Open Source 6.0 and 5.4 are supported. Users running earlier releases are encouraged to upgrade to one of these two releases. Related Links Get ScyllaDB Open Source 6.0 as binary packages (RPM/DEB), AWS AMI, GCP Image and Docker Image Upgrade from ScyllaDB 5.4 to ScyllaDB 6.0 Report an issue   New features Tablets In this release, ScyllaDB enabled Tablets, a new data distribution algorithm to replace the legacy vNodes approach inherited from Apache Cassandra. While the vNodes approach statically distributes all tables across all nodes and shards based on the token ring, the Tablets approach dynamically distributes each table to a subset of nodes and shards based on its size. In the future, distribution will use CPU, OPS, and other information to further optimize the distribution. Read Avi Kivity’s blog series on tablets In particular, Tablets provide the following: Faster scaling and topology changes. New nodes can start serving reads and writes as soon as the first Tablet is migrated. Together with Strongly Consistent Topology Updates (below), this also allows users to add multiple nodes simultaneously and scale, out or down, much faster Automatic support for mixed clusters with different core counts. Manual vNode updates are not required. More efficient operations on small tables, since such tables are placed on a small subset of nodes and shards. Read more about Tablets in the docs Using Tablets Tablets are enabled by default for new clusters. No action required. To disable Tablets for a Keyspace use     CREATE KEYSPACE ... WITH TABLETS = { 'enabled': false } For Tablets limitations in 6.0, see the discussion in the docs. Monitoring Tablets To Monitor Tablets in real time, upgrade ScyllaDB Monitoring Stack to release 4.7, and use the new dynamic Tablet panels, below. Driver Support The Following Drivers support Tablets: Java driver 4.x, from 4.18.0.2 (to be released soon) Java Driver 3.x, from 3.11.5.2 Python driver, from 3.26.6 Gocql driver, from 1.13.0 Rust Driver from 0.13.0 Legacy ScyllaDB and Apache Cassandra drivers will continue to work with ScyllaDB but will be less efficient when working with tablet-based Keyspaces. Strongly Consistent Topology Updates With Raft-managed topology enabled, all topology operations are internally sequenced consistently. A centralized coordination process ensures that topology metadata is synchronized across the nodes on each step of a topology change procedure. This makes topology updates fast and safe, as the cluster administrator can trigger many topology operations concurrently, and the coordination process will safely drive all of them to completion. For example, multiple nodes can be bootstrapped concurrently, which couldn’t be done with the previous gossip-based topology. Strongly Consistent Topology Updates is now the default for new clusters, and should be enabled after upgrade for existing clusters. In addition to Topology Updates, more Cluster metadata elements are now strongly consistent: Strongly Consistent Auth Updates. Role-Based Access Control (RBAC) commands like create role or grant permission are safe to run in parallel. As a result, there is no need to update system_auth RF or run repair when adding a DataCenter. Strongly Consistent Service Levels. Service Levels allow you to define attributes such as timeout per workload. Service levels are now strongly consistent Native Nodetool The nodetool utility provides simple command-line interface operations and attributes. ScyllaDB inherited the Java based nodetool from Apache Cassandra. In this release, the Java implementation was replaced with a backward-compatible native nodetool. The native nodetool works much faster. Unlike the Java version ,the native nodetool is part of the ScyllaDB repo, and allows easier and faster updates. With the Native Nodetool, the JMX server has become redundant and will no longer be part of the default ScyllaDB Installation or image, and can be installed separately Maintenance Mode and Socket Maintenance mode is a new mode in which the node does not communicate with clients or other nodes and only listens to the local maintenance socket and the REST API. It can be used to fix damaged nodes – for example, by using nodetool compact or nodetool scrub. In maintenance mode, ScyllaDB skips loading tablet metadata if it is corrupted to allow an administrator to fix it. The new Maintenance Socket provides a way to interact with ScyllaDB, only from within the node it runs on, while on Maintenance Mode Maintenance Socket docs. Improvements and Bug Fixes The latest release also features extensive improvements to: Bloom Filters Stability and performance Compaction Commitlog Cluster operations Materialized views Performance Edge cases Guardrails CQL Alternator (DynamoDB compatible API) REST API Tracing Monitoring Packaging and configuration For details, see the release notes on the ScyllaDB Community Forum See the detailed release notes