Introducing ScyllaDB Open Source 5.0
With the release of ScyllaDB Open Source 5.0, we’ve introduced a set of noteworthy production-ready new features – foremost of which is including support for the AWS EC2 I4i line of servers powered by the 3rd generation of Intel Xeon (“Ice Lake”) processors. ScyllaDB running on I4i servers can provide 2x the performance of comparable I3 series servers. On top of this, the new release provides fundamental optimizations to our IO scheduler, faster performance for reverse queries, and new virtual tables for easier configuration and to access nodetool-level information directly through the CQL interface.
ScyllaDB 5.0 also includes new experimental features, such as providing strongly and immediately consistent schema changes and a new approach to tombstone garbage collection, both of which make ScyllaDB easier to administer and more reliable.
With these new capabilities, ScyllaDB Open Source 5.0 is faster, more efficient, and easier to use than ever.
For those discovering us for the first time via this announcement, ScyllaDB is a monstrously fast and scalable NoSQL database designed for the data-intensive applications of this next tech cycle. It was purpose-built to resolve the challenges of legacy NoSQL databases like Apache Cassandra and Amazon DynamoDB by taking their best attributes and improving on capabilities, performance, scalability, and cost-efficiency.
If you want to learn more about the specific features in ScyllaDB 5.0, we will be hosting a webinar introducing you in depth to its capabilities on July 19, 2022.
REGISTER NOW FOR THE SCYLLADB 5.0 WEBINAR
The Journey from 4.0 to 5.0
We released ScyllaDB Open Source 4.0 on May 7, 2020. Over the course of the past two years, there have been 7,720 code commits and 2,002 issues closed on Github. Apart from the era of initial development, 2015-2016, it’s been our most consistently productive era of coding. And since the release of ScyllaDB Open Source 4.0, users have benefitted from a number of new features added in the six subsequent minor releases (4.1 to 4.6) which are worthy to review.
A graphic showing the number of commits to scylladb/scylla on Github, with the period since ScyllaDB Open Source 4.0 released in May 2020 highlighted. Note that there are between 50 to 100 new code commits most weeks, and at times we have exceeded 100 code commits per week.
Support for Arm-based Systems
With ScyllaDB Open Source 4.6 we implemented support for Arm-based architectures, including the new AWS Im4gn and Is4gen storage-optimized instances powered by Graviton2 processors. Release 4.6 also supports the low cost T4g burstable instance for development of cloud-based applications. Since ScyllaDB is now compiled to run on any Aarch64 architecture, you can even run it on an Arm-based M1-powered Macintosh using Docker for local development.
Change Data Capture (CDC)
Change Data Capture (CDC) was promoted to a production-ready feature in release 4.3. This was vital to users building enterprise-grade event streaming architectures at scale. ScyllaDB can now serve as a source of real-time data updates for technologies such as Apache Kafka, using our Debezium-based CDC source connector.
DynamoDB Compatibility (Alternator)
We also repeatedly and continuously improved on our DynamoDB-compatible interface, which we call Alternator. This included support (in 4.5) for Cross-Origin Resource Sharing (CORS), nested attribute paths, and attribute paths in ConditionExpression, FilterExpression Support, and ProjectionExpression. We also added Alternator Streams as an experimental feature in release 4.3, using our underlying Change Data Capture (CDC) infrastructure.
ScyllaDB Monitoring Stack Advisor Section
Observability was made easier than ever by the introduction of many features in ScyllaDB Monitoring Stack, foremost of which was the new Advisor section. Now you have a virtual expert watching for the proper balance of your cluster, letting you know when it spots the warning signs of unprepared statements, large cells or hot partitions.
Kubernetes Operator Improvements
Our Kubernetes story was production-ready with ScyllaDB Operator 1.0, released in January of 2020. Since that time, we have continuously improved on our capabilities and performance tuning with cloud orchestration.
Faster Shard-Aware Drivers
Also during the past two years we have learned a lot about making our shard-aware drivers even faster. Open source community contributions led to a shard-aware Python driver which you can read about here and here. Since then our developers have embraced the speed and safety of Rust, and we are now porting our database drivers to async Rust, which will provide a core of code for multiple drivers with language bindings to C/C++ and Python.
Repair-based Node Operations (RBNO)
We leveraged our row-based repair mechanism for other repair-based node operations (RBNO), such as node bootstraps, decommissions, and removals, and allowed it to use off-strategy compaction to simplify cluster management.
Granular Timeout Controls
Timeouts per operation were also added, allowing more granular control over latency requirements and freeing up server resources (4.4). We expanded upon this by adding Service Level Properties, allowing you to associate attributes to rules and users, so you can have granular control over session properties, like per-service-level timeouts and workload types (4.6).
Improved Binary Search on Promoted Index
Even fundamental algorithms such as binary search for the SSTable promoted index were improved, moving from an O(n) to an O(log n) function, reducing query search time (4.2).
SSTable Index Caching
We also added SSTable Index Caching (4.6). Up to this release, ScyllaDB only cached data from SSTables data rows (values), not the indexes themselves. As a result, if the data was not in cache readers had to touch the disk while walking the index. This was inefficient, especially for large partitions, increasing the load on the disk, and adding latency. Now, index blocks can be cached in memory, between readers, populated on access, and evicted on memory pressure – reducing the IO and decreasing latency. More info can be found in Tomasz Grabiec session in ScyllaDB Summit “SSTable Index Caching.”
SSTable Index Caching provides faster performance by avoiding unnecessary queries to SSD.
Guardrails
Lastly, administering large scale systems always has associated fundamental risks which we try to mitigate for users as much as possible. To that end in ScyllaDB Open Source 4.6 we added Guardrails, a collection of reservations administrators can control that warn or by default disallow users from making non-recommended changes to their production environments, such as enabling experimental or performance-impacting features. Note that this feature has the same name as a similar feature available in an upcoming release of Apache Cassandra, but because of operational differences between the two databases, there will be dissimilarities between the two implementations.
Production Ready Features in Release 5.0
Support for AWS EC2 I4i Series Instances
ScyllaDB now supports the new AWS EC2 I4i series instances. The I4i series provides superior performance over the I3 series due to a number of factors: the 3rd generation Intel Xeon “Ice Lake” processors, the AWS Nitro System hypervisor, and low-latency Nitro NVMe SSDs. ScyllaDB can achieve 2x throughput and lower latencies on I4i instances over comparable i3 servers.
Operations per Second (OPS) throughput results on i4i.16xlarge (64 vCPU servers) vs. i3.16xlarge with 50% Reads / 50% Writes (higher is better)
I/O Scheduler Improvements
A new I/O scheduler was integrated via a Seastar update. The new scheduler is better at restricting disk I/O in order to keep latency low. This implementation introduces a new dispatcher in the middle of a traditional producer-consumer model. The IO scheduler seeks to find what is known as the effective dispatch rate — the fastest rate at which the system can process data without running into internal queuing jams.
Improved Reverse Queries
Reverse queries are SELECT statements that use reverse order from the table schema. If no order was defined, the default order is ascending (ASC). For example, imagine rows in a partition sorted by time in ascending order. A reverse query would sort rows in descending order, with the newest rows first. Reverse queries were improved in ScyllaDB Open Source 4.6, and are further improved in 5.0, first, to return short pages to limit memory consumption, and secondly, for reverse queries to leverage ScyllaDB’s row-based cache (before 5.0 they bypassed the cache).
New Virtual Tables for Configuration and Nodetool Information
A new system.config virtual table allows querying and updating a subset of configuration parameters over CQL. These updates are not persistent, and will return to the scylla.yaml update after restart. Nodetool command information can also be accessed via virtual tables, including snapshots, protocol servers, runtime info, and a virtual table replacement for nodetool versions. Virtual tables allow remote access over CQL, including for ScyllaDB Cloud users.
Experimental Features
Schema Changes using Raft
Using the new Raft consensus protocol support in ScyllaDB, you can now perform immediate and safe schema management for DDL operations like CREATE, ALTER, DROP for KEYSPACE, TABLE, INDEX, UDT, MV, etc.
ScyllaDB’s Raft implementation implements a number of important extensions:
- Designed to handle very large clusters (1000+ nodes)
- Resilience against asymmetric network failures
- Read and write support on all cluster nodes
- Efficient multi-raft: every node can replicate many state machines
You can learn more by watching the presentation “Making Schema Changes Safe With Raft” from ScyllaDB Summit 2022.
More Robust Tombstone Garbage Collection
Tombstones (markers that indicate deleted records) that are older than the most recent repair can now be automatically purged, and newer ones will be kept. This drops tombstones more frequently if repairs are made in a timely manner, and prevents data resurrection if repairs are delayed beyond gc_grace_seconds. Note that gc_grace_seconds stems from our Cassandra compatibility legacy. We believe our new repair-based node operation (RBNO) tombstone garbage collection will provide a far more reliable and correct experience for users.
ScyllaDB Sizing Up vs. the Competition
Just as we ourselves have not been idle during the past two years, neither has the database industry as a whole. After six years Apache Cassandra finally released its long-anticipated 4.0 major release in July 2021. Since then they announced Cassandra 4.1 which is still pending release as of this writing, and provided four maintenance releases (4.0.1 through 4.0.4).
Over that same time period — July 2021 to July 2022 — ScyllaDB released three new minor versions (4.5, 4.6 and 5.0) and provided a total of fourteen maintenance releases (4.4.4 – 4.4.8; 4.5.1 – 4.5.6; and 4.6.1 – 4.6.3). This release cadence enables ScyllaDB to deliver new features and to resolve issues more swiftly. Such reliability in release deliveries was one of the reasons cited by Expedia in switching to ScyllaDB.
Yet now that the long-awaited Cassandra 4.0 is out, how does it stack up in performance in head-to-head benchmarking? Was it worth the wait for patient Cassandra users? First we compared Cassandra 4.0 to Cassandra 3.11. We found it provided modest throughput gains and far better latencies. However, we also found that it was still no match for ScyllaDB Open Source. This held true whether the measurement was of throughputs, P99 latencies, or conducting administrative tasks such as replacing a node or doubling a cluster size.
A peek into the results: the maximum throughput (measured in operations per second) achieved on 3 x i3.4xlarge machines (48 vCPUs). ScyllaDB led the pack, processing from 2x to 5x more requests than either of the Cassandras.
The time needed to add 3 nodes to an already existing 3-node cluster of i3.4xlarge machines, preloaded with 1 TB of data at RF=3. Cassandra 4.0 performed moderately better than Cassandra 3.11. but ScyllaDB still leads the pack.
The most telling difference was performing major compactions. Here, a process that took ScyllaDB a little more than half an hour could still take as most of a whole day with Cassandra 4.0 — or even longer, more than a day-and-a-half — if you used default settings.
Major compaction of 1 TB of data at RF=1 on i3.4xlarge machine. ScyllaDB demonstrates the power of sharded architecture by compacting on all cores concurrently. In our case ScyllaDB is up to 60x faster and this figure should continue to scale linearly with the number of cores.
By our thinking, if you were using Cassandra 3.11, patiently waiting for 4.0, and you want to see true gains in performance across the board, there’s no reason to hesitate — upgrade straight to ScyllaDB.
As a side note, all of these results were produced in mid-2021 using i3.4xlarge AWS instances, before the availability of the I4i family. So stay tuned to see even better numbers now in 2022.
Also, we acknowledge and understand the hesitancy in accepting any vendor’s benchmarking claims at face value. We’d love to hear of your own real-world requirements and performance testing results. Feel free to contact us directly, or share them with our user community on Slack.
ScyllaDB V and the Days Ahead
Scylla Open Source 5.0 is the first delivery of the release series that we are calling ScyllaDB V. New infrastructure under the hood, such as the Raft consensus protocol implementation, establishes the foundation for additional innovation. Our distinguished software engineer, Tomasz Grabiec described these future capabilities at his ScyllaDB Summit 2022 talk on the “Future of Consensus.” Beyond today’s Raft-based schema changes, the next developments will include:
- Flexible topology changes — concurrently adding, replacing or removing more than one node at a time from a cluster
- Elasticity via auto-scaling — reliable, fault tolerant, fast automatic cluster scaling
- Replacing gossip with Raft — allow faster, more consistent topology changes
- Strongly-consistent tables — moving from Paxos-based lightweight transactions (LWT) using 3 round trips to fast, Raft-based operations using 1-round trip
- Tablet partitioning — allows for finer control, better data distribution and load balancing, and easier management of data — faster resharding and cleanup
With future releases ScyllaDB users can look forward to even greater elasticity, resiliency, scalability and maintainability.
Get ScyllaDB Open Source 5.0 Now
ScyllaDB Open Source 5.0 is immediately available. You can check out the documentation and download it now from the links below. If this is your first foray into ScyllaDB, or into NoSQL in general, remember to sign up for our free online courses in ScyllaDB University. Or if live, instructor-led training is more your thing, you can sign up now for our ScyllaDB University LIVE Summer Session, which will be held on July 28, 2022.
LISTEN TO THE SCYLLADB OPEN SOURCE 5.0 WEBINAR RECORDING