NoSQL Summer Blockbusters: Rust, Kafka, Media Streaming & Social Media & More

July 24, 2023

Cynthia Dunlop

Working on systems where low latency at scale is critical? These on-demand NoSQL features will keep you on the edge of your seat.

If Barbenheimer has gotten you into the summer blockbuster mood, we’ve got a marquee full of low-latency database features that will keep you on the edge of your seat:

The database concurrency optimization strategies that saved Black Friday
Heroics that improved database performance 3-5x while reducing costs 50-80%
How to steer clear of data modeling mistakes that have tormented even the world’s top engineering teams
An inside look at the challenges and strategies behind the media streaming systems behind your binge watching

Grab some popcorn and tune into whatever’s most relevant for your projects or professional development.

Learning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline

Numberly operates business-critical data pipelines and applications where failure and latency means “lost money” in the best-case scenario. Most of their data pipelines and applications are deployed on Kubernetes and rely on Kafka and ScyllaDB, with Kafka acting as the message bus and ScyllaDB as the source of data for enrichment. The availability and latency of both systems are thus very important for data pipelines. While most of Numberly’s applications are developed using Python, they found a need to move high-performance applications to Rust in order to benefit from a lower-level programming language. You’ll learn lessons from Numberly’s experience, including:

Their rationale for selecting Rust
Their experience developing using a lower-level Rust code base
Observability and analyzing latency impacts with Rust
How they built a mission-critical system with Kafka & ScyllaDB
A retrospective after half a year of Rust in production

Watch Learning Rust the Hard Way for a Production Kafka + ScyllaDB Pipeline

Bonus: This NoSQL video does indeed have a sequel. Watch Building a 100% ScyllaDB Shard-Aware Application Using Rust to learn about their latest “crazy idea” to replace Kafka and Redis with a (Rust-based) ScyllaDB shard-aware application.

Dev Workshop: Build Low-Latency Rust Applications on ScyllaDB

ScyllaDB is a perfect match for Rust. Similar to the Rust programming language and the Tokio framework, ScyllaDB is built on an asynchronous, non-blocking runtime that works extremely well for building highly-reliable low-latency distributed applications. This video shows how to build a sample Rust application on our high performance native Rust client driver, including walking through the code and crafting queries to a locally running ScyllaDB cluster. You’ll learn how to

Install and compile a sample app, built on ScyllaDB’s native Rust SDK
Get a ScyllaDB cluster up and running
Connect the application to the database
Review data modeling, query types, and best practices
Manage and monitor the database for consistently low latencies

Watch Build Low-Latency Rust Applications on ScyllaDB

NoSQL Data Modeling Masterclass

Some NoSQL databases popularized the notion of “loose schema,” but getting data modeling right is actually one of the most difficult aspects of working with NoSQL. It is dangerously easy to make decisions or tradeoffs that could later impede your performance and scalability.

Designed for organizations migrating from SQL to NoSQL or optimizing any NoSQL data model, this masterclass will assist practitioners looking to advance their understanding of NoSQL data modeling. We’ll cover a range of topics, from building a solid foundation on NoSQL to correcting your course if you’re heading down a dangerous path. You’ll learn how to:

Determine what NoSQL data modeling techniques will be most performant for your specific usage patterns
Apply common key-value and key-key-value (e.g., Amazon DynamoDB, Google BigTable, Apache Cassandra, ScyllaDB) data modeling strategies in the context of a sample application
Apply NoSQL data modeling best practices to avoid mistakes that have troubled some of the world’s top engineering teams in production

Watch NoSQL Data Modeling Masterclass

NoSQL at Scale: Proven Practices & Pitfalls

Discover what’s most important for achieving low latency at extreme scale. Our Solution Architects walk you through the strategies and pitfalls learned by working on thousands of real-world distributed database projects, many reaching 1M OPS with single-digit MS latencies. In addition to offering clear recommendations, we’ll explain how we arrived at those recommendations – so you can benefit from the lessons learned by other teams. You’ll learn how to

Design and deploy a large-scale distributed database cluster with ScyllaDB
Optimize your clients’ interactions with it
Expand the cluster horizontally and globally
Ensure it survives whatever disasters the world throws at it

Watch NoSQL at Scale: Proven Practices & Pitfalls

How to Maximize Database Concurrency

Having a database that’s capable of high concurrency is one thing, but actually tapping all that potential concurrency is another. Knowing exactly how hard you can push your database is key for facing throughput challenges, like Black Friday surges, without a hitch. After wrestling with this firsthand, Optimizely Engineering developed practical strategies that can help other teams. Brian Taylor shares how they take full advantage of the high concurrency that’s possible with their NoSQL database, ScyllaDB – while also guaranteeing correctness and protecting the quality of service. You’ll learn about

Understanding concurrency and its impact on throughput and latency
Closed loop load testing, open loop load testing & the Universal Scaling Law
The type of load testing you should be performing for capacity planning
How to identify the region where your database can make the best use of concurrency
Strategies for optimizing sound concurrency based on your data dependencies

Watch How to Maximize Database Concurrency

Bonus: Read the blog with some brilliant Q & A discussion that followed the original webinar.

ShareChat’s Path to High-Performance NoSQL with ScyllaDB

To power India’s leading social media platform, ShareChat needs to deliver impressive performance at rapidly-increasing scale – but without rapidly-increasing cost. With 180M monthly active users expecting real-time engagement with 2.5B posts per month, having the right database implementation is essential.

Hear how Geetish Nayak, Staff Engineer/Architect – Platforms at ShareChat modernized the database powering their services to stay ahead of these challenges. Geetish will share the strategies they applied to improve performance 3-5x while reducing costs 50-80%. You’ll learn:

About their technical challenges related to supporting new locations, debugging, and performance at scale
What NoSQL capabilities were most critical for preparing their data architecture for the company’s next level of growth
How they measured the impact of their NoSQL migration, and what they have achieved so far
Best practices that ShareChat followed when adopting ScyllaDB

Watch ShareChat’s Path to High-Performance NoSQL with ScyllaDB

Bonus: Watch two insightful tech talks from ShareChat engineers that were featured at ScyllaDB Summit:

Overcoming Media Streaming Challenges with NoSQL

Behind every seamless and engaging viewing experience is a great database design. But what this actually means varies wildly. Databases are used for a range of media streaming use cases – each with distinctly different workloads and requirements. And then there are the challenges that span use cases. Pressure to keep costs and latencies down as usage rises. Dealing with unpredictable intra-day traffic fluctuations as well as major events with hundreds of millions of users. And removing dependencies on databases that were easy to adopt, but not sustainable at scale. Watch this video as our experts explore these topics using real-world examples. You’ll learn:

The role databases play in common media streaming use cases
The core challenges faced in media streaming, along with different options for addressing them
What’s changing in the media streaming landscape and how this might impact databases

Watch Overcoming Media Streaming Challenges with NoSQL

DBaaS in the Real World: Risks, Rewards & Tradeoffs

What do you give up – and gain – when moving to a fully-managed cloud database?
Now that database-as-a-Service (DBaaS) offerings have been “battle tested” in production, how is the reality matching up to the expectation? What can teams thinking of adopting a fully-managed DBaaS can learn from teams who have years of experience working with this deployment model?

Join this webinar to dive into the reality of working with various high-performance DBaaS offerings. We cover the following topics, all supported with real-world examples: