P99 CONF 24 Recap: Heckling on the Shoulders of Giants

As I sit here at the hotel breakfast bar contemplating what a remarkable couple of days it’s been for the fourth annual P99 CONF, I feel quite honored to have helped host it. While the coffee is strong and the presentations are fresh in my mind, let’s recap some of the great content we shared and reveal some of the behind-the-scenes efforts that made it all happen. Watch P99 CONF Talks On Demand Day 1 While we warmed up the live hosting stage, we had a fright with Error 1016 origin DNS errors on our event platform. As we scrambled to potentially host live on an alternative platform, Cloudflare saved the day and the show was back on the road. DNS issues weren’t going to stop us from launching P99 CONF! Felipe Cardeneti Mendes got things started in the lounge with hundreds of people asking great questions about ScyllaDB pre-show. Co-founder of ScyllaDB, Dor Laor, opened the show with his keynote about ScyllaDB tablets. In the first few slides we were looking at assembly, then 128 fully utilized CPU cores not long after that. By the end of the presentation, we had throughput north of 1M ops/sec – complete with ScyllaDB’s now famous predictable low latency. To help set the scene of P99 CONF, we heard from Pekka Enberg, CTO of Turso (sorry, I’m the one who overlooked the company name mistake in the original video). Pekka dove into the patterns of low latency. This generated more great conversation in chat. If you want all the details, then his book simply titled Latency is a must-read. Since parallel programming is hard, we opened up 3 stages for you to choose from following the keynotes. Felipe returned, this time as a session speaker. Proving that not all benchmarks need to be institutionalized cheating, he paired with Alan “Dormando” of Memcached to see how ScyllaDB stacks up from a caching perspective. We also heard from Luc Lenôtre, a talented engineer, who toyed with a kernel written in Rust. Luc showed us lots of flame graphs and low-level tuning of Maestro. Continuing with the Rust theme was Amos Wenger in a very interesting look at making HTTP faster with io_uring. There were other great talks from well-known companies. For example, Jason Rahman from Microsoft shared his insights on tracing Linux scheduler behavior using ftrace. Also, Christopher Peck from Uber shared their experience tuning with generational ZGC. This reflects much of the P99 CONF content – real world, production experience taming P99 latencies at scale. Another expanding theme at this year’s P99 CONF eBPF. And who better than Liz Rice from Isovalent to kick it off with her keynote, Zero-overhead Container Networking with eBPF and Netkit. I love listening to Liz explain in technical detail the concepts and benefits of using eBPF and will definitely be reading through her book–Learning eBPF on the long flight home to Australia. By the way, books! There are now so many authors associated with P99 CONF which, I think, is a testament to the quality and professionalism of the speakers. We were giving away book bundles to members of the community who were top contributors in the chat, answering and asking great questions (huge thank you!). Some of the books on offer – which you can grab for yourself – are: Database Performance at Scale – by Felipe Mendes (ScyllaDB) et. al. Latency – by Pekka Enberg (Turso) Think Distributed Systems – by Dominik Tornow (Resonate HQ) Writing for Developers: Blogs that Get Read – by Piotr Sarna (poolside) ScyllaDB in Action – by Bo Ingram (Discord) And if you’re truly a tech bookworm, see this blog post for an extensive reading list: 14 Books by P99 CONF Speakers: Latency, Wasm, Databases & More. By mid-afternoon, day 1, Gunnar Morling from decodable lightened things up with his keynote on creating the 1 billion row challenge. I’m sure you’ve heard of it, and we had another speaker, Shraddha Agrawal following up with her version in Golang. We enjoyed lots more great content in the afternoon, including Piotr Sarna (from poolside AI and co-author of Database Performance at Scale + Writing for Developers) taking us back to the long-standing database theme of the conference with performance perspectives on database drivers. Speaking of themes, Wasm returned with book authors Brian Sletten and Ramnivas Laddad looking at WebAssembly on the edge. And the two Adams from Zoo gave us unique insight into building a remote CAD solution that feels local. And showing that we can finish day 1 just as strong as we started it, Carl Lerche, creator of tokio-rs, returned to P99 CONF for the Day 1 closing keynote. This year, he highlighted how Rust – which is typically used at the infrastructure level for all the features we love, like safety, concurrency, and performance – is also applicable at higher levels in the stack. He also announced the first version of Toasty, an ORM for Rust. Day 2 The second day kicked off with Andy Pavlo from CMU and his take on the tension between the database and operating system, with a unique research project, Tigger, a database proxy that pushes a database into kernel space using eBPF. Leading on from that, we had Bryan Cantrill, CTO of Oxide and creator of DTrace, reviewing DTrace’s 21-year history with plenty of insights into the origins and evolution of this framework. Bryan has presented at every P99 CONF and is one of the many industry giants, that you can stand on the shoulders of heckle in chat. We love the dynamism that Bryan brings each year to P99 CONF. More great talks followed with Cameron from Shopify, Cary from Oracle, Vivkek from Linkedin and Richard from Datadog. We were also honored to host the creator of Postgres and Turing Award winner Michael Stonebraker. I also enjoyed Tanel Poder’s take on using eBPF off-CPU sampling to get to the bottom of database wait time. There was also more database content from Lukasz at ScyllaDB with advanced compression techniques. Avi Kivity led the afternoon’s keynote with his fascinating asymptotic performance journey through optimization and tradeoffs he’s made as CTO for ScyllaDB. Also on ScyllaDB, the Sharechat team described the evolution of their feature store. First they made it fast, as described at P99 CONF 23. Now, they’re making it cheaper to run with this impressive performance. Another blast of technical content in the afternoon with Benjamin and Tyler from American Express building card payment systems with ultra-low latency. A P99 favorite, Aleksei from TigerBeetle looked at just-in-time compaction with incredible detail. Dominik from Resonate to us through distributed async await patterns in the cloud. Peter from Percona dove into Redis alternatives. Ash from Unum Cloud gave us insights to AI powered real-time searches taking advantage of SIMD and GPU accelerated implementations. And Cristian from Uber was all about improving p99 latency in third-party APIs. Jose Fernandez from Netflix capped off the second day with his keynote on noisy neighbor detection with eBPF, which was a strong finish to another fantastic conference. What you didn’t see on stage, but happened in the recently renamed “Champagne Room,” was a mini happy birthday celebration for Cynthia – complete with a near-miss disaster champagne opening (no electrical equipment was harmed). Conference chat was lively right until the end. As I finish my second cup of coffee, I think that this is really what makes P99 CONF so special. Not only the ability to watch people from around the world share their real-life P99-related experiences, but having the chance to chat with them too. I look forward to hosting this again next year. Watch P99 CONF Talks On Demand