Скачать или смотреть Systor 25' Keynote: Prof. Niv Dayan (University of Toronto) - Scaling Storage Engines for the AI Age

Abstract:
Log-Structured Merge-trees (LSM-trees) underpin large-scale storage systems such as Bigtable, DynamoDB, and RocksDB, and are foundational to ML and AI infrastructure including feature stores, recommendation engines, and vector databases. While their write-optimized design is well-suited to write-intensive workloads, scaling both read and write performance simultaneously remains a central challenge.

This talk revisits core LSM-tree design principles and presents recent advances that upend longstanding trade-offs between read and write efficiency. We discuss the use of range filters as a replacement for traditional Bloom filters and introduce techniques for optimizing filter space allocation across levels to reduce false positives and compaction costs simultaneously. These techniques unlock new regimes of performance, enabling LSM-based systems to scale more gracefully under modern demands.

Bio:
Niv Dayan is an Assistant Professor at the University of Toronto (U of T). His research focuses on designing and analyzing data structures for database and storage systems. Before joining U of T, he was a Research Scientist at Pliops and a Technical Advisor to Speedb. He received his Ph.D. from the IT University of Copenhagen and was a postdoctoral researcher at Harvard.