Key-Key-Value Store: Generic NoSQL Datastore w/ Tombstone Reduction & Automatic Partition Splitting

🎥 Watch all the ScyllaDB Summit 2023 talks here: https://www.scylladb.com/scylladb-sum...

Discord’s Senior Software Engineer, Stephen Ma, shares how Discord built a developer-friendly key-value service atop Scylla that hides many database-specific complexities from developers. Discord wanted to shield developers from having to be involved in database specific schema design and schema migrations while still providing powerful datastore primitives that support a broad set of developer needs.

Their solution was a key-value store service that supports storing and querying entities stored as protobufs and identified by composite primary keys. The composite key consists of a parent ID and an entity ID and the service supports range scans of entity IDs within one parent ID. Their “Key-Key-Value Store” also safeguards developers against ScyllaDB performance complexities through a tombstone reduction mechanism and automatic splitting of large partitions.