High throughput data conversion for database virtualization - G. Mitra, M. Pope | Code BEAM America

✨This talk was recorded at Code BEAM America 2024. Check out our next event: https://codebeamamerica.com ✨

Hyper-Q is a database virtualization application written in Erlang. It emulates the syntax and semantics of a guest database over a host database, which involves cross-compiling SQL from one dialect to another. Additionally, it emulates functionalities that are not offered by the host database. Tuning Hyper-Q for new SQL workloads involves identifying bottlenecks in the cross-compilation process. This presentation will provide an overview of the diagnosis process for identifying bottlenecks through profiling and will also focus on some architectural changes made to address them.

The main objective of this initiative was to reduce memory consumption during processing, thereby improving the throughput of queries that can be processed through Hyper-Q. This, in turn, assists customers in transitioning from legacy data warehouses to cloud databases with minimum need for application rewrites saving time and effort.

Let's stay in touch! Follow us on:
💥 Twitter:   / codebeamio  
💥 Facebook:   / codesyncglobal  
💥 Linkedin:   / code-sync  
💥 Mastodon: https://genserver.social/codesync