David Eriksson | "High-Dimensional Bayesian Optimization"

Abstract: Bayesian optimization is a powerful paradigm for sample-efficient optimization of black-box objective functions and has been successfully used in real-world scientific and industrial applications. However, even with recent methodological advances, Bayesian optimization is generally limited to single-objective optimization in simple low-dimensional domains. In the first part of this talk, we will introduce our new Sparse Axis-Aligned Subspace BO (SAASBO) method which is very sample-efficient and can handle search spaces with hundreds of tunable parameters. We will also show a successful application of SAASBO for exploring the trade-offs between latency and accuracy for a production-scale on-device natural language understanding model at Meta. Finally, we will discuss our recent work on Multi-Objective trust Region Bayesian Optimization (MORBO) which expands the use of Bayesian optimization to high-dimensional multi-objective problems in the high-throughput setting.