Unbiased Learning to Rank: On Recent Advances in the Foundations and Applications - SIGIR23 / WSDM24

This video is based on three tutorials presented at the SIGIR 2023 conference in Taipei; Fire 2023 in India; and WSDM 2024 in Mexico.

Slides and more info can be found on the tutorial websites:
https://sites.google.com/view/sigir-2...
https://sites.google.com/view/fire-20...
https://sites.google.com/view/wsdm-20...

Authors:
Shashank Gupta, Philipp Hager, Jin Huang, Ali Vardasbi, Harrie Oosterhuis

Abstract:
Since its inception, the field of unbiased learning to rank (ULTR) has remained very active and has seen several impactful advancements in recent years. This tutorial provides both an introduction to the core concepts of the field and an overview of recent advancements in its foundations along with several applications of its methods. The tutorial is divided into four parts: Firstly, we give an overview of the different forms of bias that can be addressed with ULTR methods. Secondly, we present a comprehensive discussion of the latest estimation techniques in the ULTR field. Thirdly, we survey published results of ULTR in real-world applications. Fourthly, we discuss the connection between ULTR and fairness in ranking. We end by briefly reflecting on the future of ULTR research and its applications. This tutorial is intended to benefit both researchers and industry practitioners who are interested in developing new ULTR solutions or utilizing them in real-world applications.

Chapters:
0:00 - Tutorial Opening and Overview
2:30 - Part 1: Introduction
8:58 - Counterfactual Learning to Rank
18:35 - Inverse Propensity Scoring
34:04 - Part 2: Biases in User Interactions
34:48 - Estimating Position Bias
48:55 - Advanced User Models
56:29 - Trust Bias
1:00:58 - Item Selection Bias
1:02:11 - Item Context Biases
1:06:38 - Part 3: Estimation Methods
1:06:56 - Advanced IPS / Affine Correction
1:10:46 - Policy-Aware Estimation
1:16:40 - Intervention-Aware Estimation
1:25:27 - Two Tower Models
1:33:55 - Doubly-Robust Estimation
1:45:26 - Safety in Optimization
1:58:40 - Part 4: Survey Applications
1:59:18 - Real World Results
2:03:51 - Case Study 1: Grid Layouts
2:08:31 - Case Study 2: Beyond Clicks
2:12:24 - Practical Considerations
2:17:24 - Part 5: Unbiased to Fair LTR
2:17:40 - Introduction to Ranking Fairness
2:32:43 - Unbiased LTR and Fairness
2:39:30 - Part 6: Conclusion
2:42:07 - The Bandwagon Effect: Not Statistical Bias
2:49:13 - Limitations of Unbiased LTR Approach
2:59:18 - Future of Unbiased LTR
3:02:09 - Acknowledgements