Inference Time Compute

This tutorial aims at explaining the key technologies behind "Inference-Time Compute", which is said to be the core of OpenAI O1. I will talk about how we can teach the LLM to do "Chain-of-Thought" thinking, step-by-step reasoning by using the process reward model, self-reflection and backtracking. I will also briefly cover inference-time scaling laws and current community efforts on reproducing OpenAI O1.

The initial version of this tutorial is made at TTIC & UChicago NLP seminar on 10/24. I later incorporate content from Sasha Rush's great talk at Simons Institute and I also recommend people interested to take a look as well: https://t.co/jvEztaHf0p.