Ian Osband from DeepMind is going to talk about: 'The Price of Learning'
Policy gradient sits at the heart of modern RL, including LLM post-training. Yet on image classification, where the gradient of accuracy is exact, it loses to cross-entropy, even on accuracy. Our explanation is that learning is a sequential resource allocation problem. Data, compute and capacity are scarce, and each update sets where the next one starts. Policy gradient ignores this and spends greedily. This talk presents simple methods that price learning, and shows how they improve robustness, compute efficiency and accuracy.
Further Information
Bio:
Ian Osband leads the Science of Post-Training team at Google DeepMind. Before that he spent two years at OpenAI working on post-training for ChatGPT and reasoning models. He did his PhD at Stanford and studied maths at New College, Oxford.