Since their introduction in 2017, Transformers have revolutionized large language models and the broader field of deep learning. Central to this success is the ground-breaking self-attention mechanism. In this presentation, I’ll introduce a mathematical framework that casts this mechanism as a mean-field interacting particle system, revealing a desirable long-time clustering behaviour. This perspective leads to a trove of fascinating questions with unexpected connections to Kuramoto oscillators, sphere packing, Wasserstein gradient flows, and slow dynamics.

Bio: Philippe Rigollet is a Distinguished Professor of Mathematics at MIT, where he serves as Chair of the Applied Math Committee and Director of the Statistics and Data Science Center. His research spans multiple dimensions of mathematical data science, including statistics, machine learning, and optimization, with recent emphasis on optimal transport and its applications. See https://math.mit.edu/~rigollet/ for more information.

This talk is hosted by the AI Reading Group

Given a space with some kind of geometry, one can ask how the geometry of the space relates to its homology. This talk will survey some comparisons of geometric notions of complexity with homological notions of complexity. We will then focus on hyperbolic 3-manifolds and the main result will replace a spectral gap problem related to torsion in homology with a geometric version involving geodesic length and stable commutator length. As an application, we provide "bad" examples of hyperbolic 3-manifolds with bounded geometry but extremely small (1-form) spectral gaps.