Model-Free Policy Gradient for Discrete-Time Mean-Field Control
Abstract
We study model-free policy learning for discrete-time mean-field control (MFC) problems with finite state space and compact action space. In contrast to the extensive literature on value-based methods for MFC, policy-based approaches remain largely unexplored due to the intrinsic dependence of transition kernels and rewards on the evolving population state distribution, which prevents the direct use of likelihood-ratio estimators of policy gradients from classical single-agent reinforcement learning. We introduce a perturbation scheme on the state-distribution flow and prove that the gradient of the resulting perturbed value function converges to the true policy gradient as the perturbation magnitude vanishes. This construction yields a model-free estimator based solely on simulated trajectories and an auxiliary estimate of the sensitivity of the state distribution. Building on this framework, we develop MF-REINFORCE, a model-free policy gradient algorithm for MFC, and establish quantitative bounds on its bias and mean-squared error.
All central University information for staff is now on OxIntranet, bringing together everything you need in one place - The 'For Staff' area provides access to key systems, processes, policies and guidance.
We are currently inviting applications for a Postdoctoral Research Associate to work with Professor Terry Lyons at the Mathematical Institute, University of Oxford. These are two-year, fixed-term positions, funded by the Engineering and Physical Sciences Research Council (EPSRC). The starting date of these positions is flexible.
14:15
Cayley fibrations have singular fibres
Abstract
Calibrated fibrations are expected to play an important role in exceptional holonomy, much as special Lagrangian fibrations do in the SYZ picture for Calabi--Yau manifolds. Singular fibres are expected to be essential, and a natural question is whether they are forced by the geometry. In his PhD thesis, Baraglia showed that coassociative fibrations of compact full-holonomy G_2-manifolds must have singular fibres. The analogous problem for Cayley fibrations remained open for many years and turns out to be substantially more difficult, with its resolution relying on deep results from 4-manifold topology.
The proof takes some unexpected twists and turns, involving a Diophantine equation arising from the Spin(7)-structure, families Seiberg--Witten theory and parametrized homotopy theory. After a short crash course on Spin(7) geometry, I will explain how these pieces fit together. This is joint work with Jianfeng Lin and Viktor Majewski.
A structure theorem for properly embedded CMC surfaces and applications
Abstract