The Cambridge AI Safety Hub would like to invite the exceptional students of the Mathematical Institute to apply to the upcoming iteration of the Mentorship for Alignment Researchers (MARS), an AI safety fellowship that matches exceptional students and early-career researchers with experienced researchers and academics from AI labs, think tanks, and academia. Despite "Cambridge" in our name, the in-person sprint week of MARS V will take place in Oxford in mid-July, so we are especially excited to see Oxford students apply!
We'll have more than 20 projects spanning multiple disciplines, but a few projects we think especially interesting to mathematics students are:
- Research with Pablo Villalobos using Cartesian frames, linear logic, and category theory (up to adjunctions and monads) to model how agents reason about each other and how the reasoning of smaller agents composes into group reasoning.
- A project with Thomas Dooms and Ward Gauderis (Goodfire; Vrije Universiteit Brussel) using multilinear algebra and tensor-network methods to decompose neural-network weights directly, without relying on activation data.
- Work with Francisco Ferreira da Silva (Pivotal Research) on the geometry of LLM activations — activation plateaus, manifold structure, and Jacobian-norm minimization as a proxy for on-manifold behaviour.
- An investigation with Battista Biggio (University of Cagliari) of low-rank subspace structure in transformer weights, combining linear algebra, optimization, and probability.
- A collaboration with Shivam Raval and Luiza Corpaci (Harvard; AMD) on scalable oversight, using Lean-verified equational theories as ground truth for semantic equivalence.
Applications close on May 3rd. Students can find more information on our program's webpage, caish [dot] org [slash] mars.