Linear networks provide valuable insight into the workings of neural networks in general. In this talk, we improve the state of the art in (Bah et al., 2019) by identifying conditions under which gradient flow successfully trains a linear network, in spite of the non-strict saddle points present in the optimization landscape. We also improve the state of the art for the computational complexity of training linear networks in (Arora et al., 2018a) by establishing non-local linear convergence rates for gradient flow.

Crucially, these new results are not in the lazy training regime, cautioned against in (Chizat et al., 2019; Yehudai & Shamir, 2019). Our results require the network to have a layer with one neuron, which corresponds to the popular spiked covariance model in statistics, and subsumes the important case of networks with a scalar output. Extending these results to all linear networks remains an open problem.

References:

- Bah, B., Rauhut, H., Terstiege, U., and Westdickenberg, M. Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers.

- Arora, S., Cohen, N., Golowich, N., and Hu, W. A convergence analysis of gradient descent for deep linear neural networks.

- Chizat, L., Oyallon, E., and Bach, F. On lazy training in differentiable programming.

- Yehudai, G. and Shamir, O. On the power and limitations of random features for understanding neural networks.

# Past Forthcoming Seminars

Suspensions are composed of mixtures of particles and fluid and are ubiquitous in industrial processes (e.g. waste disposal, concrete, drilling muds, metalworking chip transport, and food processing) and in natural phenomena (e.g. flows of slurries, debris, and lava). The present talk focusses on the rheology of concentrated suspensions of non-colloidal particles. It addresses the classical shear viscosity of suspensions but also non-Newtonian behaviour such as normal-stress differences and shear-induced migration. The rheology of dense suspensions can be tackled via a diversity of approaches that are introduced. In particular, the rheometry of suspensions can be undertaken at an imposed volume fraction but also at imposed values of particle normal stress, which is particularly well suited to yield examination of the rheology close to the jamming transition. The influences of particle roughness and shape are discussed.

In this talk, we shall propose a relaxed control regularization with general exploration rewards to design robust feedback controls for multi-dimensional continuous-time stochastic exit time problems. We establish that the regularized control problem admits a H\”{o}lder continuous feedback control, and demonstrate that both the value function and the feedback control of the regularized control problem are Lipschitz stable with respect to parameter perturbations. Moreover, we show that a pre-computed feedback relaxed control has a robust performance in a perturbed system, and derive a first-order sensitivity equation for both the value function and optimal feedback relaxed control. These stability results provide a theoretical justification for recent reinforcement learning heuristics that including an exploration reward in the optimization objective leads to more robust decision making. We finally prove first-order monotone convergence of the value functions for relaxed control problems with vanishing exploration parameters, which subsequently enables us to construct the pure exploitation strategy of the original control problem based on the feedback relaxed controls. This is joint work with Christoph Reisinger (available at https://arxiv.org/abs/2001.03148).

The finite element method (FEM) is one of the great triumphs of applied mathematics, numerical analysis and software development. Recent developments in sensor and signalling technologies enable the phenomenological study of systems. The connection between sensor data and FEM is restricted to solving inverse problems placing unwarranted faith in the fidelity of the mathematical description of the system. If one concedes mis-specification between generative reality and the FEM then a framework to systematically characterise this uncertainty is required. This talk will present a statistical construction of the FEM which systematically blends mathematical description with observations.

>

Invented by Andreas Floer in 1988 to solve Arnold's

conjecture, (symplectic) Floer Homology is a machinery to relate the

existence of periodic trajectories of an Hamiltonian flow on a

symplectic manifold M with the homology groups of M, analougously to

Morse Homology. Indeed this is done by developing an infinite

dimensional Morse theoretic framework adapted to a certain functional

(the action functional) on the loop space of M, whose critical points

are the periodic trajectories of the given hamiltonian flow.

Despite the topological nature of the results, the construction is

technically quite heavy, involving hard analysis and elliptic systems of

PDEs.

Together with Andrei Agrachev and Antonio Lerario we are developing a

method to construct such infinite dimensional homology invariants using

only soft and essentially finite dimensional tools. In my talk I will

present our approach.

The main feature consists in approximating the loop space with finite

dimensional submanifolds of increasing dimension, we do this with the

language of control theory, and then interpret asymptotically the

information provided by classical Morse theory

## Further Information:

There is a beautiful mathematical theory of how independent agents tend to synchronise their behaviour when weakly coupled. Examples include how audiences spontaneously rhythmically applause and how nearby pendulum clocks tend to move in sync. Another famous example is that of the London Millennium Bridge. On the day it opened, the bridge underwent unwanted lateral vibrations that are widely believed to be due to pedestrians synchronising their footsteps.

In this talk Alan will explain how this theory is in fact naive and there is a simpler mathematical theory that is more consistent with the facts and which explains how other bridges have behaved including Bristol's Clifton Suspension Bridge. He will also reflect on the nature of mathematical modelling and the interplay between mathematics, engineering and the real world.

Alan Champneys is a Professor of Applied Non-linear Mathematics at the University of Bristol.

Please email external-relations@maths.ox.ac.uk to register.

Watch live:

https://www.facebook.com/OxfordMathematics/

https://livestream.com/oxuni/Champneys (available soon)

The Oxford Mathematics Public Lectures are generously supported by XTX Markets.