I will discuss the use of partitioned schemes for neural networks. This work is in the tradition of multrate numerical ODE methods in which different components of system are evolved using different numerical methods or with different timesteps. The setting is the training tasks in deep learning in which parameters of a hierarchical model must be found to describe a given data set. By choosing appropriate partitionings of the parameters some redundant computation can be avoided and we can obtain substantial computational speed-up. I will demonstrate the use of the procedure in transfer learning applications from image analysis and natural language processing, showing a reduction of around 50% in training time, without impairing the generalization performance of the resulting models. This talk describes joint work with Tiffany Vlaar.

A common observation that wider (in the number of hidden units/channels/attention heads) neural networks perform better motivates studying them in the infinite-width limit.

Remarkably, infinitely wide networks can be easily described in closed form as Gaussian processes (GPs), at initialization, during, and after training—be it gradient-based, or fully Bayesian training. This provides closed-form test set predictions and uncertainties from an infinitely wide network without ever instantiating it (!).

These infinitely wide networks have become powerful models in their own right, establishing several SOTA results, and are used in applications including hyper-parameter selection, neural architecture search, meta learning, active learning, and dataset distillation.

The talk will provide a high-level overview of our work at Google Brain on infinite-width networks. In the first part I will derive core results, providing intuition for why infinite-width networks are GPs. In the second part I will discuss challenges and solutions to implementing and scaling up these GPs. In the third part, I will conclude with example applications made possible with infinite width networks.

The talk does not assume familiarity with the topic beyond general ML background.

In this talk we will present some work in progress generalising Lubin-Tate formal group laws to higher dimensions. There have been some other generalisations, but ours is different in that the ring over which the formal group law is defined changes as the dimension increases. We will state some conjectures about these formal group laws, including their relationship to the Drinfeld tower over the p-adic upper half plane, and provide supporting evidence for these conjectures.

Let F be a p-adic field, and k an algebraically closed field of characteristic l different from p. In this talk, we will first give a category decomposition of Rep_k(SL_n(F)), the category of smooth k-representations of SL_n(F), with respect to the GL_n(F)-equivalent supercuspidal classes of SL_n(F), which is not always a block decomposition in general. We then give a block decomposition of the supercuspidal subcategory, by introducing a partition on each GL_n(F)-equivalent supercuspidal class through type theory, and we interpret this partition by the sense of l-blocks of finite groups. We give an example where a block of Rep_k(SL_2(F)) is defined with several SL_2(F)-equivalent supercuspidal classes, which is different from the case where l is zero. We end this talk by giving a prediction on the block decomposition of Rep_k(A) for a general p-adic group A.

Fluids sculpt many of the shapes we see in the world around us. We present a new mathematical model describing the shape evolution of a body that dissolves or melts under gravitationally stable buoyancy-driven convection, driven by thermal or solutal transfer at the solid-fluid interface. For high Schmidt number, the system is reduced to a single integro-differential equation for the shape evolution. Focusing on the particular case of a cone, we derive complete predictions for the underlying self-similar shapes, intrinsic scales and descent rates. We will present the results of new laboratory experiments, which show an excellent match to the theory. By analysing all initial power-law shapes, we uncover a surprising result that the tips of melting or dissolving bodies can either sharpen or blunt with time subject to a critical condition.

The evolution of land surfaces is partly cause by the erosion, transport and deposition of sediment. My research aims to understand the origin and evolution of landscapes, using the tools of fluid mechanics. I am particularly interested in aeolian and fluvial transport of sediments. To do this, I use a multi-method approach (theoretical/numerical analysis, laboratory experiments and field measurements). The use of simplified laboratory experiments allows me to limit the complexity of natural systems by identifying the main mechanisms controlling sediment transport.  Once these physical laws are established, I apply them to natural data to explain the morphology of the observed landscapes, and to predict their evolution.

In this seminar, I will present two examples of the application of my work. An experimental study highlighting the influence of input conditions (water and sediment flows, sediment properties) on the morphology of fluvial deposits (i.e. alluvial fan), as well as a theoretical analysis coupled with field measurements to understand the mechanisms of dune initiation.