Past Computational Mathematics and Applications Seminar

We propose and analyse two structure preserving finite volume schemes to approximate the solutions to a cross-diffusion system with self-consistent electric interactions introduced by Burger, Schlake & Wolfram (2012). This system has been derived thanks to probabilistic arguments and admits a thermodynamically motivated Lyapunov functional that is preserved by suitable two-point flux finite volume approximations. This allows to carry out the mathematical analysis of two schemes to be compared.

This is joint work with Maxime Herda and Annamaria Massimini.

The currently available quantum computers fall into the NISQ (Noisy Intermediate Scale Quantum) regime. These enable variational algorithms with a relatively small number of free parameters. We are now entering the FTQC (Fault Tolerant Quantum Computer)  regime where gate fidelities are high enough that error-correction schemes are effective. The UK Quantum Missions include the target for a FTQC device that can perform a million operations by 2028, and a trillion operations by 2035.

This talk will present the outcomes from assessments of  two quantum linear equation solvers for FTQCs– the Harrow–Hassidim–Lloyd (HHL) and the Quantum Singular Value Transform (QSVT) algorithms. These have used sample matrices from a Computational Fluid Dynamics (CFD) testcase. The quantum solvers have also been embedded with an outer non-linear solver to judge their impact on convergence. The analysis uses circuit emulation and is used to judge the FTQC requirements to deliver quantum utility.

Solving large-scale continuous-time algebraic Riccati equations is a significant challenge in various control theory applications. 

This work demonstrates that when the matrix coefficients of the equation are quasiseparable, the solution also exhibits numerical quasiseparability. This property enables us to develop two efficient Riccati solvers. The first solver is applicable to the general quasiseparable case, while the second is tailored to the particular case of banded coefficients. Numerical experiments confirm the effectiveness of the proposed algorithms on both synthetic examples and case studies from the control of partial differential equations and agent-based models. 

Numerically solving PDEs typically requires compressing infinite information into a finite system of algebraic equations. Pragmatically, we usually follow a recipe: “Assume solutions of form X; substitute into PDE Y; discard terms by rule Z.” In contrast, Lanczos’s pioneering “tau method” prescribes modifying the PDE to form an exact finite system. Crucially, any recipe-based method can be viewed as adding a small equation correction, enabling us to compare multiple schemes independently of the solver. 

This talk also addresses a paradox: PDEs often admit infinitely many solutions, but finite systems produce only a finite set. When we include a “small” correction, the missing solutions are effectively hidden. I will discuss how tau methods frame this perspective and outline proposals for systematically studying and optimising various residuals.

Cryptocurrencies such as Bitcoin have only recently become a significant part of the financial landscape. Many billions of dollars are now traded daily on limit order-book markets such as Binance, and these are probably among the most open, liquid and transparent markets there are. They therefore make an interesting platform from which to investigate myriad questions to do with market microstructure. I shall talk about a few of these, including live-trading experiments to investigate the difference between on-paper strategy analysis (typical in the academic literature) and actual trading outcomes. I shall also mention very recent work on the new Hyperliquid exchange which runs on a blockchain basis, showing how to use this architecture to obtain datasets of an unprecendented level of granularity. This is joint work with Jakob Albers, Mihai Cucuringu and Alex Shestopaloff.

Parameters in mathematical models are often impossible to determine fully or accurately, and are hence subject to uncertainty. By modelling the input parameters as stochastic processes, it is possible to quantify the uncertainty in the model outputs. 

In this talk, we employ the multilevel Monte Carlo (MLMC) method to compute expected values of quantities of interest related to partial differential equations with random coefficients. We make use of the circulant embedding method for sampling from the coefficient, and to further improve the computational complexity of the MLMC estimator, we devise and implement the smoothing technique integrated into the circulant embedding method. This allows to choose the coarsest mesh on the  first level of MLMC independently of the correlation length of the covariance function of the random  field, leading to considerable savings in computational cost.

 

Please note; this talk is hosted by Rutherford Appleton Laboratory, Harwell Campus, Didcot, OX11 0QX

Many matrices that arise in scientific computing and in data science have internal structure that can be exploited to accelerate computations. The focus in this talk will be on matrices that are either of low rank, or can be tessellated into a collection of subblocks that are either of low rank or are of small size. We will describe how matrices of this nature arise in the context of fast algorithms for solving PDEs and integral equations, and also in handling "kernel matrices" from computational statistics. A particular focus will be on randomized algorithms for obtaining data sparse representations of such matrices.

At the end of the talk, we will explore an unorthodox technique for discretizing elliptic PDEs that was designed specifically to play well with fast algorithms for dense structured matrices.

Differentiable programming is the underpinning technology for the AI revolution. It allows neural networks to be programmed in very high level user code while still achieving very high performance for both the evaluation of the network and, crucially, its derivatives. The Firedrake project applies exactly the same concepts to the simulation of physical phenomena modelled with partial differential equations (PDEs). By exploiting the high level mathematical abstraction offered by the finite element method, users are able to write mathematical operators for the problem they wish to solve in Python. The high performance parallel implementations of these operators are then automatically generated, and composed with the PETSc solver framework to solve the resulting PDE. However, because the symbolic differential operators are available as code, it is possible to reason symbolically about them before the numerical evaluation. In particular, the operators can be differentiated with respect to their inputs, and the resulting derivative operators composed in forward or reverse order. This creates a differentiable programming paradigm congruent with (and compatible with) machine learning frameworks such as Pytorch and JAX. 

In this presentation, David Ham will present Firedrake in the context of differentiable programming, and show how this enables productivity, capability and performance to be combined in a unique way. I will also touch on the mechanism that enables Firedrake to be coupled with Pytorch and JAX.

Please note this talk will take place at Rutherford Appleton Laboratory, Harwell Campus, Didcot. 

In this talk I will review some recent results concerning the qualitative behaviour of symplectic integrators applied to Hamiltonian PDEs, such as the nonlinear wave equation or Schrödinger equations. 

Additionally, I will discuss the problem of numerical resonances, the existence of modified energy and the existence and stability of numerical solitons over long times. 

These are works with B. Grébert, D. Bambusi, G. Maierhofer and K. Schratz. 

Can one recover a matrix from only matrix-vector products? If so, how many are needed? We will consider the matrix recovery problem for the class of hierarchical rank-structured matrices. This problem arises in scientific machine learning, where one wishes to recover the solution operator of a PDE from only input-output pairs of forcing terms and solutions. Peeling algorithms are the canonical method for recovering a hierarchical matrix from matrix-vector products, however their recursive nature poses a potential stability issue which may deteriorate the overall quality of the approximation. Our work resolves the open question of the stability of peeling. We introduce a robust version of peeling and prove that it achieves low error with respect to the best possible hierarchical approximation to any matrix, allowing us to analyze the performance of the algorithm on general matrices, as opposed to exactly hierarchical ones. This analysis relies on theory for low-rank approximation, as well as the surprising result that the Generalized Nystrom method is more accurate than the randomized SVD algorithm in this setting. 

The Landau equation stands as one of the fundamental equations in kinetic theory and plays a key role in plasma physics. However, computing it presents significant challenges due to the complexity of the Landau operator,  the dimensionality, and the need to preserve the physical properties of the solution. In this presentation, I will introduce deep learning assisted particle methods aimed at addressing some of these challenges. These methods combine the benefits of traditional structure-preserving techniques with the approximation power of neural networks, aiming to handle high dimensional problems with minimal training. 

 Many optimal control, estimation and design problems can be formulated as so-called dynamic optimization problems, which are optimization problems with differential equations and other constraints. State-of-the-art methods based on collocation, which enforce the differential equations at only a finite set of points, can struggle to solve certain dynamic optimization problems, such as those with high-index differential algebraic equations, consistent overdetermined constraints or problems with singular arcs. We show how numerical methods based on integrating the differential equation residuals can be used to solve dynamic optimization problems where collocation methods fail. Furthermore, we show that integrated residual methods can be computationally more efficient than direct collocation.

This seminar takes place at RAL (Rutherford Appleton Lab). 

We introduce a novel approach to global optimization  via continuous-time dynamic programming and Hamilton-Jacobi-Bellman (HJB) PDEs. For non-convex, non-smooth objective functions,  we reformulate global optimization as an infinite horizon, optimal asymptotic stabilization control problem. The solution to the associated HJB PDE provides a value function which corresponds to a (quasi)convexification of the original objective.  Using the gradient of the value function, we obtain a  feedback law driving any initial guess towards the global optimizer without requiring derivatives of the original objective. We then demonstrate that this HJB control law can be integrated into other global optimization frameworks to improve its performance and robustness. 

Deflation is a technique to remove a solution to a problem so that other solutions to this problem can subsequently be found. The most prominent instance is deflation we see in eigenvalue solvers, but recent interest has been in deflation of rootfinding problems from nonlinear PDEs with many isolated solutions (spearheaded by Farrell and collaborators). 

In this talk I’ll show you recent results on deflation techniques for optimisation algorithms with many local minima, focusing on the Gauss—Newton algorithm for nonlinear least squares problems.  I will demonstrate advantages of these techniques instead of the more obvious approach of applying deflated Newton’s method to the first order optimality conditions and present some proofs that these algorithms will avoid the deflated solutions. Along the way we will see an interesting generalisation of Woodbury’s formula to least squares problems, something that should be more well known in Numerical Linear Algebra (joint work with Güttel, Nakatsukasa and Bloor Riley).

Main preprint: https://arxiv.org/abs/2409.14438. 

WoodburyLS preprint: https://arxiv.org/abs/2406.15120

There is a mystery at the heart of operator learning: how can one recover a non-self-adjoint operator from data without probing the adjoint? Current practical approaches suggest that one can accurately recover an operator while only using data generated by the forward action of the operator without access to the adjoint. However, naively, it seems essential to sample the action of the adjoint for learning time-dependent PDEs. 

In this talk, we will first explore connections with low-rank matrix recovery problems in numerical linear algebra. Then, we will show that one can approximate a family of non-self-adjoint infinite-dimensional compact operators via projection onto a Fourier basis without querying the adjoint.

We present an exponential-integrator-based multi-index Monte Carlo (MIMC) method for the weak approximation of mild solutions to semilinear stochastic partial differential equations (SPDEs). Theoretical results on multi-index coupled solutions of the SPDE are provided, demonstrating their stability and the satisfaction of multiplicative error estimates. Leveraging this theory, we develop a tractable MIMC algorithm. Numerical experiments illustrate that MIMC outperforms alternative approaches, such as multilevel Monte Carlo, particularly in low-regularity settings.

We are now able to solve many partial differential equation problems that were well beyond reach when I started in academia. Some of this success is due to computer hardware but much is due to algorithmic advances. 

I will give a personal perspective of the development of computational methodology in this area over my career thus far. 

Large Language Models (LLMs) have demonstrated impressive achievements. However, recent research has shown that their deeper layers often contribute minimally, with effectiveness diminishing as layer depth increases. This pattern presents significant opportunities for model compression. 

In the first part of this seminar, we will explore how this phenomenon can be harnessed to improve the efficiency of LLM compression and parameter-efficient fine-tuning. Despite these opportunities, the underutilization of deeper layers leads to inefficiencies, wasting resources that could be better used to enhance model performance. 

The second part of the talk will address the root cause of this ineffectiveness in deeper layers and propose a solution. We identify the issue as stemming from the prevalent use of Pre-Layer Normalization (Pre-LN) and introduce Mix-Layer Normalization (Mix-LN) with combined Pre-LN and Post-LN as a new approach to mitigate this training deficiency.

Kinetic and mean-field equations are central to understanding complex systems across fields such as classical physics, engineering, and the socio-economic sciences. Efficiently solving these equations remains a significant challenge due to their high dimensionality and the need to preserve key structural properties of the models. 

In this talk, we will focus on recent advancements in deterministic numerical methods, which provide an alternative to particle-based approaches (such as Monte Carlo or particle-in-cell methods) by avoiding stochastic fluctuations and offering higher accuracy. We will discuss strategies for designing these methods to reduce computational complexity while preserving fundamental physical properties and maintaining efficiency in stiff regimes. 
Special attention will be given to the role of these methods in addressing multi-scale problems in rarefied gas dynamics and plasma physics. Time permitting, we will also touch on emerging techniques for uncertainty quantification in these systems.

AI tools like ChatGPT, Microsoft Copilot, GitHub Copilot, Claude and even older AI-enabled tools like Grammarly and MS Word, are becoming an everyday part of our research environment.  This last-minute opening up of a seminar slot due to the unfortunate illness of our intended speaker (who will hopefully re-schedule for next term) gives us an opportunity to discuss what this means for us as researchers; what are good helpful uses of AI, and are there uses of AI which we might view as inappropriate?  Please come ready to participate with examples of things which you have done yourselves with AI tools.

In this seminar I will begin by giving an overview of some problems in stochastic simulation and uncertainty quantification. I will then outline the Multilevel Monte Carlo for situations in which accurate simulations are very costly, but it is possible to perform much cheaper, less accurate simulations.  Inspired by the multigrid method, it is possible to use a combination of these to achieve the desired overall accuracy at a much lower cost.

Quantum computers are becoming a reality and current generations of machines are already well beyond the 50-qubit frontier. However, hardware imperfections still overwhelm these devices and it is generally believed the fault-tolerant, error-corrected systems will not be within reach in the near term: a single logical qubit needs to be encoded into potentially thousands of physical qubits which is prohibitive.

Due to limited resources, in the near term, hybrid quantum-classical protocols are the most promising candidates for achieving early quantum advantage and these need to resort to quantum error mitigation techniques. I will explain the basic concepts and introduce hybrid quantum-classical protocols are the most promising candidates for achieving early quantum advantage. These have the potential to solve real-world problems---including optimisation or ground-state search---but they suffer from a large number of circuit repetitions required to extract information from the quantum state. I will finally identify the most likely areas where quantum computers may deliver a true advantage in the near term.

Bálint Koczor

Associate Professor in Quantum Information Theory

Mathematical Institute, University of Oxford

webpage

Following the 2012 breakthrough in deep learning for classification and visions problems, the last decade has seen tremendous raise of interest in machine learning in a wider mathematical research community from foundational research through field specific analysis to applications. 

As data is at the core of any inverse problem, it was a natural direction for the field to investigate how machine learning could aid various aspects of inversion yielding numerous approaches from somewhat ad-hoc but very effective like learned unrolled methods to provably convergent learned regularisers with everything in between. In this talk I will review some on these developments through a lens of the research of our group.