Past DataSig Seminar

23 July 2020
16:00
Abstract

The random initialisation of Artificial Neural Networks (ANN) allows one to describe, in the functional space, the limit of the evolution of ANN when their width tends towards infinity. Within this limit, an ANN is initially a Gaussian process and follows, during learning, a gradient descent convoluted by a kernel called the Neural Tangent Kernel.

Connecting neural networks to the well-established theory of kernel methods allows us to understand the dynamics of neural networks, their generalization capability. In practice, it helps to select appropriate architectural features of the network to be trained. In addition, it provides new tools to address the finite size setting.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

9 July 2020
16:00
Horatio Boedihardjo
Abstract

The concept of path signatures has been widely used in several areas of pure mathematics including in applications to data science. However, we remain unable to answer even the most basic questions about it. For instance, how to fully characterise the set of (untruncated) signatures of bounded variation paths? Can certain norms on signatures be related to the length of a path, like in Fourier isometry? In this talk, we will review some known results, explain the open problems and discuss their difficulties.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

25 June 2020
16:00
to
18:00
Abstract

We present a method for obtaining approximate solutions to the problem of optimal execution, based on a signature method. The framework is general, only requiring that the price process is a geometric rough path and the price impact function is a continuous function of the trading speed. Following an approximation of the optimisation problem, we are able to calculate an optimal solution for the trading speed in the space of linear functions on a truncation of the signature of the price process. We provide strong numerical evidence illustrating the accuracy and flexibility of the approach. Our numerical investigation both examines cases where exact solutions are known, demonstrating that the method accurately approximates these solutions, and models where exact solutions are not known. In the latter case, we obtain favourable comparisons with standard execution strategies.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

4 June 2020
14:00
Abstract

The heart of modern machine learning (ML) is the approximation of high dimensional functions. Traditional approaches, such as approximation by piecewise polynomials, wavelets, or other linear combinations of fixed basis functions, suffer from the curse of dimensionality (CoD). We will present a mathematical perspective of ML, focusing on the issue of CoD. We will discuss three major issues: approximation theory and error analysis of modern ML models, dynamics and qualitative behavior of gradient descent algorithms, and ML from a continuous viewpoint. We will see that at the continuous level, ML can be formulated as a series of reasonably nice variational and PDE-like problems. Modern ML models/algorithms, such as the random feature and two-layer and residual neural network models, can all be viewed as special discretizations of such continuous problems. We will also present a framework that is suited for analyzing ML models and algorithms in high dimension, and present results that are free of CoD. Finally, we will discuss the fundamental reasons that are responsible for the success of modern ML, as well as the subtleties and mysteries that still remain to be understood.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

14 May 2020
16:00
Abstract

Abstract: Gradient descent is known to converge quickly for convex objective functions, but it can be trapped at local minimums. On the other hand, Langevin dynamic can explore the state space and find global minimums, but in order to give accurate estimates, it needs to run with small discretization step size and weak stochastic force, which in general slows down its convergence. This work shows that these two algorithms can “collaborate” through a simple exchange mechanism, in which they swap their current positions if Langevin dynamic yields a lower objective function. This idea can be seen as the singular limit of the replica-exchange technique from the sampling literature. We show that this new algorithm converges to the global minimum linearly with high probability, assuming the objective function is strongly convex in a neighbourhood of the unique global minimum. By replacing gradients with stochastic gradients, and adding a proper threshold to the exchange mechanism, our algorithm can also be used in online settings. This is joint work with Xin Tong at National University of Singapore.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

7 May 2020
16:00
James Michael Leahy

Further Information: 

[Meeting URL removed for security reasons]

Abstract

We introduce constrained variational principles for fluid dynamics on rough paths. The advection of the fluid is constrained to be the sum of a vector field which represents coarse-scale motion and a rough (in time) vector field which parametrizes fine-scale motion. The rough vector field is regarded as fixed and the rough partial differential equation for the coarse-scale velocity is derived as a consequence of being a critical point of the action functional.

 

The action functional is perturbative in the sense that if the rough vector f ield is set to zero, then the corresponding variational principle agrees with the reduced (to the vector fields) Euler-Poincare variational principle introduced in Holm, Marsden and Ratiu (1998). More precisely, the Lagrangian in the action functional encodes the physics of the fluid and is a function of only the coarse-scale velocity. 

 

By parametrizing the fine-scales of fluid motion with a rough vector field, we preserve the pathwise nature of deterministic fluid dynamics and establish a flexible framework for stochastic parametrization schemes. The main benefit afforded by our approach is that the system of rough partial differential equations we derive satisfy essential conservation laws, including Kelvin’s circulation theorem. This talk is based on recent joint work with Dan Crisan, Darryl Holm, and Torstein Nilssen.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

30 April 2020
16:45
to
18:00
Weijun Xu

Further Information: 

[Meeting URL removed for security reasons]

 

Abstract

Abstract: The signature of a path is a sequence of iterated coordinate integrals along the path. We aim at reconstructing a path from its signature. In the special case of lattice paths, one can obtain exact recovery based on a simple algebraic observation. For general continuously differentiable curves, we develop an explicit procedure that allows to reconstruct the path via piecewise linear approximations. The errors in the approximation can be quantified in terms of the level of signature used and modulus of continuity of the derivative of the path. The main idea is philosophically close to that for the lattice paths, and this procedure could be viewed as a significant generalisation. A key ingredient is the use of a symmetrisation procedure that separates the behaviour of the path at small and large scales.We will also discuss possible simplifications and improvements that may be potentially significant. Based on joint works with Terry Lyons, and also with Jiawei Chang, Nick Duffield and Hao Ni.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

30 April 2020
16:00
Adeline Fermanian

Further Information: 

Abstract

Abstract: Sequential and temporal data arise in many fields of research, such as quantitative finance, medicine, or computer vision. We will be concerned with a novel approach for sequential learning, called the signature method, and rooted in rough path theory. Its basic principle is to represent multidimensional paths by a graded feature set of their iterated integrals, called the signature. On the one hand, this approach relies critically on an embedding principle, which consists in representing discretely sampled data as paths, i.e., functions from [0,1] to R^d. We investigate the influence of embeddings on prediction accuracy with an in-depth study of three recent and challenging datasets. We show that a specific embedding, called lead-lag, is systematically better, whatever the dataset or algorithm used. On the other hand, in order to combine signatures with machine learning algorithms, it is necessary to truncate these infinite series. Therefore, we define an estimator of the truncation order and prove its convergence in the expected signature model.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).