Bridging Rough Paths and Deep Learning: New Frontiers

Overview

This workshop showcases the latest advancements in integrating rough path theory with rapidly evolving field of deep learning. A central theme will be the signature of a path—an important mathematical tool for capturing the essence of complex, irregular data—and its broad applications across various domains. The program will delve into how these techniques can be leveraged to enhance models in control system design, sequence modelling, and time series generation. Additionally, this workshop will also feature an introduction of Kolmogorov-Arnold networks and demonstrate their effectiveness as a versatile framework for time series analysis. Attendees will gain a deeper understanding of how rough path theory and signature-based methods can provide solutions to challenges in data science.

Logistics

This workshop will take place in-person on Monday 18th November at the Alan Turing Institute (British Library, 96 Euston Rd., London NW1 2DB).

Any queries should be directed to Lingyi Yang (@email).

Registration Form

To register, please complete the following Google Form by Friday 1st November:

https://docs.google.com/forms/d/1zmjWmhYpC3GEPL3546yEkFxZud40IMhXh3YN4OaGjdc/

Timetable

The final timetable is as follows:

11:15-11:30 Welcome
11:30-12:15 On the role of the signature in control design (Anna Scampicchio, ETH Zurich)
12:15-13:00 Score-Based Diffusion for Generating Paths via Signature Embeddings (Barbora Barancikova, Imperial College)
13:00-14:00 Lunch
14:00-14:45 Unravelling Kolmogorov Arnold Networks (Hugo Inzirillo, CREST, Institut Polytechnique de Paris)
14:45-15:30 Time Series Analysis with Signature-Weighted Kolmogorov-Arnold Networks (Rémi Genet, Paris Dauphine University)
15:30-16:15 Rough Transformers: Lightweight Continuous-Time Sequence Modelling with Path Signatures (Fernando Moreno-Pino, OMI, University of Oxford)

Abstracts of talks

On the role of the signature in control design

One of the cornerstones of technology consists in performing control design: i.e., given a dynamical system, develop an algorithm that acts on the inputs of the system to drive it to a desired behaviour. Control theory is an established field, encompassing a wide range of methods that have been applied to, e.g., robotics, power systems, chemical plants and biomedicine — however, controlling unknown, nonlinear dynamical systems is still an open problem. Focusing on predictive methods, which are one of the most successful techniques in modern industry, we will show that signature-based models can play a key role in tackling such a defying challenge. Specifically, we present a novel control strategy that leverages the signature and its universal approximation property, showing its potential in view of theoretically sound and practically effective predictive control of nonlinear systems with unknown dynamics.

Score-Based Diffusion for Generating Paths via Signature Embeddings (Barbora Barancikova, Imperial College)

Score-based diffusion models have recently emerged as state-of-the-art generative models for a variety of data modalities. Nonetheless, it remains unclear how to adapt these models to generate long multivariate time series. Viewing a time series as the discretization of an underlying continuous process, this talk will introduce SigDiffusion, a novel diffusion model operating on log-signature embeddings of the data. The forward and backward processes gradually perturb and denoise log-signatures preserving their algebraic structure. To efficiently recover a signal from its log-signature, new closed-form inversion formulae will be introduced expressing the coefficients obtained by expanding the signal in a given basis (e.g. Fourier or orthogonal polynomials) as explicit polynomial functions of the log-signature. Combining SigDiffusion with these inversion formulae results in highly realistic time series generation, competitive with the current state-of-the-art on various datasets of synthetic and real-world examples.

Unravelling Kolmogorov Arnold Networks (Hugo Inzirillo, CREST, Institut Polytechnique de Paris)

This presentation introduces Kolmogorov-Arnold Networks (KANs) from Ziming Liu et al., a recent development in machine learning that offers an alternative to standard Multilayer Perceptrons (MLPs). We will examine the theoretical foundations of KANs, their operational principles, and the mathematical basis for their efficacy. The discussion will include a comparative analysis of KANs and traditional neural network approaches, highlighting specific conditions under which KANs demonstrate superior performance. The latter part of the talk will address the adaptation of KANs for time series analysis. We will present our work on the Temporal Kolmogorov-Arnold Network (TKAN), which uses established techniques in sequence modeling, including recurrent architectures and memory mechanisms.

Time Series Analysis with Signature-Weighted Kolmogorov-Arnold Networks (Rémi Genet, Paris Dauphine University)

This presentation introduces the Signature-Weighted Kolmogorov-Arnold Network (SigKAN), a novel approach to time series analysis that combines KANs with path signatures. We will discuss the theoretical framework of SigKAN, focusing on its use of path signatures to capture temporal information without using standard recurrent architecture. The talk will detail how SigKAN employs a weighting strategy, utilizing KANs within a gated residual network applied to path signatures. The presentation will include a discussion of the method's efficacy compared to current state-of-the-art techniques in time series analysis, supported by empirical results on market volume prediction as well as volatility forecasting.

Rough Transformers: Lightweight Continuous-Time Sequence Modelling with Path Signatures (Fernando Moreno-Pino, OMI, University of Oxford)

Time-series data in real-world settings typically exhibit long-range dependencies and are observed at non-uniform intervals. In these settings, traditional sequence-based recurrent models struggle. To overcome this, researchers often replace recurrent architectures with Neural ODE-based models to account for irregularly sampled data and use Transformer-based architectures to account for long-range dependencies. Despite the success of these two approaches, both incur very high computational costs for input sequences of even moderate length. To address this challenge, we introduce the Rough Transformer, a variation of the Transformer model that operates on continuous-time representations of input sequences and incurs significantly lower computational costs. In particular, we propose multi-view signature attention, which uses path signatures to augment vanilla attention and to capture both local and global (multi-scale) dependencies in the input data, while remaining robust to changes in the sequence length and sampling frequency and yielding improved spatial processing. We find that, on a variety of time-series-related tasks, Rough Transformers consistently outperform their vanilla attention counterparts while obtaining the representational benefits of Neural ODE-based models, all at a fraction of the computational time and memory resources.

Speaker Bios

Anna Scampicchio is a postdoctoral researcher at the Institute for Dynamic Systems and Control, ETH Zurich. She received in 2015 the Bachelor degree in Information Engineering and in 2017 the Masters degree in Automation Engineering, both cum laude, from the University of Padova. In 2017 she was awarded with the Roberto Rocca scholarship for her career during the Masters Degree. She held a visiting position at the Department of Applied Mathematics of University of Washington, Seattle, in 2019. In 2021 she received the Ph.D. in Information Engineering from the University of Padova. Her research interests lie at the interplay among system identification, machine learning and control design.

Barbora is a second-year PhD student at the AI4Health Doctoral Training Centre at Imperial College London, supervised by Dr. Cristopher Salvi. Her research combines rough path theory with machine learning for healthcare applications. Currently, she focuses on diffusion models for synthesizing Lie groups and solving linear inverse problems on paths. Barbora holds a BSc in Mathematics and Computer Science from the University of Glasgow.

Hugo Inzirillo is a researcher and lecturer in machine learning and financial econometrics, currently teaching at Université Paris-Dauphine. With a Ph.D. from CREST, Institut Polytechnique de Paris, Hugo’s work bridges deep learning and econometrics, focusing on financial modeling and cryptocurrency. Hugo co-founded Vulturi, a tech company specializing in quantitative research and software development.

Rémi Genet is a researcher and lecturer in machine learning and computer science at Université Paris-Dauphine. He is currently pursuing his PhD in Finance, sponsored by Aplo, where his research focuses on deep learning techniques for forecasting and optimal trading execution in cryptocurrency markets. His academic work combines expertise in computer science, quantitative finance, and digital asset markets.

Dr. Fernando Moreno-Pino is a Postdoctoral Researcher at the Oxford-Man Institute of Quantitative Finance, University of Oxford. His research focuses on the intersection of Deep Learning, Probabilistic Machine Learning, and Quantitative Finance. Previously, he earned his PhD in Probabilistic Machine Learning and Deep Learning from the Signal Processing and Learning Group at Universidad Carlos III de Madrid, Spain. His research included working with heterogeneous models for high-dimensional data, exploring probabilistic machine learning methods, integrating signal processing techniques into deep-learning architectures, developing DNN methodologies for time-series modeling and forecasting, and applying ML techniques to quantitative finance-related problems.

About the event

This event has been made possible through the funding and support provided by The Alan Turing Institute for the Rough Paths Interest Group (RPIG). We run regular online seminars on a wide range of topics intersecting with rough path theory. If you are interested in learning more about the RPIG, including how to sign up, please visit:

https://www.turing.ac.uk/research/interest-groups/rough-paths-machine-learning-sequential-data