Mon, 26 Feb 2024

14:00 - 15:00
Lecture Room 3

Fantastic Sparse Neural Networks and Where to Find Them

Dr Shiwei Liu
(Maths Institute University of Oxford)
Abstract

Sparse neural networks, where a substantial portion of the components are eliminated, have widely shown their versatility in model compression, robustness improvement, and overfitting mitigation. However, traditional methods for obtaining such sparse networks usually involve a fully pre-trained, dense model. As foundation models become prevailing, the cost of this pre-training step can be prohibitive. On the other hand, training intrinsic sparse neural networks from scratch usually leads to inferior performance compared to their dense counterpart. 

 

In this talk, I will present a series of approaches to obtain such fantastic sparse neural networks by training from scratch without the need for any dense pre-training steps, including dynamic sparse training, static sparse with random pruning, and only masking no training. First, I will introduce the concept of in-time over-parameterization (ITOP) (ICML2021) which enables training sparse neural networks from scratch (commonly known as sparse training) to attain the full accuracy of dense models. By dynamically exploring new sparse topologies during training, we avoid the costly necessity of pre-training and re-training, requiring only a single training run to obtain strong sparse neural networks. Secondly, ITOP involves additional overhead due to the frequent change in sparse topology. Our following work (ICLR2022) demonstrates that even a naïve, static sparse network produced by random pruning can be trained to achieve dense model performance as long as our model is relatively larger. Moreover, I will further discuss that we can continue to push the extreme of training efficiency by only learning masks at initialization without any weight updates, addressing the over-smoothing challenge in building deep graph neural networks (LoG2022).

Fri, 06 Feb 2015
13:00
L6

Path-dependent PDE and Backward SDE

Shige Peng
(Maths Institute University of Oxford)
Abstract

In this talk we present a new type of Soblev norm defined in the space of functions of continuous paths. Under the Wiener probability measure the corresponding norm is suitable to prove the existence and uniqueness for a large type of system of path dependent quasi-linear parabolic partial differential equations (PPDE). We have establish 1-1 correspondence between this new type of PPDE and the classical backward SDE (BSDE). For fully nonlinear PPDEs, the corresponding Sobolev norm is under a sublinear expectation called G-expectation, in the place of Wiener expectation. The canonical process becomes a new type of nonlinear Brownian motion called G-Brownian motion. A similar 1-1 correspondence has been established. We can then apply the recent results of existence, uniqueness and principle of comparison for BSDE driven by G-Brownian motion to obtain the same result for the PPDE.

Fri, 06 Mar 2015
13:00
L6

Community structure in temporal multilayer networks, and its application to financial correlation networks

Marya Bazzi
(Maths Institute University of Oxford)
Abstract

Networks are a convenient way to represent systems of interacting entities. Many networks contain "communities" of nodes that are more densely connected to each other than to nodes in the rest of the network.

Most methods for detecting communities are designed for static networks. However, in many applications, entities and/or interactions between entities evolve in time.

We investigate "multilayer modularity maximization", a method for detecting communities in temporal networks. The main difference between this method and most previous methods for detecting communities in temporal networks is that communities identified in one temporal snapshot are not independent of connectivity patterns in other snapshots.  We show how the resulting partition reflects a trade-off between static community structure within snapshots and persistence of community structure between snapshots. As a focal example in our numerical experiments, we study time-dependent financial asset correlation networks.

Fri, 27 Feb 2015
13:00
L6

No arbitrage in progressive enlargement of filtration setting

Anna Aksamit
(Maths Institute University of Oxford)
Abstract

Our study addresses the question of how an arbitrage-free semimartingale model is affected when the knowledge about a random time is added. Precisely, we focus on the No-Unbounded-Profit-with-Bounded-Risk condition, which is also known in the literature as the first kind of no arbitrage. In the general semimartingale setting, we provide a sufficient condition on the random time and price process for which the no arbitrage is preserved under filtration enlargement. Moreover we study the condition on the random time for which the no arbitrage is preserved for any process. This talk is based on a joint work with Tahir Choulli, Jun Deng and Monique Jeanblanc.

Fri, 13 Feb 2015
13:00
L6

Zubov's method for controlled diffusions with state constraints

Athena Picarelli
(Maths Institute University of Oxford)
Abstract

We consider a controlled stochastic system in presence of state-constraints. Under the assumption of exponential stabilizability of the system near a target set, we aim to characterize the set of points which can be asymptotically driven by an admissible control to the target with positive probability. We show that this set can be characterized as a level set of the optimal value function of a suitable unconstrained optimal control problem which in turn is the unique viscosity solution of a second order PDE which can thus be interpreted as a generalized Zubov equation.

Fri, 30 Jan 2015
13:00
L6

Optimal investment and price dependence in a semi-static market

Pietro Siorpaes
(Maths Institute University of Oxford)
Abstract

We study the problem of maximizing expected utility from terminal wealth in a semi-static market composed of derivative securities, which we assume can be traded only at time zero, and of stocks, which can be
traded continuously in time and are modeled as locally-bounded semi-martingales.

Using a general utility function defined on the positive real line, we first study existence and uniqueness of the solution, and then we consider the dependence of the outputs of the utility maximization problem on the price of the derivatives, investigating not only stability but also differentiability, monotonicity, convexity and limiting properties.

Subscribe to Maths Institute University of Oxford