Machine Learning and Data Science Seminar

Inverse problems involve reconstructing unknown physical quantities from indirect measurements. They appear in various fields, including medical imaging (e.g., MRI, Ultrasound, CT), material sciences and molecular biology (e.g., electron microscopy), as well as remote sensing just to name a few examples. While deep neural networks are currently able to achieve state-of-the-art performance in many imaging tasks, in this talk we argue that  many inverse imaging problems cannot be solved convincingly using a black-box solution. Instead, they require a well-crafted combination of computational tools taking the underlying signal, the physical constraints and acquisition characteristics into account.

In the first part of the talk, we introduce INDigo+, a novel INN-guided probabilistic diffusion algorithm for arbitrary image restoration tasks. INDigo+ combines the perfect reconstruction property of invertible neural networks (INNs) with the strong generative capabilities of pre-trained diffusion models. Specifically, we leverage the invertibility of the network to condition the diffusion process and in this way we generate high quality restored images consistent with the measurements.

In the second part of the talk, we discuss the unfolding techniques which is an approach that allows embedding priors and models in the neural network architecture. In this context we discuss the problem of monitoring the dynamics of large populations of neurons over a large area of the brain. Light-field microscopy (LFM), a type of scanless microscopy, is a particularly attractive candidate for high-speed three-dimensional (3D) imaging which is needed for monitoring neural activity. We review fundamental aspects of LFM and then present computational methods based on deep learning for neuron localization and activity estimation from light-field data.
Finally, we look at the multi-modal case and present an application in art investigation. Often X-ray images of Old Master paintings contain information of the visible painting and of concealed sub-surface design, we therefore introduce a model-based neural network capable of separating from the “mixed X-ray”  the X-ray image of the visible painting and the X-ray of the concealed design.

This is joint work with  A. Foust, P. Song, C. Howe, H. Verinaz, J. Huang, Di You and Y. Su from Imperial College London, M. Rodrigues and W. Pu from University College London, I. Daubechies from Duke University, Barak Sober from the Hebrew University of Jerusalem and C. Higgitt and N. Daly from The National Gallery in London.

Cost-sensitive loss functions are crucial in many real-world prediction problems, where different types of errors are penalized differently; for example, in medical diagnosis, a false negative prediction can lead to worse consequences than a false positive prediction. However, traditional learning theory has mostly focused on the symmetric zero-one loss, letting cost-sensitive losses largely unaddressed. In this work, we extend the celebrated theory of boosting to incorporate both cost-sensitive and multi-objective losses. Cost-sensitive losses assign costs to the entries of a confusion matrix, and are used to control the sum of prediction errors accounting for the cost of each error type. Multi-objective losses, on the other hand, simultaneously track multiple cost-sensitive losses, and are useful when the goal is to satisfy several criteria at once (e.g., minimizing false positives while keeping false negatives below a critical threshold). We develop a comprehensive theory of cost-sensitive and multi-objective boosting, providing a taxonomy of weak learning guarantees that distinguishes which guarantees are trivial (i.e., can always be achieved), which ones are boostable (i.e., imply strong learning), and which ones are intermediate, implying non-trivial yet not arbitrarily accurate learning. For binary classification, we establish a dichotomy: a weak learning guarantee is either trivial or boostable. In the multiclass setting, we describe a more intricate landscape of intermediate weak learning guarantees. Our characterization relies on a geometric interpretation of boosting, revealing a surprising equivalence between cost-sensitive and multi-objective losses.

Attention-based models, such as Transformer, excel across various tasks but lack a comprehensive theoretical understanding, especially regarding token-wise sparsity and internal linear representations. To address this gap, we introduce the single-location regression task, where only one token in a sequence determines the output, and its position is a latent random variable, retrievable via a linear projection of the input. To solve this task, we propose a dedicated predictor, which turns out to be a simplified version of a non-linear self-attention layer. We study its theoretical properties, by showing its asymptotic Bayes optimality and analyzing its training dynamics. In particular, despite the non-convex nature of the problem, the predictor effectively learns the underlying structure. This work highlights the capacity of attention mechanisms to handle sparse token information and internal linear structures.

This is a joint work with Pierre Marion, Gérard Biau and Raphaël Berthier

TBA