Machine Learning and Data Science Seminar

Professor Hong Yan is going to talk about; 'Efficient Machine Learning Methods Based on Matrix and Tensor CUR Decomposition'

The training of large deep neural networks can be very expensive in term of long computing time, which requires costly equipment and high energy consumption. Vector, matrix and tensor multiplications are among the most time-consuming tasks. In this talk, a CUR decomposition-based method will be presented to reduce the computational complexity. Our basic strategy is to approximate a large low-rank matrix as a product of three much smaller matrices. Then multiplications with these matrices become very efficient. This can increase the speed of the self-attention operations in transformers significantly. Another reason for the long computing time in neural network training is that a large training dataset is needed. Our research group has recently studied a pattern matching method that requires only a small number of training samples. To achieve high accuracy and reliability, we use high-order pattern matching based on hypergraph and tensor models. A difficulty with this approach is that the high-order compatibility tensor is very large. Employing tensor CUR decomposition, we have tackled this problem and proposed an efficient pattern matching algorithm. Based on pattern matching, we have recently developed an intelligent indoor positioning and navigation system, which will be demonstrated in this talk.

Learning governing equations from data is a central problem in scientific machine learning. Given noisy and incomplete observations of a physical process, the goal is to recover the underlying law. This is often approached by fitting a neural network or a dictionary of candidate terms to the observed dynamics. A good fit, however, does not determine identifiability of the law, its approximability by what is computed, physical consistency, or interpretability as a formula. These depend largely on how the learning problem is posed.

This talk formulates the learning task as a regularized inverse problem in function space and studies how these properties can be established. Over a parameterized class of candidate laws, one minimizes a model residual, a data misfit, and a regularizer, with the class and the regularizer as the design choices. With a sufficiently expressive class and suitable regularization, one obtains a regularization-based notion of identifiability, under which the regularization-minimizing law consistent with the data is unique. As the approximation scale grows and the regularization parameters are chosen accordingly, minimizers of the parameterized problem converge to that law. For classes carrying the structural constraints of the underlying physical model, the learned models are physically consistent and well-posed by construction at every finite approximation scale. For symbolic networks built from rational building blocks, the recovered law is a readable formula. The emphasis of this talk is analytical, and first numerical experiments illustrate the recovery in practice. 

Overall, the results show that identifiability, consistency, and interpretability are not competing objectives, but can be unified in one analysis-driven framework.

TBA

TBA

TBA

Professor Pereyra will talk about; 'Physics-informed deep generative models: Applications to computational sensing'

This talk introduces a novel mathematical and computational framework for constructing high-dimensional Bayesian inversion methods that leverage state-of-the-art generative denoising diffusion models as highly informative priors. A central innovation is the construction of physics-informed generative models using Langevin diffusion processes and Markov chain Monte Carlo (MCMC) sampling techniques to develop stochastic neural network architectures capable of near-exact sampling. The obtained networks are modular and composed of interpretable layers that are directly related to statistical image priors and data likelihoods derived from forward observation models. The layers encoding the data likelihood function are designed for flexibility, enabling scene and instrument model parameters to be specified at inference time and seamlessly integrated with pre-trained foundational generative priors. To achieve high computational efficiency, we employ adversarial model distillation, which yields excellent sampling performance with as few as four Markov chain Monte Carlo steps, even in problems exceeding one million dimensions. Our approach is validated through non-asymptotic convergence analysis and extensive numerical experiments in computational image and video restoration. We conclude by discussing unsupervised training strategies that allow the models to be fine-tuned directly from measurement data, thereby bypassing the need for clean reference data.

The talk is based on recent work in physics-informed generative AI for Bayesian imaging: https://arxiv.org/abs/2503.12615 (ICCV 2025), which uses a distilled latent Stable Diffusion XL model trained on five billion clean images as a zero-shot prior, and  https://arxiv.org/pdf/2507.02686, which integrates pixel-based diffusion models with deep unfolding and diffusion distillation (TMLR 2025). The extension to video restoration is presented in https://arxiv.org/abs/2510.01339 (ICLR 2025). Our approach to unsupervised training of diffusion models is introduced in https://arxiv.org/abs/2510.11964.

TBA

TBA