Past Data Science Seminar

14 October 2021
14:00
David Bau
Abstract

One of the great challenges of neural networks is to understand how they work.  For example: does a neuron encode a meaningful signal on its own?  Or is a neuron simply an undistinguished and arbitrary component of a feature vector space?  The tension between the neuron doctrine and the population coding hypothesis is one of the classical debates in neuroscience. It is a difficult debate to settle without an ability to monitor every individual neuron in the brain.

 

Within artificial neural networks we can examine every neuron. Beginning with the simple proposal that an individual neuron might represent one internal concept, we conduct studies relating deep network neurons to human-understandable concepts in a concrete, quantitative way: Which neurons? Which concepts? Are neurons more meaningful than an arbitrary feature basis? Do neurons play a causal role? We examine both simplified settings and state-of-the-art networks in which neurons learn how to represent meaningful objects within the data without explicit supervision.

 

Following this inquiry in computer vision leads us to insights about the computational structure of practical deep networks that enable several new applications, including semantic manipulation of objects in an image; understanding of the sparse logic of a classifier; and quick, selective editing of generalizable rules within a fully trained generative network.  It also presents an unanswered mathematical question: why is such disentanglement so pervasive?

 

In the talk, we challenge the notion that the internal calculations of a neural network must be hopelessly opaque. Instead, we propose to tear back the curtain and chart a path through the detailed structure of a deep network by which we can begin to understand its logic.

 

  • Data Science Seminar
11 June 2021
14:00
Melanie Weber
Abstract

Many machine learning applications involve non-Euclidean data, such as graphs, strings or matrices. In such cases, exploiting Riemannian geometry can deliver algorithms that are computationally superior to standard(Euclidean) nonlinear programming approaches. This observation has resulted in an increasing interest in Riemannian methods in the optimization and machine learning community.

In the first part of the talk, we consider the task of learning a robust classifier in hyperbolic space. Such spaces have received a surge of interest for representing large-scale, hierarchical data, due to the fact that theyachieve better representation accuracy with fewer dimensions. We present the first theoretical guarantees for the (robust) large margin learning problem in hyperbolic space and discuss conditions under which hyperbolic methods are guaranteed to surpass the performance of their Euclidean counterparts. In the second part, we introduce Riemannian Frank-Wolfe (RFW) methods for constrained optimization on manifolds. Here, we discuss matrix-valued tasks for which such Riemannian methods are more efficient than classical Euclidean approaches. In particular, we consider applications of RFW to the computation of Riemannian centroids and Wasserstein barycenters, both of which are crucial subroutines in many machine learning methods.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

  • Data Science Seminar
4 June 2021
12:00
Joe Kileel
Abstract

From latent variable models in machine learning to inverse problems in computational imaging, tensors pervade the data sciences.  Often, the goal is to decompose a tensor into a particular low-rank representation, thereby recovering quantities of interest about the application at hand.  In this talk, I will present a recent method for low-rank CP symmetric tensor decomposition.  The key ingredients are Sylvester’s catalecticant method from classical algebraic geometry and the power method from numerical multilinear algebra.  In simulations, the method is roughly one order of magnitude faster than existing CP decomposition algorithms, with similar accuracy.  I will state guarantees for the relevant non-convex optimization problem, and robustness results when the tensor is only approximately low-rank (assuming an appropriate random model).  Finally, if the tensor being decomposed is a higher-order moment of data points (as in multivariate statistics), our method may be performed without explicitly forming the moment tensor, opening the door to high-dimensional decompositions.  This talk is based on joint works with João Pereira, Timo Klock and Tammy Kolda. 

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

  • Data Science Seminar
28 May 2021
12:00
Nina Otter
Abstract

One of the most successful methods in topological data analysis (TDA) is persistent homology, which associates a one-parameter family of spaces to a data set, and gives a summary — an invariant called "barcode" — of how topological features, such as the number of components, holes, or voids evolve across the parameter space. In many applications one might wish to associate a multiparameter family of spaces to a data set. There is no generalisation of the barcode to the multiparameter case, and finding algebraic invariants that are suitable for applications is one of the biggest challenges in TDA.

The use of persistent homology in applications is justified by the validity of certain stability results. At the core of such results is a notion of distance between the invariants that one associates to data sets. While such distances are well-understood in the one-parameter case, the study of distances for multiparameter persistence modules is more challenging, as they rely on a choice of suitable invariant.

In this talk I will first give a brief introduction to multiparameter persistent homology. I will then present a general framework to study stability questions in multiparameter persistence: I will discuss which properties we would like invariants to satisfy, present different ways to associate distances to such invariants, and finally illustrate how this framework can be used to derive new stability results. No prior knowledge on the subject is assumed.

The talk is based on joint work with Barbara Giunti, John Nolan and Lukas Waas. 

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

  • Data Science Seminar
12 March 2021
12:00
David Pfau
Abstract

Learning a representation from data that disentangles different factors of variation is hypothesized to be a critical ingredient for unsupervised learning. Defining disentangling is challenging - a "symmetry-based" definition was provided by Higgins et al. (2018), but no prescription was given for how to learn such a representation. We present a novel nonparametric algorithm, the Geometric Manifold Component Estimator (GEOMANCER), which partially answers the question of how to implement symmetry-based disentangling. We show that fully unsupervised factorization of a data manifold is possible if the true metric of the manifold is known and each factor manifold has nontrivial holonomy – for example, rotation in 3D. Our algorithm works by estimating the subspaces that are invariant under random walk diffusion, giving an approximation to the de Rham decomposition from differential geometry. We demonstrate the efficacy of GEOMANCER on several complex synthetic manifolds. Our work reduces the question of whether unsupervised disentangling is possible to the question of whether unsupervised metric learning is possible, providing a unifying insight into the geometric nature of representation learning.

 

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

  • Data Science Seminar
5 March 2021
12:00
Abstract

Low rank orthogonal tensor approximation (LROTA) is an important problem in tensor computations and their applications. A classical and widely used algorithm is the alternating polar decomposition method (APD). In this talk, I will first give very a brief introduction to tensors and their decompositions. After that, an improved version named iAPD of the classical APD will be proposed and all the following four fundamental properties of iAPD will be discussed : (i) the algorithm converges globally and the whole sequence converges to a KKT point without any assumption; (ii) it exhibits an overall sublinear convergence with an explicit rate which is sharper than the usual O(1/k) for first order methods in optimization; (iii) more importantly, it converges R-linearly for a generic tensor without any assumption; (iv) for almost all LROTA problems, iAPD reduces to APD after finitely many iterations if it converges to a local minimizer. If time permits, I will also present some numerical experiments.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

  • Data Science Seminar
26 February 2021
12:00
Nina Otter
Abstract

Magnitude is an isometric invariant of metric spaces that was introduced by Tom Leinster in 2010, and is currently the object of intense research, since it has been shown to encode many invariants of a metric space such as volume, dimension, and capacity.

Magnitude homology is a homology theory for metric spaces that has been introduced by Hepworth-Willerton and Leinster-Shulman, and categorifies magnitude in a similar way as the singular homology of a topological space categorifies its Euler characteristic.

In this talk I will first introduce magnitude and magnitude homology. I will then give an overview of existing results and current research in this area, explain how magnitude homology is related to persistent homology, and finally discuss new stability results for magnitude and how it can be used to study point cloud data.

This talk is based on  joint work in progress with Miguel O’Malley and Sara Kalisnik, as well as the preprint https://arxiv.org/abs/1807.01540.

  • Data Science Seminar
19 February 2021
12:00
Abstract

Digital data capture is the backbone of all modern day systems and “Digital Revolution” has been aptly termed as the Third Industrial Revolution. Underpinning the digital representation is the Shannon-Nyquist sampling theorem and more recent developments include compressive sensing approaches. The fact that there is a physical limit to which sensors can measure amplitudes poses a fundamental bottleneck when it comes to leveraging the performance guaranteed by recovery algorithms. In practice, whenever a physical signal exceeds the maximum recordable range, the sensor saturates, resulting in permanent information loss. Examples include (a) dosimeter saturation during the Chernobyl reactor accident, reporting radiation levels far lower than the true value and (b) loss of visual cues in self-driving cars coming out of a tunnel (due to sudden exposure to light). 

 

To reconcile this gap between theory and practice, we introduce the Unlimited Sensing framework or the USF that is based on a co-design of hardware and algorithms. On the hardware front, our work is based on a radically different analog-to-digital converter (ADC) design, which allows for the ADCs to produce modulo or folded samples. On the algorithms front, we develop new, mathematically guaranteed recovery strategies.  

 

In the first part of this talk, we prove a sampling theorem akin to the Shannon-Nyquist criterion. We show that, remarkably, despite the non-linearity in sensing pipeline, the sampling rate only depends on the signal’s bandwidth. Our theory is complemented with a stable recovery algorithm. Beyond the theoretical results, we will also present a hardware demo that shows our approach in action.

 

Moving further, we reinterpret the unlimited sensing framework as a generalized linear model that motivates a new class of inverse problems. We conclude this talk by presenting new results in the context of single-shot high-dynamic-range (HDR) imaging, sensor array processing and HDR tomography based on the modulo Radon transform.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

  • Data Science Seminar
20 November 2020
12:00
Peter Markowich
Abstract

We present a partial differential equation framework for deep residual neural networks and for the associated learning problem. This is done by carrying out the continuum limits of neural networks with respect to width and depth. We study the wellposedness, the large time solution behavior, and the characterization of the steady states of the forward problem. Several useful time-uniform estimates and stability/instability conditions are presented. We state and prove optimality conditions for the inverse deep learning problem, using standard variational calculus, the Hamilton-Jacobi-Bellmann equation and the Pontryagin maximum principle. This serves to establish a mathematical foundation for investigating the algorithmic and theoretical connections between neural networks, PDE theory, variational analysis, optimal control, and deep learning.

This is based on joint work with Hailiang Liu.

  • Data Science Seminar
13 November 2020
12:00
Afonso Bandeira
Abstract

When faced with a data analysis, learning, or statistical inference problem, the amount and quality of data available fundamentally determines whether such tasks can be performed with certain levels of accuracy. With the growing size of datasets however, it is crucial not only that the underlying statistical task is possible, but also that is doable by means of efficient algorithms. In this talk we will discuss methods aiming to establish limits of when statistical tasks are possible with computationally efficient methods or when there is a fundamental «Statistical-to-Computational gap›› in which an inference task is statistically possible but inherently computationally hard. We will focus on Hypothesis Testing and the ``Low Degree Method'' and also address hardness of certification via ``quiet plantings''. Guiding examples will include Sparse PCA, bounds on the Sherrington Kirkpatrick Hamiltonian, and lower bounds on Chromatic Numbers of random graphs.

The join button will be published on the right (Above the view all button) 30 minutes before the seminar starts (login required).

  • Data Science Seminar

Pages