Kullback-Leibler Approximation Of Probability Measures

6 March 2014
Professor Andrew Stuart
Many problems in the physical sciences require the determination of an unknown function from a finite set of indirect measurements. Examples include oceanography, oil recovery, water resource management and weather forecasting. The Bayesian approach to these problems is natural for many reasons, including the under-determined and ill-posed nature of the inversion, the noise in the data and the uncertainty in the differential equation models used to describe complex mutiscale physics. The object of interest in the Bayesian approach is the posterior probability distribution on the unknown field [1]. \\ \\ However the Bayesian approach presents a computationally formidable task as it results in the need to probe a probability measure on separable Banach space. Monte Carlo Markov Chain methods (MCMC) may be used to achieve this [2], but can be prohibitively expensive. In this talk I will discuss approximation of probability measures by a Gaussian measure, looking for the closest approximation with respect to the Kullback-Leibler divergence. This methodology is widely used in machine-learning [3]. In the context of target measures on separable Banach space which themselves have density with respect to a Gaussian, I will show how to make sense of the resulting problem in the calculus of variations [4]. Furthermore I will show how the approximate Gaussians can be used to speed-up MCMC sampling of the posterior distribution [5]. \\ \\ [1] A.M. Stuart. "Inverse problems: a Bayesian perspective." Acta Numerica 19(2010) and http://arxiv.org/abs/1302.6989 \\ [2] S.L.Cotter, G.O.Roberts, A.M. Stuart and D. White, "MCMC methods for functions: modifying old algorithms to make them faster". Statistical Science 28(2013). http://arxiv.org/abs/1202.0709 \\ [3] C.M. Bishop, "Pattern recognition and machine learning". Springer, 2006. \\ [4] F.J. Pinski G. Simpson A.M. Stuart H. Weber, "Kullback-Leibler Approximations for measures on infinite dimensional spaces." http://arxiv.org/abs/1310.7845 \\ [5] F.J. Pinski G. Simpson A.M. Stuart H. Weber, "Algorithms for Kullback-Leibler approximation of probability measures in infinite dimensions." In preparation.
  • Computational Mathematics and Applications Seminar