When cast in a Bayesian setting, the solution to an inverse problem is given as a distribution over the space where the quantity of interest lives. When the quantity of interest is in principle a field then the discretization is very high-dimensional. Formulating algorithms which are defined in function space yields dimension-independent algorithms, which overcome the so-called curse of dimensionality. These algorithms are still often too expensive to implement in practice but can be effectively used offline and on toy-models in order to benchmark the ability of inexpensive approximate alternatives to quantify uncertainty in very high-dimensional problems. Inspired by the recent development of pCN and other function-space samplers [1], and also the recent independent development of Riemann manifold methods [2] and stochastic Newton methods [3], we propose a class of algorithms [4,5] which combine the benefits of both, yielding various dimension-independent and likelihood-informed (DILI) sampling algorithms. These algorithms can be effective at sampling from very high-dimensional posterior distributions.
[1] S.L. Cotter, G.O. Roberts, A.M. Stuart, D. White. "MCMC methods for functions: modifying old algorithms to make them faster," Statistical Science (2013).
[2] M. Girolami, B. Calderhead. "Riemann manifold Langevin and Hamiltonian Monte Carlo methods," Journal of the Royal Statistical Society: Series B (Statistical Methodology) 73 (2), 123–214 (2011).
[3] J. Martin, L. Wilcox, C. Burstedde, O. Ghattas. "A stochastic newton mcmc method for large-scale statistical inverse problems with application to seismic inversion," SIAM Journal on Scientific Computing 34(3), 1460–1487 (2012).
[4] K. J. H. Law. "Proposals Which Speed Up Function-Space MCMC," Journal of Computational and Applied Mathematics, in press (2013). http://dx.doi.org/10.1016/j.cam.2013.07.026
[5] T. Cui, K.J.H. Law, Y. Marzouk. Dimension-independent, likelihood- informed samplers for Bayesian inverse problems. In preparation.