Efficiency of serial and parallel block-coordinate descent methods for minimizing composite functions
Abstract
We develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an $\epsilon$-accurate solution with probability at least $1-\rho$ in at most $O[(2n/\epsilon)\log(1/\rho)]$ iterations, where $n$ is the dimension of the problem. For strongly convex functions the method converges linearly. This extends recent results of Nesterov [Efficiency of coordinate descent methods on huge-scale optimization problems, CORE Discussion Paper \#2010/2], which cover the smooth case, to composite minimization, while at the same time improving the complexity by the factor of 4 and removing $\epsilon$ from the logarithm. More importantly, in contrast with the aforementioned work in which the authors achieve the results by applying their method to a regularized version of the objective function with an unknown scaling factor, we show that this is not necessary, thus achieving true iteration complexity bounds. In the smooth case we also allow for arbitrary probability vectors and non-Euclidean norms. Time permitting, we will also mention new iteration complexity results for a parallel version of the method.
\\
\\
In the second part of the talk we demonstrate numerically that the algorithm is able to solve huge-scale $\ell_1$-regularized support vector machine and least squares problems with billion variables. Finally, we present preliminary computational results with a GPU-accelerated parallel version of the method, on truss topology design instances, achieving speedups of up to two orders of magnitude when compared to a single-core implementation in C.
\\
\\
References:
\\
P. Richtarik and M. Takac, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, 2011; http://arxiv.org/abs/1107.2848
\\
P. Richtarik and M. Takac, Efficient serial and parallel coordinate descent methods for huge-scale truss topology design, 2011; http://www.optimization-online.org/DB_HTML/2011/08/3118.html
\\
P. Richtarik and M. Takac, Efficiency of randomized coordinate descent methods on minimization problems with a composite objective function, Proceedings of SPARS11
The numerical computation of violent liquid motion
Abstract
Liquid impact is a key issue in various industrial applications (seawalls, offshore structures, breakwaters, sloshing in tanks of liquefied natural gas vessels, wave energy converters, offshore wind turbines, etc). Numerical simulations dealing with these applications have been performed by many groups, using various types of numerical methods. In terms of the numerical results, the outcome is often impressive, but the question remains of how relevant these results are when it comes to determining impact pressures. The numerical models are too simplified to reproduce the high variability of the measured pressures. In fact, for the time being, it is not possible to simulate accurately both global and local effects. Unfortunately it appears that local effects predominate over global effects when the behaviour of pressures is considered.
\\
\\
Having said this, it is important to point out that numerical studies can be quite useful to perform sensitivity analyses in idealized conditions such as a liquid mass falling under gravity on top of a horizontal wall and then spreading along the lateral sides. Simple analytical models inspired by numerical results on idealized problems can also be useful to predict trends.
\\
\\
The talk is organized as follows: After an introduction on some of the industrial applications, it will be explained to what extent numerical studies can be used to improve our understanding of impact pressures. Results on a liquid mass hitting a wall obtained by various numerical codes will be shown.
14:15
Penrose geometries, null geodesics and gravity
Abstract
This talk will be based on arxiv:1106.5254.
Penrose geometries, null geodesics and gravity
Abstract
This talk will be based on arxiv:1106.5254.
RBFs on Spheres
Abstract
In this talk, I will discuss various aspects of approximation by radial basis functions on spheres. After a short introduction to the subject of scattered data approximation on spheres and optimal recovery, I will particularly talk about error analysis, a hybrid approximation scheme involving polynomials and radial basis functions and, if time permits, solving nonlinear parabolic equations on spheres.
Several kinds of Chebyshev polynomials in higher dimensions
Abstract
Chebyshev polynomials are arguably the most useful orthogonal polynomials for computational purposes. In one dimension they arise from the close relationship that exists between Fourier series and polynomials. We describe how this relationship generalizes to Fourier series on certain symmetric lattices, that exist in all dimensions. The associated polynomials can not be seen as tensor-product generalizations of the one-dimensional case. Yet, they still enjoy excellent properties for interpolation, integration, and spectral approximation in general, with fast FFT-based algorithms, on a variety of domains. The first interesting case is the equilateral triangle in two dimensions (almost). We further describe the generalization of Chebyshev polynomials of the second kind, and many new kinds are found when the theory is completed. Connections are made to Laplacian eigenfunctions, representation theory of finite groups, and the Gibbs phenomenon in higher dimensions.
Analysis of a multiscale method for nonlinear nonmonotone elliptic problems
Abstract
Following the framework of the heterogeneous multiscale method, we present a numerical method for nonlinear elliptic homogenization problems. We briefly review the numerical, relying on an efficient coupling of macro and micro solvers, for linear problems. A fully discrete analysis is then given for nonlinear (nonmonotone) problems, optimal convergence rates in the H1 and L2 norms are derived and the uniqueness of the method is shown on sufficiently fine macro and micro meshes.
Numerical examples confirm the theoretical convergence rates and illustrate the performance and versatility of our approach.
IDR -- A New Class of Krylov Subspace Solvers: Benefits and Drawbacks
Abstract
This talk is about the Induced Dimension Reduction (IDR) methods developed by Peter Sonneveld and, more recently, Martin van Gijzen. We sketch the history, outline the underlying principle, and give a few details about different points of view on this class of Krylov subspace methods. If time permits, we briefly outline some recent developments in this field and the benefits and drawbacks of these and IDR methods in general.