Newton-MR methods for nonconvex optimization

In this talk, we introduce Newton-MR variants for solving nonconvex optimization problems. Unlike the overwhelming majority of Newton-type methods, which rely on conjugate gradient method as the primary workhorse for their respective sub-problems, Newton-MR employs minimum residual (MINRES) method. With certain useful monotonicity properties of MINRES as well as its inherent ability to detect non-positive curvature directions as soon as they arise, we show that our algorithms come with desirable properties including the optimal first and second-order worst-case complexities. Numerical examples demonstrate the performance of our proposed algorithms.