CMS data and workflow management system
 Fanfani, A Bonacorsi, B Ibacchi, W Codispoti, G De Filippis, N Pompili, A My, S Abbrescia, M Maggi, G Donvito, G Silvestris, L Calzolari, F Sarkar, S Spiga, D Cinquili, M Lacaprara, S Biasotto, M Farina, F Merlo, M Belforte, S Kavka, C Sala, L Harvey, J Hufnagel, D Fanzago, F Corvo, M Magini, N Rehn, J Toteva, Z Feichtinger, D Tuura, L Eulisse, G Bockelman, B Lundstedt, C Egeland, R Evans, D Mason, D Gutsche, O Sexton-Kennedy, L Dagenhart, D Afaq, A Guo, Y Kosyakov, S Lueking, L Sekhri, V Fisk, I McBride, P Bauerdick, L Bakken, J Rossman, P Wicklund, E Wu, Y Jones, C Kuznetsov, V Riley, D Dolgert, A Van Lingen, F Narsky, I Paus, C Klute, M Gomez-Ceballos, G Piedra-Gomez, J Miller, M Mohapatra, A Lazaridis, C Bradley, D Elmer, P Wildish, T Wuerthwein, F Letts, J Bourilkov, D Kim, B Smith, P Hernandez, J Caballero, J Delgado, A Flix, J Cabrillo-Bartolome, I Kasemann, M Flossdorf, A Stadie, H Kreuzer, P Khomitch, A Hof, C Zeidler, C Kalini, S Trunov, A Saout, C Felzmann, U Metson, S Newbold, D Geddes, N Brew, C Jackson, J Wakefield, S De Weirdt, S Adler, V Maes, J Van Mulders, P Villella, I Astroparticle, Particle and Space Physics, Detectors and Medical Physics Applications - Proceedings of the 10th Conference 441-445 (01 Jan 2008)
Hierarchical Quantification of Synergy in Channels
 Perrone, P Ay, N Frontiers in Robotics and AI volume 2 35 (08 Jan 2016)
Bimonoidal Structure of Probability Monads
 Fritz, T Perrone, P Electronic Notes in Theoretical Computer Science volume 341 121-149 (Dec 2018)
Stochastic order on metric spaces and the ordered Kantorovich monad
 Fritz, T Perrone, P Advances in Mathematics volume 366 107081 (Jun 2020)
Monads, Partial Evaluations, and Rewriting
 Fritz, T Perrone, P Electronic Notes in Theoretical Computer Science volume 352 129-148 (Oct 2020)
Partition functions and fibering operators on the Coulomb branch of 5d SCFTs
 Closset, C Magureanu, H Journal of High Energy Physics volume 2023 issue 1 35 (10 Jan 2023)
Thu, 11 May 2023

14:00 - 15:00
Lecture Room 3

A coordinate descent algorithm on the Stiefel manifold for deep neural network training

Estelle Massart
(UC Louvain)
Abstract

We propose to use stochastic Riemannian coordinate descent on the Stiefel manifold for deep neural network training. The algorithm rotates successively two columns of the matrix, an operation that can be efficiently implemented as a multiplication by a Givens matrix. In the case when the coordinate is selected uniformly at random at each iteration, we prove the convergence of the proposed algorithm under standard assumptions on the loss function, stepsize and minibatch noise. Experiments on benchmark deep neural network training problems are presented to demonstrate the effectiveness of the proposed algorithm.

Subscribe to