Nilsequences are sequences coming from Lie groups which play the role of additive characters in higher order Fourier analysis. In this talk, I will define these and give some basic examples without assuming any prior knowledge. I'll use this to state an equidistribution result due to Green and Tao, and compare what happens in this setting to the familiar case of sequences in the torus.
 

Professor Ruoyu Sun will talk about: 'Understanding and Improving LLM Training via Hessian and Spectral Analysis' 

In the first part, we investigate the approximate block-diagonal Hessian structure of neural networks. We identify the conditions under which this structure emerges and give the first rigorous proofs based on random matrix theory. From this structural perspective, we explain why Adam works far better than SGD on Transformers. Following this structural guideline, we design the memory-efficient optimizer Adam-mini; Normuon is another optimizer developed under the same principle.

 In the second part, we adopt a spectral perspective to study and refine normalization layers for neural network training. We propose a preconditioning (PC) layer, an advanced weight-centric module built with low-degree polynomial preconditioning for scalable spectral control. Theoretically, for deep linear networks, we prove that bounding each layer's singular values ensures geometric convergence of gradient descent to global minima. Empirically, PC delivers consistent efficiency gains over a standard Transformer baseline in Llama2-1B pretraining.