This talk compares neural network-based methods with classical numerical methods from a theoretical perspective. Through several representative examples, we examine both the potential and the limitations of deep neural networks in scientific computing and, more broadly, in machine learning.

 

We begin by comparing ReLU deep neural networks with polynomials and piecewise polynomial spaces, focusing on their structures and expressive power. We then revisit the curse of dimensionality and discuss whether deep neural networks truly offer advantages over traditional numerical methods for high-dimensional problems. Next, we consider the use of deep neural networks for solving partial differential equations, with particular emphasis on the challenge of achieving high accuracy. Finally, we examine multigrid methods and explore whether their underlying principles can help us better understand, design, and train deep neural network models with possible implications for broader AI applications.