The deep neural network has achieved impressive results in various applications, and is involved in more and more branches of science. However, there are still few theories supporting its empirical success. In particular, we miss the mathematical tool to explain the advantage of certain structures of the network, and to have quantitive error bounds. In our recent work, we used a regularised relaxed control problem to model the deep neural network. We managed to characterise its optimal control by the invariant measure of a mean-field Langevin system, which can be approximated by the marginal laws. Through this study we understand the importance of the pooling for the deep nets, and are capable of computing an exponential convergence rate for the (stochastic) gradient descent algorithm.
