Machine learning is beginning to have an impact on some of the hardest problems in mathematics and theoretical physics. In this talk I will discuss several examples where machine learning has helped to tackle questions that are otherwise computationally or conceptually challenging, including problems in knot theory and low-dimensional topology, optimisation in large discrete spaces, the generation of mathematical conjectures, and the study of Calabi-Yau geometries arising in string theory. Along the way, I will discuss both what machine learning can and cannot do in these settings, and how ideas from physics, such as symmetry, geometry, and statistical mechanics, have influenced the development of modern machine learning itself.