Networks are fascinating because of their ability to describe complex structures found in a broad variety of systems, from arts and humanities, via the life sciences to the physical science and mathematics. Perhaps even more startling is the variety of approaches that different disciplines have contributed to the study of networks. All of these approaches have a common goal: finding simplicity in complexity. Yet complexity science has no single overarching theory of what simplicity means and how and why it can be found. In this talk I will present some well known methods and results to highlight different approaches to finding simplicity that computer science, physics and mathematics have developed. I will then highlight some less-known connections and argue that an overarching theory of simplicity may be within reach.