In this talk I will present network-theoretic tools [1-2] to filter information in large-scale datasets and I will show that these are powerful tools to study complex datasets. In particular I will introduce correlation-based information filtering networks and the planar filtered graphs (PMFG) and I will show that applications to financial data-sets can meaningfully identify industrial activities and structural market changes [3-4].
It has been shown that by making use of the 3-clique structure of the PMFG a clustering can be extracted allowing dimensionality reduction that keeps both local information and global hierarchy in a deterministic manner without the use of any prior information [5-6]. However, the algorithm so far proposed to construct the PMFG is numerically costly with O(N3) computational complexity and cannot be applied to large-scale data. There is therefore scope to search for novel algorithms that can provide, in a numerically efficient way, such a reduction to planar filtered graphs. I will introduce a new algorithm, the TMFG (Triangulated Maximally Filtered Graph), that efficiently extracts a planar subgraph which optimizes an objective function. The method is scalable to very large datasets and it can take advantage of parallel and GPUs computing [7]. Filtered graphs are valuable tools for risk management and portfolio optimization too [8-9] and they allow to construct probabilistic sparse modeling for financial systems that can be used for forecasting, stress testing and risk allocation [10].
Filtered graphs can be used not only to extract relevant and significant information but more importantly to extract knowledge from an overwhelming quantity of unstructured and structured data. I will provide a practitioner example by a successful Silicon Valley start-up, Yewno. The key idea underlying Yewno’s products is the concept of the Knowledge Graph, a framework based on filtered graph research, whose goal is to extract signals from evolving corpus of data. The common principle is that a methodology leveraging on developments in Computational linguistics and graph theory is used to build a graph representation of knowledge [11], which can be automatically analysed to discover hidden relations between components in many different complex systems. This Knowledge Graph based framework and inference engine has a wide range of applications, including finance, economics, biotech, law, education, marketing and general research.
[1] T. Aste, T. Di Matteo, S. T. Hyde, Physica A 346 (2005) 20.
[2] T. Aste, Ruggero Gramatica, T. Di Matteo, Physical Review E 86 (2012) 036109.
[3] M. Tumminello, T. Aste, T. Di Matteo, R. N. Mantegna, PNAS 102, n. 30 (2005) 10421.
[4] N. Musmeci, Tomaso Aste, T. Di Matteo, Journal of Network Theory in Finance 1(1) (2015) 1-22.
[5] W.-M. Song, T. Di Matteo, and T. Aste, PLoS ONE 7 (2012) e31929.
[6] N. Musmeci, T. Aste, T. Di Matteo, PLoS ONE 10(3): e0116201 (2015).
[7] Guido Previde Massara, T. Di Matteo, T. Aste, Journal of Complex networks 5 (2), 161 (2016).
[8] F. Pozzi, T. Di Matteo and T. Aste, Scientific Reports 3 (2013) 1665.
[9] N. Musmeci, T. Aste and T. Di Matteo, Scientific Reports 6 (2016) 36320.
[10] Wolfram Barfuss, Guido Previde Massara, T. Di Matteo, T. Aste, Phys.Rev. E 94 (2016) 062306.
[11] Ruggero Gramatica, T. Di Matteo, Stefano Giorgetti, Massimo Barbiani, Dorian Bevec and Tomaso Aste, PLoS One (2014) PLoS ONE 9(1): e84912.