Dual Nonlinear Filters and Entropy Production

The talk will describe recent collaborative work between the speaker and Professor Sanjoy Mitter of MIT on connections between continuous-time nonlinear filtering theory, and nonequilibrium statistical mechanics. The study of nonlinear filters from a (Shannon) information- theoretic viewpoint reveals two flows of information, dubbed 'supply' and 'dissipation'. These characterise, in a dynamic way, the dependencies between the past, present and future of the signal and observation processes. In addition, signal and nonlinear filter processes exhibit a number of symmetries, (in particular they are jointly and marginally Markov), and these allow the construction of dual filtering problems by time reversal. The information supply and dissipation processes of a dual problem have rates equal to those of the original, but with supply and dissipation exchanging roles. The joint (signal-filter) process of a nonlinear filtering problem is unusual among Markov processes in that it exhibits one-way flows of information between components. The concept of entropy flow in the stationary distribution of a Markov process is at the heart of a modern theory of nonequilibrium statistical mechanics, based on stochastic dynamics. In this, a rate of entropy flow is defined by means of time averages of stationary ergodic processes. Such a definition is inadequate in the dynamic theory of nonlinear filtering. Instead a rate of entropy production can be defined, which is based on only the (current) local characteristics of the Markov process. This can be thought of as an 'entropic derivative'. The rate of entropy production of the joint process of a nonlinear filtering problem contains an 'interactive' component equal to the sum of the information supply and dissipation rates. These connections between nonlinear filtering and statistical mechanics allow a certain degree of cross- fertilisation between the fields. For example, the nonlinear filter, viewed as a statistical mechanical system, is a type of perpetual motion machine, and provides a precise quantitative example of Landauer's Principle. On the other hand, the theory of dissipative statistical mechanical systems can be brought to bear on the study of sub-optimal filters. On a more philosophical level, we might ask what a nonlinear filter can tell us about the direction of thermodynamic time.