Bayesian approaches to brain function

Bayesian Brain is a term used in behavioural and neuroscience that refers to an explanation of the brain's cognitive abilities based on internal probabilistic models updated by neural processing of sensory information using methods approximating those of Bayesian probability.<ref>Kenji Doya (Editor), Shin Ishii (Editor), Alexandre Pouget (Editor), Rajesh P. N. Rao (Editor) (2007), Bayesian Brain: Probabilistic Approaches to Neural Coding, The MIT Press; 1 edition (Jan 1 2007)<\ref>

This field of study has its historical roots in numerous disciplines including Machine Learning, Experimental Psychology and Bayesian Probability.

As early as the 1860s, with the work of Herman Helmholtz in experimental psychology the brain's ability to extract perceptual information from sensory data was modeled in terms of an inference machine.<ref>Helmholtz, H. (1860/1962). Handbuch der physiologischen optik (Southall, J. P. C. (Ed.), English trans.),Vol. 3. New York: Dover.<\ref>

The basic challenge, explaining the organization and processing of sensory data into an accurate internal model of the outside world, was taken up in research on Unsupervised Learning, in particular the Analysis by Synthesis approach, branches of Machine Learning. , In 1983 Geoffrey Hinton and colleagues proposed the brain could be seen as a machine making decisions based on the uncertainties of the outside world. During the 1990s researchers including Peter Dayan, Geoffrey Hinton and Richard Zemel proposed that the brain represents knowledge of the world in terms of probabilities and made specific proposals for tractable neural processes that could manifest such a 'Helmholtz Machine'. , , Also during the 1990s some researchers such as Geoffrey Hinton and Karl Friston began examining the concept of 'free energy' as a calculably tractable measure of the discrepancy between actual features of the world and representations of those features captured by neural network models. Bayesian probability, as developed by Laplace, Bayes, Jeffries, Cox and Jaynes has developed mathematical techniques and procedures for treating probability as the degree of plausibility which should be assigned to a given supposition or hypothesis based on the available evidence. In 1988 E.T. Jaynes presented a framework for using Bayesian Probability to model mental processes. A synthesis of these researches has recently been attempted by Karl Friston. Using Variational Bayesian methods, he has shown how mental models of the outside world could be updated by sensory information and may be driven to minimize free energy or the discrepancy between the mental model formed and events as they actually occur. According to Friston: The free-energy considered here represents a bound on the surprise inherent in any exchange with the environment, under expectations encoded by its state or configuration. A system can minimise free-energy by changing its configuration to change the way it samples the environment, or to change its expectations. These changes correspond to action and perception, respectively, and lead to an adaptive exchange with the environment that is characteristic of biological systems. This treatment implies that the system’s state and structure encode an implicit and probabilistic model of the environment. This area of research was summarized and made more widely available to the public in a 2008 article in New Scientist that reported excitement within the research community for a potentially unifying theory of brain function of great explanatory scope.

Karl Friston provides a sense of the potential explanatory power of the theory:

This model of brain function can explain a wide range of anatomical and physiological aspects of brain systems; for example, the hierarchical deployment of cortical areas, recurrent architectures using forward and backward connections and functional asymmetries in these connections (Angelucci et al., 2002a; Friston, 2003). In terms of synaptic physiology, it predicts associative plasticity and, for dynamic models, spike-timing-dependent plasticity. In terms of lectrophysiology it accounts for classical and extra-classical receptive field effects and long-latency or endogenous components of evoked cortical responses (Rao and Ballard, 1998; Friston, 2005). It predicts the attenuation of responses encoding prediction error with perceptual learning and explains many phenomena like repetition suppression, mismatch negativity and the P300 in electroencephalography. In psychophysical terms, it accounts for the behavioural correlates of these physiological phenomena, e.g., priming, and global precedence (see Friston, 2005 for an overview).

It is fairly easy to show that both perceptual inference and learning rest on a minimisation of free energy (Friston, 2003) or suppression of prediction error (Rao and Ballard, 1998).