Deep learning

Deep Learning is a sub-field within [Machine Learning] that uses deep architectures to model complex relationships among data. Such models have proven to be effective feature extractors over high dimensional, structured data (Hinton - Scholarpedia, 2009).

One of the earliest successful implementations (Hinton et al. 2006) involves learning the distribution of high level image (or possibly other data) features using successive layers of binary latent variables. The approach uses a [Restricted Boltzmann Machine|Boltzmann Machine] (Smolensky, 1986) to model each new layer of higher level features. Each new layer guarantees an increase on the lower-bound of the log likelihood of the data, thus improving the model, if trained properly. Once sufficiently many layers have been learned the deep architecture may be used as a generative model by reproducing the data by sampling down the model from the top level features.

References

• Hinton, G. E.; Osindero, S.; Teh, Y. (2006). "A fast learning algorithm for deep belief nets" (PDF). Neural Computation. 18 (7): 1527–1554. doi:10.1162/neco.2006.18.7.1527. PMID 16764513.
• Smolensky, P. (1986). Information processing in dynamical systems: Foundations of harmony theory. Vol. 1. pp. 194–281. {{cite book}}: |journal= ignored (help)

External links

• Deep Learning