Jump to content

Multilinear principal component analysis

Edit links
From Wikipedia, the free encyclopedia
This is an old revision of this page, as edited by Argospicy (talk | contribs) at 23:09, 30 June 2016. The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Multilinear Principal Component Analysis (MPCA) concerns dimension reduction of multidimensional data. Such data is represented by an array indexed by three or more indices, referred to as a tensor. Multilinear PCA is a family of algorithms and approaches that extend principal component analysis (PCA) to handle data of multidimensional format. Just as for usual PCA, the aim of such methods is to compress data, and to facilitate its analysis. For example, it can be used for data visualization.

MPCA can also refer to one particular algorithm of the same name. This is one approach that can be taken to extend the PCA to multi-dimensional data. It works in the following way. One computes a set of orthonormal matrices associated with each mode of a data tensor, and the data is expressed with respect to this matrix basis. The matrices are analogues of the vector principal components that occur in usual PCA. The transformation aims to account for as much of the variance in the data as possible, given a restricted number of basis matrices.

As a family of methods, higher dimensional PCA can be traced back to the Tucker decomposition[1] and Peter Kroonenberg's "M-mode PCA/3-mode PCA" work.[2] In 2000, De Lathauwer etal. restated Tucker and Kroonenberg's work in their SIAM paper entitled Multilinear Singular Value Decomposition,[3] (HOSVD) and in their paper "On the Best Rank-1 and Rank-(R1, R2, ..., RN ) Approximation of Higher-order Tensors".[4]

Circa 2001, Vasilescu reframed the data analysis, recognition and synthesis problems as multilinear tensor problems based on the insight that most observed data are the compositional consequence of several causal factors of data formation, and are well suited for multi-modal data tensor analysis. The power of the tensor framework was showcased in a visually and mathematically compelling manner by decomposing and representing joint angels, or images in terms of their causal factors of data formation in the following works: Human Motion Signatures [5] (CVPR 2001, ICPR 2002), face recognition - TensorFaces, [6] [7] (ECCV 2002, CVPR 2003, etc.) and computer graphics -- TensorTextures[8](Siggraph 2004).

Historically, MPCA has been referred to as "M-mode PCA", a terminology which was coined by Peter Kroonenberg in 1980.[2] In 2005, Vasilescu and Terzopoulos introduced the Multilinear PCA[9] terminology as a way to better differentiate between linear and multilinear tensor decomposition, as well as, to better differentiate between the work[5][6][7][8] that computed 2nd order statistics associated with each data tensor mode(axis), and subsequent work on Multilinear Independent Component Analysis[9] that computed higher order statistics associated with each tensor mode/axis.

Multilinear PCA may be applied to compute the causal factors of data formation,or as signal processing tool on data tensors whose individual observation have either been vectorized [5] [6] [7] ,[8] or whose observations are treated as matrix [10] and concatenated into a data tensor.


The algorithm

The MPCA solution follows the alternating least square (ALS) approach.[2] It is iterative in nature. As in PCA, MPCA works on centered data. Centering is a little more complicated for tensors, and it is problem dependent.

Feature selection

MPCA features: Supervised MPCA feature selection is used in object recognition[11] while unsupervised MPCA feature selection is employed in visualization task.[12]

Extensions

Various extensions of MPCA have been developed: [13]

  • Uncorrelated MPCA (UMPCA) [14] In contrast, the uncorrelated MPCA (UMPCA) generates uncorrelated multilinear features.[14]
  • Boosting+MPCA[15]
  • Non-negative MPCA (NMPCA) [16]
  • Robust MPCA (RMPCA) [17]

Resources

References

  1. ^ Tucker, Ledyard R (September 1966). "Some mathematical notes on three-mode factor analysis". Psychometrika. 31 (3): 279–311. doi:10.1007/BF02289464.
  2. ^ a b c P. M. Kroonenberg and J. de Leeuw, Principal component analysis of three-mode data by means of alternating least squares algorithms, Psychometrika, 45 (1980), pp. 69–97.
  3. ^ L.D. Lathauwer, B.D. Moor, J. Vandewalle (2000) "A multilinear singular value decomposition", SIAM Journal of Matrix Analysis and Applications, 21 (4), 1253–1278
  4. ^ L. D. Lathauwer, B. D. Moor, J. Vandewalle (2000) "On the best rank-1 and rank-(R1, R2, ..., RN ) approximation of higher-order tensors", SIAM Journal of Matrix Analysis and Applications 21 (4), 1324–1342.
  5. ^ a b c M. A. O. Vasilescu (2002) "Human Motion Signatures: Analysis, Synthesis, Recognition," Proceedings of International Conference on Pattern Recognition (ICPR 2002), Vol. 3, Quebec City, Canada, Aug, 2002, 456-460.
  6. ^ a b c M.A.O. Vasilescu, D. Terzopoulos (2002) "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision -- ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447-460.
  7. ^ a b c M.A.O. Vasilescu, D. Terzopoulos (2003) "Multilinear Subspace Analysis for Image Ensembles, M. A. O. Vasilescu, D. Terzopoulos, Proc. Computer Vision and Pattern Recognition Conf. (CVPR '03), Vol.2, Madison, WI, June, 2003, 93-99.
  8. ^ a b c M.A.O. Vasilescu, D. Terzopoulos (2004) "TensorTextures: Multilinear Image-Based Rendering", M. A. O. Vasilescu and D. Terzopoulos, Proc. ACM SIGGRAPH 2004 Conference Los Angeles, CA, August, 2004, in Computer Graphics Proceedings, Annual Conference Series, 2004, 336-342.
  9. ^ a b M. A. O. Vasilescu, D. Terzopoulos (2005) "Multilinear Independent Component Analysis", "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, June 2005, vol.1, 547-553."
  10. ^ H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, (2008) "MPCA: Multilinear principal component analysis of tensor objects", IEEE Trans. Neural Netw., 19 (1), 18–39
  11. ^ , M. A. O. Vasilescu, D. Terzopoulos (2003) "Multilinear Subspace Analysis of Image Ensembles", "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI, June, 2003"
  12. ^ H. Lu, H.-L. Eng, M. Thida, and K.N. Plataniotis, "Visualization and Clustering of Crowd Video Content in MPCA Subspace," in Proceedings of the 19th ACM Conference on Information and Knowledge Management (CIKM 2010) , Toronto, ON, Canada, October, 2010.
  13. ^ Lu, Haiping; Plataniotis, K.N.; Venetsanopoulos, A.N. (2011). "A Survey of Multilinear Subspace Learning for Tensor Data" (PDF). Pattern Recognition. 44 (7): 1540–1551. doi:10.1016/j.patcog.2011.01.004.
  14. ^ a b H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, "Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning," IEEE Trans. Neural Netw., vol. 20, no. 11, pp. 1820–1836, Nov. 2009.
  15. ^ H. Lu, K. N. Plataniotis and A. N. Venetsanopoulos, "Boosting Discriminant Learners for Gait Recognition using MPCA Features", EURASIP Journal on Image and Video Processing, Volume 2009, Article ID 713183, 11 pages, 2009. doi:10.1155/2009/713183.
  16. ^ Y. Panagakis, C. Kotropoulos, G. R. Arce, "Non-negative multilinear principal component analysis of auditory temporal modulations for music genre classification", IEEE Trans. on Audio, Speech, and Language Processing, vol. 18, no. 3, pp. 576–588, 2010.
  17. ^ K. Inoue, K. Hara, K. Urahama, "Robust multilinear principal component analysis", Proc. IEEE Conference on Computer Vision, 2009, pp. 591–597.
Retrieved from "https://en.wikipedia.org/w/index.php?title=Multilinear_principal_component_analysis&oldid=727738058"