Edit filter log

'{{context|date=June 2012}} '''Multilinear Principal Component Analysis''' (MPCA) is a multilinear extension of [[principal component analysis]] (PCA). Its origin can be traced back to the [[Tucker decomposition]]<ref>{{Cite journal|last1=Tucker| first1=Ledyard R | authorlink1 = Ledyard R Tucker | title = Some mathematical notes on three-mode factor analysis | journal = [[Psychometrika]] | volume = 31 | issue = 3 | pages = 279–311 |date=September 1966 | doi = 10.1007/BF02289464 }}</ref>. It was developed by Peter Kroonenberg <ref>P. M. Kroonenberg and J. de Leeuw, [http://www.springerlink.com/content/c8551t1p31236776/ Principal component analysis of three-mode data by means of alternating least squares algorithms], Psychometrika, 45 (1980), pp. 69–97.</ref>. Tucker and Kroonenberg's work was restated by De Lathauwer etal. in clear and concise numerical computational terms in their 2000 SIAM papers entitled [[Multilinear Singular Value Decomposition]],<ref name="HOSVD">L.D. Lathauwer, B.D. Moor, J. Vandewalle (2000) [http://portal.acm.org/citation.cfm?id=354398 "A multilinear singular value decomposition"], ''SIAM Journal of Matrix Analysis and Applications'', 21 (4), 1253–1278</ref> (HOSVD) and in their paper on the best rank-(R1, R2, ..., RN ) approximation of higher-order tensors.<ref>L. D. Lathauwer, B. D. Moor, J. Vandewalle (2000) [http://portal.acm.org/citation.cfm?id=354405 "On the best rank-1 and rank-(R1, R2, ..., RN ) approximation of higher-order tensors"], ''SIAM Journal of Matrix Analysis and Applications'' 21 (4), 1324–1342.</ref> Historically, MPCA has been referred to as "M-mode PCA", a terminology which was coined by Peter Kroonenberg <ref>P. M. Kroonenberg and J. de Leeuw, [http://www.springerlink.com/content/c8551t1p31236776/ Principal component analysis of three-mode data by means of alternating least squares algorithms], Psychometrika, 45 (1980), pp. 69–97.</ref>. In 2005, Vasilescu and Terzopoulos introduced the the Multilinear PCA<ref name="MPCA-MICA2005"> M. A. O. Vasilescu, D. Terzopoulos (2005) [http://www.media.mit.edu/~maov/mica/mica05.pdf "Multilinear Independent Component Analysis"], "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, June 2005, vol.1, 547-553."</ref> terminology as a way to better differentiate between linear and multilinear tensor decomposition, as well as to better differentiate from subsequent work on Multilinear Independent Component Analysis<ref name="MPCA-MICA2005"> M. A. O. Vasilescu, D. Terzopoulos (2005) [http://www.media.mit.edu/~maov/mica/mica05.pdf "Multilinear Independent Component Analysis"], "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, June 2005, vol.1, 547-553."</ref>. Multilinear PCA could be applied to data tensors whose individual observation have either been vectorized for analysis, recognition <ref name="MPCA2008">M.A.O. Vasilescu, D. Terzopoulos (2002) [http://www.media.mit.edu/~maov/tensorfaces/eccv02_corrected.pdf "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision -- ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447-460. </ref><ref name="MPCA2002">M.A.O. Vasilescu, D. Terzopoulos (2002) [http://www.media.mit.edu/~maov/tensorfaces/eccv02_corrected.pdf "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision -- ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447-460. </ref><ref name="MPCA2003">M.A.O. Vasilescu, D. Terzopoulos (2003) [http://www.media.mit.edu/~maov/tensorfaces/eccv02_corrected.pdf <ref name="Vasilescu2003">M.A.O. Vasilescu, D. Terzopoulos (2002) [http://www.media.mit.edu/~maov/tensorfaces/cvpr03.pdf "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision -- ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447-460. </ref><ref name="MPCA2008">M.A.O. Vasilescu, D. Terzopoulos (2002) [http://www.media.mit.edu/~maov/tensorfaces/eccv02_corrected.pdf "Multilinear Analysis of Image Ensembles: TensorFaces," Proc. 7th European Conference on Computer Vision (ECCV'02), Copenhagen, Denmark, May, 2002, in Computer Vision -- ECCV 2002, Lecture Notes in Computer Science, Vol. 2350, A. Heyden et al. (Eds.), Springer-Verlag, Berlin, 2002, 447-460.</ref> or they are treated as matrix <ref name="MPCA2008">H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, (2008) [http://www.dsp.utoronto.ca/~haiping/Publication/MPCA_TNN08_rev2010.pdf "MPCA: Multilinear principal component analysis of tensor objects"], ''IEEE Trans. Neural Netw.'', 19 (1), 18–39</ref> and concatenated into a data tensor. MPCA computes a set of orthonormal matrices associated with each mode of the data tensor which are analogous to the orthonoramal row and column space of the matrix SVD. This transformation aims to capture as high a variance as possible, accounting for as much of the variability in the data as possible, subject to the constraint of mode-wise orthogonality. == The algorithm == As in PCA, MPCA works on centered data. The MPCA solution follows the alternating least square (ALS) approach.<ref>P. M. Kroonenberg and J. de Leeuw, [http://www.springerlink.com/content/c8551t1p31236776/ Principal component analysis of three-mode data by means of alternating least squares algorithms], Psychometrika, 45 (1980), pp. 69–97.</ref> Thus, is iterative in nature and it proceeds by decomposing the original problem to a series of multiple projection subproblems. Each subproblem is a classical PCA problem, which can be easily solved. It should be noted that while PCA with orthogonal transformations produces uncorrelated features/variables, this is not the case for MPCA. Due to the nature of tensor-to-tensor transformation, MPCA features are not uncorrelated in general although the transformation in each mode is orthogonal.<ref name="UMPCA">H. Lu, K. N. Plataniotis, and A. N. Venetsanopoulos, "[http://www.dsp.utoronto.ca/~haiping/Publication/UMPCA_TNN09.pdf Uncorrelated multilinear principal component analysis for unsupervised multilinear subspace learning]," IEEE Trans. Neural Netw., vol. 20, no. 11, pp. 1820–1836, Nov. 2009.</ref> In contrast, the uncorrelated MPCA (UMPCA) generates uncorrelated multilinear features.<ref name="UMPCA"/> == Feature selection == MPCA produces tensorial features. Supervised MPCA feature selection is used in object recognition<ref name="MPCA">, M. A. O. Vasilescu, D. Terzopoulos (2003) [http://www.cs.toronto.edu/~maov/tensorfaces/cvpr03.pdf "Multilinear Subspace Analysis of Image Ensembles"], "Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR’03), Madison, WI, June, 2003"</ref> while unsupervised MPCA feature selection is employed in visualization task.<ref>H. Lu, H.-L. Eng, M. Thida, and K.N. Plataniotis, "[http://www.dsp.utoronto.ca/~haiping/Publication/CrowdMPCA_CIKM2010.pdf Visualization and Clustering of Crowd Video Content in MPCA Subspace]," in Proceedings of the 19th ACM Conference on Information and Knowledge Management (CIKM 2010) , Toronto, ON, Canada, October, 2010.</ref> == Extensions == Various extensions of MPCA have been developed:<ref>{{cite journal |first=Haiping |last=Lu |first2=K.N. |last2=Plataniotis |first3=A.N. |last3=Venetsanopoulos |url=http://www.dsp.utoronto.ca/~haiping/Publication/SurveyMSL_PR2011.pdf |title=A Survey of Multilinear Subspace Learning for Tensor Data |journal=Pattern Recognition |volume=44 |number=7 |pages=1540–1551 |year=2011 |doi=10.1016/j.patcog.2011.01.004 }}</ref> *Uncorrelated MPCA (UMPCA) <ref name="UMPCA"/> *[[Boosting (meta-algorithm)|Boosting]]+MPCA<ref>H. Lu, K. N. Plataniotis and A. N. Venetsanopoulos, "[http://www.hindawi.com/journals/ivp/2009/713183.html Boosting Discriminant Learners for Gait Recognition using MPCA Features]", EURASIP Journal on Image and Video Processing, Volume 2009, Article ID 713183, 11 pages, 2009. {{doi|10.1155/2009/713183}}.</ref> *Non-negative MPCA (NMPCA) <ref>Y. Panagakis, C. Kotropoulos, G. R. Arce, "Non-negative multilinear principal component analysis of auditory temporal modulations for music genre classification", IEEE Trans. on Audio, Speech, and Language Processing, vol. 18, no. 3, pp. 576–588, 2010.</ref> *Robust MPCA (RMPCA) <ref>K. Inoue, K. Hara, K. Urahama, "Robust multilinear principal component analysis", Proc. IEEE Conference on Computer Vision, 2009, pp. 591–597.</ref> == Resources == * '''Matlab code''': [http://www.mathworks.com/matlabcentral/fileexchange/26168 MPCA]. * '''Matlab code''': [http://www.mathworks.com/matlabcentral/fileexchange/35432 UMPCA (including data)]. ==References== {{Reflist}} [[Category:Dimension reduction]] [[Category:Machine learning]] [[Category:Multivariate statistics]]'