Matrix normal distribution

Matrix normal

Notation	${\mathcal {MN}}_{n,p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$
Parameters	$\mathbf {M}$ location (real $n\times p$ matrix) $\mathbf {U}$ scale (positive-definite real $n\times n$ matrix) $\mathbf {V}$ scale (positive-definite real $p\times p$ matrix)
Support	$\mathbf {X} \in \mathbb {R} ^{n\times p}$
PDF	${\frac {\exp \left(-{\frac {1}{2}}\,\mathrm {tr} \left[\mathbf {V} ^{-1}(\mathbf {X} -\mathbf {M} )^{T}\mathbf {U} ^{-1}(\mathbf {X} -\mathbf {M} )\right]\right)}{(2\pi )^{np/2}|\mathbf {V} |^{n/2}|\mathbf {U} |^{p/2}}}$
Mean	$\mathbf {M}$
Variance	$\mathbf {U}$ (among-row) and $\mathbf {V}$ (among-column)

In statistics, the matrix normal distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables.

Definition

The probability density function for the random matrix X (n × p) that follows the matrix normal distribution ${\mathcal {MN}}_{n,p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$ has the form:

$p(\mathbf {X} |\mathbf {M} ,\mathbf {U} ,\mathbf {V} )={\frac {\exp \left(-{\frac {1}{2}}\,\mathrm {tr} \left[\mathbf {V} ^{-1}(\mathbf {X} -\mathbf {M} )^{T}\mathbf {U} ^{-1}(\mathbf {X} -\mathbf {M} )\right]\right)}{(2\pi )^{np/2}|\mathbf {V} |^{n/2}|\mathbf {U} |^{p/2}}}$

where $\mathrm {tr}$ denotes trace and M is n × p, U is n × n and V is p × p.

The matrix normal is related to the multivariate normal distribution in the following way:

$\mathbf {X} \sim {\mathcal {MN}}_{n\times p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} ),$

if and only if

$\mathrm {vec} (\mathbf {X} )\sim {\mathcal {N}}_{np}(\mathrm {vec} (\mathbf {M} ),\mathbf {V} \otimes \mathbf {U} )$

where $\otimes$ denotes the Kronecker product and $\mathrm {vec} (\mathbf {M} )$ denotes the vectorization of $\mathbf {M}$.

Proof

The equivalence between the above matrix normal and multivariate normal density functions can be shown using several properties of the trace and Kronecker product, as follows. We start with the argument of the exponent of the matrix normal PDF:

${\begin{aligned}&\;\;\;\;-{\frac {1}{2}}{\text{tr}}\left[\mathbf {V} ^{-1}(\mathbf {X} -\mathbf {M} )^{T}\mathbf {U} ^{-1}(\mathbf {X} -\mathbf {M} )\right]\\&=-{\frac {1}{2}}{\text{vec}}\left(\mathbf {X} -\mathbf {M} \right)^{T}{\text{vec}}\left(\mathbf {U} ^{-1}(\mathbf {X} -\mathbf {M} )\mathbf {V} ^{-1}\right)\\&=-{\frac {1}{2}}{\text{vec}}\left(\mathbf {X} -\mathbf {M} \right)^{T}\left(\mathbf {V} ^{-1}\otimes \mathbf {U} ^{-1}\right){\text{vec}}\left(\mathbf {X} -\mathbf {M} \right)\\&=-{\frac {1}{2}}\left[{\text{vec}}(\mathbf {X} )-{\text{vec}}(\mathbf {M} )\right]^{T}\left(\mathbf {V} \otimes \mathbf {U} \right)^{-1}\left[{\text{vec}}(\mathbf {X} )-{\text{vec}}(\mathbf {M} )\right]\end{aligned}}$

which is the argument of the exponent of the multivariate normal PDF. The proof is completed by using the determinant property: $|\mathbf {V} \otimes \mathbf {U} |=|\mathbf {V} |^{n}|\mathbf {U} |^{p}.$

Properties

If $\mathbf {X} \sim {\mathcal {MN}}_{n\times p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$, then we have the following properties:

Expected values

The mean, or expected value is:

$E[\mathbf {X} ]=\mathbf {M}$

and we have the following second-order expectations:

$E[(\mathbf {X} -\mathbf {M} )(\mathbf {X} -\mathbf {M} )^{T}]=\mathbf {U} \operatorname {tr} (\mathbf {V} )$

$E[(\mathbf {X} -\mathbf {M} )^{T}(\mathbf {X} -\mathbf {M} )]=\mathbf {V} \operatorname {tr} (\mathbf {U} )$

where $\operatorname {tr}$ denotes trace.

More generally, for appropriately dimensioned matrices A,B,C:

${\begin{aligned}E[\mathbf {X} \mathbf {A} \mathbf {X} ^{T}]&=\mathbf {U} \operatorname {tr} (\mathbf {A} ^{T}\mathbf {V} )+\mathbf {MAM} ^{T}\\E[\mathbf {X} ^{T}\mathbf {B} \mathbf {X} ]&=\mathbf {V} \operatorname {tr} (\mathbf {U} \mathbf {B} ^{T})+\mathbf {M} ^{T}\mathbf {BM} \\E[\mathbf {X} \mathbf {C} \mathbf {X} ]&=\mathbf {U} \mathbf {C} ^{T}\mathbf {V} +\mathbf {MCM} \end{aligned}}$

Transformation

Transpose transform:

$\mathbf {X} ^{T}\sim {\mathcal {MN}}_{p\times n}(\mathbf {M} ^{T},\mathbf {V} ,\mathbf {U} )$

Linear transform: let D (r-by-n), be of full rank r ≤ n and C (p-by-s), be of full rank s ≤ p, then:

$\mathbf {DXC} \sim {\mathcal {MN}}_{n\times p}(\mathbf {DMC} ,\mathbf {DUD} ^{T},\mathbf {C} ^{T}\mathbf {VC} )$

Example

Let's imagine a sample of n independent p-dimensional random variables identically distributed according to a multivariate normal distribution:

$\mathbf {Y} _{i}\sim {\mathcal {N}}_{p}({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}){\text{ with }}i\in \{1,\ldots ,n\}$.

When defining the n × p matrix $\mathbf {X}$ for which the ith row is $\mathbf {Y} _{i}$, we obtain:

$\mathbf {X} \sim {\mathcal {MN}}_{n\times p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$

where each row of $\mathbf {M}$ is equal to ${\boldsymbol {\mu }}$, that is $\mathbf {M} =\mathbf {1} _{n}\times {\boldsymbol {\mu }}^{T}$, $\mathbf {U}$ is the n × n identity matrix, that is the rows are independent, and $\mathbf {V} ={\boldsymbol {\Sigma }}$.

Relation to other distributions

Dawid (1981) provides a discussion of the relation of the matrix-valued normal distribution to other distributions, including the Wishart distribution, Inverse Wishart distribution and matrix t-distribution, but uses different notation from that employed here.

Maximum Likelihood Parameter Estimation

Given k matrices of size n × p, denoted $\mathbf {X} _{1},\mathbf {X} _{2},\ldots ,\mathbf {X} _{k}$ the maximum-likelihood estimate of the parameters of a matrix normal distribution can be obtained by maximizing the likelihood,

$\prod _{i=1}^{k}{\mathcal {MN}}_{n\times p}(\mathbf {X} _{i}\mid \mathbf {M} ,\mathbf {U} ,\mathbf {V} ).$

The solution for the mean has a closed form:

$\mathbf {M} ={\frac {1}{k}}\sum _{i=1}^{k}\mathbf {X} _{i}$

but covariance parameters do not. However at the maximum, they must satisfy:

$\mathbf {U} ={\frac {1}{kp}}\sum _{i=1}^{k}(\mathbf {X} _{i}-\mathbf {M} )\mathbf {V} ^{-1}(\mathbf {X} _{i}-\mathbf {M} )'$

and

$\mathbf {V} ={\frac {1}{kn}}\sum _{i=1}^{k}(\mathbf {X} _{i}-\mathbf {M} )'\mathbf {U} ^{-1}(\mathbf {X} _{i}-\mathbf {M} )$

which allows an iterative solution. See for example and references therein. It should be noted that the covariance parameters are non-identifiable in the sense that for any scale factor, $s>0$,

${\mathcal {MN}}_{n\times p}(\mathbf {X} _{i}\mid \mathbf {M} ,\mathbf {U} ,\mathbf {V} )={\mathcal {MN}}_{n\times p}(\mathbf {X} _{i}\mid \mathbf {M} ,s\mathbf {U} ,{\frac {1}{s}}\mathbf {V} )$

See also

• Multivariate normal distribution.

References

• Dawid, A.P. (1981). "Some matrix-variate distribution theory: Notational considerations and a Bayesian application". Biometrika. 68 (1): 265–274. doi:10.1093/biomet/68.1.265. JSTOR 2335827. MR 0614963.
• Dutilleul, P (1999). "The MLE algorithm for the matrix normal distribution". Journal of Statistical Computation and Simulation. 64 (2): 105–123. doi:10.1080/00949659908811970.
• Arnold, S.F. (1981), The theory of linear models and multivariate analysis, New York: John Wiley & Sons, ISBN 0471050652