Matrix normal distribution

Matrix normal

Notation	${\mathcal {MN}}_{n,p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$
Parameters	$\mathbf {M}$ location (real $n\times p$ matrix) $\mathbf {U}$ scale (positive-definite real $n\times n$ matrix) $\mathbf {V}$ scale (positive-definite real $p\times p$ matrix)
Support	$\mathbf {X} \in \mathbb {R} ^{n\times p}$
PDF	${\frac {\exp \left(-{\frac {1}{2}}\,\mathrm {tr} \left[\mathbf {V} ^{-1}(\mathbf {X} -\mathbf {M} )^{T}\mathbf {U} ^{-1}(\mathbf {X} -\mathbf {M} )\right]\right)}{(2\pi )^{np/2}|\mathbf {V} |^{n/2}|\mathbf {U} |^{p/2}}}$
Mean	$\mathbf {M}$
Variance	$\mathbf {U}$ (among-row) and $\mathbf {V}$ (among-column)

In statistics, the matrix normal distribution is a probability distribution that is a generalization of the multivariate normal distribution to matrix-valued random variables.

Definition

The probability density function for the random matrix X (n × p) that follows the matrix normal distribution ${\mathcal {MN}}_{n,p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$ has the form:

$p(\mathbf {X} |\mathbf {M} ,\mathbf {U} ,\mathbf {V} )={\frac {\exp \left(-{\frac {1}{2}}\,\mathrm {tr} \left[\mathbf {V} ^{-1}(\mathbf {X} -\mathbf {M} )^{T}\mathbf {U} ^{-1}(\mathbf {X} -\mathbf {M} )\right]\right)}{(2\pi )^{np/2}|\mathbf {V} |^{n/2}|\mathbf {U} |^{p/2}}}$

where M is n × p, U is n × n and V is p × p.

The matrix normal is related to the multivariate normal distribution in the following way:

$\mathbf {X} \sim {\mathcal {MN}}_{n\times p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} ),$

if and only if

$\mathrm {vec} (\mathbf {X} )\sim {\mathcal {N}}_{np}(\mathrm {vec} (\mathbf {M} ),\mathbf {V} \otimes \mathbf {U} )$

where $\otimes$ denotes the Kronecker product and $\mathrm {vec} (\mathbf {M} )$ denotes the vectorization of $\mathbf {M}$.

Properties

If $\mathbf {X} \sim {\mathcal {MN}}_{n\times p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$, then we have the following properties:

Expected values

The mean, or expected value is:

$E[\mathbf {X} ]=\mathbf {M}$

and we have the following second-order expectations:

$E[(\mathbf {X} -\mathbf {M} )(\mathbf {X} -\mathbf {M} )^{T}]=\mathbf {U} \operatorname {tr} (\mathbf {V} )$

$E[(\mathbf {X} -\mathbf {M} )^{T}(\mathbf {X} -\mathbf {M} )]=\mathbf {V} \operatorname {tr} (\mathbf {U} )$

where $\operatorname {tr}$ denotes trace.

More generally, for appropriately dimensioned matrices A,B,C:

$E[\mathbf {X} \mathbf {A} \mathbf {X} ^{T}]=\mathbf {U} \operatorname {tr} (\mathbf {A} ^{T}\mathbf {V} )+\mathbf {MAM} ^{T}$

$E[\mathbf {X} ^{T}\mathbf {B} \mathbf {X} ]=\mathbf {V} \operatorname {tr} (\mathbf {U} \mathbf {B} ^{T})+\mathbf {M} ^{T}\mathbf {BM}$

$E[\mathbf {X} \mathbf {C} \mathbf {X} ]=\mathbf {U} \mathbf {C} ^{T}\mathbf {V} +\mathbf {MCM}$

Transformation

Transpose transform:

$\mathbf {X} ^{T}\sim {\mathcal {MN}}_{n\times p}(\mathbf {M} ^{T},\mathbf {V} ,\mathbf {U} )$

Linear transform: let D (r-by-n), be of full rank r ≤ n and C (p-by-s), be of full rank s ≤ p, then:

$\mathbf {DXC} \sim {\mathcal {MN}}_{n\times p}(\mathbf {DMC} ,\mathbf {DUD} ^{T},\mathbf {C} ^{T}\mathbf {VC} )$

Example

Let's imagine a sample of n independent p-dimensional random variables identically distributed according to a multivariate normal distribution:

$\mathbf {Y} _{i}\sim {\mathcal {N}}_{p}({\boldsymbol {\mu }},{\boldsymbol {\Sigma }}){\text{ with }}i\in \{1,\ldots ,n\}$.

When defining the n × p matrix $\mathbf {X}$ for which the ith row is $\mathbf {Y} _{i}$, we obtain:

$\mathbf {X} \sim {\mathcal {MN}}_{n\times p}(\mathbf {M} ,\mathbf {U} ,\mathbf {V} )$

where each row of $\mathbf {M}$ is equal to ${\boldsymbol {\mu }}$, that is $\mathbf {M} =\mathbf {1} _{n}\times {\boldsymbol {\mu }}^{T}$, $\mathbf {U}$ is the n × n identity matrix, that is the rows are independent, and $\mathbf {V} ={\boldsymbol {\Sigma }}$.

Relation to other distributions

Dawid (1981) provides a discussion of the relation of the matrix-valued normal distribution to other distributions, including the Wishart distribution, Inverse Wishart distribution and matrix t-distribution, but uses different notation from that employed here.

See also

• Multivariate normal distribution.

References

• Dawid, A.P. (1981). "Some matrix-variate distribution theory: Notational considerations and a Bayesian application". Biometrika. 68 (1): 265–274. doi:10.1093/biomet/68.1.265. JSTOR 2335827. MR 0614963.
• Dutilleul, P (1999). "The MLE algorithm for the matrix normal distribution". Journal of Statistical Computation and Simulation. 64 (2): 105–123. doi:10.1080/00949659908811970.
• Arnold, S.F. (1981), The theory of linear models and multivariate analysis, New York: John Wiley & Sons, ISBN 0471050652