Generalized inverse

"Pseudoinverse" redirects here. For the Moore–Penrose inverse, sometimes referred to as "the pseudoinverse", see Moore–Penrose inverse.

In mathematics, and in particular, algebra, a generalized inverse of an element x is an element y that has some properties of an inverse element but not necessarily all of them. Generalized inverses can be defined in any mathematical structure that involves associative multiplication, that is, in a semigroup. This article describes generalized inverses of a matrix $A$.

A matrix $A^{\mathrm {g} }\in \mathbb {R} ^{n\times m}$ is a generalized inverse of a matrix $A\in \mathbb {R} ^{m\times n}$ if $AA^{\mathrm {g} }A=A.$

The purpose of constructing a generalized inverse of a matrix is to obtain a matrix that can serve as an inverse in some sense for a wider class of matrices than invertible matrices. A generalized inverse exists for an arbitrary matrix, and when a matrix has a regular inverse, this inverse is its unique generalized inverse.

Motivation

Consider the linear system

$Ax=y$

where $A$ is an $n\times m$ matrix and $y\in {\mathcal {R}}(A),$ the column space of $A$. If $A$ is nonsingular (which implies $n=m$) then $x=A^{-1}y$ will be the solution of the system. Note that, if $A$ is nonsingular, then

$AA^{-1}A=A.$

Now suppose $A$ is rectangular ($n\neq m$), or square and singular. Then we need a right candidate $G$ of order $m\times n$ such that for all $y\in {\mathcal {R}}(A),$

$AGy=y.$

That is, $x=Gy$ is a solution of the linear system $Ax=y$. Equivalently, we need a matrix $G$ of order $m\times n$ such that

$AGA=A.$

Hence we can define the generalized inverse or g-inverse as follows: Given an $m\times n$ matrix $A$, an $n\times m$ matrix $G$ is said to be a generalized inverse of $A$ if $AGA=A.$‍ The matrix $A^{-1}$ has been termed a regular inverse of $A$ by some authors.

Penrose conditions

The Penrose conditions are used to classify different generalized inverses $A^{\mathrm {g} }\in \mathbb {R} ^{n\times m}$ of $A\in \mathbb {R} ^{m\times n}$:

1. $AA^{\mathrm {g} }A=A$
2. $A^{\mathrm {g} }AA^{\mathrm {g} }=A^{\mathrm {g} }$
3. $\left(AA^{\mathrm {g} }\right)^{*}=AA^{\mathrm {g} }$
4. $\left(A^{\mathrm {g} }A\right)^{*}=A^{\mathrm {g} }A,$

where ${}^{*}$ indicates conjugate transpose.

Definitions

An $I$-inverse of $A$, where $I\subset \{1,2,3,4\}$, is a generalized inverse of $A$ which satisfies the Penrose conditions listed in $I$. A generalized inverse of $A$ is a $\{1\}$-inverse of $A$. A reflexive generalized inverse of $A$ is a $\{1,2\}$-inverse of $A$. A pseudoinverse of $A$ is a $\{1,2,3,4\}$-inverse of $A$. We denote the pseudoinverse of $A$ by $A^{+}$.

When $A$ is non-singular, any generalized inverse $A^{\mathrm {g} }=A^{-1}$ and is unique. Otherwise, there are an infinite number of $I$-inverses for a given $I$ with less than 4 elements. However, the pseudoinverse is unique.

Types

Specific kinds of generalized inverses are:

• Moore–Penrose inverse or pseudoinverse, after the pioneering works by E. H. Moore and Roger Penrose.
• One-sided inverse
  • A right inverse of $A$ is $A^{g}\in \mathbb {R} ^{n\times m}$ such that $AA^{g}=I_{m}$, where $I_{m}$ is the $m\times m$ identity matrix. A right inverse is a $\{1,2,3\}$-inverse of $A$, and exists if ${\text{rank}}(A)=m$.
  • A left inverse of $A$ is $A^{g}\in \mathbb {R} ^{n\times m}$ such that $A^{g}A=I_{n}$, where $I_{n}$ is the $n\times n$ identity matrix. A left inverse is a $\{1,2,4\}$-inverse of $A$, and exists if $\operatorname {rank} (A)=n$.

• Bott–Duffin inverse
• Drazin inverse

Characterization by singular value decomposition

Let $A\in \mathbb {R} ^{m\times n}$, and $A=U{\begin{bmatrix}\Sigma _{1}&0\\0&0\end{bmatrix}}V^{\textsf {T}}$ be its singular-value decomposition. Then for any generalized inverse $A^{g}$, there exist matrices $X$, $Y$, and $Z$ such that

$$A^{g}=V{\begin{bmatrix}\Sigma _{1}^{-1}&X\\Y&Z\end{bmatrix}}U^{\textsf {T}}.$$

Conversely, any choice of $X$, $Y$, and $Z$ for matrix of this form is a generalized inverse of $A$. The $\{1,2\}$-inverses are exactly those for which $Z=Y\Sigma _{1}X$, the $\{1,3\}$-inverses are exactly those for which $X=0$, and the $\{1,4\}$-inverses are exactly those for which $Y=0$. In particular, the pseudoinverse is given by $X=Y=Z=0$:

$$A^{+}=V{\begin{bmatrix}\Sigma _{1}^{-1}&0\\0&0\end{bmatrix}}U^{\textsf {T}}.$$

Examples

Reflexive generalized inverse

Let

$A={\begin{bmatrix}1&2&3\\4&5&6\\7&8&9\end{bmatrix}},\quad G={\begin{bmatrix}-{\frac {5}{3}}&{\frac {2}{3}}&0\\[4pt]{\frac {4}{3}}&-{\frac {1}{3}}&0\\[4pt]0&0&0\end{bmatrix}}.$

Since $\det(A)=0$, $A$ is singular and has no regular inverse. However, $A$ and $G$ satisfy conditions (1) and (2), but not (3) or (4). Hence, $G$ is a reflexive generalized inverse of $A$.

One-sided inverse

Let

$A={\begin{bmatrix}1&2&3\\4&5&6\end{bmatrix}},\quad A_{\mathrm {R} }^{-1}={\begin{bmatrix}-{\frac {17}{18}}&{\frac {8}{18}}\\[4pt]-{\frac {2}{18}}&{\frac {2}{18}}\\[4pt]{\frac {13}{18}}&-{\frac {4}{18}}\end{bmatrix}}.$

Since $A$ is not square, $A$ has no regular inverse. However, $A_{\mathrm {R} }^{-1}$ is a right inverse of $A$. The matrix $A$ has no left inverse.

Inverse of other semigroups (or rings)

The element b is a generalized inverse of an element a if and only if $a\cdot b\cdot a=a$, in any semigroup (or ring, since the multiplication function in any ring is a semigroup).

The generalized inverses of the element 3 in the ring $\mathbb {Z} /12\mathbb {Z}$ are 3, 7, and 11, since in the ring $\mathbb {Z} /12\mathbb {Z}$:

$3\cdot 3\cdot 3=3$

$3\cdot 7\cdot 3=3$

$3\cdot 11\cdot 3=3$

The generalized inverses of the element 4 in the ring $\mathbb {Z} /12\mathbb {Z}$ are 1, 4, 7, and 10, since in the ring $\mathbb {Z} /12\mathbb {Z}$:

$4\cdot 1\cdot 4=4$

$4\cdot 4\cdot 4=4$

$4\cdot 7\cdot 4=4$

$4\cdot 10\cdot 4=4$

If an element a in a semigroup (or ring) has an inverse, the inverse must be the only generalized inverse of this element, like the elements 1, 5, 7, and 11 in the ring $\mathbb {Z} /12\mathbb {Z}$.

In the ring $\mathbb {Z} /12\mathbb {Z}$, any element is a generalized inverse of 0, however, 2 has no generalized inverse, since there is no b in $\mathbb {Z} /12\mathbb {Z}$ such that $2\cdot b\cdot 2=2$.

Construction

The following characterizations are easy to verify:

• A right inverse of a non-square matrix $A$ is given by $A_{\mathrm {R} }^{-1}=A^{\mathsf {T}}\left(AA^{\mathsf {T}}\right)^{-1}$, provided A has full row rank.

• A left inverse of a non-square matrix $A$ is given by $A_{\mathrm {L} }^{-1}=\left(A^{\mathsf {T}}A\right)^{-1}A^{\mathsf {T}}$, provided A has full column rank.

• If $A=BC$ is a rank factorization, then $G=C_{\mathrm {R} }^{-1}B_{\mathrm {L} }^{-1}$ is a g-inverse of $A$, where $C_{\mathrm {R} }^{-1}$ is a right inverse of $C$ and $B_{\mathrm {L} }^{-1}$ is left inverse of $B$.

• If $A=P{\begin{bmatrix}I_{r}&0\\0&0\end{bmatrix}}Q$ for any non-singular matrices $P$ and $Q$, then $G=Q^{-1}{\begin{bmatrix}I_{r}&U\\W&V\end{bmatrix}}P^{-1}$ is a generalized inverse of $A$ for arbitrary $U,V$ and $W$.

• Let $A$ be of rank $r$. Without loss of generality, let

$A={\begin{bmatrix}B&C\\D&E\end{bmatrix}},$

where $B_{r\times r}$ is the non-singular submatrix of $A$. Then,

$G={\begin{bmatrix}B^{-1}&0\\0&0\end{bmatrix}}$

is a generalized inverse of $A$.

• $A^{(1,4)}AA^{(1,3)}=A^{+}$for any $\{1,4\}$-inverse $A^{(1,4)}$and $\{1,3\}$-inverse $A^{(1,3)}$. In particular, $A^{+}AA^{(1,3)}=A^{+}$for any $\{1,3\}$-inverse $A^{(1,3)}$.

Uses

Any generalized inverse can be used to determine whether a system of linear equations has any solutions, and if so to give all of them. If any solutions exist for the n × m linear system

$Ax=b$,

with vector $x$ of unknowns and vector $b$ of constants, all solutions are given by

$x=A^{\mathrm {g} }b+\left[I-A^{\mathrm {g} }A\right]w$,

parametric on the arbitrary vector $w$, where $A^{\mathrm {g} }$ is any generalized inverse of $A$. Solutions exist if and only if $A^{\mathrm {g} }b$ is a solution, that is, if and only if $AA^{\mathrm {g} }b=b$. If A has full column rank, the bracketed expression in this equation is the zero matrix and so the solution is unique.

Transformation consistency properties

In practical applications it is necessary to identify the class of matrix transformations that must be preserved by a generalized inverse. For example, the Moore–Penrose inverse, $A^{+},$ satisfies the following definition of consistency with respect to transformations involving unitary matrices U and V:

$(UAV)^{+}=V^{*}A^{+}U^{*}$.

The Drazin inverse, $A^{\mathrm {D} }$ satisfies the following definition of consistency with respect to similarity transformations involving a nonsingular matrix S:

$\left(SAS^{-1}\right)^{\mathrm {D} }=SA^{\mathrm {D} }S^{-1}$.

The unit-consistent (UC) inverse, $A^{\mathrm {U} },$ satisfies the following definition of consistency with respect to transformations involving nonsingular diagonal matrices D and E:

$(DAE)^{\mathrm {U} }=E^{-1}A^{\mathrm {U} }D^{-1}$.

The fact that the Moore–Penrose inverse provides consistency with respect to rotations (which are orthonormal transformations) explains its widespread use in physics and other applications in which Euclidean distances must be preserved. The UC inverse, by contrast, is applicable when system behavior is expected to be invariant with respect to the choice of units on different state variables, e.g., miles versus kilometers.

See also

• Block matrix pseudoinverse
• Proofs involving the Moore–Penrose inverse
• Regular semigroup

References

• Ben-Israel, Adi; Greville, Thomas N.E. (2003). Generalized inverses: Theory and applications (2nd ed.). New York, NY: Springer. doi:10.1007/b97366. ISBN 978-0-387-00293-4.
• Campbell, S. L.; Meyer, Jr., C. D. (1991). Generalized Inverses of Linear Transformations. Dover. ISBN 978-0-486-66693-8.
• Horn, Roger A.; Johnson, Charles R. (1985), Matrix Analysis, Cambridge University Press, ISBN 978-0-521-38632-6.
• James, M. (June 1978). "The generalised inverse". Mathematical Gazette. 62 (420): 109–114. doi:10.2307/3617665. JSTOR 3617665.
• Nakamura, Yoshihiko (1991). Advanced Robotics: Redundancy and Optimization. Addison-Wesley. ISBN 978-0201151985.
• Rao, C. Radhakrishna; Mitra, Sujit Kumar (1971). Generalized Inverse of Matrices and its Applications. New York: John Wiley & Sons. pp. 240. ISBN 978-0-471-70821-6.
• Zheng, B; Bapat, R. B. (2004). "Generalized inverse A(2)T,S and a rank equation". Applied Mathematics and Computation. 155 (2): 407–415. doi:10.1016/S0096-3003(03)00786-0.