Logarithmic norm

In mathematics, the logarithmic norm is a real-valued functional on operators, and is derived from either an inner product, a vector norm, or its induced operator norm. The logarithmic norm was independently introduced by Germund Dahlquist and Sergei Lozinskiĭ in 1958, for square matrices. It has since been extended to nonlinear operators and unbounded operators as well. The logaritmic norm has a wide range of applications, in particular in matrix theory, differential equations and numerical analysis.

Original Definition

Let $A$ be a square matrix and $\|\cdot \|$ be an induced matrix norm. The associated logarithmic norm $\mu$ of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle A} is defined

$\mu (A)=\lim \limits _{h\rightarrow 0^{+}}{\frac {\|I+hA\|-1}{h}}$

Here $I$ is the identity matrix of the same dimension as $A$, and $h$ is a real, positive number. The limit as $h\rightarrow 0^{-}$ equals $-\mu (-A)$, and is in general different from the logarithmic norm $\mu (A)$, as $-\mu (-A)\leq \mu (A)$ for all matrices.

While $\|A\|$ is always positive if $A\neq 0$, the logarithmic norm $\mu (A)$ may also take negative values, e.g. when $A$ is negative definite. Therefore, the logarithmic norm does not satisfy the axioms of a norm. The name logarithmic norm, which does not appear in the original reference, seems to originate from logarithmic differentiation, as the maximal growth rate of the logarithm of the norm of solutions to the differential equation

${\dot {x}}=Ax$

is $\mu (A)$. This is expressed by the differential inequality

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \frac{\mathrm d}{\mathrm d t} \log \|x\| \leq \mu(A),}

which is directly related to Grönwall's lemma.

Alternative definitions

If the vector norm is an inner product norm, as in a Hilbert space, then the logarithmic norm is the smallest number $\mu (A)$ such that for all $x$

$\Re \langle x,Ax\rangle \leq \mu (A)\cdot \|x\|^{2}$

Unlike the original definition, the latter expression also allows $A$ to be unbounded. Thus differential operators too can have logarithmic norms, allowing the use of the logarithmic norm both in algebra and in analysis. The modern, extended theory therefore prefers a definition based on inner products or duality. Both the operator norm and the logarithmic norm are then associated with extremal values of quadratic forms as follows:

$\|A\|^{2}=\sup _{x\neq 0}{\frac {\langle Ax,Ax\rangle }{\langle x,x\rangle }}\,;\qquad \mu (A)=\sup _{x\neq 0}{\frac {\Re \langle x,Ax\rangle }{\langle x,x\rangle }}$

Properties

Basic properties of the logarithmic norm of a matrix include:

1. $\mu (zI)=\Re \,(z)$
2. $\mu (A)\leq \|A\|$
3. $\mu (\gamma A)=\gamma \mu (A)\,$ for scalar Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \gamma > 0 }
4. $\mu (A+zI)=\mu (A)+\Re \,(z)$
5. $\mu (A+B)\leq \mu (A)+\mu (B)$
6. $\alpha (A)\leq \mu (A)\,$ where $\alpha (A)$ is the maximal real part of the eigenvalues of $A$
7. $\|\mathrm {e} ^{tA}\|\leq \mathrm {e} ^{t\mu (A)}\,$ for $t\geq 0$
8. $\mu (A)<0\,\Rightarrow \,\|A^{-1}\|\leq -1/\mu (A)$

Example logarithmic norms

The logarithmic norm of a matrix can be calculated as follows for the three most common norms. In these formulas, $a_{ij}$ represents the element on the $i$th row and $j$th column of a matrix $A$.

• $\mu _{1}(A)=\sup \limits _{j}(\Re (a_{jj})+\sum \limits _{i,i\neq j}|a_{ij}|)$

• $\displaystyle \mu _{2}(A)=\alpha (A+A^{\mathrm {T} })/2$

• $\mu _{\infty }(A)=\sup \limits _{i}(\Re (a_{ii})+\sum \limits _{j,j\neq i}|a_{ij}|)$

Applications in matrix theory and spectral theory

The logarithmic norm is related to the extreme values of the Rayleigh quotient. It holds that

$-\mu (-A)\leq {\frac {x^{\mathrm {T} }Ax}{x^{\mathrm {T} }x}}\leq \mu (A),$

and both extreme values are taken for some vectors $x\neq 0$. This also means that every eigenvalue Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \lambda_k} of $A$ satisfies

$-\mu (-A)\leq \Re \,\lambda _{k}\leq \mu (A)$.

More generally, the logarithmic norm is related to the numerical range of a matrix.

A matrix with $-\mu (-A)>0$ is positive definite, and one with $\mu (A)<0$ is negative definite. Such matrices have inverses. The inverse of a negative definite matrix is bounded by

$\|A^{-1}\|\leq -{\frac {1}{\mu (A)}}.$

Both the bounds on the inverse and on the eigenvalues hold irrespective of the choice of vector (matrix) norm. Some results only hold for inner product norms, however. For example, if $R$ is a rational function with the property

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \real \, (z)\leq 0 \, \Rightarrow \, |R(z)|\leq 1}

then, for inner product norms,

$\mu (A)\leq 0\,\Rightarrow \,\|R(A)\|\leq 1.$

Thus the matrix norm and logarithmic norms may be viewed as generalizing the modulus and real part, respectively, from complex numbers to matrices.

Applications in stability theory and numerical analysis

The logarithmic norm plays an important role in the stability analysis of a continuous dynamical system Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \dot x = Ax} . Its role is analogous to that of the matrix norm for a discrete dynamical system Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle x_{n+1} = Ax_n} .

In the simplest case, when $A$ is a scalar complex constant $\lambda$, the discrete dynamical system has stable solutions when Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle |\lambda|\leq 1} , while the differential equation has stable solutions when $\Re \,\lambda \leq 0$. When $A$ is a matrix, the discrete system has stable solutions if $\|A\|\leq 1$. In the continuous system, the solutions are of the form $\mathrm {e} ^{tA}x(0)$. They are stable if $\|\mathrm {e} ^{tA}\|\leq 1$ for all $t\geq 0$, which follows from property 7 above, if $\mu (A)\leq 0$. In the latter case, $\|x\|$ is a Lyapunov function for the system.

Runge-Kutta methods for the numerical solution of ${\dot {x}}=Ax$ replace the differential equation by a discrete equation $x_{n+1}=R(hA)\cdot x_{n}$, where the rational function $R$ is characteristic of the method, and $h$ is the time step size. If Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle |R(z)|\leq 1} whenever $\Re \,(z)\leq 0$, then a stable differential equation, having $\mu (A)\leq 0$, will always result in a stable (contractive) numerical method, as Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \|R(hA)\|\leq 1} . Runge-Kutta methods having this property are called A-stable.

Retaining the same form, the results can, under additional assumptions, be extended to nonlinear systems as well as to semigroup theory, where the crucial advantage of the logarithmic norm is that it discriminates between forward and reverse time evolution and can establish whether the problem is well posed. Similar results also apply in the stability analysis in control theory, where there is a need to discriminate between positive and negative feedback.

Applications to elliptic differential operators

In connection with differential operators it is common to use inner products and integration by parts. In the simplest case we consider functions satisfying $u(0)=u(1)=0$ with inner product

$\langle u,v\rangle =\int _{0}^{1}uv\,\mathrm {d} x.$

Then it holds that

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \langle u,u''\rangle = -\langle u',u'\rangle \leq -\pi^2\|u\|^2,}

where the equality on the left represents integration by parts, and the inequality to the right is a Sobolev inequality. In the latter, equality is attained for the function $\sin \,\pi x$, implying that the constant $-\pi ^{2}$ is the best possible. Thus

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle \langle u, Au\rangle \leq -\pi^2 \|u\|^2}

for the differential operator $A=\mathrm {d} ^{2}/\mathrm {d} x^{2}$, which implies that

$\mu ({\frac {\mathrm {d} ^{2}}{\mathrm {d} x^{2}}})=-\pi ^{2}.$

As an operator satisfying $\langle u,Au\rangle >0$ is called elliptic, the logarithmic norm quantifies the (strong) ellipticity of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle -\mathrm d^2/\mathrm dx^2} . Thus, if $A$ is elliptic, then $\mu (-A)<0$, and is invertible given proper data.

If a finite difference method is used to solve $-u''=f$, the problem is replaced by an algebraic equation Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle Tu=f} . The matrix $T$ will typically inherit the ellipticity, i.e., $-\mu (-T)>0$, showing that $T$ is positive definite and therefore invertible.

These results carry over to the Poisson equation as well as to other numerical methods such as the Finite element method.

Extensions to nonlinear maps

For nonlinear operators the operator norm and logarithmic norm are defined in terms of the inequalities

$l(f)\cdot \|u-v\|\leq \|f(u)-f(v)\|\leq L(f)\cdot \|u-v\|,$

where $L(f)$ is the least upper bound Lipschitz constant of $f$, and $l(f)$ is the greatest lower bound Lipschitz constant; and

$m(f)\cdot \|u-v\|^{2}\leq \langle u-v,f(u)-f(v)\rangle \leq M(f)\cdot \|u-v\|^{2},$

where $M(f)$ is the leat upper bound logarithmic Lipschitz constant of $f$, and $l(f)$ is the greatest lower bound logarithmic Lipschitz constant. Here it holds that $m(f)=-M(-f)$ (compare above) and, analogously, $l(f)=L(f^{-1})^{-1}$.

For nonlinear operators that are Lipschitz continuous, it then holds that

$M(f)=\lim _{h\rightarrow 0^{+}}{\frac {L(I+hf)-1}{h}}.$

An operator having either $m(f)>0$ or $M(f)<0$ is called uniformly monotone. An operator satisfying Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "http://localhost:6011/en.wikipedia.org/v1/":): {\displaystyle L(f)<1} is called contractive. This extension offers many connections to fixed point theory, and critical point theory.

The theory becomes completely analogous to the theory of the logarithmic norm for matrices, but is more complicated as the domains of the operators need to be given close attention, as in the case with unbounded operators. Property 8 of the logarithmic norm above carries over, independently of the choice of vector norm, and it holds that

$M(f)<0\,\Rightarrow \,L(f^{-1})\leq -{\frac {1}{M(f)}},$

which quantifies the Uniform Monotonicity Theorem due to Browder & Minty (1963).