View source for Hilbert projection theorem

{{Short description|On closed convex subsets in Hilbert space}}
{{One source|date=February 2020}}
In mathematics, the '''Hilbert projection theorem''' is a famous result of [[convex analysis]] that says that for every vector <math>x</math> in a [[Hilbert space]] <math>H</math> and every nonempty closed convex <math>C \subseteq H,</math> there exists a unique vector <math>m \in C</math> for which <math>\|c - x\|</math> is minimized over the vectors <math>c \in C</math>; that is, such that <math>\|m - x\| \leq \|c - x\|</math> for every <math>c \in C.</math>

== Finite dimensional case ==

Some intuition for the theorem can be obtained by considering the [[first order condition]] of the optimization problem.

Consider a finite dimensional real Hilbert space <math>H</math> with a subspace <math>C</math> and a point <math>x.</math> If <math>m \in C</math> is a {{em|[[Minimum point|minimizer]]}} or {{em|[[minimum point]]}} of the function <math>N : C \to \R</math> defined by <math>N(c) := \|c - x\|</math> (which is the same as the minimum point of <math>c \mapsto \|c - x\|^2</math>), then derivative must be zero at <math>m.</math> 

In matrix derivative notation:<ref>{{cite web|last1=Petersen|first1=Kaare|title=The Matrix Cookbook|url=https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf|access-date=9 January 2021}}</ref>
<math display=block>\begin{aligned}
\partial \lVert x - c \rVert^2 &= \partial \langle c - x, c - x \rangle \\
&= 2 \langle c - x, \partial c\rangle
\end{aligned}</math>
Since <math>\partial c</math> is a vector in <math>C</math> that represents an arbitrary tangent direction, it follows that <math>m - x</math> must be orthogonal to every vector in <math>C.</math>

== Statement ==

{{math theorem|name=Hilbert projection theorem|math_statement=For every vector <math>x</math> in a [[Hilbert space]] <math>H</math> and every nonempty closed convex <math>C \subseteq H,</math> there exists a unique vector <math>m \in C</math> for which <math>\lVert x - m \rVert</math> is equal to <math>\delta := \inf_{c \in C} \|x - c\|.</math> 

If the closed subset <math>C</math> is also a [[vector subspace]] of <math>H</math> then this minimizer <math>m</math> is the unique element in <math>C</math> such that <math>x - m</math> is [[Orthogonal vectors (inner product space)|orthogonal]] to <math>C.</math> 
}}

=== Detailed elementary proof ===

{{math proof|title=Proof that a minimum point <math>y</math> exists|drop=hidden|proof= 
Let <math>\delta := \inf_{c \in C} \|x - c\|</math> be the distance between <math>x</math> and <math>C,</math> <math>\left(c_n\right)_{n=1}^{\infty}</math> a sequence in <math>C</math> such that the distance squared between <math>x</math> and <math>c_n</math> is less than or equal to <math>\delta^2 + 1/n.</math> Let <math>n</math> and <math>m</math> be two integers, then the following equalities are true:
<math display=block>\left\|c_n - c_m\right\|^2 = \left\|c_n - x\right\|^2 + \left\|c_m - x\right\|^2 - 2 \left\langle c_n - x \, , \, c_m - x\right\rangle</math>
and
<math display=block>4 \left\|\frac{c_n + c_m}2 - x\right\|^2 = \left\|c_n - x\right\|^2 + \left\|c_m - x\right\|^2 + 2 \left\langle c_n - x \, , \, c_m - x\right\rangle</math>
Therefore
<math display=block>\left\|c_n - c_m\right\|^2 = 2 \left\|c_n - x\right\|^2 + 2\left\|c_m - x\right\|^2 - 4\left\|\frac{c_n + c_m}2 - x\right\|^2</math>
(This equation is the same as [[Median (geometry)#Length of a median expressed in terms of lengths of sides|the formula <math>a^2 = 2 b^2 + 2 c^2 - 4 M_a^2</math>]] for the length <math>M_a</math> of a [[Median (geometry)|median]] in a triangle with sides of length <math>a, b,</math> and <math>c,</math> where specifically, the triangle's vertices are <math>x, c_m, c_n</math>).

By giving an upper bound to the first two terms of the equality and by noticing that the midpoint of <math>c_n</math> and <math>c_m</math> belong to <math>C</math> and has therefore a distance greater than or equal to <math>\delta</math> from <math>x,</math> it follows that:
<math display=block>\|c_n - c_m\|^2 \; \leq \; 2\left(\delta^2 + \frac{1}{n}\right) + 2\left(\delta^2 + \frac{1}{m}\right) - 4\delta^2 = 2\left(\frac{1}{n} + \frac{1}{m}\right)</math>

The last inequality proves that <math>\left(c_n\right)_{n=1}^{\infty}</math> is a [[Cauchy sequence]]. Since <math>C</math> is complete, the sequence is therefore convergent to a point <math>m \in C,</math> whose distance from <math>x</math> is minimal.
<math>\blacksquare</math>
}}

{{math proof|title=Proof that <math>m</math> is unique|drop=hidden|proof= 
Let <math>m_1</math> and <math>m_2</math> be two [[minimum point]]s. Then:
<math display=block>\|m_2 - m_1\|^2 = 2\|m_1 - x\|^2 + 2\|m_2 - x\|^2 - 4 \left\|\frac{m_1 + m_2}2 - x\right\|^2</math>

Since <math>\frac{m_1 + m_2}2</math> belongs to <math>C,</math> we have <math>\left\|\frac{m_1 + m_2} 2 - x\right\|^2 \geq \delta^2</math> and therefore
<math display=block>\|m_2 - m_1\|^2 \leq 2 \delta^2 + 2 \delta^2 - 4 \delta^2 = 0.</math>

Hence <math>m_1 = m_2,</math> which proves uniqueness. 
<math>\blacksquare</math>
}}

{{math proof|title=Proof of characterization of minimum point when <math>C</math> is a closed vector subspace|drop=hidden|proof=
Assume that <math>C</math> is a closed vector subspace of <math>H.</math> It must be shown the minimizer <math>m</math> is the unique element in <math>C</math> such that <math>\langle m - x, c \rangle = 0</math> for every <math>c \in C.</math> 
 
Proof that the condition is sufficient: 
Let <math>z \in C</math> be such that <math>\langle z - x, c \rangle = 0</math> for all <math>c \in C.</math> 
If <math>c \in C</math> then <math>c - z \in C</math> and so 
<math display=block>\|c-x\|^2 = \|(z-x) + (c-z)\|^2 = \|z-x\|^2 + \|c-z\|^2 + 2 \langle z-x, c-z \rangle = \|z-x\|^2 + \|c-z\|^2</math> 
which implies that <math>\|z-x\|^2 \leq \|c-x\|^2.</math> 
Because <math>c \in C</math> was arbitrary, this proves that <math>\|z-x\| = \inf_{c \in C} \|c - x\|</math> and so <math>z</math> is a minimum point. 

Proof that the condition is necessary: 
Let <math>m \in C</math> be the minimum point. Let <math>c \in C</math> and <math>t \in \R.</math>
Because <math>m + t c \in C,</math> the minimality of <math>m</math> guarantees that <math>\|m-x\| \leq \|(m + t c) - x\|.</math> Thus
<math display=block>\|(m + t c) - x\|^2 - \|m-x\|^2 = 2t\langle m-x, c\rangle + t^2 \|c\|^2</math>
is always non-negative and <math>\langle m-x, c\rangle</math> must be a real number. 
If <math>\langle m - x, c\rangle \neq 0</math> then the map <math>f(t) := 2t\langle m - x, c\rangle + t^2 \|c\|^2</math> has a minimum at <math>t_0 := - \frac{\langle m - x, c\rangle}{\|c\|^2}</math> and moreover, <math>f\left(t_0\right) < 0,</math> which is a contradiction. 
Thus <math>\langle m - x, c\rangle = 0.</math> 
<math>\blacksquare</math>
}}

=== Proof by reduction to a special case ===

It suffices to prove the theorem in the case of <math>x = 0</math> because the general case follows from the statement below by replacing <math>C</math> with <math>C - x.</math> 

{{math theorem|name=Hilbert projection theorem (case <math>x = 0</math>){{sfn|Rudin|1991|pp=306–309}}|math_statement=For every nonempty closed convex subset <math>C \subseteq H</math> of a [[Hilbert space]] <math>H,</math> there exists a unique vector <math>m \in C</math> such that <math>\inf_{c \in C} \| c \| = \| m \|.</math> 

Furthermore, letting <math>d := \inf_{c \in C} \| c \|,</math> if <math>\left(c_n\right)_{n=1}^{\infty}</math> is {{em|any}} sequence in <math>C</math> such that <math>\lim_{n \to \infty} \left\|c_n\right\| = d</math> in <math>\R</math><ref group=note>Because the norm <math>\| \cdot \| : H \to \R</math> is continuous, if <math>\lim_{n \to \infty} x_n</math> converges in <math>H</math> then necessarily <math>\lim_{n \to \infty} \left\|x_n\right\|</math> converges in <math>\R.</math> But in general, the converse is not guaranteed. However, under this theorem's hypotheses, knowing that <math>\lim_{n \to \infty} \left\|c_n\right\| = d</math> in <math>\R</math> {{em|is sufficient}} to conclude that <math>\lim_{n \to \infty} c_n</math> converges in <math>H.</math></ref> then <math>\lim_{n \to \infty} c_n = m</math> in <math>H.</math>
}}

{{math proof|drop=hidden|proof= 
Let <math>C</math> be as described in this theorem and let
<math display=block>d := \inf_{c \in C} \| c \|.</math>
This theorem will follow from the following lemmas.

{{math theorem|name=Lemma 1|math_statement=If <math>c_{\bull} := \left(c_n\right)_{n=1}^{\infty}</math> is {{em|any}} sequence in <math>C</math> such that <math>\lim_{n \to \infty} \left\|c_n\right\| = d</math> in <math>\R</math> then there exists some <math>c \in C</math> such that <math>\lim_{n \to \infty} c_n = c</math> in <math>H.</math> Furthermore, <math>\|c\| = d.</math>
}}

{{collapse top|title=Proof of Lemma 1|left=true}}
[[File:Parallelogram law.svg|thumb|Vectors involved in the parallelogram law: <math>\|x + y\|^2 + \|x - y\|^2 = 2 \|x\|^2 + 2 \|y\|^2.</math>]]
Because <math>C</math> is convex, if <math>m, n \in \N</math> then <math>\frac{1}{2}\left(c_m + c_n\right) \in C</math> so that by definition of the infimum, <math>d \leq \left\| \frac{1}{2}\left(c_m + c_n\right) \right\|,</math> which implies that <math>4d^2 \leq \left\|c_m + c_n\right\|^2.</math> 
By the [[parallelogram law]],
<math display=block>\left\|c_m + c_n\right\|^2 + \left\|c_m - c_n\right\|^2 = 2 \left\|c_m\right\|^2 + 2 \left\|c_n\right\|^2</math>
where <math>4d^2 \leq \left\|c_m + c_n\right\|^2</math> now implies 
<math display=block>4 d^2 + \left\|c_m - c_n\right\|^2 ~\leq~ 2 \left\|c_m\right\|^2 + 2 \left\|c_n\right\|^2</math>
and so
<math display=block>\begin{alignat}{4}
\left\|c_m - c_n\right\|^2 
~\leq~ 2 \left\|c_m\right\|^2 + 2 \left\|c_n\right\|^2 - 4 d^2
\end{alignat}</math>
The assumption <math>\lim_{n \to \infty} \left\|c_n\right\| = d</math> implies that the right hand side (RHS) of the above inequality can be made arbitrary close to <math>0</math> by making <math>m</math> and <math>n</math> sufficiently large.<ref group=note>Explicitly, this means that given any <math>\epsilon > 0</math> there exists some integer <math>N > 0</math> such that "the quantity" is <math>\,\leq \epsilon</math> whenever <math>m, n \geq N.</math> Here, "the quantity" refers to the inequality's right hand side <math>2 \left\|c_m\right\|^2 + 2 \left\|c_n\right\|^2 - 4 d^2</math> and later in the proof, "the quantity" will also refer to <math>\left\|c_m - c_n\right\|^2</math> and then <math>\left\|c_m - c_n\right\|.</math> By definition of "[[Cauchy sequence]]," <math>\left(c_n\right)_{n=1}^{\infty}</math> is Cauchy in <math>H</math> if and only if "the quantity" <math>\left\|c_m - c_n\right\|</math> satisfies this aforementioned condition.</ref> The same must consequently also be true of the inequality's left hand side <math>\left\|c_m - c_n\right\|^2</math> and thus also of <math>\left\|c_m - c_n\right\|,</math> which proves that <math>\left(c_n\right)_{n=1}^{\infty}</math> is a Cauchy sequence in <math>H.</math> 

Since <math>H</math> is complete, there exists some <math>c \in H</math> such that <math>\lim_{n \to \infty} c_n = c</math> in <math>H.</math> 
Because every <math>c_n</math> belongs to <math>C,</math> which is a [[Closed set|closed subset]] of <math>H,</math> their limit <math>c</math> must also belongs to this closed subset, which proves that <math>c \in C.</math> 
Since the norm <math>\| \,\cdot\, \| : H \to \R</math> is a continuous function, <math>\lim_{n \to \infty} c_n = c</math> in <math>H</math> implies that <math>\lim_{n \to \infty} \left\|c_n\right\| = \|c\|</math> in <math>\R.</math> But <math>\lim_{n \to \infty} \left\|c_n\right\| = d</math> also holds (by assumption) so that <math>\|c\| = d</math> (because limits in <math>\R</math> are unique). 
<math>\blacksquare</math>
{{collapse bottom}}

{{math theorem|name=Lemma 2|math_statement=A sequence <math>\left(c_n\right)_{n=1}^{\infty}</math> satisfying the hypotheses of Lemma 1 exists. 
}}

{{collapse top|title=Proof of Lemma 2|left=true}}
<!-- NOTE TO EDITORS.
A reader of this article who is looking at the proof of this Lemma might be newer to the subject, which is why this proof is so very detailed. 
--->
The existence of the sequence follows from the definition of the [[infimum]], as is now shown. 
The set <math>S := \{ \| c \| : c \in C \}</math> is a non-empty subset of non-negative real numbers and <math>d := \inf_{c \in C} \| c \| = \inf S.</math> 
Let <math>n \geq 1</math> be an integer. 
Because <math>\inf S < d + \frac{1}{n},</math> there exists some <math>s_n \in S</math> such that <math>s_n < d + \frac{1}{n}.</math> 
Since <math>s_n \in S,</math> <math>d = \inf S \leq s_n</math> holds (by definition of the infimum). Thus <math>d \leq s_n < d + \frac{1}{n}</math> and now the [[squeeze theorem]] implies that <math>\lim_{n \to \infty} s_n = d</math> in <math>\R.</math> (This first part of the proof works for any non-empty subset of <math>S \subseteq \R</math> for which <math>d := \inf_{s \in S} s</math> is finite). 

For every <math>n \in \N,</math> the fact that <math>s_n \in S = \{ \| c \| : c \in C \}</math> means that there exists some <math>c_n \in C</math> such that <math>s_n = \left\| c_n \right\|.</math> 
The convergence <math>\lim_{n \to \infty} s_n = d</math> in <math>\R</math> thus becomes <math>\lim_{n \to \infty} \left\|c_n\right\| = d</math> in <math>\R.</math> 
<math>\blacksquare</math>
{{collapse bottom}}

Lemma 2 and Lemma 1 together prove that there exists some <math>c \in C</math> such that <math>\|c\| = d.</math> 
Lemma 1 can be used to prove uniqueness as follows. 
Suppose <math>b \in C</math> is such that <math>\|b\| = d</math> and denote the sequence <math display=block>b, c, b, c, b, c, \ldots</math> by <math>\left(c_n\right)_{n=1}^{\infty}</math> so that the [[subsequence]] <math>\left(c_{2n}\right)_{n=1}^{\infty}</math> of even indices is the constant sequence <math>c, c, c, \ldots</math> while the subsequence <math>\left(c_{2n - 1}\right)_{n=1}^{\infty}</math> of odd indices is the constant sequence <math>b, b, b, \ldots.</math> 
Because <math>\left\|c_n\right\| = d</math> for every <math>n \in \N,</math> <math>\lim_{n \to \infty} \left\|c_n\right\| = \lim_{n \to \infty} d = d</math> in <math>\R,</math> which shows that the sequence <math>\left(c_n\right)_{n=1}^{\infty}</math> satisfies the hypotheses of Lemma 1. 
Lemma 1 guarantees the existence of some <math>x \in C</math> such that <math>\lim_{n \to \infty} c_n = x</math> in <math>H.</math> 
Because <math>\left(c_n\right)_{n=1}^{\infty}</math> converges to <math>x,</math> so do all of its subsequences. 
In particular, the subsequence <math>c, c, c, \ldots</math> converges to <math>x,</math> which implies that <math>x = c</math> (because limits in <math>H</math> are unique and this constant subsequence also converges to <math>c</math>). Similarly, <math>x = b</math> because the subsequence <math>b, b, b, \ldots</math> converges to both <math>x</math> and <math>b.</math> Thus <math>b = c,</math> which proves the theorem. <math>\blacksquare</math>
}}

== Consequences ==

{{math theorem|name=Proposition|math_statement=If <math>C</math> is a closed vector subspace of a Hilbert space <math>H</math> then<ref group=note>Technically, <math>H = K \oplus K^{\bot}</math> means that the addition map <math>K \times K^{\bot} \to H</math> defined by <math>(k, p) \mapsto k + p</math> is a surjective [[linear isomorphism]] and [[homeomorphism]]. See the article on [[complemented subspace]]s for more details.</ref>
<math display=block>H = C \oplus C^{\bot}.</math>
}}

{{collapse top|title=Proof{{sfn|Rudin|1991|pp=307−309}}|left=true}}
{{em|Proof that <math>C \cap C^{\bot} = \{ 0 \}</math>}}: 

If <math>c \in C \cap C^{\bot}</math> then 
<math>0 = \langle \,c, \,c\, \rangle = \|c\|^2,</math> 
which implies <math>c = 0.</math> 
<math>\blacksquare</math>

{{hr|1}}

{{em|Proof that <math>C^{\bot}</math> is a closed vector subspace of <math>H</math>}}: 

Let <math>P := \prod_{c \in C} \mathbb{F}</math> where <math>\mathbb{F}</math> is the underlying scalar field of <math>H</math> and define 
<math display=block>\begin{alignat}{4}
L : \,& H && \to    \,&& P \\
      & h && \mapsto\,&& \left(\langle \,h, \,c\, \rangle\right)_{c \in C} \\
\end{alignat}</math>
which is continuous and linear because this is true of each of its coordinates <math>h \mapsto \langle h, c \rangle.</math> 
The set <math>C^{\bot} = L^{-1}(0) = L^{-1}\left(\{ 0 \}\right)</math> is closed in <math>H</math> because <math>\{ 0 \}</math> is closed in <math>P</math> and <math>L : H \to P</math> is continuous. 
The kernel of any linear map is a vector subspace of its domain, which is why <math>C^{\bot} = \ker L</math> is a vector subspace of <math>H.</math> 
<math>\blacksquare</math>

{{hr|1}}

{{em|Proof that <math>C + C^{\bot} = H</math>}}: 

Let <math>x \in H.</math> 
The Hilbert projection theorem guarantees the existence of a unique <math>m \in C</math> such that <math>\|x - m\| \leq \|x - c\| \text{ for all } c \in C</math> (or equivalently, for all <math>x - c \in x - C</math>). 
Let <math>p := x - m</math> so that <math>x = m + p \in C + p</math> and it remains to show that <math>p \in C^{\bot}.</math> 
The inequality above can be rewritten as:
<math display=block>\|p\| \leq \|z\| \quad \text{ for all } z \in x - C.</math>
Because <math>m \in C</math> and <math>C</math> is a vector space, <math>m + C = C</math> and <math>C = - C,</math> which implies that <math>x - C = x + C = p + m + C = p + C.</math> 
The previous inequality thus becomes
<math display=block>\|p\| \leq \|z\| \quad \text{ for all } z \in p + C.</math>
or equivalently,
<math display=block>\|p\| \leq \|p + c\| \quad \text{ for all } c \in C.</math> 
But this last statement is true if and only if <math>\langle \,p, c\, \rangle = 0</math> every <math>c \in C.</math> Thus <math>p \in C^{\bot}.</math> 
<math>\blacksquare</math>
{{collapse bottom}}

== Properties ==

'''Expression as a global minimum'''

The statement and conclusion of the Hilbert projection theorem can be expressed in terms of global minimums of the following functions. Their notation will also be used to simplify certain statements.

Given a non-empty subset <math>C \subseteq H</math> and some <math>x \in H,</math> define a function 
<math display=block>d_{C,x} : C \to [0, \infty) \quad \text{ by } c \mapsto \|x - c\|.</math> 
A {{em|[[global minimum point]]}} of <math>d_{C,x},</math> if one exists, is any point <math>m</math> in <math>\,\operatorname{domain} d_{C,x} = C\,</math> such that 
<math display=block>d_{C,x}(m) \,\leq\, d_{C,x}(c) \quad \text{ for all } c \in C,</math> 
in which case <math>d_{C,x}(m) = \|m - x\|</math> is equal to the {{em|global minimum value}} of the function <math>d_{C, x},</math> which is:
<math display=block>\inf_{c \in C} d_{C,x}(c) = \inf_{c \in C} \|x - c\|.</math> 

'''Effects of translations and scalings'''

When this global minimum point <math>m</math> exists and is unique then denote it by <math>\min(C, x);</math> explicitly, the defining properties of <math>\min(C, x)</math> (if it exists) are:
<math display=block>\min(C, x) \in C \quad \text { and } \quad \left\|x - \min(C, x)\right\| \leq \|x - c\| \quad \text{ for all } c \in C.</math>
The Hilbert projection theorem guarantees that this unique minimum point exists whenever <math>C</math> is a non-empty closed and convex subset of a Hilbert space. 
However, such a minimum point can also exist in non-convex or non-closed subsets as well; for instance, just as long is <math>C</math> is non-empty, if <math>x \in C</math> then <math>\min(C, x) = x.</math> 

If <math>C \subseteq H</math> is a non-empty subset, <math>s</math> is any scalar, and <math>x, x_0 \in H</math> are any vectors then
<math display=block>\,\min\left(s C + x_0, s x + x_0\right) = s \min(C, x) + x_0</math>
which implies:
<math display=block>\begin{alignat}{6}
\min&(s C, s x) &&= s &&\min(C, x) \\
\min&(- C, - x) &&= - &&\min(C, x) \\
\end{alignat}</math>
<math display=block>\begin{alignat}{6}
\min\left(C + x_0, x + x_0\right) &= \min(C, x) + x_0 \\
\min\left(C - x_0, x - x_0\right) &= \min(C, x) - x_0 \\
\end{alignat}</math>
<math display=block>\begin{alignat}{6}
\min&(C, - x)   {}  &&= \min(C + x, 0) - x \\
\min&(C, 0)     \;+\; x\;\;\;\; &&= \min(C + x, x) \\
\min&(C - x, 0) {}  &&= \min(C, x) - x \\
\end{alignat}</math>

'''Examples'''

The following counter-example demonstrates a continuous linear isomorphism <math>A : H \to H</math> for which <math>\,\min(A(C), A(x)) \neq A(\min(C, x)).</math> 
Endow <math>H := \R^2</math> with the [[dot product]], let <math>x_0 := (0, 1),</math> and for every real <math>s \in \R,</math> let <math>L_s := \{ (x, s x) : x \in \R \}</math> be the line of slope <math>s</math> through the origin, where it is readily verified that <math>\min\left(L_s, x_0\right) = \frac{s}{1+s^2}(1, s).</math> 
Pick a real number <math>r \neq 0</math> and define <math>A : \R^2 \to \R^2</math> by <math>A(x, y) := (r x, y)</math> (so this map scales the <math>x-</math>coordinate by <math>r</math> while leaving the <math>y-</math>coordinate unchanged). 
Then <math>A : \R^2 \to \R^2</math> is an invertible continuous linear operator that satisfies <math>A\left(L_s\right) = L_{s/r}</math> and <math>A\left(x_0\right) = x_0,</math> 
so that <math>\,\min\left(A\left(L_s\right), A\left(x_0\right)\right) = \frac{s}{r^2 + s^2} (1, s)</math> and <math>A\left(\min\left(L_s, x_0\right)\right) = \frac{s}{1 + s^2} \left(r, s\right).</math> 
Consequently, if <math>C := L_s</math> with <math>s \neq 0</math> and if <math>(r, s) \neq (\pm 1, 1)</math> then <math>\,\min(A(C), A\left(x_0\right)) \neq A\left(\min\left(C, x_0\right)\right).</math>

== Iterated projections ==
For any closed convex nonempty subset <math>C \subset H</math>, let <math>P_C: H \to C</math> be the projection function.

If there are multiple closed convex subsets <math>C_1, C_2, \dots, C_n</math>, then one can approximate the projection operator <math>P_{C_1 \cap \dots \cap C_n}</math> by applying <math>P_{C_1}, P_{C_2}, \dots, P_{C_n}</math> in sequence, then do it again and again. That is, one can approximate <math>(P_{C_n} \dots P_{C_2} P_{C_1})^k \to P_{C_1 \cap \dots \cap C_n}</math> as <math>k \to \infty</math>. The [[Kaczmarz method]] is a commonly used special case. Such methods can be computationally effective. For example, if <math>C</math> is a complicated shape, then projecting directly to <math>C</math> may be difficult. However, <math>C</math> can be approximated as an intersection of simple objects like [[Half-space (geometry)|half-spaces]], [[Hyperplane|hyperplanes]], finite-dimensional subspaces, or [[Cone|cones]].<ref>{{Citation |last=Deutsch |first=Frank |title=The Method of Alternating Projections |date=2001 |work=Best Approximation in Inner Product Spaces |pages=193–235 |url=http://link.springer.com/10.1007/978-1-4684-9298-9_9 |place=New York, NY |publisher=Springer New York |doi=10.1007/978-1-4684-9298-9_9 |isbn=978-1-4419-2890-0|url-access=subscription }}</ref>

If <math>C</math> is a closed subspace, then it is convex. In this case, the projection function <math>P: H \to C</math> is an orthogonal projection (a continuous linear operator that is self-adjoint). A classic theorem states that, if <math>C_1, \dots, C_n</math> are closed subspaces, then<ref>{{Cite journal |last=Netyanun |first=Anupan |last2=Solmon |first2=Donald C. |date=August 2006 |title=Iterated Products of Projections in Hilbert Space |url=https://www.tandfonline.com/doi/full/10.1080/00029890.2006.11920347 |journal=The American Mathematical Monthly |language=en |volume=113 |issue=7 |pages=644–648 |doi=10.1080/00029890.2006.11920347 |issn=0002-9890|url-access=subscription }}</ref> <math display="block">\lim_{k \to \infty}\|(P_{C_1} \cdots P_{C_n})^kx - P_{C_1 \cap \dots \cap C_n} x\| = 0, \quad \forall x \in H</math>

== See also ==

* {{annotated link|Orthogonal complement}}
* {{annotated link|Orthogonal projection}}
* {{annotated link|Orthogonality principle}}
* {{annotated link|Riesz representation theorem}}

== Notes ==

{{reflist|group=note}}

== References ==

{{reflist}}

== Bibliography ==

* {{cite book|last=Rudin|first=Walter|authorlink=Walter Rudin|title=Real and Complex Analysis|edition=Third|year=1987}}
* {{Rudin Walter Functional Analysis|edition=2}} <!-- {{sfn|Rudin|1991|p=}} -->

{{Functional analysis}}
{{Hilbert space}}

{{DEFAULTSORT:Hilbert Projection Theorem}}
[[Category:Convex analysis]]
[[Category:Theorems in functional analysis]]