Convex analysis

Convex analysis is the branch of mathematics devoted to the study of properties of convex functions and convex sets, often with applications in convex minimization, a subdomain of optimization theory.

Convex sets

A subset $C\subseteq X$ of some vector space $X$ is called convex if it satisfies any of the following equivalent conditions:

If $0\leq r\leq 1$ is real and $x,y\in C$ then $rx+(1-r)y\in C.$ ^[1]
If $0<r<1$ is real and $x,y\in C$ with $x\neq y,$ then $rx+(1-r)y\in C.$
$(r+s)C=rC+sC$ for all positive real $r>0$ and $s>0.$ ^[2]

Convex functions

A convex function is any extended real-valued function $f:X\to \mathbb {R} \cup \{\pm \infty \}$ that satisfies the hypothesis of Jensen's inequality; that is, for any $x,y\in X$ and any $λ \in [0, 1]$ ,

f(\lambda x+(1-\lambda )y)\leq \lambda f(x)+(1-\lambda )f(y).

^[1]

Equivalently, a convex function is any (extended) real valued function such that its epigraph

\left\{(x,r)\in X\times \mathbf {R} :f(x)\leq r\right\}

is a convex set.^[1]

Convex conjugate

The convex conjugate of an extended real-valued (not necessarily convex) function $f:X\to \mathbb {R} \cup \{\pm \infty \}$ is $f^{*}:X^{*}\to \mathbb {R} \cup \{\pm \infty \}$ where $X^{*}$ is the dual space of $X,$ and^[3]

f^{*}\left(x^{*}\right)=\sup _{x\in X}\left\{\left\langle x^{*},x\right\rangle -f(x)\right\}.

Biconjugate

The biconjugate of a function $f:X\to \mathbb {R} \cup \{\pm \infty \}$ is the conjugate of the conjugate, typically written as $f^{**}:X\to \mathbb {R} \cup \{\pm \infty \}.$ The biconjugate is useful for showing when strong or weak duality hold (via the perturbation function).

For any $x\in X,$ the inequality $f^{**}(x)\leq f(x)$ follows from the Fenchel–Young inequality. For proper functions, $f=f^{**}$ if and only if $f$ is convex and lower semi-continuous by Fenchel–Moreau theorem.^[3]^[4]

Convex minimization

A convex minimization (primal) problem is one of the form

find

\inf _{x\in M}f(x)

when given a convex function

f:X\to \mathbb {R} \cup \{\pm \infty \}

and a convex subset

M\subseteq X.

Dual problem

In optimization theory, the duality principle states that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem.

In general given two dual pairs separated locally convex spaces $\left(X,X^{*}\right)$ and $\left(Y,Y^{*}\right).$ Then given the function $f:X\to \mathbb {R} \cup \{\pm \infty \},$ we can define the primal problem as finding $x$ such that

\inf _{x\in X}f(x).

If there are constraint conditions, these can be built into the function $f$ by letting $f=f+I_{\mathrm {constraints} }$ where $I$ is the indicator function. Then let $F:X\times Y\to \mathbb {R} \cup \{\pm \infty \}$ be a perturbation function such that $F(x,0)=f(x).$ ^[5]

The dual problem with respect to the chosen perturbation function is given by

\sup _{y^{*}\in Y^{*}}-F^{*}\left(0,y^{*}\right)

where $F^{*}$ is the convex conjugate in both variables of $F.$

The duality gap is the difference of the right and left hand sides of the inequality^[6]^[5]^[7]

\sup _{y^{*}\in Y^{*}}-F^{*}\left(0,y^{*}\right)\leq \inf _{x\in X}F(x,0).

This principle is the same as weak duality. If the two sides are equal to each other, then the problem is said to satisfy strong duality.

There are many conditions for strong duality to hold such as:

$F=F^{**}$ where $F$ is the perturbation function relating the primal and dual problems and $F^{**}$ is the biconjugate of $F$ ;^{[citation needed]}
the primal problem is a linear optimization problem;
Slater's condition for a convex optimization problem.^[8]^[9]

Lagrange duality

For a convex minimization problem with inequality constraints,

\min {}_{x}f(x)

subject to

g_{i}(x)\leq 0

for

i=1,\ldots ,m.

the Lagrangian dual problem is

\sup {}_{u}\inf {}_{x}L(x,u)

subject to

u_{i}(x)\geq 0

for

i=1,\ldots ,m.

where the objective function $L(x,u)$ is the Lagrange dual function defined as follows:

L(x,u)=f(x)+\sum _{j=1}^{m}u_{j}g_{j}(x)

Notes

^ ^a ^b ^c Rockafellar, R. Tyrrell (1997) [1970]. Convex Analysis. Princeton, NJ: Princeton University Press. ISBN 978-0-691-01586-6.
^ Rudin 1991, p. 38.
^ ^a ^b Zălinescu 2002, pp. 75–79.
^ Borwein, Jonathan; Lewis, Adrian (2006). Convex Analysis and Nonlinear Optimization: Theory and Examples (2 ed.). Springer. pp. 76–77. ISBN 978-0-387-29570-1.
^ ^a ^b Boţ, Radu Ioan; Wanka, Gert; Grad, Sorin-Mihai (2009). Duality in Vector Optimization. Springer. ISBN 978-3-642-02885-4.
^ Zălinescu 2002, pp. 106–113.
^ Csetnek, Ernö Robert (2010). Overcoming the failure of the classical generalized interior-point regularity conditions in convex optimization. Applications of the duality theory to enlargements of maximal monotone operators. Logos Verlag Berlin GmbH. ISBN 978-3-8325-2503-3.
^ Borwein, Jonathan; Lewis, Adrian (2006). Convex Analysis and Nonlinear Optimization: Theory and Examples (2 ed.). Springer. ISBN 978-0-387-29570-1.
^ Boyd, Stephen; Vandenberghe, Lieven (2004). Convex Optimization (pdf). Cambridge University Press. ISBN 978-0-521-83378-3. Retrieved October 3, 2011.

References

Hiriart-Urruty, J.-B.; Lemaréchal, C. (2001). Fundamentals of convex analysis. Berlin: Springer-Verlag. ISBN 978-3-540-42205-1.
Rudin, Walter (1991). Functional Analysis. International Series in Pure and Applied Mathematics. Vol. 8 (Second ed.). New York, NY: McGraw-Hill Science/Engineering/Math. ISBN 978-0-07-054236-5. OCLC 21163277.
Singer, Ivan (1997). Abstract convex analysis. Canadian Mathematical Society series of monographs and advanced texts. New York: John Wiley & Sons, Inc. pp. xxii+491. ISBN 0-471-16015-6. MR 1461544.
Stoer, J.; Witzgall, C. (1970). Convexity and optimization in finite dimensions. Vol. 1. Berlin: Springer. ISBN 978-0-387-04835-2.
A.G. Kusraev; S.S. Kutateladze (1995). Subdifferentials: Theory and Applications. Dordrecht: Kluwer Academic Publishers. ISBN 978-94-011-0265-0.
Zălinescu, Constantin (30 July 2002). Convex Analysis in General Vector Spaces. River Edge, N.J. London: World Scientific Publishing. ISBN 978-981-4488-15-0. MR 1921556. OCLC 285163112 – via Internet Archive.

External links

Media related to Convex analysis at Wikimedia Commons

[Rockafellar-1] Rockafellar, R. Tyrrell (1997) [1970]. Convex Analysis. Princeton, NJ: Princeton University Press. ISBN 978-0-691-01586-6.

[FOOTNOTERudin199138-2] Rudin 1991, p. 38.

[FOOTNOTEZălinescu200275–79-3] Zălinescu 2002, pp. 75–79.

[BorweinLewis-4] Borwein, Jonathan; Lewis, Adrian (2006). Convex Analysis and Nonlinear Optimization: Theory and Examples (2 ed.). Springer. pp. 76–77. ISBN 978-0-387-29570-1.

[BWG-5] Boţ, Radu Ioan; Wanka, Gert; Grad, Sorin-Mihai (2009). Duality in Vector Optimization. Springer. ISBN 978-3-642-02885-4.

[FOOTNOTEZălinescu2002106–113-6] Zălinescu 2002, pp. 106–113.

[Csetnek_2010-7] Csetnek, Ernö Robert (2010). Overcoming the failure of the classical generalized interior-point regularity conditions in convex optimization. Applications of the duality theory to enlargements of maximal monotone operators. Logos Verlag Berlin GmbH. ISBN 978-3-8325-2503-3.

[borwein-8] Borwein, Jonathan; Lewis, Adrian (2006). Convex Analysis and Nonlinear Optimization: Theory and Examples (2 ed.). Springer. ISBN 978-0-387-29570-1.

[boyd-9] Boyd, Stephen; Vandenberghe, Lieven (2004). Convex Optimization (pdf). Cambridge University Press. ISBN 978-0-521-83378-3. Retrieved October 3, 2011.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

v t e Convex analysis and variational analysis
Basic concepts	Convex combination Convex function Convex set
Topics (list)	Choquet theory Convex geometry Convex metric space Convex optimization Duality Lagrange multiplier Legendre transformation Locally convex topological vector space Simplex
Maps	Convex conjugate Concave (Closed K- Logarithmically Proper Pseudo- Quasi-) Convex function Invex function Legendre transformation Semi-continuity Subderivative
Main results (list)	Carathéodory's theorem Ekeland's variational principle Fenchel–Moreau theorem Fenchel-Young inequality Jensen's inequality Hermite–Hadamard inequality Krein–Milman theorem Mazur's lemma Shapley–Folkman lemma Robinson–Ursescu Simons Ursescu
Sets	Convex hull (Orthogonally, Pseudo-) Convex set Effective domain Epigraph Hypograph John ellipsoid Lens Radial set/Algebraic interior Zonotope
Series	Convex series related ((cs, lcs)-closed, (cs, bcs)-complete, (lower) ideally convex, (Hx), and (Hwx))
Duality	Dual system Duality gap Strong duality Weak duality
Applications and related	Convexity in economics