Value function

The value function of an optimization problem gives the value attained by the objective function at a solution, depending on the parameters of the problem. In an economic context, where the objective function usually represents utility, the value function is conceptually equivalent to the indirect utility function.

In a problem of optimal control, the value function is implicitly defined as the supremum of the objective function taken over the set of admissible controls. The parameters of the problem are the initial conditions of the system: the initial value of the state variable and the initial time. Generally an optimal control problem may be written:

$J(u;x_{0})=\int _{t_{0}}^{t_{1}}L(t,x(t),u(t))\,\mathrm {d} t$

to be maximized over all admissible controls $u\in U$ for which the corresponding trajectory of $x(t)$, ${\dot {x}}(t)=f(t,x(t),u(t))$ with initial condition $x(t_{0})=x_{0}$ and some terminal constraint. Then the value function is defined as

$V(x_{0})=\sup _{u\in U}J(u;x_{0})$

By Bellman's principle of optimality, which roughly states that any optimal policy at time $t$ taking the current state $x(t)$ as initial condition must be optimal for the remaining problem, gives rise to an important functional recurrence equation, known as the Bellman equation.

Although unknown until a solution to the optimization problem is found, the value function itself can be used to find a solution.