Ridge function

Not to be confused with ridge regression.

In mathematics, a ridge function is any function $f:\mathbb {R} ^{d}\rightarrow \mathbb {R}$ that can be written as the composition of an univariate function $g:\mathbb {R} \rightarrow \mathbb {R}$, that is called a profile function, with an affine transformation, given by a direction vector $a\in \mathbb {R} ^{d}$ with shift $b\in \mathbb {R}$.

Then, the ridge function reads $f(x)=g(x^{\top }a+b)$ for $x\in \mathbb {R} ^{d}$.

Coinage of the term 'ridge function' is often attributed to B.F. Logan and L.A. Shepp.

Relevance

A ridge function is not susceptible to the curse of dimensionality, making it an instrumental tool in various estimation problems. This is a direct result of the fact that ridge functions are constant in $d-1$ directions: Let $a_{1},\dots ,a_{d-1}$ be $d-1$ independent vectors that are orthogonal to $a$, such that these vectors span $d-1$ dimensions. Then

$f\left({\boldsymbol {x}}+\sum _{k=1}^{d-1}c_{k}{\boldsymbol {a}}_{k}\right)=g\left({\boldsymbol {x}}\cdot {\boldsymbol {a}}+\sum _{k=1}^{d-1}c_{k}{\boldsymbol {a}}_{k}\cdot {\boldsymbol {a}}\right)=g\left({\boldsymbol {x}}\cdot {\boldsymbol {a}}+\sum _{k=1}^{d-1}c_{k}0\right)=g({\boldsymbol {x}}\cdot {\boldsymbol {a}})=f({\boldsymbol {x}})$

for all $c_{i}\in \mathbb {R} ,1\leq i<d$. In other words, any shift of ${\boldsymbol {x}}$ in a direction perpendicular to ${\boldsymbol {a}}$ does not change the value of $f$.

Ridge functions play an essential role in amongst others projection pursuit, generalized linear models, and as activation functions in neural networks. For a survey on ridge functions, see. For books on ridge functions, see.