Griewank function

First order Griewank function

In mathematics, the Griewank function is often used in testing of optimization. It is defined as follows:

$1+{\frac {1}{4000}}\sum _{i=1}^{n}x_{i}^{2}-\prod _{i=1}^{n}\cos \left({\frac {x_{i}}{\sqrt {i}}}\right)$

The following paragraphs display the special cases of first, second and third order Griewank function, and their plots.

First-order Griewank function

$g:=1+(1/4000)\cdot x_{1}^{2}-\cos(x_{1})$

The first order Griewank function has multiple maxima and minima.

Let the derivative of Griewank function be zero：

${\frac {1}{2000}}\cdot x_{1}+\sin(x_{1})=0$

Find its roots in the interval [−100..100] by means of numerical method,

In the interval [−10000,10000], the Griewank function has 6365 critical points.

Second-order Griewank function

2nd order Griewank function 3D plot

2nd-order Griewank function contour plot

$1+{\frac {1}{4000}}x_{1}^{2}+{\frac {1}{4000}}x_{2}^{2}-\cos(x_{1})\cos \left({\frac {1}{2}}x_{2}{\sqrt {2}}\right)$

Third-order Griewank function

Third-order Griewank function Maple animation

$\left\{1+{\frac {1}{4000}}\,x_{1}^{2}+{\frac {1}{4000}}\,x_{2}^{2}+{\frac {1}{4000}}\,{x_{3}}^{2}-\cos(x_{1})\cos \left({\frac {1}{2}}x_{2}{\sqrt {2}}\right)\cos \left({\frac {1}{3}}x_{3}{\sqrt {3}}\right)\right\}$

Non-Smooth Variant

A non-smooth version of the Griewank function has also been developed to simulate optimization problems with piecewise smooth landscapes. This variant modifies the cosine product term to introduce non-smooth transitions, providing a more challenging test for algorithms that rely on gradient information, such as sub-gradient algorithms. The addition of non-smoothness makes this function particularly useful for testing modern optimization methods designed to handle irregular objective functions arising from applications in machine learning

Non-Smooth Variant of the Griewank Function

A non-smooth version of the Griewank function has been developed to emulate the characteristics of objective functions frequently encountered in machine learning (ML) optimization problems. These functions often exhibit piecewise smooth or non-smooth behavior due to the presence of regularization terms, activation functions, or constraints in learning models.

The non-smooth variant of the Griewank function is defined as:

$f(x)=1+{\frac {1}{4000}}\sum _{i=1}^{n}x_{i}^{2}-\prod _{i=1}^{n}P_{i}(x_{i}){\text{ with }}P_{i}(x_{i})=\left|\cos \left({\frac {x_{i}}{2{\sqrt {i}}}}\right)\right|-\left|\sin \left({\frac {x_{i}}{2{\sqrt {i}}}}\right)\right|.$

By incorporating non-smooth elements, such as absolute values in the cosine and sine terms, this function mimics the irregularities present in many ML loss landscapes. It offers a robust benchmark for evaluating optimization algorithms, especially those designed to handle the challenges of non-convex, non-smooth, or high-dimensional problems, including sub-gradient, hybrid, and evolutionary methods.

The function's resemblance to practical ML objective functions makes it particularly valuable for testing the robustness and efficiency of algorithms in tasks such as hyperparameter tuning, neural network training, and constrained optimization.