General regression neural network

Generalized regression neural network (GRNN) is a variation to radial basis neural networks. GRNN was suggested by D.F. Specht in 1991.

GRNN can be used for regression, prediction, and classification. GRNN can also be a good solution for online dynamical systems.

GRNN represents an improved technique in the neural networks based on the nonparametric regression. The idea is that every training sample will represents a mean to a radial basis neuron.

Mathematical representation

$Y(x)={\frac {\sum _{k=1}^{N}y_{k}K(x,x_{k})}{\sum _{k=1}^{N}K(x,x_{k})}}$

where:

• $Y(x)$ is the prediction value of input $x$
• $y_{k}$ is the activation weight for the pattern layer neuron at $k$
• $K(x,x_{k})$ is the Radial basis function kernel (Gaussian kernel) as formulated below.

Gaussian Kernel

$K(x,x_{k})=e^{d_{k}/2\sigma },\qquad d_{k}=(x-x_{k})^{T}(x-x_{k})$

where $d_{k}$ is the squared euclidean distance between the training samples $x_{k}$ and the input $x.$

Implementation

GRNN has been implemented in many software including MATLAB, R- programming language and Python (programming language).

Advantages and disadvantages

Similar to RBFNN GRNN has the following advantages:

• single-pass learning so no backpropagation is required.
• high accuracy in the estimation since it uses Gaussian functions.
• it can handle noises in the inputs.

The main disadvantages of GRNN are:

• Its size can grow to huge size which computationally expensive.
• There is no optimal method to improve it.