Hyper basis function network

In machine learning, a Hyper basis function network, or HyperBF network, is a generalization of radial basis function (RBF) networks concept, where the Mahalanobis-like distance is used instead of standard Euclidian distance measure. The activation function at the HyperBF network takes the following form

$\rho _{j}(||x-\mu _{j}||)=e^{(x-\mu _{j})^{T}R_{j}(x-\mu _{j})}$

where $R_{j}$ is a positive definite $d\times d$ matrix. Depending on the application, the following types of matrices $R_{j}$ are usually considered

• $R_{j}={\frac {1}{2\sigma ^{2}}}\mathbb {I} _{d\times d}$, where $\sigma >0$. This case corresponds to the regular RBF network.
• $R_{j}={\frac {1}{2\sigma _{j}^{2}}}\mathbb {I} _{d\times d}$, where $\sigma _{j}>0$. In this case, the basis functions are radially symmetric, but are scaled with different width.
• $R_{j}=diag\left({\frac {1}{2\sigma _{j1}^{2}}},...,{\frac {1}{2\sigma _{jz}^{2}}}\right)\mathbb {I} _{d\times d}$, where $\sigma _{ji}>0$. Every neuron has an elliptic shape with a varying size.
• Positive definite matrix, but not diagonal.

As at the RBF network case, the output of the network is a scalar function of the input vector, $\phi :\mathbb {R} ^{n}\to \mathbb {R}$, is given by

$\phi =\sum _{j=1}^{N}a_{i}\rho _{j}(||x-\mu _{j}||)$

Training HyperBF networks can be computationally challenging. Moreover, the high degree of freedom of HyperBF leads to overfitting and poor generalization. However, HyperBF networks have an important advantage that a small number of neurons is enough for learning complex functions.