Radial basis function kernel

In machine learning, the (Gaussian) radial basis function kernel, or RBF kernel, is a popular kernel function. It is the most popular kernel function used in support vector machine classification

The RBF kernel on two samples x and x', represented as feature vectors in some input space, is defined as

$K(\mathbf {x} ,\mathbf {x'} )=\exp(-{\frac {||\mathbf {x} -\mathbf {x'} ||_{2}^{2}}{2\sigma ^{2}}})$

$\textstyle ||\mathbf {x} -\mathbf {x'} ||_{2}^{2}$ may be recognized as the squared Euclidean distance between the two feature vectors. $\sigma$ is a free parameter. An equivalent, but simpler, definition involves a parameter $\textstyle \gamma ={\frac {-1}{2\sigma ^{2}}}$:

$K(\mathbf {x} ,\mathbf {x'} )=\exp(\gamma ||\mathbf {x} -\mathbf {x'} ||_{2}^{2})$

Since the value of the RBF kernel decreases with distance and ranges between zero (in the limit) and one (when x = x', it has a ready interpretation as a similarity measure.

Approximations

Because support vector machines and other models employing the kernel trick do not scale well to large numbers of training samples or large numbers of features in the input space, several approximations to the RBF kernel (and similar kernels) have been devised. Typically, these take the form z(x), i.e. a function transforming a single vector independently of other vectors (e.g. the support vectors in an SVM), such that

$z(\mathbf {x} )z(\mathbf {x'} )\approx \varphi (\mathbf {x} )\varphi (\mathbf {x'} )=K(\mathbf {x} ,\mathbf {x'} )$

One way to construct such a z is to randomly sample from the Fourier transformation of the kernel.

See also

• Polynomial kernel
• Radial basis function network