Kernel-independent component analysis

Kernel independent component analysis (Kernel ICA) is an efficient algorithm for independent component analysis based on estimating source components, which are represented in a reproducing kernel Hilbert space based on optimizing a generalized variance contrast function. Those contrast functions use the notion of mutual information as a measure of statistical independence.

Main idea

Kernel ICA is based on the idea that correlations between two random variables can be represented in a reproducing kernel Hilbert space (RKHS), denoted by ${\mathcal {F}}$, associated with a feature map $L_{x}:{\mathcal {F}}\mapsto \mathbb {R}$ defined for a fixed $x\in \mathbb {R}$. The ${\mathcal {F}}$-correlation between two random variables $X$ and $Y$ is defined

$\rho _{\mathcal {F}}(X,Y)=\max _{f,g\in {\mathcal {F}}}{\text{corr}}(\langle \ L_{X},f\rangle ,\langle \ L_{Y},g\rangle )$

where the functions $f,g:\mathbb {R} \to \mathbb {R}$ range over ${\mathcal {F}}$ and

${\text{corr}}(\langle \ L_{X},f\rangle ,\langle \ L_{Y},g\rangle ):={\frac {{\text{cov}}(f(X),g(Y))}{{\text{var}}(f(X))^{1/2}{\text{var}}(g(Y))^{1/2}}}$

for fixed $f,g\in {\mathcal {F}}$. Note that the reproducing property implies that $f(x)=\langle \ L_{x},f\rangle$ for fixed $x\in \mathbb {R}$ and $f\in {\mathcal {F}}$.<ref name = "Saitoh">Saitoh, Saburou (1988). Theory of Reproducing Kernels and Its Applications. Longman. ISBN 0582035643.