Alternating conditional expectations

ACE algorithm is an algorithm to find the optimal transformations between the response variable and predictor variables in regression analysis.

Introduction

In statistics, nonlinear transformation of variables is commonly used in practice in regression problems. Alternating conditional expectations(ACE) is one of these method to find those transformations that produce the best fitting additive model. Knowledge of such transformations aids in the interpretation and understanding of the relationship between the response and predictors.

ACE transform the response variable $Y$ and its predictor variables, $X_{i}$ to minimize the fraction of variance not explained. The transformation is nonlinear and is obtained from data in an iterative way.

Mathematical Description

Let $Y,X_{1},\dots ,X_{p}$ be random variables. We use $X_{1},\dots ,X_{p}$ to predict $Y$. Suppose $\theta (Y),\varphi _{1}(X_{1}),\dots ,\varphi _{p}(X_{p})$ are mean-zero functions and with these transformation functions, the fraction of variance of $\theta (Y)$ not explained is

$e^{2}(\theta ,\varphi _{1},\dots ,\varphi _{p})={\frac {E[\theta (Y)-\sum _{i=1}^{p}\varphi _{i}(X_{i})]^{2}}{E\theta ^{2}(Y)}}$

Generally, the optimal transformations that minimize the unexplained part are difficult to compute directly. As an alternative, ACE is an iterative method to calculate the optimal transformations. The procedure of ACE has the following steps:

1. Hold $\phi _{1}(X_{1}),\dots ,\phi _{p}(X_{p})$ fixed, minimizing $e^{2}$gives $\theta _{1}(Y)=E[\sum _{i=1}^{p}\varphi _{i}(X_{i})|Y]$
2. Normalize $\theta _{1}(Y)$ to unit variance.
3. For each $k$, fix other $\varphi _{i}(X_{i})$ and $\theta (Y)$, minimizing $e^{2}$ and the solution is:: ${\tilde {\varphi }}_{k}=E[\theta (Y)-\sum _{i\neq k}\varphi _{i}(X_{i})|X_{k}]$
4. Iterate the above three steps until $e^{2}$ is within error tolerance.

Bivariate Case

The optical transformation $\theta ^{*}(Y),\varphi ^{*}(X)$for $p=1$ satisfies

$\rho ^{*}(X,Y)=\rho ^{*}(\theta ^{*},\varphi ^{*})=\max _{\theta ,\varphi }\rho [\theta (Y),\varphi (X)]$

where $\rho$ is Pearson correlation coefficient. $\rho ^{*}(X,Y)$ is known as the maximal correlation between $X$ and $Y$. It can be used as a general measure of dependence.

In the bivariate case, ACE algorithm can also be regarded as a method for estimating the maximal correlation between two variables.