Alternating conditional expectations

ACE algorithm is an algorithm to find the optimal transformations between the response variable and predictor variables in regression analysis.

Introduction

Let $Y,X_{1},\dots ,X_{p}$ be random variables. We use $X_{1},\dots ,X_{p}$ to predict $Y$. Suppose $\theta (Y),\varphi _{1}(X_{1}),\dots ,\varphi _{p}(X_{p})$ are mean-zero functions and with these transformation functions, the fraction of variance of $\theta (Y)$ not explained is

$e^{2}(\theta ,\varphi _{1},\dots ,\varphi _{p})={\frac {E[\theta (Y)-\sum _{i=1}^{p}\varphi _{i}(X_{i})]^{2}}{E\theta ^{2}(Y)}}$

Generally, the optimal transformations that minimize the unexplained part are difficult to compute directly. As an alternative, ACE is an iterative method to calculate the optimal transformations. The procedure of ACE has the following steps:

1. Hold $\phi _{1}(X_{1}),\dots ,\phi _{p}(X_{p})$ fixed, minimizing $e^{2}$gives $\theta _{1}(Y)=E[\sum _{i=1}^{p}\varphi _{i}(X_{i})|Y]$
2. Normalize $\theta _{1}(Y)$ to unit variance.
3. For each $k$, fix other $\varphi _{i}(X_{i})$ and $\theta (Y)$, minimizing $e^{2}$ and the solution is:: ${\tilde {\varphi }}_{k}=E[\theta (Y)-\sum _{i\neq k}\varphi _{i}(X_{i})|X_{k}]$
4. Iterate the above three steps until $e^{2}$ is within error tolerance.