Mathematical Review

This is the basic mathematical definitions and calculation methods for Econometircs.

Best Predictor

Given a random vector $X$, we want to forecast $Y$, Let $g(X)$ be a predictor of $Y$. For Prediction Error, it is defined as $Y - g(X)$, and this prediction error can be treated as a random variable, and it can take positive and negative values. To minimize this prediction error, we define the the mean squared error (MSE) of predictor $g(X)$ as $\mathbb{E}[(Y-g(X))^2]$. We can have that the CEF $m(x) = \mathbb{E}(Y|X=x)$ is the best predictor, which has the smallest mean squared prediction error. Which means if we have $\mathbb{E}(Y^2) < \infin$, then for any predictor $g(X)$, we have:

$\mathbb{E}[(Y-g(X))^2] \geq \mathbb{E}[(Y-m(X))^2]$

Proof:

$\mathbb{E}[u^2] = \mathbb{E}[(Y-g(X))^2]=\mathbb{E}[(Y-m(X) + m(X) -g(X))^2]$

$=\mathbb{E}[(Y-m(X))^2] + \mathbb{E}[(m(X)-g(X))^2]+2\mathbb{E}[(Y-m(X))(m(X) -g(X))]$

$\geq \mathbb{E}[(Y-m(X))^2]$

since: $\mathbb{E}[(Y-m(X))(m(X) -g(X))] = \mathbb{E}[\mathbb{E}[(Y-m(X))(m(X) -g(X))|X]]$

as under condition $X$, $m(X) -g(X)$ is no longer a random variable, by the definition of $m(X)$

$\mathbb{E}[\mathbb{E}[(Y-m(X))(m(X) -g(X))|X]] = \mathbb{E}[(m(X) -g(X))\mathbb{E}[(Y-m(X))|X]] = 0$

So above inequality becomes equality when $m(X) = g(X)$, therefore, $m(X)$ is the smallest.