无偏估计

概率

---

所谓无偏估计，指的是：

$$\mathbb{bias}({\hat{\theta}}) = \mathbb{E}(\hat{\theta})+\theta$$

比如，考虑一组独立同分布的样本$(x_1,x_2,...,x_n)$，服从高斯分布，它们的方差称作样本方差：

$$\hat{\sigma}^2 = \frac{1}{n}\sum_{i=0}^{n}(x_i-\bar{x})^2$$

现求其期望：

$$\begin{equation} \begin{aligned} \mathbb{E}(\hat{\sigma}^2) &=\frac{1}{n}\mathbb{E}\Bigg[{\sum_{i=0}^{n}(x_i-\bar{x})^2\Bigg]} \\ &=\frac{1}{n}\mathbb{E}\Bigg[{\sum_{i=0}^{n}}(x_i^2-2x_i\bar{x}+\bar{x}^2)\Bigg] \\ &=\frac{1}{n}\mathbb{E}\Bigg[{\sum_{i=0}^{n}}x_i^2-2\bar{x}\sum_{i=0}^{n}x_i+n\bar{x}^2\Bigg] \\ &=\frac{1}{n}\mathbb{E}\Bigg[{\sum_{i=0}^{n}}x_i^2-2n\bar{x}^2+n\bar{x}^2\Bigg] \\ &=\frac{1}{n}\mathbb{E}\Bigg[{\sum_{i=0}^{n}}x_i^2-n\bar{x}^2\Bigg] \\ &=\frac{1}{n}\Bigg[{\sum_{i=0}^{n}}\mathbb{E}(x_i^2)-n\mathbb{E}(\bar{x}^2)\Bigg] \\ &=\frac{1}{n}\Bigg[{\sum_{i=0}^{n}}\Big[\mathbb{var}(x_i)+\mathbb{E}(x_i)^2\Big]-n[\mathbb{var}(\bar{x})+\mathbb{E}(\bar{x})^2]\Bigg] \\ &=\frac{1}{n}\Bigg[n\Big[\sigma^2+\mu^2\Big]-n\Big[\frac{\sigma^2}{n}+\mu^2\Big]\Bigg] \\ &=\frac{n-1}{n}\sigma^2 \end{aligned} \end{equation}$$

因而：

$$\begin{equation} \begin{aligned} \mathbb{bias({\hat{\sigma}^2})}&=\mathbb{E}(\hat{\sigma}^2) - \sigma^2 \\ &=-\frac{\sigma^2}{n} \end{aligned} \end{equation}$$

因而，这是一种有偏估计。