TU Wien:Statistik und Wahrscheinlichkeitstheorie UE (Bura)/Übungen 2019W/5.4
- Unbiasedness of the empirical variance
Let n ≥ 2 and be i.i.d. (independent and identically distributed) random varia- bles, with . Calculate the expectation of the empirical variance
What would have been the expectation if in we had scaled with n instead of n − 1?
Lösungsvorschlag von Ikaly
Vorlage:Main The sample variance of a random variable demonstrates two aspects of estimator bias: firstly, the naive estimator is biased, which can be corrected by a scale factor; second, the unbiased estimator is not optimal in terms of mean squared error (MSE), which can be minimized by using a different scale factor, resulting in a biased estimator with lower MSE than the unbiased estimator. Concretely, the naive estimator sums the squared deviations and divides by n, which is biased. Dividing instead by n − 1 yields an unbiased estimator. Conversely, MSE can be minimized by dividing by a different number (depending on distribution), but this results in a biased estimator. This number is always larger than n − 1, so this is known as a shrinkage estimator, as it "shrinks" the unbiased estimator towards zero; for the normal distribution the optimal value is n + 1.
then S2 is a biased estimator of σ2, because
To continue, we note that by subtracting from both sides of , we get
Meaning, (by cross-multiplication) . Then, the previous becomes: