14.1 - Point Estimation

A point estimator of some population parameter θ is a single numerical value of a statistic.

Example

Suppose that X1, …, Xn are random samples from a continuous population with the mean value θ.

  • X1+X2, X1Xn, X1/(X1+X2+…+Xn), and \(\bar{X}\) are examples of statistics.
  • Let \(\hat{\theta}=\bar{x}\). Then \(\hat{\theta}\) is a point estimator of θ.
  • x1 + x2, x1xn, x1 can also be point estimators of θ, albeit poor ones.

Which one is the best?

Desirable Properties of Point Estimators

Let \(\hat{\theta}\) be a point estimator of a population parameter θ.

Bias: The difference between the expected value of the estimator \(E[\hat{\theta}]\) and the true value of θ, i.e.

\( Bias(\hat{\theta}) = E[\hat{\theta}]-\theta\)

When \(E[\hat{\theta}] = \theta\), \(\hat{\theta}\)  is called an unbiased estimator.

Variance is calculated by \(Var(\hat{\theta}) = E\left[\hat{\theta}-E[\hat{\theta}] \right]^2\).

Unbiased estimators that have minimum variance are called best unbiased estimators.

The Mean Squared Error (MSE) of \(\hat{\theta}\) is defined as

\(MSE(\hat{\theta}) =E[\hat{\theta}-\theta]^2=\left(Bias(\hat{\theta})   \right)^2+Var(\hat{\theta})\)

We want to choose the estimator which has the smallest MSE among all possible point estimators.

Bias-Variance Tradeoff: Modifying an estimator to reduce its bias increases its variance, and vice versa.

Balancing bias and variance is a central issue in data mining.

 Unknown Parameters, Statistics, and Point Estimators

table

Self-check

Think About It!

Let X1, X2 denote a random sample from a population having mean μ (μ ≠ 0) and variance 1. Consider the following estimators of μ:

\(\hat{\mu}_1=\frac{X_1+X_2}{2}\), \(\hat{\mu}_2=\frac{X_1-X_2}{2}\).

Find the answers to these questions below and then click the icon on the left to reveal the answer.

   1. Is either estimator unbiased?

 

   2. Which estimator is “BEST”? Explain the reason.