1.1 - Definitions

We'll start the lesson with some formal definitions. In doing so, recall that we denote the \(n\) random variables arising from a random sample as subscripted uppercase letters:

\(X_1, X_2, \cdots, X_n\)

The corresponding observed values of a specific random sample are then denoted as subscripted lowercase letters:

\(x_1, x_2, \cdots, x_n\)

Parameter Space
The range of possible values of the parameter \(\theta\) is called the parameter space \(\Omega\) (the greek letter "omega").

For example, if \(\mu\) denotes the mean grade point average of all college students, then the parameter space (assuming a 4-point grading scale) is:

\(\Omega=\{\mu: 0\le \mu\le 4\}\)

And, if \(p\) denotes the proportion of students who smoke cigarettes, then the parameter space is:

\(\Omega=\{p:0\le p\le 1\}\)

Point Estimator
The function of \(X_1, X_2, \cdots, X_n\), that is, the statistic \(u=(X_1, X_2, \cdots, X_n)\), used to estimate \(\theta\) is called a point estimator of \(\theta\).

For example, the function:

\(\bar{X}=\dfrac{1}{n}\sum\limits_{i=1}^n X_i\)

is a point estimator of the population mean \(\mu\). The function:

\(\hat{p}=\dfrac{1}{n}\sum\limits_{i=1}^n X_i\)

(where \(X_i=0\text{ or }1)\) is a point estimator of the population proportion \(p\). And, the function:

\(S^2=\dfrac{1}{n-1}\sum\limits_{i=1}^n (X_i-\bar{X})^2\)

is a point estimator of the population variance \(\sigma^2\).

Point Estimate
The function \(u(x_1, x_2, \cdots, x_n)\) computed from a set of data is an observed point estimate of \(\theta\).

For example, if \(x_i\) are the observed grade point averages of a sample of 88 students, then:

\(\bar{x}=\dfrac{1}{88}\sum\limits_{i=1}^{88} x_i=3.12\)

is a point estimate of \(\mu\), the mean grade point average of all the students in the population.

And, if \(x_i=0\) if a student has no tattoo, and \(x_i=1\) if a student has a tattoo, then:

\(\hat{p}=0.11\)

is a point estimate of \(p\), the proportion of all students in the population who have a tattoo.

Now, with the above definitions aside, let's go learn about the method of maximum likelihood.