Looking at the data, how will we find things that will work, or which model should we use? These are key questions. The variance for the estimators will be an important indicator.

The Idea Behind Regression Estimation

When the auxiliary variable x is linearly related to y but does not pass through the origin, a linear regression estimator would be appropriate. This does not mean that the regression estimate cannot be used when the intercept is close to zero. The two estimates, regression, and ratio may be quite close in such cases and you can choose the one you want to use.

In addition, if multiple auxiliary variables have a linear relationship with y, multiple regression estimates may be appropriate.

To estimate the mean and total of y-values, denoted as \(\mu\) and \(\tau\), one can use the linear relationship between y and known x-values.

Let's start with a simple example:

\(\hat{y}=a+bx\) , which is our basic regression equation.
Then,

\(b=\dfrac{\sum\limits_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{\sum\limits_{i=1}^n(x_i-\bar{x})^2}\) and

\(a=\bar{y}-b\bar{x}\)

Then to estimate the mean for y, substitute as follows:

\(x=\mu_x,\quad a=\bar{y}-b\bar{x},\text{then}\)
\(\hat{\mu}_L=(\bar{y}-b\bar{x})+b\mu_x\)
\(\hat{\mu}_L=\bar{y}+b(\mu_x-\bar{x}),\quad \hat{\mu}_L=a+b\mu_x\)

Note that even though  \(\hat{\mu}_L\) is not unbiased under simple random sampling, it is roughly so (asymptotically unbiased) for large samples.

Thus, the Mean Square Error of the estimate, denoted as MSE (\(\hat{\mu}_L\)) is not the same as Var(\(\hat{\mu}_L\))due to the bias but can be roughly estimated by the following when the sample size is large:

\begin{align}
\hat{V}ar(\hat{\mu}_L) &=\dfrac{N-n}{N \times n}\cdot \dfrac{\sum\limits_{i=1}^n(y_i-a-bx_i)^2}{n-2}\\
&= \dfrac{N-n}{N \times n}\cdot MSE\\
\end{align}

where MSE is the MSE of the linear regression model of y on x.

It follows that:

\(\hat{\tau}_L=N\cdot \hat{\mu}_L=N\bar{y}+b(\tau_x-N\bar{x})\)

\begin{align}
\hat{V}ar(\hat{\tau}_L) &= N^2 \hat{V}ar(\hat{\mu}_L) \\
&= \dfrac{N \times (N-n)}{n} \cdot MSE\\
\end{align}

Compare the regression estimate to the estimate \(\bar{y}\)

To compare the regression estimate to the estimate \(\bar{y}\), (which does not use auxiliary result of x), we see that:

\(\hat{V}ar(\bar{y})=\dfrac{N-n}{N}\cdot \dfrac{s^2}{n}\)

\(s^2\) for y values is: \((15.11)^2\)

Recall that the 95% confidence interval using regression estimate is 80.63 \(\pm\) 6.28; a much shorter confidence interval.

This regression estimate is more precise than \(\bar{y}\).

Additionally, we have another estimator that we can look at.

Compare \(\hat{\mu}_L\) to the ratio estimator \(\hat{\mu}_r\)

Next, Minitab was used to find out the mean and standard deviation for X and Y.

The ratio estimate is inappropriate for this example. However, just to show a counter-example, we can compute the variance of the ratio estimate using the following Minitab printout and compare this to the regression estimate.

\(\hat{\mu}_r=r\mu_x=\dfrac{\bar{y}}{\bar{x}}\cdot \mu_x=\dfrac{76}{46}\cdot 52=85.91\)

Next, we need to figure out the variance and for this, we need the MSE while using a ratio estimate. From the Minitab output, we have the SS / n-1, therefore, the

\(s^2_r=\dfrac{1}{10-1} \sum\limits_{i=1}^{10} (y_i-rx_i)^2=283.33\) (this is huge!)

Now we can compute the variance:

Now we can compute a 95% confidence interval for \(\mu\).

We can see that the ratio estimate is even worse than \(\bar{y}\) when it is used in an inappropriate situation.

The width of the interval is larger than the one for the regression estimate.

The moral of this story here is, "Use the right model!"