3a.1 - The Overall Mean Model

The design matrix for the overall mean model is:

\(\mathbf{Y}=\begin{bmatrix}
2\\
1\\
3\\
4\\
6\\
5
\end{bmatrix}
\mathbf{X}=\begin{bmatrix}
1\\
1\\
1\\
1\\
1\\
1
\end{bmatrix}
\boldsymbol{\beta}=(\mu) \;\;
\boldsymbol{\epsilon}=\begin{bmatrix}
\epsilon_1\\
\epsilon_2\\
\epsilon_3\\
\epsilon_4\\
\epsilon_5\\
\epsilon_6
\end{bmatrix}\)

The estimates of the parameters in \(\boldsymbol{\beta}\) are obtained as:

\(b= (\mathbf{X}^{'} \mathbf{X} ) ^{-1} \mathbf{X}^{' } \mathbf{Y}\)

In this simplest of cases, we can see how this works. The term \(\mathbf{X}^{'} \mathbf{X}\) would be:

\(\begin{bmatrix}
1 & 1 & 1 & 1 &1 & 1
\end{bmatrix}\ast \begin{bmatrix}
1\\
1\\
1\\
1\\
1\\
1
\end{bmatrix}=1+1+1+1+1+1=6 = n\)

exponent, indicating that we will be dividing by \(\mathbf{X}^{'} \mathbf{X}\).

The term \(\mathbf{X}^{'} \mathbf{Y}\) would be:

 \(\begin{bmatrix}
2 & 1 & 3 & 4 &5 & 6
\end{bmatrix}\ast \begin{bmatrix}
1\\
1\\
1\\
1\\
1\\
1
\end{bmatrix}=2+1+3+4+5+6=21 = \Sigma Y_i\)

So in this case, the estimate \(\mathbf{b}\) is simply the overall mean = \(\mu_{.}= 21/6 = 3.5\)

To perform these matrix operations in SAS IML, we will open a regular SAS editor window, and then copy and paste three components from the file (IML Design Matrices):

 SAS: Overall Mean Model

  1. Step 1

    Procedure initiation, and specification of the dependent variable vector, \(\mathbf{Y}\).

    For our example we have:

    /* Initiate IML, define response variable */
    proc iml;
    y={
    2, 
    1,
    3,
    4,
    6,
    5};
  2. Step 2

    We then enter a design matrix (\(\mathbf{X}\)). For the Overall Mean model and our example data, we have:

    
    x={
    1,
    1,
    1,
    1,
    1,
    1};
  3. Step 3

    We can now copy and paste a program for the matrix computations to generate results (regression coefficients and ANOVA output):

    beta=inv(x`*x)*x`*y; 
    beta_label={"Beta_0","Beta_1","Beta_2","Beta_3"};
    print beta [label="Regression Coefficients"
    			rowname=beta_label];
    n=nrow(y); 
    p=ncol(x); 
    j=j(n,n,1);
    ss_tot = (y`*y) - (1/n)*(y`*j)*y;
    ss_trt = (beta`*(x`*y)) - (1/n)*(y`*j)*y;
    ss_error = ss_tot - ss_trt; 
    total_df=n-1;
    trt_df=p-1;
    error_df=n-p; 
    MS_trt = ss_trt/(p-1);
    MS_error = ss_error / error_df;
    F=ms_trt/ms_error; 
    
    empty={.}; 
    row_label= {"Treatment", "Error", "Total"}; 
    col_label={"df" "SS" "MS" "F"}; 
    trt_row= trt_df || ss_trt || ms_trt || F; 
    error_row= error_df || ss_error || ms_error || empty;
    tot_row=total_df || ss_tot || empty || empty; 
    aov = trt_row // error_row // tot_row;
    print aov [label="ANOVA"
    			rowname=row_label
    			colname=col_label];

Here is a quick video walk-through to show you the process for how you can do this in SAS. (Right-click and select 'Show All' if your browser does not display the entire screencast window.)

The program can then be run to produce the following output:

Regression Coefficients
Beta_0 3.5
ANOVA
  df SS MS F
Treatment 0 0    
Error 5 17.5 3.5  
Total 5 17.5    

We see the estimate of the regression coefficient for \(\beta_0\) is indeed the overall mean of the response variable. In this simple case, where the treatment factor has not entered the model, the only item of interest from the ANOVA table would be the \(SS_{Error}\) for later use in the General Linear F test.

To check to see the internal calculations, we can add the following few lines to the end of the calculation code.

 /* (Optional) Intermediates in the matrix computations */
xprimex=x`*x; print xprimex;
xprimey=x`*y; print xprimey;
xprimexinv=inv(x`*x); print xprimexinv;
check=xprimexinv*xprimex; print check;
SumY2=beta`*(x`*y); print SumY2;
CF=(1/n)*(y`*j)*y; print CF;

This additional code produces the following output:

xprimex
6
xprimey
21
xprimexinv
0.1666667
check
1
SumY2
73.5
SumY_2
73.5

From this we can verify the computations for the \(SS_{treatment}\Sigma Y_{i}^{2} – \frac{(\Sigma Y_i)^2}{n}=\Sigma Y^{2}-CF = 0\).

The ‘check’ calculation is confirming that \(\mathbf{X}^{-1} \mathbf{X} = 1\), which in fact defines the matrix division operation. In this simple case, it amounts to simple division by n, but in other models that we will work with, the matrix division process is more complicated. The inverse (\(\mathbf{X}^{-1}\)) is defined by the product \(\mathbf{X}^{-1} \mathbf{X} = \mathbf{I}\), where \(\mathbf{I}\) is the identity matrix (a diagonal matrix of ‘1’s). Here in this simple case there is only a single diagonal element.