Recall the \(3\times2\times2\) table that we examined in Lesson 5 that classifies 800 boys according to S = socioeconomic status, B = whether a boy is a scout, and D = juvenile delinquency status.

Because the outcome variable D is binary, we can express many models of interest using binary logistic regression. Before handling the full three-way table, let us consider the \(2\times2\) marginal table for B and D as we did in Lesson 5. We concluded that the boy scout status (B) and the delinquent status (D) are associated and that (from the table below)

the estimated log-odds ratio is

\(\log\left(\dfrac{33( 360)}{64 (343)}\right)=-0.6140\)

with a standard error of \(\sqrt{\dfrac{1}{33}+\dfrac{1}{343}+\dfrac{1}{64}+\dfrac{1}{360}}=0.2272\). That is, we estimate that being a boy scout is associated a lower log-odds of delinquency by 0.614; the odds-ratio is 0.541. Now let’s fit the logistic regression model

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0+\beta_1 x\)

where \(x\) is a dummy variable

\(x\) = 0 if non-scout,
\(x\) = 1 if scout.

options nocenter nodate nonumber linesize=72;

data scout1;
   input S $ y n;
   cards;
scout     33 376
nonscout  64 424
;

proc logist data=scout1;
  class S / param=ref ref=first;
  model y/n = S / scale=none;
  run;

model y/n = / scale=none;

#### corresponds to the data scout1 in scout.SAS
#### Fits a Logistic regression with S="scout" or "nonscout"

S=factor(c("scout","nonscout"))
Sscout=(S=="scout")
Snonscout=(S=="nonscout")
y=c(33,64)
n=c(376,424)
count=cbind(y,n-y)
result=glm(count~Sscout+Snonscout,family=binomial("logit"))
summary(result)

Coefficients: (1 not defined because of singularities)
              Estimate Std. Error z value Pr(>|z|)    
(Intercept)    -1.7272     0.1357 -12.732  < 2e-16 ***
SscoutTRUE     -0.6140     0.2272  -2.702  0.00688 ** 
SnonscoutTRUE       NA         NA      NA       NA    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

   Null deviance: 7.6126e+00  on 1  degrees of freedom
Residual deviance: 4.0190e-14  on 0  degrees of freedom
AIC: 15.083

Now let us do a similar analysis for the \(3\times2\) table that classifies subjects by S and D:

Two odds ratios of interest are

\((53(236)) / (34( 212)) = 1.735\)
\((53( 255)) / (10 (212)) = 6.375\)

We estimate that the odds of delinquency for the S = low group are 1.735 times as high as for the S = medium group, and 6.375 times as high as for the S = high group. The estimated log odds ratios are

\(\log 1.735 = .5512\) and \(\log 6.375 = 1.852\)

and the standard errors are

\(\sqrt{\dfrac{1}{53}+\dfrac{1}{212}+\dfrac{1}{34}+\dfrac{1}{236}}=0.2392\)

\(\sqrt{\dfrac{1}{53}+\dfrac{1}{212}+\dfrac{1}{10}+\dfrac{1}{255}}=0.3571\)

Now let us replicate this analysis using logistic regression. First, we re-express the data in terms of \y_i\) = number of delinquents and \(n_i\) = number of boys for the \(i\)th row (SES group):

Then we define a pair of dummy indicators,

\(x_1=1\) if S=medium,
\(x_1=0\) otherwise,

\(x_2=1\) if S=high,
\(x_2=0\) otherwise.

Let \(\pi\) = probability of delinquency. Then, the model

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0+\beta_1 x_1+\beta_2 x_2\)

says that the log-odds of delinquency are \(\beta_0\) for S = low, \(\beta_0+\beta_1\) for S = medium, and \(\beta_0+\beta_2\) for S = high.

options nocenter nodate nonumber linesize="72";
data new;
	input S $ y n;
    cardsl
low		53 265
medium		34 270
hight		10 265
;

proc logist data=new;
class S / order data param=ref ref=first;
model y/n = S scale=none;
run;

S=factor(c("low","medium","high"))
y=c(53,34,10)
n=c(265,270,265)
count=cbind(y,n-y)
Smedium=(S=="medium")
Shigh=(S=="high")
result=glm(count~Smedium+Shigh,family=binomial("logit"))
summary(result)

Call:
glm(formula = count ~ Smedium + Shigh, family = binomial("logit"))

Deviance Residuals: 
[1]  0  0  0

Coefficients:
            Estimate Std. Error z value Pr(>|z|)    
(Intercept)  -1.3863     0.1536  -9.027  < 2e-16 ***
SmediumTRUE  -0.5512     0.2392  -2.304   0.0212 *  
ShighTRUE    -1.8524     0.3571  -5.188 2.13e-07 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 3.6252e+01  on 2  degrees of freedom
Residual deviance: 6.8390e-14  on 0  degrees of freedom
AIC: 20.942

Number of Fisher Scoring iterations: 3

Notice that we did not say anything here about the scout status. We have "ignored" that information because we collapsed over that variable. Next, we will see how this plays out with logistic regression.

In this example, we could have also arranged the input data like this:

A SAS program for fitting the same model is shown below.

data new;
   input S $ B $ y n;
   cards;
low      scout      11   54
low      nonscout   42  211
medium   scout      14  118
medium   nonscout   20  152
high     scout       8  204
high     nonscout    2   61
;

proc logist data=new;
  class S / order=data param=ref ref=first;
  model y/n = S / scale=none;
  run;

The parameter estimates from this new program are exactly the same as before:

But the overall fit statistics are different! Before, we had \(X^2=0\) and \(G^2=0\) because the model was saturated (there were three parameters and \(N = 3\) lines of data). But now, the fit statistics are:

The model appears to fit very well, but it is no longer saturated. What happened? Recall that \(X^2\) and \(G^2\) are testing the null hypothesis that the current model is correct, versus the alternative of a saturated model. When we disaggregated the data by levels of B, using six input lines rather than three, the current model did not change but the saturated model did; the saturated model was enlarged to six parameters.

Another way to interpret the overall \(X^2\) and \(G^2\) goodness-of-fit tests is that they are testing the significance of all omitted covariates. If we collapse the data over B and use only three lines of data, then SAS is unaware of the existence of B. But if we disaggregate the data by levels of B and do not include it in the model, then SAS has the opportunity to test the fit of the current model—in which the probability of delinquency varies by S alone—against the saturated alternative in which the probability of delinquency varies by each combination of the levels of S and B. When the data are disaggregated, the goodness-of-fit tests are actually testing the hypothesis that D is unrelated to B once S has been taken into account—i.e., that D and B are conditionally independent given S.

Here’s another way to think about it. The current model has three parameters:

• an intercept, and
• two indicators for S.

But the alternative has six parameters. We can think of these six parameters as an intercept and five dummies to distinguish among the six rows of data, but then it's not easy to see how the current model becomes a special case of it. So, we think of the six parameters as

• an intercept,
• two dummies for S,
• one dummy for B, and
• two interaction terms for SB.

Now it has become clear that the current model is a special case of this model in which the coefficients for B and the SB interactions are zero. The overall \(X^2\) and \(G^2\) statistics for the disaggregated data are testing the joint significance of the B dummy and the SB interactions.

So, should we aggregate, or should we not? If the current model is true, then it doesn’t matter; we get exactly the same estimated coefficients and standard errors either way. But dis-aggregating gives us the opportunity to test the significance of the omitted terms for B and SB.

Therefore, it often makes sense to dis-aggregate your dataset by variables that are not included in the model, because it gives you the opportunity to test the overall fit of your model. But that strategy has limits. If you dis-aggregate the data too much, the \(n_i\)s may become too small to reliably test the fit by \(X^2\) and \(G^2\).

In this part of the lesson we will consider different binary logistic regression models for three-way tables and their link to log-linear models. Let us return to the \(3\times2\times2\) table:

As we discussed in Lesson 5, there are many different models that we could fit to this table. But if we think of D as a response and B and S as potential predictors, we can focus on a subset of models that make sense.

Let \(\pi\) be the probability of delinquency. The simplest model to consider is the null or intercept-only model,

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0\) (1)

in which D is unrelated to B or S. If we were to fit this model in PROC LOGISTIC using the disaggregated data (all six lines), we would find that the\(X^2\) and \(G^2\) statistics are identical to those we obtained in Lesson 5 from testing the null hypothesis "S and B are independent of D". That is, testing the overall fit of model (1), i.e., intercept only model, is equivalent to testing the fit of the log-linear model (D, SB), because (1) says that D is unrelated to S and B but makes no assumptions about whether S and B are related.

After (1), we may want to fit the logit model that has a main effect for B,

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0+\beta_1 x_1\) (2)

where

\(x_1=1\) if B=scout,
\(x_1=0\) otherwise.

If the data are disaggregated into six lines, the goodness-of-fit tests for model (2) will be equivalent to the test for (DB, SB) log-linear model, which says that D and S are conditionally independent given B. This makes sense, because (2) says that S has no effect on D once B has been taken into account.

The model that has main effects for S,

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0+\beta_2 x_2+\beta_3 x_3\) (3)

where

\(x_2=1\) if S = medium,
\(x_2=0\) otherwise,

\(x_3=1\) if S = high,
\(x_2=0\) otherwise,

says that B has no effect on D once S has been taken into account. The goodness-of-fit tests for (3) are equivalent to testing the null hypothesis that (DS, BS) fits, i.e. that D and B are conditionally independent given S.

The logit model

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0+\beta_1 x_1+\beta_2 x_2+\beta_3 x_3\) (4)

has main effects for both B and S, corresponds to the model of homogeneous association which we discussed in Lesson 5. We could not fit the model at that time, because the ML estimates have no closed-form solution, but we calculated CMH statistic and Breslow-Day statistic. But with logistic regression software, fitting this model is no more difficult than for any other model.

This model says that the effect of B on D, when expressed in terms of odds ratios, is identical across the levels of S. Equivalently, it says that the odds ratios describing the relationship between S and D are identical across the levels of B. If this model does not fit, we have evidence that the effect of B on D varies across the levels of S, or that the effect of S on D varies across the levels of B.

Finally, the saturated model can be written as

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0+\beta_1 x_1+\beta_2 x_2+\beta_3x_3+\beta_4x_1x_2+\beta_5 x_1x_3\)

which has the main effects for B and S and their interactions. This model has \(X^2=G^2=0\) with zero degrees of freedom (see scout.sas).