6.2.4 - Multi-level Predictor

The concepts discussed with binary predictors extend to predictors with multiple levels. In this lesson we consider \(Y_i\) a binary response, \(x_i\) a discrete explanatory variable (with \(k = 3\) levels, and make connections to the analysis of \(2\times3\) tables. But the basic ideas extend to any \(2\times J\) table.

We begin by replicating the analysis of the original \(3\times2\) table with logistic regression.

First, we re-express the data in terms of \(Y_i=\) number of smoking students, and \(n_i=\) number of students for the three groups based on parents behavior (we can think of \(i\) as an index for the rows in the table).

Then we decide on a baseline level for the explanatory variable \(x\) and create \(k − 1\)indicators if \(x\) is a categorical variable with \(k\) levels. For our example, we set "Neither" as the baseline for the parent smoking predictor and define a pair of indicators that take one of two values, specifically,

\(x_{1i}=1\) if parent smoking = "One" ,
\(x_{1i}=0\) otherwise

\(x_{2i}=1\) if parent smoking = "Both",
\(x_{2i}=0\) otherwise.

If these two indicator variables were added as columns in the table above, \(x_{1i}\) would have values \((0,1,0)\), and \(x_{2i}\) would have values \((1,0,0)\). Next, we let \(\pi_i\) denote the probability of student smoking for the \(i\)th row group so that finally, the model is

\(\log\left(\dfrac{\pi_i}{1-\pi_i}\right)=\beta_0+\beta_1 x_{1i}+\beta_2 x_{2i}\)

for \(i=1,\ldots,3\). Thus, the log-odds of a student smoking is \(\beta_0\) for students whose parents don't smoke ("neither" group), \(\beta_0+\beta_1\) for students with one smoking parent ("one" group), and \(\beta_0+\beta_2\) for students with both smoking parents. In particular, \(\beta_1\)  is the difference in log odds or, equivalently, the log odds ratio for smoking when comparing students with one smoking parent against students with neither smoking parent. Similarly, \(\beta_2\) represents that when comparing students with two smoking parents against students with neither smoking parent.

Based on the tabulated data, where we saw \(1.42=416 ( 1168)/(188(  1823))\) and \(1.80=400(1168)/(188(1380))\), we're not surprised to see \(\hat{\beta}_1=\log(1.42)=0.351\) and \(\hat{\beta}_2=\log(1.80)=0.588\). The estimated intercept should be \(\hat{\beta}_0=\log(188/1168)=-1.826\).

The goodness-of-fit statistics \(X^2\) and \(G^2\) from this model are both zeroes because the model is saturated. Suppose that we fit the intercept-only model as before by removing the predictors from the model statement:

proc logistic data=smoke descending;
class s1 s2 (ref=first)/ param=ref;
model y/n = / scale=none lackfit;
output out=predict pred=prob reschi=pearson resdev=deviance;
title 'Logistic regression for intercept-only model';
run;

    Null deviance:  3.8366e+01  on 2  degrees of freedom
Residual deviance: -3.7215e-13  on 0  degrees of freedom
AIC: 28.165

> ## hosmerlem() takes the vector of successes, predicted vector of success and g=# of groups as input
> ## produce the vector of predicted success "yhat"
> yhat=rowSums(response)*predict(smoke.logistic, type="response")
> yhat
1 2 3
400 416 188

> hosmerlem(response[,1], yhat, g=3) ## here run 3 groups
X^2 Df P(>Chi)
"-1.00593633284243e-24" "1" "."

We have shown that analyzing a \(2\times 3\) table for associations is equivalent to a binary logistic regression with two dummy variables as predictors. For \(2\times J\) tables, we would fit a binary logistic regression with \(J − 1\) indicator variables.

Starting with the (full) model

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0+\beta_1 x_1+\beta_2 x_2\)

the null hypothesis \(H_0\colon\beta_1=\beta_2=0\) specifies the intercept-only (reduced) model:

\(\log\left(\dfrac{\pi}{1-\pi}\right)=\beta_0\)

In general, this test has degrees of freedom equal to the number of slope parameters, which is 2 in this case. Large chi-square statistics lead to small p-values and provide evidence to reject the intercept-only model in favor of the full model.

The likelihood-ratio statistic is

\(G^2 = −2 (\log \mbox{ likelihood from reduced model}−(−2 \log \mbox{ likelihood from full model}))\)

and the degrees of freedom is \(k=\) the number of parameters differentiating the two models. The p-value is \(P(\chi^2_k \geq G^2)\).

For our example, \(G^2 = 5176.510 − 5138.144 = 38.3658\) with \(3 − 1 = 2\) degrees of freedom. Notice that this matches

Likelihood Ratio 38.3658 2 <.0001

from the "Testing Global Hypothesis: BETA=0" section. Here is the model we just looked at in SAS.

data smoke;
input s1 s2 $ y n ;
cards;
0 1 400 1780
1 0 416 2239
0 0 188 1356
;
proc logistic data=smoke descending;
class s1 s2 (ref=first)/ param=ref;
model y/n = s1 s2 / scale=none lackfit;
output out=predict pred=prob reschi=pearson resdev=deviance;
run;

proc logistic data=smoke descending;
class s1 s2 (ref=first)/ param=ref;
model y/n = s1 / scale=none lackfit;
output out=predict pred=prob reschi=pearson resdev=deviance;
run;

Compare this to

model y/n = s1 /scale=none lackfit;

For the test of the significance of a single variable \(x_j\)

\(H_0\colon\beta_j=0\) versus \(H_A\colon\beta_j\ne0\)

we can use the (Wald) test statistic and p-value. A large value of \(z\) (relative to standard normal) or \(z^2\) (relative to chi-square with 1 degree of freedom) indicates that we can reject the null hypothesis and conclude that \(\beta_j\) is not 0.

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept)  -1.82661    0.07858 -23.244  < 2e-16 ***
parentsmoke1  0.34905    0.09554   3.654 0.000259 ***
parentsmoke2  0.58823    0.09695   6.067  1.3e-09 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Confidence Intervals: An approximate \((1 − \alpha)100\)% confidence interval for \(\beta_j\) is given by

\(\hat{\beta}_j \pm z_{(1-\alpha/2)} \times SE(\hat{\beta}_j)\)

For example, a 95% CI for \(\beta-1\) is

\(0.3491 \pm 1.96 (0.0955) = (0.16192, 0.5368)\)

Then, the 95% CI for the odds-ratio of a student smoking, if one parent is smoking in comparison to neither smoking, is

\((\exp(0.16192), \exp(0.5368)) = (1.176, 1.710)\)

Since this interval does not include the value 1, we can conclude that student and parents' smoking behaviors are associated. Furthermore,

• The estimated conditional odds ratio of a student smoking between one parent smoking and neither smoking is \(\exp(\beta_1) = \exp(0.3491) = 1.418\).
• The estimated conditional odds ratio of a student smoking between both parents smoking and neither smoking is \(\exp(\beta_2) = \exp(0.5882) = 1.801\).
• The estimated conditional odds ratio of a student smoking between both parents smoking and one smoking is \(\exp(\beta_2)/\exp(\beta_1) = 1.801/1.418 = 1.27 = \exp(\beta_2-\beta_1) = \exp(0.5882 − 0.3491) = 1.27\). That is, compared with a student who has only one parent smoking, a student who has both parents smoking has an odds of smoking 1.27 times as high.