10.2 - Quantitative Predictors: Orthogonal Polynomials

Polynomial trends in the response with respect to a quantitative predictor can be evaluated by using orthogonal polynomial contrasts, a special set of linear contrasts. This is an alternative to the regression analysis illustrated in the previous section, which may be affected by multicollinearity. Note that centering to remedy multicollinearity is effective only for quadratic polynomials. Therefore this simple technique of trend analysis performed via orthogonal polynomial coding will prove to be beneficial for higher-order polynomials. Orthogonal polynomials have the property that the cross-products defined by the numerical coefficients of their terms add to zero.

The orthogonal polynomial coding can be applied only when the levels of quantitative predictor are equally spaced. The method partitions the quantitative factor in the ANOVA table into independent single degrees of freedom comparisons. The comparisons are called orthogonal polynomial contrasts or comparisons.

Orthogonal polynomials are equations such that each is associated with a power of the independent variable (e.g. \(x\), linear; \(x^2\), quadratic; \(x^3\), cubic, etc.) In other words, orthogonal polynomials are coded forms of simple polynomials. The number of possible comparisons is equal to \(k-1\), where \(k\) is the number of quantitative factor levels. For example, if \(k=3\), only two comparisons are possible allowing for testing of linear and quadratic effects.

Using orthogonal polynomials to fit the desired model to the data allows us to eliminate collinearity while obtaining the same information as with simple polynomials. 

A typical polynomial model of order \(k-1\) would be:

\(y = \beta_0 + \beta_1 x + \beta_2 x^2 + \cdots + \beta_{(k-1)} x^{(k-1)} + \epsilon\)

The simple polynomials used are \(x, x^2, \dots, x^{(k-1)}\). We can obtain orthogonal polynomials as linear combinations of these simple polynomials. If the levels of the predictor variable, \(x\) are equally spaced then one can easily use coefficient tables to determine the orthogonal polynomial coefficients that can be used to set up an orthogonal polynomial model. 

If we are to fit the \((k-1)^{th}\) order polynomial using orthogonal contrasts coefficients, the general equation can be written as 

\(y_{ij} = \alpha_0 + \alpha_1 g_{1i}(x) + \alpha_2 g_{2i}(x) + \cdots + \alpha_{(k-1)} g_{(k-1)i}(x)+\epsilon_{ij}\)

where \(g_{pi}(x)\) is a polynomial in \(x\) of degree \(p\) for the \(i^{th}\) level treatment factor. The parameter \(\alpha_p\) will depend on the coefficients \(\beta_p\). Using the properties of the function \(g_{pi}(x)\), one can show that the first five orthogonal polynomial are of the following form:

Mean: \(g_0(x)=1\)

Linear: \(g_1(x)=\lambda_1 \left( \dfrac{x-\bar{x}}{d}\right)\)

Quadratic: \(g_2(x)=\lambda_2 \left( \left(\dfrac{x-\bar{x}}{d}\right)^2-\left( \dfrac{t^2-1}{12} \right)\right)\)

Cubic: \(g_3(x)=\lambda_3 \left( \left(\dfrac{x-\bar{x}}{d}\right)^3- \left(\dfrac{x-\bar{x}}{d}\right)\left( \dfrac{3t^2-7}{20} \right)\right)\)

Quartic: \(g_4(x) = \lambda_4 \left( \left( \dfrac{x - \bar{x}}{d} \right)^4 - \left( \dfrac{x - \bar{x}}{d} \right)^2 \left( \dfrac{3t^2 - 13}{14} \right)+ \dfrac{3 \left(t^2 - 1\right) \left(t^2 - 9 \right)}{560} \right)\)

where \(t\) = number of levels of the factor, \(x\) = value of the factor level, \(\bar{x}\) = mean of the factor levels, and \(d\) = distance between factor levels.

In the next section, we will illustrate how the orthogonal polynomial contrast coefficients are generated, and the Factor SS is partitioned. This method will be required to fit polynomial regression models with terms greater than the quadratic because even after centering there will still be multicollinearity between \(x\) and \(x^3\) as well as between \(x^2\) and \(x^4\).