The simplest case is \(2^k\) where \(k = 2\). We will define a new notation which is known as Yates notation. We will refer to our factors using the letters A, B, C, D, etc. as arbitrary labels of the factors. In the chemical process case, A is the concentration of the reactant and B is the amount of catalyst, both of which are quantitative. The yield of the process is our response variable.

Since there are two levels of each of two factors, \(2^k\) equals four. Therefore, there are four treatment combinations and the data are given below:

You can see that we have 3 observations at each of \(4 = 2^k\) combinations for \(k = 2\). So we have \(n = 3\) replicates.

The table above gives the data with the factors coded for each of the four combinations and below is a plot of the region of experimentation in two dimensions for this case.

The Yates notation used for denoting the factor combinations is as follows:

The use of this Yates notation indicates the high level of any factor simply by using the small letter of that level factor. This notation actually is used for two purposes. One is to denote the total sum of the observations at that level. In the case below \(b = 60\) is the sum of the three observations at the level b.

This shortcut notation, using the small letters, shows which level for each of our k factors we are at just by its presence or absence.

We will also connect this to our previous notation for the two-factor treatment design:

\(Y_{ijk} = \mu + \alpha_{i} + \beta_{j} + (\beta \beta)_{ij} + e_{ijk}\)

What is the primary goal of these screening experiments?

The goal is to decide which of these factors is important. After determining which factors are important, then we will typically plan for a secondary experiment where the goal is to decide what level of the factors gives us the optimal response. Thus the screening \(2^k\) experiment is the first stage, generally, of an experimental sequence. In the second stage, one is looking for a response surface or an experiment to find the optimal level of the important factors.

The definition of an effect in the \(2^k\) context is the difference in the means between the high and the low level of a factor. From this notation, A is the difference between the averages of the observations at the high level of A minus the average of the observations at the low level of A.

Therefore, \(A=\bar{y}_{A^+}-\bar{y}_{A^-}\), in the example above:

\(A = 190/6 - 140/6 = 50/6 = 8.33\)

Similarly, \(B=\bar{y}_{B^+}-\bar{y}_{B^-}\), is similar only looking in the other direction. In our example:

\(B = 150/6 - 180/6 = 25 - 30 = -5\)

and finally, \(AB=\dfrac{ab+(1)}{2n}-\dfrac{a+b}{2n}\)

\(AB = [(90 + 80)/6 - (100 + 60)/6] = 10/6 = 1.67\)

Therefore in the Yates notation, we define an effect as the difference in the means between the high and the low levels of a factor whereas in previous models we defined an effect as the coefficients of the model, which are the differences between the marginal mean and the overall mean. To restate this, in terms of A, the A effect is the difference between the means at the high levels of A versus the low levels of A, whereas the coefficient, \(\alpha_i\), in the model is the difference between the marginal mean and the overall mean. So the Yates "effect" is twice the size of the estimated coefficient α_i in the model, which is also usually called the effect of factor A.

The confusion is all in the notation used in the definition.

Let's look at another example in order to reinforce your understanding of the notation for these types of designs. Here is an example in three dimensions, with factors A, B and C. Below is a figure of the factors and levels as well as the table representing this experimental space.

How can we apply what we learned in the preceding section?

In general for \(2^k\) factorials the effect of each factor and interaction is:

\(\text{Effect} = (1/2^{(k-1)}n)\) [contrast of the totals]

We also defined the variance as follows:

\(\text{Variance(Effect)} = \sigma^2 / 2^{(k-2)}n\)

The true but unknown residual variance \(\sigma^2\), which is also called the within-cell variance, can be estimated by the MSE.

If we want to test an effect, for instance, say A = 0, then we can construct a t-test which is the effect over the square root of the estimated variance of the effect as follows:

\(t^{\ast}=\dfrac{Effect}{\sqrt{\frac{MSE}{n2^{k-2}}}} \sim t(2^k (n-1))\)

where ~ means that it has a t distribution with \(2^{k}(n-1)\) degrees of freedom.

Finally, here is the equation for the sum of squares due to an effect to complete the story here:

\(\text{SS(Effect)} = \text{(contrast of totals)}^{2} / 2^{k}n\)

Where does all of this come from? Each effect in a \(2^k\) model has one degree of freedom. In the simplest case, we have two main effects and one interaction. They each have 1 degree of freedom. So the t statistic is the ratio of the effect over its estimated standard error (standard deviation of the effect). You will recall that if you have a t statistic with \(\nu\) degrees of freedom and square it, you get an F distribution with one and \(\nu\) degrees of freedom.

\(t^2(v)=F(1,v)\)

We can use this fact to confirm the formulas just developed. We see that the

\((t^{\ast}(\nu))^2=\dfrac{(Effect)^2}{MSE/n2^{k-2}}\)

and from the definition of an F-test, when the numerator has 1 degree of freedom:

\(F(1,\nu)=\dfrac{SS(Effect)/1}{MSE}=\dfrac{(contrast)^2}{2^kn(MSE)}\)

But from the definition of an Effect, we can write \(\text{(Effect)}^2 = \text{(contrast)}^2 / (n2^{k-1})^{2}\) and thus \(F(1, \nu) = (t*(\nu))^2\) which you can show by some algebra or by calculating an example.

Hint: Multiply \(F(1, \nu)\) by \((2^{(k-1)}n)^{2} / (2^{(k-1)}n)^{2}\) and simplify.

Once you have these contrasts, you can easily calculate the effect, you can calculate the estimated variance of the effect and the sum of squares due to the effect as well.

Let's use Minitab to help us create a factorial design and then add data so that we can analyze it. The data come from Figure 6.1.

In Minitab we use the software under Stat > Design of Experiments to create our full factorial design. We will come back to this command another time to look at fractional factorial and other types of factorial designs.

In the example that was shown above, we did not randomize the runs but kept them in standard order for the purpose of seeing more clearly the order of the runs. In practice, you would want to randomize the order of run when you are designing the experiment.

Once we have created a factorial design within the Minitab worksheet we then need to add the response data so that the design can be analyzed. These response data, Yield, are the individual observations, not the totals. So, we again go to the Stat > DOE > Factorial Menu where we will analyze the data set from the factorial design.

We began with the full model with all the terms included, both the main effects and all of the interactions. From here we were able to determine which effects were significant and should remain in the model and which effects were not significant and can be removed to form a simpler reduced model.

Similar to the previous example, in this second industrial process example we have three factors, A equals Gap, B = Flow, C = Power and our response y = Etch Rate. (The data are from Table 6-4 in the text.) Once again in Minitab we will create a similar layout for a full factorial design for three factors with two replicates which gives us 16 observations. Next, we add the response data, Etch Rate, to this worksheet and analyze this data set. These are the results we get:

The analysis of variance shows the individual effects and the coefficients, (which are half of the effects), along with the corresponding t-tests. Now we can see from these results that the A effect and C effect are highly significant. The B effect is not significant. In looking at the interactions, AB, is not significant, BC is not significant, and the ABC are not significant. However the other interaction, AC is significant.

This is a nice example to illustrate the purpose of a screening design. You want to test a number of factors to see which ones are important. So what have we learned here? Two of these factors are clearly important, A and C. But B appears not to be important either as a main effect or within any interaction. It simply looks like random noise. B was the rate of gas flow across the edging process and it does not seem to be an important factor in this process, at least for the levels of the factor used in the experiment.

The analysis of variance summary table results show us that the main effects overall are significant. That is because two of them, A and C, are highly significant. The two-way interactions overall are significant. That is because one of them is significant. So, just looking at this summary information wouldn't tell us what to do except that we could drop the 3-way interaction.

Now we can go back to Minitab and use the Analyze command under Design of Experiments and we can remove all the effects that were seemingly not important such as any term having to do with B in the model. In running this new reduced model we get:

For this model, all three terms are significant.

These are \(2^k\) factorial designs with one observation at each corner of the "cube". An unreplicated \(2^k\) factorial design is also sometimes called a "single replicate" of the \(2^k\) experiment.

You would find these types of designs used where k is very large or the process, for instance, is very expensive or takes a long time to run. In these cases, for the purpose of saving time or money, we want to run a screening experiment with as few observations as possible. When we introduced this topic we wouldn't have dreamed of running an experiment with only one observation. As a matter of fact, the general rule of thumb is that you would have at least two replicates. This would be a minimum in order to get an estimate of variation - but when we are in a tight situation, we might not be able to afford this due to time or expense. We will look at an example with one observation per cell, no replications, and what we can do in this case.

Where are we going with this? We have first discussed factorial designs with replications, then factorial designs with one replication, now factorial designs with one observation per cell and no replications, which will lead us eventually to fractional factorial designs. This is where we are headed, a steady progression to designs with more and more factors, but fewer observations and less direct replication.

Let's look at the situation where we have one observation per cell. We need to think about where the variation occurs within this design. These designs are very widely used. However, there are risks…if there is only one observation at each corner, there is a high chance of an unusual response observation spoiling the results. What about an outlier? There would be no way to check if this was the case and thus it could distort the results fairly significantly. You have to remind yourself that these are not the definitive experiments but simply just screening experiments to determine which factors are important.  

In these experiments one really cannot model the "noise" or variability very well. These experiments cannot really test whether or not the assumptions are being met - again this is another shortcoming, or the price of the efficiency of these experiment designs.

When choosing the levels of your factors, we only have two options - low and high. You can pick your two levels low and high close together or you can pick them far apart. As most of you know from regression the further apart your two points are the less variance there is in the estimate of the slope. The variance of the slope of a regression line is inversely related the distance between the extreme points. You can reduce this variance by choosing your high and low levels far apart.

However, consider the case where the true underlying relationship is curved, i.e., more like this:

... and you picked your low and high level as illustrated above, then you would have missed capturing the true relationship. Your conclusion would probably be that there is no effect of that factor. You need to have some understanding of what your factor is to make a good judgment about where the levels should be. In the end, you want to make sure that you choose levels in the region of that factor where you are actually interested and are somewhat aware of a functional relationship between the factor and the response. This is a matter of knowing something about the context for your experiment.

How do we analyze our experiment when we have this type of situation? We must realize that the lack of replication causes potential problems in statistical testing:

• Replication provides an estimate of "pure error" (a better phrase is an internal estimate of error), and
• With no replication, fitting the full model results in zero degrees of freedom for error.

Potential solutions to this problem might be:

• Pooling high-order interactions to estimate error, (something we have done already in randomized block design),
• Normal probability plotting of effects (Cuthbert and Daniels, 1959), and/or
• Dropping entire factors from the model and other methods.

Let's use the dataset (Ex6-2.csv) and work at finding a model for this data with Minitab...

 Video Tutorial

Even with just one observation per cell, by carefully looking at the results we can come to some understanding as to which factors are important. We do have to take into account that these actual p-values are not something that you would consider very reliable because you are fitting this sequence of models, i.e., fishing for the best model. We have optimized with several decisions that invalidates the actual p-value of the true probability that this could have occurred by chance.

This is one approach to assume that some interactions are not important and use this to test lower order terms of the model and finally come up with a model that is more focused. Based on this for this example that we have just looked at, we can conclude that following factors are important, A, C, D, (of the main effects) and AC and AD of the two-way interactions.

Now I suggest you try this procedure and then go back and check to see what the final model looks like. Here is what we get when we drop factor B and all the interactions that we decided were not important:

The important factors didn't change much here. However, we have slightly higher degrees of freedom for error. But now what the design looks like, by having dropped B totally, is that we now have a \(2^3\) design with 2 replicates per cell. We have moved from a four-factor with one observation per cell, to a three-factor with two observations per cell.

So, we have looked at two strategies here. The first is to take a higher order interaction out of the model and use them as the estimate of error. Next, what we did at the end of the process is drop that factor entirely. If a particular factor in the screening experiment turns out to be not important either as a main effect or as part of any interaction we can remove it. This is the second strategy, and for instance in this example we took out factor B completely from the analysis.

Having included all the terms back into a full model we have shown how to produce a normal plot. Remember that all of these effects are 1 degree of freedom contrasts of the original data, each one of these is a linear combination of the original observations, which are normally distributed with constant variance. Then these 15 linear combinations or contrasts are also normally distributed with some variance. If we assume that none of these effects are significant, the null hypothesis for all of the terms in the model, then we simply have 15 normal random variables, and we will do a normal random variable plot for these. That is what we will ask Minitab to plot for us. We get a normal probability plot, not of the residuals, not of the original observations but of the effects. We have plotted these effects against what we would expect if they were normally distributed.

In the middle - the points in black, they are pretty much in a straight line - they are following a normal distribution. In other words, their expectation or percentile is proportionate to the size of the effect. The ones in red are like outliers and stand away from the ones in the middle and indicate that they are not just random noise but there must be an actual affect. Without making any assumptions about any of these terms this plot is an overall test of the hypothesis based on simply assuming all of the effects are normal. This is a very helpful - a good quick and dirty first screen - or assessment of what is going on in the data, and this corresponds exactly with what we found in our earlier screening procedures.

Let's look at another plot - the Pareto plot. This is simply a plot that can quickly show you what is important. It looks at the size of the effects and plots the effect size on a horizontal axis ranked from largest to smallest effect.

Having dropped some of the terms out of the model, for instance the three and four way interactions, Minitab plots the remaining effects, but now it is the standardized effect. Basically it is plotting the t-value, the effect over its standard deviation and then plotting it in ranked order. It also displays the t critical point as a red line at alpha = 0.05.

Another Minitab command that we can take a look at is the subcommand called Factorial Plots. Here we can create plots for main effects telling Minitab which factors you want to plot. As well you can plot two-way interactions. Here is a plot of the interactions (which are more interesting to interpret), for the example we've been looking at:

You can see that the C and D interaction plot the lines are almost parallel and therefore do not indicate interaction effects that are significant. However the other two combinations, A and C and A and D, indicate that significant interaction exists. If you just looked at the main effects plot you would likely miss the interactions that are obvious here.

We have reduced the model to include only those terms that we found were important. Now we want to check the residuals in order to make sure that our assumptions are not out of line with any conclusions that we are making. We can ask Minitab to produce a Four in One residuals plot which, for this example, looks like this:

In visually checking the residuals we can see that we have nothing to complain about. There does not seem to be any great deviation in the normal probability plot of the residuals. There's nothing here that is very alarming and it seems acceptable. In looking at the residuals versus the fitted values plot in the upper right of this four in one plot - except for the lower values on the left where there are smaller residuals and you might be somewhat concerned here, the rest do not set off any alarms - but we will come back to this later.

We may also want contour plots of all pairs of our numeric factors. These can be very helpful to understand and present the relationship between several factors on the response. The contour plots below for our example show the color coded average response over the region of interest. The effect of these changes in colors is to show the twist in the plane.

In the D*C plot area you can see that there is no curvature in the colored areas, hence no evidence of interaction. However, if you look at C*A display you can see that if C is low you get a dramatic change. If C is high it makes very little difference. In other words, the response due to A depends on the level of C. This is what the interaction means and it shows up nicely in this contour plot.

Finally, we can also ask Minitab to give us a surface plot. We will set this up the same way in Minitab and this time Minitab will show the plot in three dimensions, two variables at a time.

The surface plot shows us the same interaction effect in three dimensions in the twisted plane. This might be a bit easier to interpret. In addition you can ask Minitab to provide you with 3-D graphical tools that will allow you to grab these boxes and twist them around so that you can look at these boxes in space from different perspectives. Pretty cool! Give it a try. These procedures are all 'illustrated in the "Inspect" Flash movie at the beginning of this section.

When you look at the graph of the residuals as shown below you can see that the variance is small at the low end and the variance is quite large on the right side producing a fanning effect. Consider the family of transformations that can be applied to the response \(y_{ij}\).

Transformations towards the bottom of the list are stronger in how they shrink large values more than they shrink small values that are represented on the plot. This pattern of the residuals is one clue to get you to be thinking about the type of transformations you would select.

The other consideration and thinking about transformations of the response \(y_{ij}\) is what it does to the relationship itself. Some of you will recall from other classes the Tukey one-degree-of-freedom test for interaction. This is a test for interaction where you have one observation per cell such as with a randomized complete block design. But with one observation per cell and two treatments our model would be :

\(Y_{ijk}= \mu+\alpha_{i}+\beta_{j}+(\alpha\beta)_{ij}+\epsilon_{ijk}\)

where,

There is no estimate of pure error so we cannot fit the old model. The model proposed by Tukey's has one new parameter (γ) gamma :

\(Y_{ij}= \mu+\alpha_{i}+\beta_{j}+\gamma\alpha_{i}\beta_{j}+\epsilon_{ij}\)

This single parameter, gamma, is the 1 degree of freedom term and so our error,\(\epsilon_{ij}\), has (a-1)(b-1) -1 degrees of freedom. This model allows for just a single additional parameter which is based on a multiplicative effect on the two factors.

Now, when is this applicable?

Let's go back to the drill rate example (Ex6-3.MTW | Ex6-3.csv) where we saw the fanning effect in the plot of the residuals. In this example B, C and D were the three main effects and there were two interactions BD and BC. From Minitab we can reproduce the normal probability plot for the full model.

But let's first take a look at the residuals versus our main effects B, C and D.

All three of these residuals versus the main effects show same pattern, the large predicted values tend to have larger variation.

Next, what we really want to look at is the factorial plots for these three factors, B, C and D and the interactions among these, BD and BC.

What you see in the interaction plot above is a pattern that is non-parallel showing there is interaction present. But, from what you see in the residual graph what would you expect to see on this factor plot?

The tell-tale pattern that is useful here is an interaction that does not have crossing lines - a fanning effect - and it is exactly the same pattern that allows the Tukey model to fit. In both cases, it is a pattern of interaction that you can remove by transformation. If we select a transformation that will shrink the large values more than it does the small values and the overall result would be that we would see less of this fan effect in the residuals.

We can look at either the square root or log transformation. It turns out that the log transformation is the one that seems to fit the best. On a log scale it looks somewhat better - it might not be perfect but it is certainly better than what we had before.

Let's also look at the analysis of variance.

The overall main effects are still significant. But the two 2-way interactions effects combined are no longer significant, and individually, the interactions are not significant here either. So, the log transformation which improved the unequal variances pulled the higher responses down more than the lower values and therefore resulted in more of a parallel shape. What's good for variance is good for a simple model. Now we are in a position where we can drop the interactions and reduce this model to a main effects only model.

Now our residual plots are nearly homoscedastic for B, C and D. See below...

Serendipity - good things come in packages! When you pick the correct transformation, you sometimes achieve constant variance and a simpler model.

Many times you can find a transformation that will work for your data - giving you a simpler analysis but it doesn't always work.

Transformations are typically performed to:

• Stabilize variance - to achieve equal variance
• Improve normality - this is often violated because it is easy to have an outlier when variance is large which can be 'reined in' with a transformation
• Simplify the model

Sometimes transformations will solve a couple of these problems.

Is there always a transformation that can be applied to equalize variance? Not really ... there are two approaches to solving this question. First, we could use some non-parametric method. Although non-parametric methods have fewer assumptions about the distribution, you still have to worry about how you are measuring the center of the distribution. When you have a non-parametric situation you may have a different shaped distribution in different parts of the experiment. You have to be careful about using the mean in one case, and the media in another ... but that is one approach.

The other approach is a weighted analysis, where you weight the observations according to the inverse of their variance. There are situations where you have unequal variation for maybe a known reason or unknown reason, but if you have repeated observations and you can get weights, then you can do a weighted analysis.

It is this course author's experience many times you can find a transformation when you have this kind of pattern. Also, sometimes when you have unequal variance you just have a couple of bad outliers, especially when you only have one or a few observations per cell. In this case it is difficult to the distinguish whether you have a couple of outliers or the data is heteroscedastic - it is not always clear.

Prior (theoretical) knowledge or experience can often suggest the form of a transformation. However, another method for the analytical selection of lambda for the exponent used in the transformation is the Box-Cox (1964). This method simultaneously estimates the model parameters and the transformation parameter lambda.

Box-Cox method is implemented in some statistical software applications.