For many treatment mechanism (TM) studies, sample size is not an important issue because usually only a few subjects are enrolled to investigate treatment mechanisms. Here you are taking a lot of measurements on a few subjects in order to find out what might be going on with your treatment.

As presented last week, dose-finding (DF) and dose-ranging studies typically involve a design scheme, such as a modified Fibonacci design or continual reassessment. An example of phase I cytotoxic drug trials is as follows. A set of doses is determined a priori, such as 100 mg, 200 mg, 300 mg, 500 mg, 800 mg, etc. Subjects are recruited into the DF study in groups of three. The first group receives the lowest dose of 100 mg. If none of the subjects experience the effect (toxicity, side effect, etc.), then the next group of three subjects is escalated to the next dose of 200 mg. If one of the three subjects at 100 mg experiences the effect, however, then the next group of three subjects will receive the same dose of 100 mg. Whenever six subjects at the same dose reveal at least two subjects that experience the effect, then the study is terminated and the chosen dose for a safety and efficacy study is the previous dose level.

With such mechanisms in place to determine initial dosage levels, selection of. the study sample size is not a major consideration. In fact, the final sample size is dependent on patient outcomes.

The U.S. FDA mandates that efficacy is proven prior to the approval of a drug. Efficacy means that the tested dose of the drug is effective at ameliorating the treated condition. Phase II trials evaluate the potential for efficacy; Phase III trials confirm efficacy. These trials can also be referred to as safety and activity studies.

A typical goal of a safety and efficacy (SE) study is to estimate certain clinical endpoints with a specified amount of precision. Confidence intervals are useful for reflecting the amount of precision, and the width of a confidence interval is a function of sample size.

The simplest example occurs when the outcome response is binary (success or failure). Let p denote the true (but unknown) proportion of successes in the population that will be estimated from a sample.

The sample size is denoted as n and the number of observed successes is r. Thus, the point estimate of p is:

\(\hat{p}=\frac{r}{n}\)

If the sample size is large enough, then the \(100(1 - \alpha)\%\) confidence interval can be approximated as:

\( \hat{p}\pm z_{1-\alpha/2}\sqrt{\hat{p}(1-\hat{p})/n}\)

Prior to the conduct of the study, however, the point estimate is undetermined so that an educated guess is necessary for the purposes of a sample size calculation.

If it is desirable for the confidence interval to have limits of \( \hat{p} \pm \delta \) .

for a \(100(1 - \alpha)\%\) confidence interval, and the researcher has a reasonable guess as to the value of p, reworking through the sample size equation, the target sample size is:

\( n=z_{1-\alpha/2}^{2}p(1-p)/\delta^2 \)

If a researcher guesses that \(p ≈ 0.4\) and wants a 95% confidence interval to have limits of \(\delta = 0.10\), then the required sample size is

\(n = \dfrac{(1.96)^{2}(0.4)(0.6)}{(0.10)^2} = 92\)

Notice that \(p(1 - p)\) is maximized when \(p = 0.5\). Therefore, because p has to be guessed, it is more conservative to use p = 0.5 in the sample size calculation. In the above example this yields \(n = \dfrac{(1.96)^2(0.5)(0.5)}{(0.10)^2} = 96\), a slightly larger sample size.

Notice that the sample size is a quadratic function of precision. If \(\delta = 0.05\) is desired instead of 0.10 in the above example, then

\(n = \dfrac{(1.96)^2(0.5)(0.5)}{(0.05)^2} = 384\)

If you want the confidence interval to be tighter remember that splitting the width of the confidence interval in half will involve quadrupling the number of subjects in the sample size!

The normal approximation for calculating the \(100(1 - \alpha)\%\) confidence for p works well if

\( n \hat{p}\left(1-\hat{p}\right) \ge 5 \)

Otherwise, exact binomial methods should be used.

In the exact binomial method, the lower \(100(\alpha/2)\%\) confidence limit for \(p\) is determined as the value \(p_L\) that satisfies

\(\alpha/2=\sum_{k=r}^{n}C(n,k)\ast (p_L)^k(1-p_L)^{n-k} \)

The upper \(100(1 - \alpha/2)\%\) confidence limit for \(p\) is determined as the value \(p_U\) that satisfies

\(\alpha/2=\sum_{k=0}^{r}C(n,k)\ast (p_U)^k(1-p_U)^{n-k} \)

SAS PROC FREQ provides the exact and asymptotic \(100(1 - \alpha)\%\) confidence intervals for a binomial proportion, p.

An approach for discarding an ineffective treatment in an SE study, based on the exact binomial method, is as follows. Suppose that the lowest success rate acceptable to an investigator for the treatment is 0.20. Suppose that the investigator decides to administer the treatment consecutively to a series of patients. When can the investigator terminate the SE trial if he continues to find no treatment successes?

For a clinical endpoint that can be approximated by a normal distribution in an SE study, the \(100(1 - \alpha)\%\) confidence interval for the population mean, \(\mu\), is

\( \bar{Y} \pm \left [ t_{n-1, 1-\alpha/2}s/\sqrt{n} \right ] \)

where

\( \bar{Y}=\sum_{i=1}^{n}Y_i/n \) is the sample mean,

\( t_{n-1, 1-\alpha/2} \) is the appropriate percentile from the \(t_{n-1}\) distribution, and

\( s^2= \sum_{i=1}^{n}(Y_{i} - \bar{Y})^2 / (n-1) \) is the sample variance and estimates \(σ^{2}\).

If σ is known, then a z-percentile can replace the t-percentile in the \(100(1 - \alpha)\%\) confidence interval for the population mean, \(\mu\), that is,

\( \bar{Y} \pm \left( z_{1- \alpha /2}\sigma / \sqrt{n} \right) \)

If n is relatively large, say n ≥ 60, then \(z_{1 - \alpha/2} ≈ t_{n - 1,1 - \alpha/2}\).

If it is desired for the \(100(1 - \alpha)\%\) confidence interval to be

\( \bar{Y} \pm \delta \)

then

\( n= z_{1-\alpha/2}^{2}\sigma^2/\delta^2 \)

For example, the necessary sample size for estimating the mean reduction in diastolic blood pressure, where \(σ = 5\) mm Hg and \(δ = 1\) mm Hg, is \(n = \dfrac{1.96^{2} \times 5^{2}}{1^2} = 96\).

Suppose that a comparative treatment efficacy (CTE) trial consists of comparing two independent treatment groups with respect to the means of the primary clinical endpoint. Let \(\mu_1\) and \(\mu_2\) denote the unknown population means of the two groups, and let \(\sigma\) denote the known standard deviation common to both groups. Also, let \(n_1\) and \(n_2\) denote the sample sizes of the two groups.

The treatment difference in means is \(\Delta = \mu_1 -\mu_2\) and the null hypothesis is \(H_0\colon \Delta = 0\). The test statistic is

\( Z = \left( \bar{Y}_1 - \bar{Y}_2 \right) / \sigma \sqrt{\frac{1}{n_1}+\frac{1}{n_2}} \)

which follows a standard normal distribution when the null hypothesis is true. If the alternative hypothesis is two-sided, i.e., \(H_1 \colon \Delta \ne 0\), then the null hypothesis is rejected for large values of |Z|.

Under a particular alternative where there might be some difference \(\Delta, \Delta = \mu_1 - \mu_2\),

\( Z = \left( \bar{Y}_1 - \bar{Y}_2 - \Delta \right)/ \sigma \sqrt{\frac{1}{n_1}+\frac{1}{n_2}} \)

Suppose we let \(AR = \dfrac{n_1}{n_2}\) denote the allocation ratio \(\left(AR\right)\), (in most cases we will assign \(AR = 1\) to get equal sample sizes). If we wish to a have large enough sample size to detect an effect size Δ with a two-sided, α-significance level test with \(100 \left(1 - \beta \right)\%\) statistical power, then

\( n_2 = \left( \frac{AR+1}{AR}\right) \left( z_{1-\alpha/2}+z_{1-\beta} \right)^2\sigma^2/\Delta^2 \)

and \(n_1 = AR \times n_2\).

Note this formula matches the sample size formula in our FFDRG text on p. 180, assuming equal allocation to the two treatment groups and multiplying the result here by 2 to get 2N, which FFDRG uses to denote the total sample size.

If the alternative hypothesis is one-sided, then \(Z_{1 - α}\) replaces \(Z_{1 - \frac{\alpha}{2}}\) in either formula.

Notice that the sample size expression contains \(\left(\dfrac{\sigma}{\Delta}\right)^2\), the square of the effect size expressed in standard deviation units. Thus, sample size is a quadratic function of the effect size and precision. As the variance gets larger, it has a quadratic effect on the sample size. For example, reducing the effect size by one-half quadruples the required sample size.

Although this sample size formula assumes that the standard deviation is known so that a z test can be applied, it works relatively well when the standard deviation must be estimated and a t-test applied. A preliminary guess of σ must be available, however, either from a small pilot study or a report in the literature. For smaller sample sizes \(\left(n_1 ≤ 30, n_2 ≤ 30 \right)\) percentiles from a t distribution can be substituted, although this results in both sides of the formula involving \(n_2\) so that it must be solved iteratively:

\( n_2 = \left( \dfrac{AR+1}{AR}\right) \left( t_{n_1+n_2-2,1-\alpha/2}+t_{n_1+n_2-2,1-\beta} \right)^2\sigma^2/\Delta^2 \)

What if the primary response variable is binary?

When the outcome in a CTE trial is a binary response and the objective is to compare the two groups with respect to the proportion of success, the results can be expressed in a 2 × 2 table as

There are a variety of methods for performing the statistical test of the null hypothesis \(H_0\colon p_1 = p_2\), such as a z-test using a normal approximation, a \(χ^2\) test (basically, a square of the z-test), a \(χ^2\) test with continuity correction, and Fisher's exact test.

The normal and \(χ^2\) approximations for comparing two proportions are relatively accurate when these conditions are met:

\( \dfrac{n_1(r_1+r_2)}{(n_1+n_2}\ge 5, \dfrac{n_2(r_1+r_2)}{(n_1+n_2}\ge 5, \dfrac{n_1(n_1+n_2-r_1-r_2)}{(n_1+n_2}\ge 5, \dfrac{n_2(n_1+n_2-r_1-r_2)}{(n_1+n_2}\ge 5 \)

Basically when the expected number in each cell is greater than 5, the normal or Chi Square approximation is useful.

Otherwise, Fisher's exact test is recommended. All of these tests are available in SAS PROC FREQ of SAS and will be discussed later in the course.

A sample size formula for comparing the proportions \(p_1\) and \(p_2\) using the normal approximation is given below:

\( n_2=\left( \dfrac{AR+1}{AR}\right)\left( z_{1-\alpha/2}+z_{1-\beta}\right)^2\bar{p}(1-\bar{p})/(p_1-p_2)^2 \)

where \(p_1 - p_2\) represents the effect size and

\( \bar{p}= (AR \cdot p_1+p_2) / (AR+1) \)

is the weighted average of the proportions.

For many clinical trials, the response is time to an event. The methods of analysis for this type of variable are generally referred to as survival analysis methods. The basic approach is to compare survival curves.

With an event time endpoint, it is mathematically convenient to compare treatment groups (and curves) with respect to the hazard ratio. The survival function for a treatment group is characterized by \(\lambda\), the hazard rate. At time t, \(\lambda(t)\) for a treatment group, is defined as the instantaneous risk of the event (or failure) occurring at time t. In other words, given that a subject has survived the event up to time t, the hazard at time t is the probability of the event occurring within the next instant. You can think of the hazard as the slope of the survival curve.

The hazard ratio is defined as the ratio of two hazard functions, \(\lambda_1(t)\) and \(\lambda_2(t)\), corresponding to two treatment groups. Typically, we assume proportional hazards, i.e., \(\Lambda= \dfrac{\lambda_1(t)}{\lambda_2(t)}\) is a constant function independent of time. The graphs on the next two slides illustrate the concept of proportional hazards.

A hazard function may be constant, increasing, or decreasing over time, or even be a more complex function of time. In trials in which survival time is the outcome, an increasing hazard function indicates that the instantaneous risk of death increases throughout the trial.

An example where the hazard function might be decreasing involves the disease ARDS (adult respiratory distress syndrome), whereby the risk of death is highest during the early stage of the disease.

A sample size formula for comparing the hazards of two groups via the logrank test (discussed later in the course) is expressed in terms of the total number of events, E, that need to occur. For a two-sided, α-level significance test with \(100 \left(1 - \beta \right)\%\) statistical power, hazard ratio Λ, and allocation ratio AR,

\( E=\left( \dfrac{(AR+1)^2}{AR}\right) \dfrac{\left( z_{1-\alpha/2} + z_{1-\beta} \right)^2}{ \left(log_e(\Lambda) \right)^2} \)

Since we do not expect all persons in the trial to experience an event, the sample size must be larger than the required number of events.

Suppose that \(p_1\) and \(p_2\) represent the anticipated event rates in the two treatment groups. Then the sample sizes can be determined from \(n_2 = \dfrac{E}{(AR \times p_1 + p_2)}\) and \(n_1 = AR \times n_2\)

If a hazard function is assumed to be constant during the follow-up period [0, T], then it can be expressed as \(\lambda(t) = \lambda = \dfrac{-log_e(1 - p)}{T}\). In such a situation, the hazard ratio for comparing two groups is \(\Lambda = \dfrac{log_e(1 - p_1)}{log_e(1 - p_2)}\).

A constant hazard rate, \(\lambda(t) = \lambda\) for all time points t, corresponds to an exponential survival curve, i.e., survival at time \(t = exp(-\lambda t)\).

Survival curves plot the probability of the event occurring to a subject over time.

Expanded Safety (ES) trials are phase IV trials designed to estimate the frequency of uncommon adverse events that may have been undetected in earlier studies. These studies may be nonrandomized.

Typically, we assume that the study population is large, the probability of an adverse event is small (because it did not crop up in prior trials), and all participants in the cohort of size m are followed for approximately the same length of time. Under these assumptions, we can model the probability of exactly d events occurring based on a Poisson probability function, i.e.,

\( Pr \left[ D = d \right] = ( \xi m)^d exp(- \xi m)/d! \)

where \( \xi \) is the adverse event rate.

The cohort should be large enough to have a high probability of observing at least one event when the event rate is \( \xi \). Thus, we want

\( \beta = Pr \left[D \ge 1 \right] = 1 - Pr \left[ D = 0 \right] = 1 - exp(-\xi m) \)

to be relatively large. With respect to the cohort size, this means that m should be selected such that

\( m = -log_e(1 - \beta)/ \xi \)

When calculating a sample size, we may need to adjust our calculations due to multiple primary comparisons or for nonadherence to therapy or to consider the anticipated dropout rate.

• If there is more than one primary outcome variable (for example, co-primary outcomes) or more than one primary comparison (for example, 3 treatment groups), then the significance level should be adjusted to account for the multiple comparisons in order not to inflate the overall false-positive rate.

For example, suppose a clinical trial will involve two treatment groups and a placebo group. The investigator may decide that there are two primary comparisons of interest, namely, each treatment group compared to placebo. The simplest adjustment to the significance level for each test is the Bonferroni correction, which uses \(\dfrac{\alpha}{2}\) instead of \(\alpha\).

In general, if there are K comparisons of primary interest, then the Bonferroni correction is to use a significance level of \(\dfrac{\alpha}{K}\) for each of the K comparisons. The Bonferroni correction is not the most powerful or most sophisticated multiple comparison adjustment, but it is a conservative approach and easy to apply.

In the case of multiple primary endpoints, an adjustment to the significance level may not be necessary, depending on how the investigator plans to interpret the results. For example, suppose there are two primary outcome variables. If the investigator plans to claim “success of the trial” if either endpoint yields a statistically significant treatment effect, then an adjustment to the significance level is warranted. If the investigator plans to claim “success of the trial” only if both endpoints yield statistically significant treatment effects, then an adjustment to the significance level is not necessary. Thus, an adjustment to the significance level in the presence of multiple primary endpoints depends on whether it is an “or” or an “and” situation.

(A composite outcome, such as "time to stroke, MI or major cardiovascular event", is different from a co-primary outcome. In this situation, when the composite results in one statistical analysis, there is no need for adjustment.)

• Another consideration is nonadhereance to the protocol (noncompliance). All participants randomized to therapy are expected to be included in the primary statistical analysis, an intention-to-treat analysis. Intention-to-treat analysis will compare the treatments using all data from subjects in the group to which they were originally assigned, regardless of whether or not they followed the protocol, stayed on therapy,etc. Some participants will choose to withdraw from a trial before it is complete. Every effort will be made to continue obtaining data from all randomized subjects; for those who withdraw from the study completely and do not provide data, an imputation procedure may be required to represent their missing data in subsequent data analyses. Some participants assigned to active therapy discontinue therapy but continue to provide data (therapeutic drop-outs). Some on a placebo or control add an active therapy (drop-ins) and continue to be observed. The nonadherence (noncompliance) can lead to a dilution of the treatment effect and lead to lower power for the study as well as biased estimates of treatment effects.

Thus, a further adjustment to the sample size estimate may be made based on the anticipated drop-out and drop-in rates in each arm (See Wittes (2002). A similar formula is on p. 179 FFDRG.

\(N^∗= \dfrac{N}{((1- R_O - R_I))^2}\) where N is the sample size without regard to nonadherence and N* is the adjusted number for that treatment arm.

\(R_O\) and \(R_I\) represent the proportion of participants anticipated to discontinue test therapy and the proportion in the control who will add or change to a more effective therapy, respectively.

• In other situations, an adjustment may be made to increase the sample size to account for the anticipated number of subjects who will drop-out of the study altogether so that there is sufficient power with the remaining observations to detect a certain difference. This adjustment is made by dividing the calculated sample size N by (1-W) where W is the proportion expected to withdraw.

\(N^{**}=\dfrac{N}{(1-W)}\)

Let's work an example.

In this lesson, among other things, we learned to:

• Identify studies for which sample size is an important issue
• Estimate the sample size required for a confidence interval for p for given \(\delta\) and \(\alpha\), using normal approximation and Fisher's exact methods
• Estimate the sample size required for a confidence interval for μ for given \(\delta\) and \(\alpha\), using normal approximation when the sample size is relatively large
• Estimate the sample size required for a test of \(H_0 \colon \mu_1 = \mu_2\) to have \((1 - \beta)\%\) power for given \(\delta\) and \(\alpha\), using normal approximation, with equal or unequal allocation.
• Estimate the sample size required for a test of \(H_0 \colon p_1 = p_2\) for given \(\delta\) and \(\alpha\) and \(\beta\), using normal approximation and Fisher’s exact methods
• Use a SAS program to estimate the number of events required for a logrank comparison of two hazard functions to have \((1 - \beta)\%\) power with given \(\alpha\)
• Use Poisson probability methods to determine the cohort size required to have a certain probability of detecting a rare event that occurs at a \(\text{rate} = \xi\).
• Adjust sample size requirements to account for multiple comparisons and the anticipated noncompliance rates.