6a.2 - Safety and Efficacy Studies

The U.S. FDA mandates that efficacy is proven prior to the approval of a drug. Efficacy means that the tested dose of the drug is effective at ameliorating the treated condition. Phase II trials evaluate the potential for efficacy; Phase III trials confirm efficacy. These trials can also be referred to as safety and activity studies.

A typical goal of a safety and efficacy (SE) study is to estimate certain clinical endpoints with a specified amount of precision. Confidence intervals are useful for reflecting the amount of precision, and the width of a confidence interval is a function of sample size.

The simplest example occurs when the outcome response is binary (success or failure). Let p denote the true (but unknown) proportion of successes in the population that will be estimated from a sample.

The sample size is denoted as n and the number of observed successes is r. Thus, the point estimate of p is:

\(\hat{p}=\frac{r}{n}\)

If the sample size is large enough, then the \(100(1 - \alpha)\%\) confidence interval can be approximated as:

\( \hat{p}\pm z_{1-\alpha/2}\sqrt{\hat{p}(1-\hat{p})/n}\)

Prior to the conduct of the study, however, the point estimate is undetermined so that an educated guess is necessary for the purposes of a sample size calculation.

If it is desirable for the confidence interval to have limits of \( \hat{p} \pm \delta \) .

for a \(100(1 - \alpha)\%\) confidence interval, and the researcher has a reasonable guess as to the value of p, reworking through the sample size equation, the target sample size is:

\( n=z_{1-\alpha/2}^{2}p(1-p)/\delta^2 \)

If a researcher guesses that \(p ≈ 0.4\) and wants a 95% confidence interval to have limits of \(\delta = 0.10\), then the required sample size is

\(n = \dfrac{(1.96)^{2}(0.4)(0.6)}{(0.10)^2} = 92\)

Notice that \(p(1 - p)\) is maximized when \(p = 0.5\). Therefore, because p has to be guessed, it is more conservative to use p = 0.5 in the sample size calculation. In the above example this yields \(n = \dfrac{(1.96)^2(0.5)(0.5)}{(0.10)^2} = 96\), a slightly larger sample size.

Notice that the sample size is a quadratic function of precision. If \(\delta = 0.05\) is desired instead of 0.10 in the above example, then

\(n = \dfrac{(1.96)^2(0.5)(0.5)}{(0.05)^2} = 384\)

If you want the confidence interval to be tighter remember that splitting the width of the confidence interval in half will involve quadrupling the number of subjects in the sample size!

The normal approximation for calculating the \(100(1 - \alpha)\%\) confidence for p works well if

\( n \hat{p}\left(1-\hat{p}\right) \ge 5 \)

Otherwise, exact binomial methods should be used.

In the exact binomial method, the lower \(100(\alpha/2)\%\) confidence limit for \(p\) is determined as the value \(p_L\) that satisfies

\(\alpha/2=\sum_{k=r}^{n}C(n,k)\ast (p_L)^k(1-p_L)^{n-k} \)

The upper \(100(1 - \alpha/2)\%\) confidence limit for \(p\) is determined as the value \(p_U\) that satisfies

\(\alpha/2=\sum_{k=0}^{r}C(n,k)\ast (p_U)^k(1-p_U)^{n-k} \)

SAS PROC FREQ provides the exact and asymptotic \(100(1 - \alpha)\%\) confidence intervals for a binomial proportion, p.

Example

Using PROC FREQ in SAS for determining an exact confidence interval for a binomial proportion Section

6.1_binomial_proportion.sas (from Piantadosi, 2005) This is a program that illustrates the use of PROC FREQ in SAS for determining an exact confidence interval for a binomial proportion.

***********************************************************************
* This is a program that illustrates the use of PROC FREQ in SAS for  *
* determining an exact confidence interval for a binomial proportion. *
***********************************************************************;

proc format;
value succfmt 1='yes' 2='no';
run;

data Example_1;
input success count;
format success succfmt.;
cards;
1 03
2 16
;
run;

proc freq data=Example_1;
tables success/binomial alpha=0.05;
weight count/zeros;
title "Exact and Asymptotic 95% Confidence Intervals for a Binomial Proportion";
run;

In the example above, \(n = 19\) is the sample size and \(r = 3\) successes are observed in a binomial trial. The point estimate of p is

\( \hat{p} = 0.16 \)

Note, however, that

\( n\hat{p}(1-\hat{p}) = 19(0.16)(0.84)=2.55 <5 \)

The 95% confidence interval for p, based on the exact method, is [0.03, 0.40]. The 95% confidence interval for p, based on the normal approximation, is [-0.01, 0.32], which is modified to [0.00, 0.32] because p represents the probability of success that is supposed to be restricted to lie within the [0, 1] interval. Even with the correction to the lower endpoint, the confidence interval based on the normal approximation does not appear to be very accurate in this example.

Now it's your turn!

Modify the SAS program above to reflect 11 successes out of 75 trials. Run the program. Do the results round to (0.08, 0.25) for the 95% exact confidence limits?

If an investigator estimates \(p = 0.15\) and wants a 95% exact confidence interval with \(\delta = 0.1\), what sample size is needed? One way to solve this is to use SAS PROC FREQ in a "guess and check" manner. In this case, \(n = 73\) with 11 successes will result in a 95% exact confidence interval of (0.07, 0.25). It may impossible to exactly achieve the desired \(\delta\), but an estimate of the required sample size can be provided.

Using the exact confidence interval for a binomial proportion is the better option if you are not sure you are working in a standard normally distributed population.