One of the principal objectives of dose-finding (DF) studies is to assess the distribution and elimination of drug in the human system. What level of the drug is appropriate?

Pharmacokinetic (PK) models, also known as compartmental models, provide useful analytical approaches for DF studies. The objective of a PK model is to account for the absorption, distribution, metabolism, and excretion of a drug in the human system. An example of a two-compartment PK model is as follows and the objective is to estimate the rates into, between, and out of the two compartments.

Estimates are made for each of these areas, i.e., absorption rate, distribution rate, etc. for the drug in question.

The main objectives of most safety and efficacy (SE) studies with a new treatment are to estimate the frequency of adverse reactions and estimate the probability of treatment success. These types of endpoints often are expressed in binary form, presence/absence of adverse reaction, success/failure of treatment, etc., although this is not always the case.

Adverse reactions are often classified on an ordinal scale, such as absent/mild/moderate/severe. The primary efficacy endpoint also may be measured on an ordinal scale, such as failure/partial success/success, or it may be a time-to-event variable or measured on a continuous scale, such as a measurement of blood pressure. There are many ways to assess efficacy.

Estimates of risk can be useful in SE studies. Suppose that an SE study consists of a placebo group and a treatment group, and that probability of an adverse reaction is an important investigation. Let \(p_1\) and \(p_2\) denote the respective probabilities of an adverse reaction for the treatment and placebo groups. Three common parameters of risk are as follows:

\(\text{Risk Difference }= \Delta = p_1-p_2\)

\(\text{Relative Risk }= \dfrac{p_1}{p_2}\)

\(\text{Odds Ratio }= \theta = \dfrac{p_1/ (1-p_1)}{p_2/(1-p_2)}\)

For relative risk a number significantly different from 1 indicates a difference between the two groups in the risk for the event. The odds ratio indicates the relative odds of the event occurring between two groups.. Because both are ratios, the relative risk and the odds ratio are assessed in terms of their distance from 1.0.

When would the odds ratio and the relative risk be about the same? When \(p_1\) and \(p_2\) are relatively small, for instance, when you are dealing with a very rare event.

The odds ratio is useful and convenient for assessing risk when the response outcome is binary, but it does have some limitations.

Notice in the table above that while the absolute risk difference is constant, the relative risk varies greatly, as does the odds ratio. Thus, the magnitudes of the odds ratio and relative risk are strongly influenced by the initial probability of the condition.

When the outcome in a CTE trial is a binary response and the objective is to compare the two groups with respect to the proportion of success, the results can be expressed in a 2 × 2 table as

The estimated relative risk is \(\dfrac{\frac{r_1}{n_1}}{\frac{r_2}{n_2}} (\dfrac{r_1 }{n_1})/ (\dfrac{r_2}{n_2})\) and the estimated odds ratio is:

\(\hat{\theta}=\frac{r_1/(n_1-r_1)}{r_2/(n_2-r_2)}=\frac{r_1*(n_2-r_2)}{r_2*(n_1-r_1)}\)

There are a variety of methods for performing the statistical test of the null hypothesis \(H_0 \colon \theta = 1\) (or \H_0 \colon \Delta = 0\) , such as a z-test using a normal approximation, a \(\chi^2\) test (basically, a square of the z-test), a \(\chi^2\) test with continuity correction, and Fisher's exact test.

The normal and \(\chi^2\) approximations for testing \(H_0 \colon \theta = 1\) are relatively accurate if these conditions hold:

\(\frac{n_1(r_1+r_2)}{n_1+n_2} \ge 5, \frac{n_2(r_1+r_2)}{n_1+n_2} \ge 5, \frac{n_1(n_1+n_2-r_1-r_2)}{n_1+n_2} \ge 5, \frac{n_2(n_1+n_2-r_1-r_2)}{n_1+n_2} \ge 5\)

This expression is basically what we would have calculated for the expected values in the 2 × 2 table. The first part of the expression is the probability of success times the probability of being in group 1 times the number of subjects.

Otherwise, Fisher's exact test is recommended.

If the above condition is met, then the log_e-transformed estimated odds ratio has an approximate normal distribution:

\( log_e(\hat{\theta}) \sim N \left( \mu=log_e(\theta), \sigma^2=\frac{1}{r_1}+\frac{1}{r_2}+\frac{1}{n_1-r_1}+\frac{1}{n_2-r_2} \right)\)

Therefore, an approximate \(100(1 - \alpha)\%\) confidence interval for the log odds ratio is possible and would look like:

\(log_e(\hat{\theta}) \pm z_{1-\alpha/2}\sqrt{\frac{1}{r_1}+\frac{1}{r_2}+\frac{1}{n_1-r_1}+\frac{1}{n_2-r_2}}\)

An approximate \(100(1 - \alpha)\%\) confidence interval for the odds ratio is constructed by exponentiation the endpoints of the \(100(1 - \alpha)\%\) confidence interval for the log odds ratio. The computer can do this for you.

Sometimes a safety and efficacy study is stratified according to some factor, such as clinical center, disease severity, gender, etc. In such a situation, it still may be desirable to estimate the odds ratio while accounting for strata effects. The Mantel-Haenszel test for the odds ratio assumes that the odds ratio is equal across all strata, although the rates, \(p_1\) and \(p_2\), may differ across strata. This procedure calculates the odds ratio within each stratum and then combines the strata estimates into one estimate of the common odds ratio. For example,

In some safety and efficacy studies, it is of interest to determine if an increase in the dose yields an increase (or decrease) in the response. The statistical analysis for such a situation is called a dose-response or trend analysis. We want to see a trend here, not just a difference in groups. Typically, patients in a dose-response study are randomized to K + 1 treatment groups (a placebo dose and K increasing doses of the drug). The response variables of interest may be binary, ordinal, or continuous (in some circumstances, the response variable may be a time-to-event variable). In some instances, trend tests can be sensitive and reveal a mild trend where pair-wise comparisons would not be able to find significant differences and not be as helpful.

For the sake of illustration, suppose that the response is continuous and that we want to determine if there is a trend in the K + 1 population means.

A one-sided hypothesis testing framework for investigating an increasing trend is

A one-sided hypothesis testing framework for investigating a decreasing trend is

A two-sided hypothesis testing framework for investigating a trend is

More than likely we would use one of the one-sided tests as you probably have a hunch about the effect that will result.

For a continuous response, an appropriate test is the Jonckheere-Terptsra (JT) trend test that was developed in the 1950s. The JT trend test is based on a sum of Mann-Whitney-Wilcoxon tests :

\(JT=\sum_{k=0}^{K-1}\sum_{k'=1}^{K}MWW_{kk'}\)

where \(MWW,_kk´\) is the Mann-Whitney-Wilcoxon test for comparing group k to group \(k´, 0 ≤ k < k´ ≤ K\). Essentially, each of the pairs of groups is compared against one another and then summed up. In this way this test looks for trends.

If \(Y_ki , I = 1, \dots , n_k \), denote the observations from group k, and \(Y_k'i', i´ = 1, \dots , n_k'\) , denote the observations from group \(k´\), then

\(MWW_{kk'}=\sum_{i=1}^{n_k}\sum_{i'=1}^{n_k'}sign(Y_{k'i'}-Y_{ki}\)

Note that each MWW should be constructed in a consistent manner. For example, when comparing an observation from a lower dose group versus an observation higher dose group, take the difference of the latter minus the former.

As an example of how the JT statistic is constructed, suppose there are four dose groups in a study (placebo, low dose, mid-dose, and high dose). Then the JT trend test is the sum of six Mann-Whitney-Wilcoxon test statistics:

Values of the statistic JT near-zero support

Large positive values of the statistic JT support

Large negative values of the statistics JT support

The JT trend test actually is testing hypotheses about population medians, but if the underlying probability distribution is symmetric, the population mean and the population median are equal to one another. The JT trend test is available in PROC FREQ of SAS.

The parametric version of the JT trend test, based on the assumption of normal data, is to substitute the difference between sample means for the Mann-Whitney-Wilcoxon statistics. The numerator for the parametric test is as follows:

\(\sum_{k=0}^{K-1}\sum_{k'=k+1}^{K}(\bar{Y}_{k'}-\bar{Y}_{k})\)

Next, we assume that the \(K + 1\) groups have a homogeneous population variance, \(\sigma^2\) . The population variance is estimated by the pooled sample variance, \(s^2\) , and it has d degrees of freedom:

\(s^2=\frac{1}{d}\sum_{k=0}^{K}\sum_{i=1}^{n_k}(Y_{ki}-\bar{Y}_{k})^2, d=\sum_{k=0}^{K}(n_k-1)\)

Letting \(c_k = 2k - K, k = 0, 1, \dots , K\), the numerator reduces to:

\(\sum_{k=0}^{K}c_k \bar{Y}_{k}\)

Then the trend statistic is:

\(T=\left( \sum_{k=0}^{K}c_k \bar{Y}_{k} \right)/\left( \sqrt{s^2 \sum_{k=0}^{K}\dfrac{c_{k}^{2}}{n_{k}^{2}}} \right)\)

For example, if \(K = 3\) (placebo, low dose, mid dose, and high dose), then \(c_0 = -3, c_1 = -1, c_2 = 1, c_3 = 3\). Notice, however, that if there are an odd number of groups, then the middle group has a coefficient of zero. For example, with \(K = 2\) (placebo, low dose, and high dose) \(c_0 = - 1, c_1 = 0, c_2 = 1\). This is not ideal and there are better trend tests than JT and T for continuous data.

To use the actual dose values (denoted as \(d_0, d_1, \dots , d_K\)) in the parametric test, set \(c_k = d_k - \text{mean}(d_0, d_1, \dots , d_K), k = 0, 1, \dots , K\).

The T trend statistic can be constructed by using the CONTRAST statement in SAS PROC GLM.

The JT trend test works well for binary and ordinal data, as well as being available for continuous data.

Another trend test for binary data is the Cochran-Armitage (CA) trend test. The difference between the JT and CA trend tests is that for the latter test, the actual dose levels can be specified. In other words, instead of designating the dose levels as low, mid, or high, the actual numerical dose levels can be used in the CA trend test, such as 20 mg, 60, 180 mg.

The CA trend test, however, can yield unusual results if there is unequal spacing among the dose levels. If the dose levels are equally spaced and the sample sizes are equal (\(n_0 = n_1 = ... = n_K\)), then the JT and CA trend tests yield exactly the same results. Each of these parameters needs to be taken into account to make sure you are applying the best test for your data.

In many clinical trials involving serious diseases, such as cancer and AIDS, a primary objective is to evaluate the survival experience of the cohort. In clinical trials not involving serious diseases, survival may not be an outcome, but other time-to-event outcomes may be important. Examples include time to hospital discharge, time to disease relapse, time to getting another migraine, time to progression of disease, etc.

The Kaplan-Meier survival curve is a nonparametric technique for estimating the probability of survival, even in the presence of censoring (e.g. study is completed before the patient experiences the event), at any point in time. This statistical approach is nonparametric because it does not assume any particular distribution for the data, such as lognormal, exponential, or Weibull. It is a "robust" procedure because it is not adversely affected by one or more unusual data points.

In order to construct the Kaplan-Meier survival curve, the actual failure times need to be ordered from smallest to largest. In a sample size of n patients, denote these times of failure as \(t_1, t_2, \dots , t_K\). For convenience, let \(t_0 = 0\) denote the start time and let \(t_K+1 = ∞\).

At the \(k^th\) failure time, \(t_k\), the number of failures, \(d_k\), are noted as well as the number of patients who were at risk for failure immediately prior to \(t_k, n_k\). Notice that patients who are lost to follow-up (censored) prior to time \(t_j\) are not included in \(n_k\).

The algebraic formula for the Kaplan-Meier survival probability at time t is:

\(\hat{S}(t)=1, t_0 \le t \le t_1\)

\(\hat{S}(t)= \prod_{k'=1}^{k}\left( 1-\frac{d_{k'}}{n_{k'}} \right), t_k \le t \le t_{k+1}, k=1, 2, \dots , K \)

The calculation of S(t) utilizes conditional probability: the probability of surviving at time t, given that the person has survived up to time t. The Kaplan-Meier curve depicts S(t), the probability of surviving beyond time t.

An example with an initial sample of n = 100 patients is as follows:

Note that the probability estimate does not change until a failure event occurs. Also, censored values do not affect the numerator, but do affect the denominator. Thus, the Kaplan-Meier survival curve gives the appearance of a step function when graphed.

A graphical display of the Kaplan-Meier survival curve is as follows:

Each step down represents the occurrence of an event.

For comparative treatment efficacy (CTE) trials, the primary endpoints often are measured on a continuous scale. The sample mean (sample standard deviation), the sample median (sample interquartile range), or the sample geometric mean (sample coefficient of variation) serve as reasonable descriptive statistics in such circumstances.

The sample mean (sample standard deviation) is suitable if the data are normally distributed or symmetric without heavy tails. The sample median (sample interquartile range) is suitable for symmetric or asymmetric data. The sample geometric mean (sample coefficient of variation) is suitable when the data are log-normally distributed.

Usually, two-sample t-tests or Wilcoxon rank tests are applied to compare the two randomized groups. In some instances, baseline measurements (prior to randomized treatment assignment) of the primary endpoints are taken.

Suppose \(Y_{i1}\) and \(Y_{i2}\) denote the baseline and final measurements of the endpoint, respectively, for the \(i^{th}\) subject, \(i = 1, 2, \dots, n\). Instead of statistically analyzing the \(Y_{i2}s\), there could be an increase in precision by analyzing the change (or gain) in the response, namely, the \(Y_{i}s\) where \(Y_i = Y_{i2} - Y_{i1}\).

Suppose that the variance for each \(Y_{i1}\) and \(Y_{i2}\) is \(\sigma^2\) and that the correlation between \(Y_{i1}\) and \(Y_{i2}\) is \(ρ\) (we assume that subjects are independent of each other but that the pair of measurements within each subject are correlated).

This leads to

\(Var(Y_{i2} - Y_{i1}) = Var(Y_{i2}) + Var(Y_{i1}) - 2Cov(Y_{i2},Y_{i1}) = 2\sigma^2(1 - \rho)\)

Therefore,

\(Var(Y_{i2}) = \sigma^2 \text{ and } Var(Y_{i2} - Y_{i1}) = 2\sigma^2(1 - \rho)\)

If \(ρ> \dfrac{1}{2}\), which often is the case for repeated measurements within patients, then \(Var(Y_{i2} - Y_{i1}) < Var(Y_{i2})\). Thus, there may be more precision if the \(Y_{i2} - Y_{i1}\) are analyzed instead of the \(Y_{i2}\). This happens all the time. Using the patient as their own control is a good thing. We are interested in the differences that are occurring, therefore we will subtract the treatment period measurements from the baseline data for the patient. A two-sample t test or the Wilcoxon rank sum test can be applied to the change-from-baseline measurements if the CTE trial consists of two randomized groups, such as placebo and an experimental therapy.

An alternative approach with baseline measurements is analysis of covariance (ANCOVA). In this situation, the baseline measurement, \(Y_{i1}\), serves as a covariate, so that the final measurement for a subject is adjusted by the baseline measurement. A linear model that describes this for a two-armed trial with placebo and experimental treatment groups is as follows. The expected value for the \(i^{th}\) patient, \(i = 1, 2, \dots, n\), is:

\( E(Y_{i2}) = \mu_p + T_i( \mu_E - \mu_P) + Y_{i1}\beta \)

where \(\mu_P\) is the population mean for the placebo group, \(\mu_E\) is the population mean for the experimental treatment group, \(T_i = 0\) if the \(i^{th}\) patient is in the placebo group and 1 if in the experimental treatment group, and \(\beta\) is the slope for the baseline measurement.

The expectations for subjects in the placebo group and experimental treatment group, respectively, can be rewritten as:

\( E(Y_{i2} - Y_{i1}\beta) = \mu_P \text{ and } E(Y_{i2} - Y_{i1}\beta) = \mu_E \)

These expectations are analogous to the expectations for the change-from baseline measurements:

\( E(Y_{i2} - Y_{i1}) = \mu_P \text{ and } E(Y_{i2} - Y_{i1}) = \mu_E\)

The only difference between the two approaches is that in the change-from-baseline measurements, \(\beta\) is set equal to 1.0. In the ANCOVA approach, \(\beta\) is estimated in the analysis and may differ from 1.0. Thus, ANCOVA approach is more flexible and can yield slightly more statistical power and efficiency.

If the primary endpoint in a CTE trial is a time-to-event variable, then it will be of interest to compare the survival curves of the randomized treatment arms. Again, we will focus on a nonparametric approach that corresponds to comparing the Kaplan-Meier survival curves rather than a parametric approach.

The Mantel-Haenszel test can be adapted here in terms comparing two groups, say P and E for placebo and experimental treatment. In this situation, the Mantel-Haenszel test is called the logrank test.

The assumptions for the logrank test are that (1) the censoring patterns are the same for the two treatment groups, and (2) the hazard functions for the two treatment groups are proportional.

For each of the K distinct failure times across the two randomized groups at times \(t_1, t_2, \dots , t_K\), a 2 × 2 table is constructed. For failure time \(t_k , k = 1, 2, … , K\), the table is:

The logrank statistic constructs an observed minus expected score, under the assumption that the null hypothesis of equal event rates is true, for each of the K tables and then sums over all tables:

\(O-E=\sum_{k=1}^{K}\left( \frac{n_{Pk}d_{Ek}-n_{Ek}d_{Pk}}{n_{Pk}+n_{Ek}} \right)\)

The variance expression for the O - E score is as follows:

\(V_L=Var(O-E)=\sum_{k=1}^{K}\left( \frac{(d_{Pk}+d_{Ek})(n_{Pk}+n_{Ek}-d_{Pk}-d_{Ek})n_{Pk}n_{Ek}}{(n_{Pk}+n_{Ek}-1)(n_{Pk}+n_{Ek})^2} \right)\)

Then the logrank statistic is:

\(Z_L=(O-E)/\sqrt{V_L}\)

which has an approximate standard normal distribution.

The generalized Wilcoxon test also is a nonparametric test for comparing survival curves and it is an extension of the Wilcoxon rank-sum test in the presence of censoring. It also requires that the censoring patterns for the two treatment groups be the same, but it does not assume proportional hazards.

The first step in constructing the generalized Wilcoxon statistic is to pool the two samples of survival times (including censored values) and order them from lowest to highest. For the \(i^{th}\) observation in the ordered sample with survival (or censored) time\(t_i\), construct a score, \(U_i\), which represents the number of survival (or censored) times less than \(t_i\) minus the number of survival (or censored) times greater than \(t_i\). The \(U_i\) are summed over the experimental treatment group and a variance calculated, i.e.,

\(U=\sum_{i=1}^{n_E}U_i \text {and }V_U = Var(U)=\left( \frac{n_Pn_E}{(n_P+n_E)(n_P+n_E-1)}\right)\sum_{i=1}^{n_P+n_E}U_{i}^{2}\)

such that:

\(Z_U=(O-E)/\sqrt{V_U}\)

has an approximate standard normal distribution.

An example of constructing the \(U_i\) scores ("+" reflects censoring):

Then U = (-7) + 2 + (-2) + 3 = -4.

Although p-values are useful for hypothesis tests that are specified a priori, they provide poor summaries of clinical effects. In particular, they do not convey the magnitude of a clinical effect. The size of a p-value depends on the magnitude of the estimated treatment effect and its estimated variability (also a function of sample size). Thus, the p-value partially reflects the size of the trial, which has no biological interpretation. In addition, the p-value can mask the magnitude of the treatment effect, which does have biological importance. P-values only quantify the type I error and do not characterize the biologically important effects in the trial. Thus, p-values should not be used to describe the strength of evidence in a trial. Investigators have to look at the magnitude of the treatment effect.

Confidence intervals are more appropriate for describing the strength of evidence in a clinical trial, although they also are affected by the sample size. Most major journals now require this approach as it is many times more informative than simply just the p-value.

One of the most difficult statistical tasks is assessing the precision of an estimator, i.e., determining the variance of an estimator can be more difficult than determining the appropriate estimator. In complicated situations, the bootstrap method can be applied to estimate the variance of an estimator. The bootstrap is essentially a resampling plan.

For example, suppose an investigator collects a sample of N observations, denoted as \(Y_1, Y_2, \dots , Y_N\), and wants to estimate the median, \(η\), and get an expression for its variance. If the investigator does not want to make any assumptions about the distribution of the sample, then an explicit expression for the variance of the sample median does not exist.

The bootstrap can be used to construct a variance estimate of the sample median.

The bootstrap process consists of constructing B data sets, each with N observations, from the original data set. Each bootstrap sample is constructed by sampling with replacement from the original data set. This means that when constructing a bootstrap sample, N observations are generated one at a time where each \(Y_i\) has \(dfrac{1}{N}\) probability of being selected. Here is an example of the resampling of the original data:

Original sample: 17, 25, 16, 32, 27, 19, 25, 23, 22, 30

Thus, for \(b = 1, \dots,B\), the bootstrap sample \(Y_{b1}, Y_{b2}, \dots , Y_{bN}\)  is constructed and the sample median within the bth bootstrap sample is formed as:

\(\hat{\eta}_b= \text{ median }(Y_{b1},Y_{b2}, ... , Y_{bN})\)

From the B estimates of the median we construct the estimated variance as:

\(S^2=\dfrac{1}{B-1}\sum_{b=1}^{B}(\hat{\eta}_b-\bar{\eta})^2 \text{ where}\bar{\eta}=\frac{1}{B}\sum_{b=1}^{B}\hat{\eta}_b\)

to get a sense about how these medians are varying over the 100 samples. The variance estimate can then be used to construct a Z statistic for hypothesis testing, i.e.,

\(Z=(\hat{\eta}-\eta)/S\)

Some statisticians at first were leery of this approach, essentially using one sample to create many others samples from this original, i.e., "pulling oneself up by your bootstraps". The bootstrap process, however, over time has shown to have sound statistical properties. The disadvantage of this approach has to do with the random selection with replacement which could result in slight variations in results. The FDA, for instance, requires definitive results. This is simply a non-parametric approach for estimating the variance of a sample.

Clinical trial data provide the opportunity for exploratory analyses, which are analyses in addition to those specified by the primary objectives in the protocol. A trial design is usually not well-suited for all of the exploratory analyses that are performed, so the results may not have much validity.

The results from exploratory analyses should not be regarded as confirmatory, but rather as hypothesis-generating for future research. As a general rule, the same data should not be used both to generate a new hypothesis and to test that hypothesis. Unfortunately, many investigators do not follow this principle.

Data in sufficient quantity and detail can be made to yield some effect. A few statistical sayings attest to this, such as "the data will confess to anything if tortured enough." It has been well documented that increasing the number of hypothesis tests inflates the Type I error rate. Exploratory analyses typically fall into this category and the chances of finding statistically significant results, when none truly exists, can be very high.

Subset analyses are a form of exploratory analysis that are very popular with clinical trial data. For example, after performing the primary statistical analyses, the investigators might decide to compare treatment groups within certain subsets, such as male subjects, female subjects, minority subjects, subjects over the age of 50, subjects with serum cholesterol above 220, etc. Unless it is planned ahead of time, such analyses should remain exploratory.

In this lesson, among other things, we learned:

• State the objectives of a pharmacokinetic model.
• Use a SAS program to calculate a confidence interval for an odds ratio
• Use a SAS program to perform a Mantel-Haenszel.analysis to estimate an odds ratio adjusted for strata effects.
• Recognize when odds-ratios or relative risks differ significantly between groups.
• Modify a SAS program to perform JT and Cochran-Armitage tests for trend.
• Interpret a Kaplan-Meier survival curve.
• Interpret SAS output comparing survival curves.
• Describe the process of bootstrapping to estimate the variability of an estimator.