Lesson 7: The Study Cohort
Lesson 7: The Study CohortOverview
In a multi-center trial, even when study eligibility criteria are carefully described and followed precisely by different investigators at different locations, there can be enough patient heterogeneity and differences in protocol interpretation that the results can vary greatly across institutions. Thus, the differences in results actually could be due to different selection factors at the different institutions. Recruitment strategies might be different. Due to this, different patients are recruited into the study.
Eligibility criteria also define the accrual rate for a trial. Although tighter eligibility criteria lead to a more homogeneous trial, they yield a slower accrual rate. It might be more difficult to meet all of the criteria you specify using strict eligibility criteria.
Objectives
- Compare the benefits and limitations of narrowly defined eligibility criteria to broadly defined eligibility criteria
- Recognize the impact of the 'healthy worker effect'.
- Write inclusion and exclusion criteria that are less subject to misinterpretation.
- Consider the advantages and disadvantages of a run-in period or extended baseline for a study.
- Use a simple method to monitor patient accrual.
- Recognize barriers to patient participation in clinical trials
- Distinguish between an efficacy study and an effectiveness trial.
References:
Gotay, CC. (1991). Accrual to cancer clinical trials: directions from the research literature. Soc. Sci. Med. 33: 569-577.
Piantadosi Steven. (2005) The study cohort, Treatment allocation. In: Piantadosi Steven. Clinical Trials: A Methodologic Perspective. 2nd ed. Hobaken, NJ: John Wiley and Sons, Inc.
7.1 - Defining the Study Cohort
7.1 - Defining the Study CohortIt can be more difficult to isolate a biological effect of a treatment if the investigator uses broadly-defined cohorts, i.e., patients with a variety of disease types and/or severity. It is easier to isolate the biological effect of a treatment in a narrowly-defined cohort because of patient homogeneity. The researcher's job is to balance these factors. Every situation is different and the researcher needs to think carefully when defining the selection criteria.
Although a narrowly-defined cohort may have some external validity for others with the same disease if the treatment appears to be beneficial, in general, it will lack external validity because the study results may not apply to patients with slightly altered versions of the disease. Again, these are examples of the competing demands that the researcher must keep in mind.
Epidemiologists have defined the “healthy worker effect” as the phenomenon that the general health of employed individuals is better than average. For example, employed individuals may be unsuitable controls for a case-control study if the cases are hospitalized, patients. Similarly, individuals who volunteer for clinical trials may have more favorable outcomes than those who refuse to participate, even if the treatment is ineffective. This selection effect is known as the “trial participant effect” and it can be strong. For a randomized trial, however, this may not be a problem unless selection effects somehow impact treatment assignment.
Because of the possible effects of prognostic (variables that can affect the outcome) and selection factors on differences in outcome, the eligibility criteria for the study cohort need to be defined carefully.
Two contrasting philosophies in defining these criteria are as follows:
- Define very narrow eligibility criteria so that the study cohort is relatively homogeneous, which may yield an outcome variable that has less variability and result in a smaller sample size; however, the results may not have external validity.
- Define very broad eligibility criteria, and accommodate the larger amount of variability by incorporating a larger sample size, which will provide much more external validity. (This is easy for a statistician to say!)
In many instances, endpoints/outcomes are more easily evaluated if certain complicating factors are prevented by patient exclusions. For example, habitual smokers typically are excluded from asthma trials because their lung function may be impaired by smoking as well as by their asthma. The smoking behavior may confound the results of the study. Exclusions also may be invoked for ethical reasons if the treatment is not expected to benefit a certain subgroup of patients. For example, some oncology (cancer) trials might exclude patients whose life expectancy does not exceed six months.
The difficulties with interpretation of inclusion and exclusion criteria can be minimized via quantitative expressions. For example, inclusion criteria should specify the range of allowable serum chemistry variables, instead of just stating that "we will require normal lab values". Different hospitals are going to have different interpretations of what “normal” is. Obviously, you need to be specific.
Once the decisions are made about the study cohort and other design issues resolved, the protocol approved and study medications obtained, the investigator begins what can be the most difficult task in a clinical trial - recruitment! Despite the most optimistic beliefs about the existence of available patients out there, a host of factors can make the recruitment of patients challenging.
(You may notice in this section we have defined a study ‘cohort’ for the trial. This doesn’t mean however that every clinical trial is a cohort study in the sense of a long-term study following a defined group of patients.)
7.2 - Assessing Accrual
7.2 - Assessing AccrualIt is unfortunate that some clinical trials are terminated early due to low accrual, which is a waste of resources and time for all those involved. Investigators often overestimate the accrual rate because they may not account for (1) the restrictions imposed by the eligibility criteria and (2) the refusal by some eligible patients to participate.
A famous saying which speaks to the challenges associated with recruitment among clinical trialists is “The incidence of a disease diminishes when you initiate a study on it.” (source unknown)
Run-in periods or extended baseline periods are helpful in assessing which eligible patients will adhere to the protocol. For example, patients can be administered a placebo during the run-in period and monitored for treatment compliance. At the completion of the run-in period, those patients who meet the treatment compliance criteria are then randomized to treatment, whereas those who do not are discontinued in the study. Another advantage of incorporating a run-in period is that it may provide the opportunity for patients to be stabilized via a standard medication prior to randomization.
One criticism of incorporating a run-in period is that it could decrease the external validity of the trial because in the real world some patients will not be very compliant. Thus, a trial based on very compliant patients may overestimate the effectiveness of the treatment.
Sometimes it is possible to conduct a formal survey of patients prior to the onset of a trial to determine the proportion that would consider participation. This might indicate to the researcher the approximate proportion of patients that would consider participating and enable realistic timetables for completing trials..
In any event, it is extremely important to monitor accrual on a regular basis throughout the course of a trial. An accrual graph with target and the actual number of recruited patients helps monitor the process. This task typically falls on the statistician. Here is an example of a plot monitoring the accrual of patients.
The target assumes a constant accrual of patients. There was a lag in the number of patients at the beginning that were recruited but it caught up with the target for recruitment by the end of the study. This struggle in the number of patients recruited is very typical. Recruitment is always a struggle. Everyone on the research team needs to help with this process.
7.3 - Other Considerations
7.3 - Other ConsiderationsAmong adult cancer patients in the USA, less than 3% of these patients participate in clinical trials. (Gotay, 1991) Since the process of clinical trials leads to improvements in cancer therapy over time, it would seem that cancer patients would be motivated to participate in increasing numbers over time. But this has not happened. Most diseases, except for AIDS and some pediatric conditions, exhibit similar types of participation rates. The three general reasons for lack of participation are categorized as physician-, patient-, or administrative-related.
The reasons physicians give for failing to enroll patients in clinical trials are the perception that the trial may compromise the physician-patient relationship and the difficulties with informed consent. Many consent forms are cumbersome, intimidating, and not written at an appropriate reading level. The 'experts' say that these documents should be written at an 8th-grade reading level. Using plain language is important. Also, many patients are mistrustful of the medical establishment, although they may trust their individual physicians. Often, ethnic minority groups express even stronger concerns about participation in clinical trials.
There is a distinction between an efficacy trial and an effectiveness trial. In an efficacy trial, the study cohort is relatively homogeneous and the objective is to test a biological question. In an effectiveness trial, the study cohort is relatively heterogeneous and the objective is to assess effectiveness of a treatment. An effectiveness trial tends to be very large and expensive but has much more external validity because of broad eligibility criteria and a heterogeneous population. Most clinical trials are effectiveness studies.
Example
An example of an efficacy study is the trial conducted by the Asthma Clinical Research Network (ACRN), entitled “Dose of Inhaled Corticosteroids with Equisystemic Effects (DICE).” The primary objective of the trial was to investigate dose-response effects of various inhaled corticosteroids (ICS) on cortisol production by the adrenal glands. Subjects with mild-moderate asthma were recruited. There were many exclusion criteria, such as obesity, pregnancy or lactation, no oral or injectable steroids during the past twelve months, no ICS or nasal steroids during the past six months, and no topical steroids during the past two months. Subjects were randomized to either one of six different ICS (n = 24 per group) or placebo (n = 12). ICS dose was doubled on a weekly basis (0d, 1d, 2d, 4d, and 8d, where d is a pre-selected low dose for each ICS). Subjects stayed overnight at a hospital at the end of each week, during which blood was drawn hourly and analyzed to determine the concentration of cortisol.
This study was examining a very specific biological question. The primary objective of the trial was to establish whether increasing the dose for each ICS yields a decrease in plasma cortisol (adrenal suppression). The researchers were interested in looking at dose response curves. The DICE trial was not powered to compare the dose-response curves of each ICS.
DICE is strictly an efficacy trial with very narrow eligibility criteria. Furthermore, the protocol specified that the intent-to-treat paradigm would not be followed. Subjects were dropped post-randomization if they received other forms of steroids, became pregnant, or were non-compliant with dose schedules and/or visit schedules.
On the other hand, over the past 20 years, there has been great interest in the gender and ethnic composition of cohorts in clinical trials. Part of this interest is due to ensuring external validity of the results of the trials. For many years caucasian males were the only patients recruited for the purpose of assuring homogeneity. This has been broadened by both the FDA and NIH in their application process. The broader eligibility requirements will help to ensure broader external validity.
The NIH typically requires one-half female participation and one-third ethnic minority participation in CTE trials that it sponsors. Obviously, there are exceptions to this based on the disease of interest. Required representation in clinical trials, however, could be a hindrance to acquiring new knowledge if it consumes too many resources.
7.4 - Summary
7.4 - SummaryIn this lesson, among other things, we learned:
- Compare the benefits and limitations of narrowly defined eligibility criteria to broadly defined eligibility criteria
- Recognize the impact of the 'healthy worker effect'.
- Write inclusion and exclusion criteria that are less subject to misinterpretation.
- Consider the advantages and disadvantages of a run-in period or extended baseline for a study.
- Use a simple method to monitor patient accrual.
- Recognize barriers to patient participation in clinical trials
- Distinguish between an efficacy study and an effectiveness trial.