11.2 - Safety and Efficacy (Phase II) Studies: The Odds Ratio

11.2 - Safety and Efficacy (Phase II) Studies: The Odds Ratio

The main objectives of most safety and efficacy (SE) studies with a new treatment are to estimate the frequency of adverse reactions and estimate the probability of treatment success. These types of endpoints often are expressed in binary form, presence/absence of adverse reaction, success/failure of treatment, etc., although this is not always the case.

Adverse reactions are often classified on an ordinal scale, such as absent/mild/moderate/severe. The primary efficacy endpoint also may be measured on an ordinal scale, such as failure/partial success/success, or it may be a time-to-event variable or measured on a continuous scale, such as a measurement of blood pressure. There are many ways to assess efficacy.

Estimates of risk can be useful in SE studies. Suppose that an SE study consists of a placebo group and a treatment group, and that probability of an adverse reaction is an important investigation. Let \(p_1\) and \(p_2\) denote the respective probabilities of an adverse reaction for the treatment and placebo groups. Three common parameters of risk are as follows:

\(\text{Risk Difference }= \Delta = p_1-p_2\)

\(\text{Relative Risk }= \dfrac{p_1}{p_2}\)

\(\text{Odds Ratio }= \theta = \dfrac{p_1/ (1-p_1)}{p_2/(1-p_2)}\)

For relative risk a number significantly different from 1 indicates a difference between the two groups in the risk for the event. The odds ratio indicates the relative odds of the event occurring between two groups.. Because both are ratios, the relative risk and the odds ratio are assessed in terms of their distance from 1.0.

When would the odds ratio and the relative risk be about the same? When \(p_1\) and \(p_2\) are relatively small, for instance, when you are dealing with a very rare event.

The odds ratio is useful and convenient for assessing risk when the response outcome is binary, but it does have some limitations.

\(p_1\) \(p_2\) Risk Diff Rel Risk Odds Ratio
0.25 0.05 0.20 5.00 6.33
0.30 0.10 0.20 3.00 3.86
0.45 0.25 0.20 1.80 2.45
0.70 0.50 0.20 1.40 2.33

Notice in the table above that while the absolute risk difference is constant, the relative risk varies greatly, as does the odds ratio. Thus, the magnitudes of the odds ratio and relative risk are strongly influenced by the initial probability of the condition.

When the outcome in a CTE trial is a binary response and the objective is to compare the two groups with respect to the proportion of success, the results can be expressed in a 2 × 2 table as

  Group #1 Group #2
Success \(r_1\) \(r_2\)
Failure \(n_1 - r_1\) \(n_2 - r_2\)

The estimated relative risk is \(\dfrac{\frac{r_1}{n_1}}{\frac{r_2}{n_2}} (\dfrac{r_1 }{n_1})/ (\dfrac{r_2}{n_2})\) and the estimated odds ratio is:

\(\hat{\theta}=\frac{r_1/(n_1-r_1)}{r_2/(n_2-r_2)}=\frac{r_1*(n_2-r_2)}{r_2*(n_1-r_1)}\)

There are a variety of methods for performing the statistical test of the null hypothesis \(H_0 \colon \theta = 1\) (or \H_0 \colon \Delta = 0\) , such as a z-test using a normal approximation, a \(\chi^2\) test (basically, a square of the z-test), a \(\chi^2\) test with continuity correction, and Fisher's exact test.

The normal and \(\chi^2\) approximations for testing \(H_0 \colon \theta = 1\) are relatively accurate if these conditions hold:

\(\frac{n_1(r_1+r_2)}{n_1+n_2} \ge 5, \frac{n_2(r_1+r_2)}{n_1+n_2} \ge 5, \frac{n_1(n_1+n_2-r_1-r_2)}{n_1+n_2} \ge 5, \frac{n_2(n_1+n_2-r_1-r_2)}{n_1+n_2} \ge 5\)

This expression is basically what we would have calculated for the expected values in the 2 × 2 table. The first part of the expression is the probability of success times the probability of being in group 1 times the number of subjects.

Otherwise, Fisher's exact test is recommended.

If the above condition is met, then the loge-transformed estimated odds ratio has an approximate normal distribution:

\( log_e(\hat{\theta}) \sim N \left( \mu=log_e(\theta), \sigma^2=\frac{1}{r_1}+\frac{1}{r_2}+\frac{1}{n_1-r_1}+\frac{1}{n_2-r_2} \right)\)

Therefore, an approximate \(100(1 - \alpha)\%\) confidence interval for the log odds ratio is possible and would look like:

\(log_e(\hat{\theta}) \pm z_{1-\alpha/2}\sqrt{\frac{1}{r_1}+\frac{1}{r_2}+\frac{1}{n_1-r_1}+\frac{1}{n_2-r_2}}\)

An approximate \(100(1 - \alpha)\%\) confidence interval for the odds ratio is constructed by exponentiation the endpoints of the \(100(1 - \alpha)\%\) confidence interval for the log odds ratio. The computer can do this for you.

SAS® Example

Using PROC FREQ in SAS for performing statistical inference with the odds ratio in a two-way frequency table

An investigator conducted a small safety and efficacy study comparing the treatment to the placebo with respect to adverse reactions. The data are as follows:

  Treatment Placebo
adverse reaction 12 4
no adverse reaction 32 40 
***********************************************************************
* This is a program that illustrates the use of PROC FREQ in SAS for  *
* performing statistical inference with the odds ratio in a two-way   *
* frequency table.                                                    *
***********************************************************************;

proc format;
value groupfmt 0='placebo' 1='treatment';
value noyesfmt 0='no' 1='yes';
run;

data adverse_reactions;
input group advreact count;
format group groupfmt. advreact noyesfmt.;
cards;
0 0 40
0 1  4
1 0 32
1 1 12
;
run;

proc freq data=adverse_reactions;
tables group*advreact/chisq measures;
exact chisq measures;
weight count;
title "Statistical Inference Using the Odds Ratio";
run;

The estimated odds ratio is calculated as:

\(\hat{\theta}=\dfrac{(12)(40)}{(32)(4)}=3.75\)

and the approximate 95% confidence interval for the \(log_e\) odds ratio is

\(1.32 \pm (1.96 \times 0.62) = (0.10, 2.54)\)

so the 95% confidence interval for \(\theta\) is (1.10, 12.68).

Because the approximate 95% confidence interval for \(\theta\) does not contain 1.0, the null hypothesis of \(H_0 \colon \theta = 1\) is rejected at the 0.05 significance level.

Even though this data table satisfies the criteria for \(\text{log}_e\) estimated odds ratio to follow an approximate normal distribution, there still is a discrepancy between the approximate results and the exact results.

From PROC FREQ of SAS, the exact 95% exact confidence interval for \(\theta\) is (1.00, 17.25). Because the 95% confidence interval for \(\theta\) does contain 1.0, \(H_0 \colon \theta = 1\) is not rejected at the 0.05 significance level based on Fisher's exact test.

Legend
[1]Link
↥Has Tooltip/Popover
 Toggleable Visibility