15.7 - Exponential Regression Example

One simple nonlinear model is the exponential regression model

\(\begin{equation*}
y_{i}=\beta_{0}+\beta_{1}\exp(\beta_{2}x_{i,1}+\ldots+\beta_{p+1}x_{i,1})+\epsilon_{i},
\end{equation*}\)

where the \(\epsilon_{i}\) are iid normal with mean 0 and constant variance \(\sigma^{2}\). Notice that if \(\beta_{0}=0\), then the above is intrinsically linear by taking the natural logarithm of both sides.

Exponential regression is probably one of the simplest nonlinear regression models. An example where an exponential regression is often utilized is when relating the concentration of a substance (the response) to elapsed time (the predictor).

To illustrate, consider the example on long-term recovery after discharge from hospital from page 514 of Applied Linear Regression Models (4th ed) by Kutner, Nachtsheim, and Neter. The response variable, Y, is the prognostic index for long-term recovery and the predictor variable, X, is the number of days of hospitalization. The proposed model is the two-parameter exponential model:

\(\begin{equation*}
Y_{i}=\theta_{0}\exp(\theta_{1}X_i)+\epsilon_{i},
\end{equation*}\)

where the \(\epsilon_i\) are independent normal with constant variance.

We'll use Minitab's nonlinear regression routine to apply the Gauss-Newton algorithm to estimate \(\theta_0\) and \(\theta_1\). Before we do this, however, we have to find initial values for \(\theta_0\) and \(\theta_1\). One way to do this is to note that we can linearize the response function by taking the natural logarithm:

\(\begin{equation*}
\log(\theta_{0}\exp(\theta_{1}X_i)) = \log(\theta_{0}) + \theta_{1}X_i.
\end{equation*}\)

Thus we can fit a simple linear regression model with response, \(\log(Y)\), and predictor, \(X\), and the intercept (\(4.0372\)) gives us an estimate of \(\log(\theta_{0})\) while the slope (\(-0.03797\)) gives us an estimate of \(\theta_{1}\). (We then calculate \(\exp(4.0372)=56.7\) to estimate \(\theta_0\).)

 Minitab: Nonlinear Regression Model

Now we can fit the nonlinear regression model:

  1. Select Stat > Regression > Nonlinear Regression, select prog for the response, and click "Use Catalog" under "Expectation Function."
  2. Select the "Exponential" function with 1 predictor and 2 parameters in the Catalog dialog box and click OK to go to the "Choose Predictors" dialog.
  3. Select days to be the "Actual predictor" and click OK to go back to the Catalog dialog box, where you should see "Theta1 * exp( Theta2 * days )" in the "Expectation Function" box.
  4. Click "Parameters" and type "56.7" next to "Theta1" and "-0.038" next to "Theta2" and click OK to go back to the Nonlinear Regression dialog box.
  5. Click "Options" to confirm that Mintab will use the Gauss-Newton algorithm (the other choice is Levenberg-Marquardt) and click OK to go back to the Nonlinear Regression dialog box.
  6. Click "Graphs" to confirm that Mintab will produce a plot of the fitted curve with data and click OK to go back to the Nonlinear Regression dialog box.
  7. Click OK to obtain the following output:
Nonlinear Regression: Prog = Theta1 * exp(Theta2 * days)
Method
Algorithm Gauss-Newton
Max iterations 200
Tolerance 0.00001
Starting Values for Parameters
Parameter Value
Theta1 56.7
Theta2 -0.038
Equation

prog = 58.6066 * exp(-0.0395865 * days)

Parameter Estimates
Parameter Estimate SE Estimate
Theta1 58.6066 1.47216
Theta2 0.0396 0.00171

prog = Theta1 * exp(Theta2 * days)

Summary
Iterations 5
Final SSE 49.4593
DFE 13
MSE 3.80456
S 1.95053
Fitted line plot