Linear Discriminant Analysis is for homogeneous variance-covariance matrices. However, not all cases come from such simplified situations. Quadratic Discriminant Analysis is used for heterogeneous variance-covariance matrices:

$$\Sigma_i \ne \Sigma_j$$ for some $$i \ne j$$

Again, this allows the variance-covariance matrices to depend on the population.

$$s^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{\Sigma_i}|}-\frac{1}{2}{\mathbf{(x-\mu_i)'\Sigma^{-1}_i(x - \mu_i)}}+\log{p_i}$$

This is a function of population mean vectors and the variance-covariance matrices for the ith group. Similarly, we will determine a separate quadratic score function for each of the groups.

This is of course a function of the unknown population mean vector for group i and the variance-covariance matrix for group i. These will have to be estimated from the training data. As before, we replace the unknown values of $$\boldsymbol{\mu_i}$$,$$\mathbf{\Sigma_i}$$, and $$p_i$$ by their estimates to obtain the estimated quadratic score function as shown below:

$$\hat{s}^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{S_i}|}-\frac{1}{2}{\mathbf{(x-\bar{x}_i)'S^{-1}_i(x - \bar{x}_i)}}+\log{p_i}$$

All natural logs are used in this function.

Decision Rule: Our decision rule remains the same as well. We will classify the sample unit into the population that has the largest quadratic score function.

$$\hat{s}^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{S_i}|}-\frac{1}{2}{\mathbf{(x-\bar{x})'S^{-1}_i(x -\bar{x})}}+\log{p_i}$$

Let's illustrate this using the Swiss Banknotes example...

 [1] Link ↥ Has Tooltip/Popover Toggleable Visibility