10.6 - Quadratic Discriminant Analysis

Linear Discriminant Analysis is for homogeneous variance-covariance matrices. However, not all cases come from such simplified situations. Quadratic Discriminant Analysis is used for heterogeneous variance-covariance matrices:

\(\Sigma_i \ne \Sigma_j\) for some \(i \ne j\)

Again, this allows the variance-covariance matrices to depend on the population.

Quadratic discriminant analysis calculates a Quadratic Score Function:

\(s^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{\Sigma_i}|}-\frac{1}{2}{\mathbf{(x-\mu_i)'\Sigma^{-1}_i(x - \mu_i)}}+\log{p_i}\)

This is a function of population mean vectors and the variance-covariance matrices for the ith group. Similarly, we will determine a separate quadratic score function for each of the groups.

This is of course a function of the unknown population mean vector for group i and the variance-covariance matrix for group i. These will have to be estimated from the training data. As before, we replace the unknown values of \(\boldsymbol{\mu_i}\),\(\mathbf{\Sigma_i}\), and \(p_i\) by their estimates to obtain the estimated quadratic score function as shown below:

\(\hat{s}^Q_i (\mathbf{x}) = -\frac{1}{2}\log{|\mathbf{S_i}|}-\frac{1}{2}{\mathbf{(x-\bar{x}_i)'S^{-1}_i(x - \bar{x}_i)}}+\log{p_i}\)

All natural logs are used in this function.

Let's illustrate this using the Swiss Banknotes example...