2.3 - Population Variance

Linear combinations not only have a population mean but they also have a population variance. The population variance of a linear combination is expressed as the following double sum of j = 1 to p and k = 1 to p over all pairs of variables.

\(var(Y) =\sum_{j=1}^{p}\sum_{k=1}^{p}c_jc_k\sigma_{jk}= c ^ { \prime } \Sigma c\)

In each term within the double sum, the product of the paired coefficients \(c_{j}\) times \(c_{k}\) is multiplied by the covariance between the \(j^{th}\) and \(k^{th}\) variables. If \(\Sigma\) is the variance-covariance matrix of \(\mathbf{X}\), then \(Var(Y) = c ^ { \prime } \Sigma c\) .

Expressions of vectors and matrices of this form are called quadratic forms.

When using this expression, the covariance between the variables and itself, or \(\sigma_{jj}\) is simply equal to the variance of the \(j^{th}\) variable, or \(\sigma_{j}^{2 }\).

\( \sigma_{jj} = \sigma^2_j\)

The variance of the random variable y can be estimated by the sample variances or s squared Y. This is obtained by substituting the sample variances and covariances for the population variances and covariances as shown in the expression below.

\(s^2_Y = \sum_{j=1}^{p}\sum_{k=1}^{p}c_jc_ks_{jk} =c ^ { \prime } S c\)

A simplified calculation can be found below. This involves two terms.

The first term involves summing over all the variables. Here we take the squared coefficients and multiply them by their respective variances. In the second term, we sum over all unique pairs of variables j less than k. Again take the product of \(c_{j}\) times \(c_{k}\) times the covariances between variables j and k. Since each unique pair appears twice in the original expression, we must multiply the sum by 2.