options ls=78;
title "PCA - Correlation Matrix - Places Rated";

 /* After reading in the places data, the (base 10) log transformations are taken.
  * This is an optional step and not required for the pca analysis.
  */

data places;
  infile "D:\Statistics\STAT 505\data\places.csv" firstobs=2 delimiter=',';
  input climate housing health crime trans educate arts recreate econ id;
  climate=log10(climate);
  housing=log10(housing);
  health=log10(health);
  crime=log10(crime);
  trans=log10(trans);
  educate=log10(educate);
  arts=log10(arts);
  recreate=log10(recreate);
  econ=log10(econ);
  run;

 /* The princomp procedure performs pca on the correlation matrix.
  * The out=a option saves results to a data set named 'a'.
  */

proc princomp data=places out=a;
  var climate housing health crime trans educate arts recreate econ;
  run;

 /* The corr procedure is used to calculate pairwise correlations
  * between the first 3 principal components and the original variables.
  */

proc corr data=a;
  var prin1 prin2 prin3 climate housing health crime trans educate arts 
      recreate econ;
  run;

proc gplot data=a;
  axis1 length=5 in;
  axis2 length=5 in;
  plot prin2*prin1 / vaxis=axis1 haxis=axis2;
  run;

Analysis

We need to focus on the eigenvalues of the correlation matrix that correspond to each of the principal components. In this case, the total variation of the standardized variables is equal to p, the number of variables. After standardization, each variable has a variance equal to one, and the total variation is the sum of these variations, in this case, the total variation will be 9.

The eigenvalues of the correlation matrix are given in the second column in the table below.  The proportion of variation explained by each of the principal components as well as the cumulative proportion of the variation explained are also provided.

Step 1

Examine the eigenvalues to determine how many principal components to consider:

The first principal component explains about 37% of the variation. Furthermore, the first four principal components explain 72%, while the first five principal components explain 82% of the variation. Compare these proportions with those obtained using non-standardized variables. This analysis is going to require a larger number of components to explain the same amount of variation as the original analysis using the variance-covariance matrix. This is not unusual.

In most cases, the required cut-off is pre-specified; i.e. how much of the variation to be explained is pre-determined. For instance, I might state that I would be satisfied if I could explain 70% of the variation. If we do this, then we would select the components necessary until you get up to 70% of the variation. This would be one approach. This type of judgment is arbitrary and hard to make if you are not experienced with these types of analysis. The goal - to some extent - also depends on the type of problem at hand.

Another approach would be to plot the differences between the ordered values and look for a break or a sharp drop. The only sharp drop that is noticeable in this case is after the first component. One might based on this, select only one component. However, one component is probably too few, particularly because we have only explained 37% of the variation. Consider the scree plot based on the standardized variables.

The scree plot for standardized variables (correlation matrix)

Step 2

Next, we can compute the principal component scores using the eigenvectors. This is a formula for the first principal component:

\(\begin{array} \hat{Y}_1 & = & 0.158 \times Z_{\text{climate}} + 0.384 \times Z_{\text{housing}} + 0.410 \times Z_{\text{health}}\\ & & + 0.259 \times Z_{\text{crime}} + 0.375 \times Z_{\text{transportation}} + 0.274 \times Z_{\text{education}} \\ && 0.474 \times Z_{\text{arts}} + 0.353 \times Z_{\text{recreation}} + 0.164 \times Z_{\text{economy}}\end{array}\)

And remember, this is now a function of the standardized data, not of the raw data.

The magnitudes of the coefficients give the contributions of each variable to that component. Because the data have been standardized, they do not depend on the variances of the corresponding variables.

Step 3

Let's look at the coefficients for the principal components. In this case, because the data are standardized, the relative magnitude of each coefficient can be directly assessed within a column.  Each column here corresponds with a column in the output of the program labeled Eigenvectors.

Interpretation of the principal components is based on which variables are most strongly correlated with each component. In other words, we need to decide which numbers are large within each column. In the first column, we see that Health and Arts are large. This is very arbitrary. Other variables might have also been included as part of this first principal component.