1、Chapter 8,Principal Components,A principal component analysis is concerned with explaining the variance-covariance structure of a set of variables through a few linear combinations of these variables. Its general objective are (1) data reduction and (2) interpretation.,8.1 INTRODUCTION,Algebraically
2、, principal components are particu-lar linear combinations of the random variab-les , , , . Geometrically, these linear combinations represent the selection of a new coordinate system obtained by rotating the origi-nal system with , , , as the coordina-te axes. The new axes represent the directions
3、with maximum variability. As we shall see, principal components dep-end solely on the covariance matrix (or the,8.2 POPULATION PRINCIPAL COMPONENTS,correlation matrix ) of , , , . Let the random vector have the convariance matrix with eigenvalues . Consider the linear combinations,Then, we obtainThe
4、 principal components are those uncorrelated linear combinations , , , whose variances in (8.2) are as large as possible. The first principal component is the linear combination with maximum variance. That is, it maximizes . It is clear that,can be increased by multiplying any by some constant. To e
5、liminate this indete-rminacy, it is convenient to restrict attention to coefficient vectors of unit length. We therefore define First principal component=linear combination that maximizes subject to Second principal component=linear combination that maximizes subject to,and At the ith step, th princ
6、ipal component=linear combination that maximizes subject to and for Result 8.1. Let be the convariance matrix associated with the random vector . Let have the eigenvalue-eigenvector pairs , , ,Where . Then the th principal comp-onent is given by,With these choices,If some are equal, the choices of t
7、he correspon-ding coefficient vectors, , and hence , are not unique.,perpendicular to (that is, , ) gives . Now, the eigenvectors of are orthogonal if all the eigenvalues , , , are distinct. If the eigenvalues are not all distinct, the eigenvectors corresponding to common eigenval-ues may be chosen
8、to be orthogonal. Therefore, for any two eigenvectors and , , .,Result 8.2.,This is the proportion of total variance explained by the kth principal component. Proportion of totalpopulation variancedue to kth principal component,If most (for instance, 80 to 90%) of the total popu-lation variance, for
9、 large , can be attributed to thethe first one, two, or three components, then these components can “replace” the original variables without much loss of information. Each component of the coefficient vector also merits inspection. The magnitude of measures the importance of the kth variable to the
10、ith principal component,irrespective of the other variables. In particular, is proportional to the correlation coefficient betw-een and , because of ( ). Result8.3. Set , it may yield,Example 8.1 (Calculating the population principal components) p.350 , , , , , , , , ,Ex. Euclid space , , ,Step 1, .
11、,Step 2,Step 3,Principal Components Obtained from Standardized Vari-ables,In matrix notation,where the diagonal standard deviation matrix is defined in (2-35). Clearly, and Result 8.4.,The total (standardized variables) population variance is simply , the sum of the diagonal elements of the matrix .
12、 Using (8.7) with in place of , we find that the proportion of total variance explained by the kth principal component of is Proportion of total population variance due to kth principal component,where the s are the eigenvalues of .Example 8.2 (Principal components obtained from covariance and corre
13、lation matrices are different)P.433,Standardization is not inconsequential.Variables should probably be standardized if they are meas-ured on scales with widely differing ranges or if the units of measurement are not commensurate.,8.3 SUMMARIZING SAMPLE VARIATION BY PRINCIPLE COMPONENTS,Example 8.3 (Summarizing sample variability wi-th two sample principle components)P.439,
