1、Chapter 12,Clustering, Distance Methods, and Ordination,12.1 INTRODUCTION,Cluster analysis is a more primitive technique in that no assumptions are made concerning the nu-mber of groups or the group structure.,12.2 SIMILARITY MEASURES,Most efforts to produce a rather simple group stru-cture from a c
2、omplex data set require a measure of “closeness,” or “similarity.” There is often a great deal of subjectivity involved in the choice of a si-milarity measure. Important considerations inclu-de the nature of the variables (discrete, continuous, binary), scales of measurement (nominal, ordinal, inter
3、val, ratio), and subject matter knowledge.,Distances and Similarity Coefficients for Pairs of ItemsRecall that the Euclidean (straight-line) distance between two p-dimensional observations (items) and is, from (1.12),The statistical distance between the same two obs-ervations is of the form see (1.2
4、3) Another distance measures is the Minkowski metricFor , measures the “city-block” distan-ce between two points in p dimensions. For ,becomes the Euclidean distance. In general, varying m changes the weight given to larger and smaller differences. Additionally, the measures of “distance” or “dissim
5、ilarity” are given by the Canberra metric and Czekanowski coefficient. We have Canberra metric:,Czekanowski coefficient:,Example 12.1 (Clustering by shading a distance matrix)P.671,Example 12.2 (Calculating the values of a similar-ity coefficient)P.675,Similarities and Association Measures for Pairs
6、 of VariablesSimilarity measures for variables often take the form of sample correlation coefficients.,Concluding Comments on SimilarityIt appears that most practitioners use distances see (12.1) through (12.5) or the coefficients in Table 12.2 to cluster items and correlations to cluster variables.
7、,Example 12.3 (Measuring the similarities of 11 la-nguages)P.677The meanings of words change with the course of history. However, the meaning of the numbers 1,2,3, represents one conspicuous exception. Thus, a first comparison of languages might be based on the numerals alone.,12.3 HIERARCHICAL CLUS
8、TERING METHODS,Hierarchical clustering techniques proceed by eit-her a series of successive mergers or a series of successive divisions.,Single LinkageGroups are formed from the individual entities by merging nearest neighbors, where the term nearest neighbor connotes the smallest distance or larges
9、t similarity. Initially, we must find the smallest distance in and merge the corresponding objects, say, and , to get the cluster .,Example 12.4 (Clustering using single linkage)P.681 To illustrate the single linkage algorithm, we consider the hypothetical distance between pairs of five objects as f
10、ollows:,Treating each object as a cluster, we commence clustering by merging the two closest items. Since,objects 5 and 3 are merged to form the cluster (35). To implement the next level of clustering, we ne-ed the distances between the cluster (35) and the remaining objects, 1, 2, and 4. The neares
11、t neighb-or distances are,Deleting the rows and columns of D correspondi-ng to objects 3 and 5, and adding a row and colu-mn for the cluster (35), we obtain the new distan-ce matrix,The smallest distance between pairs of clusters is now , and we merge cluster (1) with cl-uster (35) to get the next c
12、luster, (135). Calculat-ing we find that the distance matrix for the next level of clustering is,The minimum nearest neighbor distance between pairs of clusters is , and we merge objects 4 and 2 to get the cluster (24). At this point we have two distinct clusters, (135) and (24). Their nearest neigh
13、bor distance is,The final distance matrix becomesConsequently, clusters (135) and (24) are merged to form a single cluster of all five objects, (12345), when the nearest neighbor distance reaches 6.,Example 12.5 (Single linkage clustering of 11 la-nguages)P.683,Complete LinkageComplete linkage clust
14、ering proceeds in much the same manner as single linkage clusterings, with one important exception: At each stage, the dista-nce (similarity) between clusters is determined by the distance (similarity) between the two elements, one from each cluster, that are most distant.,Example 12.6 (Clustering u
15、sing complete linkage)P.685,Example 12.7 (Complete linkage clustering of 11 languages)P.686,Example 12.8 (Clustering variables using compl-ete linkage)P.687,Average LinkageAverage linkage treats the distance between two clusters as the average distance between all pairs of items where one member of
16、a pair belongs to each cluster.where is the distance between object i in the cl-,uster and object k in the cluster , and and are the number of items in clusters and , respectively.,Example 12.9 (Average linkage clustering of 11 languages)P.689,Example 12.10 (Average linkage clustering of pu-blic uti
17、lities)P.690,Wards Hierarchical Clustering MethodWard 29 considered hierarchical clustering proc-edures based on minimizing the loss of informat-ion from joining two groups. This method is usu-ally implemented with loss of information taken to be an increase in an error sum of squares criterion, . F
18、irst, for a given cluster k, let be the sum of the squared deviation of every item in the clus-ter from the clustr mean (centroid). If there are cu-,rrently K clusters, define as the sum of the or the . At each step in analysis, the union of every possible pair of clusters is considered, and the two
19、 clusters wh-ose combination results in the smallest increase in (minimum loss of information) are joined.where is the multivariate measurement associ-,ated with the jth item and is the mean of all the items.,Example 12.11 (Clustering pure malt scotch whis-kles)P.691,Final Comments-Hierarchical Proc
20、edures,12.4 NONHIERARCHICAL CLUSTERING METHODS,Nonhierarchical clustering techniques are design-ed to group items, rather than variables, into a col-lection of K clusters. The number of clusters, K, may either be specified in advance or determined as part of the clustering procedure.,K-means MethodM
21、acQueen 22 suggests the term K-means for de-scribing an algorithm of his that assigns each item to the cluster having the nearest centroid (mean). In his simplest version, the process is composed of these three steps: 1. Partition the items into K initial clusters. 2. Proceed through the list of ite
22、ms, assigning an item to the cluster whose centroid (mean) is neare-,st. Recalculate the centroid for the cluster receiv-ing the new item and for the cluster losing the it-em. 3. Repeat Step 2 until no more reassignments take place.,Example 12.12 (Clustering using the K-means method)P.695,Example 12
23、.13 (K-means clustering of public util-ities)P.696,Final Comments-Nonhierarchical Procedures,12.5 MULTIDIMENSIONAL SCALING,The Basic AlgorithmFor items, there are similarities (distances) between pairs of different items. These similarities constitute the basic data. (In cases wh-ere the similaritie
24、s cannot be easily quantified as, for example, the similarity between two colors, the rank orders of the similarities are the basic data.),Example 12.14 (Multidimensional scaling of U.S. cities)P.702,Example 12.15 (Multidimensional scaling of public utilities)P.704,Example 12.16 (Multidimensional sc
25、aling of univ-ersities)P.706,12.6 CORRESPONDENCE ANALYSIS,Developed by the French, correspondence analy-sis is a graphical procedure for representing assoc-iations in a stable of frequencies or counts.,Example 12.17 (Correspondence analysis of arch-aeological data)P.709,Algebraic Development of Correspondence Anal-ysis,Final Comments,12.7 BIPLOTS FOR VIEWING SAMPLING UNITS AND VARIABLES,12.8 PROCRUSTES ANALYSIS: A METHOD FOR COMPARING CONFIGURAYIONS,
