1、A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,1,Transform and Conquer,This group of techniques solves a problem by a transformation to a simpler/more convenient instance of the same problem (
2、instance simplification) a different representation of the same instance (representation change) a different problem for which an algorithm is already available (problem reduction),A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Sad
3、dle River, NJ. All Rights Reserved.,2,Instance simplification - Presorting,Solve a problems instance by transforming it into another simpler/easier instance of the same problemPresorting Many problems involving lists are easier when list is sorted, e.g. searching computing the median (selection prob
4、lem) checking if all elements are distinct (element uniqueness) Also: Topological sorting helps solving some problems for dags. Presorting is used in many geometric algorithms.,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle
5、River, NJ. All Rights Reserved.,3,How fast can we sort ?,Efficiency of algorithms involving sorting depends on efficiency of sorting.Theorem (see Sec. 11.2): log2 n! n log2 n comparisons are necessary in the worst case to sort a list of size n by any comparison-based algorithm. Note: About nlog2 n c
6、omparisons are also sufficient to sort array of size n (by mergesort).,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,4,Searching with presorting,Problem: Search for a given K in A0n-1 Presort
7、ing-based algorithm:Stage 1 Sort the array by an efficient sorting algorithmStage 2 Apply binary search Efficiency: (nlog n) + O(log n) = (nlog n) Good or bad? Why do we have our dictionaries, telephone directories, etc. sorted?,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd e
8、d., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,5,Element Uniqueness with presorting,Presorting-based algorithmStage 1: sort by efficient sorting algorithm (e.g. mergesort)Stage 2: scan array to check pairs of adjacent elementsEfficiency: (nlog n) + O(n) = (nlog n
9、) Brute force algorithm Compare all pairs of elements Efficiency: O(n2) Another algorithm? Hashing,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,Instance simplification Gaussian Elimination,G
10、iven: A system of n linear equations in n unknowns with an arbitrary coefficient matrix. Transform to: An equivalent system of n linear equations in n unknowns with an upper triangular coefficient matrix. Solve the latter by substitutions starting with the last equation and moving up to the first on
11、e. a11x1 + a12x2 + + a1nxn = b1 a1,1x1+ a12x2 + + a1nxn = b1 a21x1 + a22x2 + + a2nxn = b2 a22x2 + + a2nxn = b2an1x1 + an2x2 + + annxn = bn annxn = bn,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reser
12、ved.,Gaussian Elimination (cont.),The transformation is accomplished by a sequence of elementary operations on the systems coefficient matrix (which dont change the systems solution): for i 1 to n-1 do replace each of the subsequent rows (i.e., rows i+1, , n) by a difference between that row and an
13、appropriate multiple of the i-th row to make the new coefficient in the i-th column of that row 0,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,Example of Gaussian Elimination,Solve 2x1 - 4x2
14、 + x3 = 6 3x1 - x2 + x3 = 11 x1 + x2 - x3 = -3 Gaussian elimination2 -4 1 6 2 -4 1 6 3 -1 1 11 row2 (3/2)*row1 0 5 -1/2 2 1 1 -1 -3 row3 (1/2)*row1 0 3 -3/2 -6 row3(3/5)*row2 2 -4 1 60 5 -1/2 20 0 -6/5 -36/5Backward substitutionx3 = (-36/5) / (-6/5) = 6x2 = (2+(1/2)*6) / 5 = 1x1 = (6 6 + 4*1)/2 = 2,
15、A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,Pseudocode and Efficiency of Gaussian Elimination,Stage 1: Reduction to an upper-triangular matrix for i 1 to n-1 dofor j i+1 to n do for k i to
16、n+1 do Aj, k Aj, k - Ai, k * Aj, i / Ai, i /improve! Stage 2: Back substitutions for j n downto 1 dot 0for k j +1 to n dot t + Aj, k * xk xj (Aj, n+1 - t) / Aj, j Efficiency: (n3) + (n2) = (n3),A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, I
17、nc. Upper Saddle River, NJ. All Rights Reserved.,10,Searching Problem,Problem: Given a (multi)set S of keys and a search key K, find an occurrence of K in S, if any Searching must be considered in the context of: file size (internal vs. external) dynamics of data (static vs. dynamic)Dictionary opera
18、tions (dynamic data): find (search) insert delete,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,11,Taxonomy of Searching Algorithms,List searching sequential search binary search interpolatio
19、n searchTree searching binary search tree binary balanced trees: AVL trees, red-black trees multiway balanced trees: 2-3 trees, 2-3-4 trees, B treesHashing open hashing (separate chaining) closed hashing (open addressing),A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch.
20、 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,12,Binary Search Tree,Arrange keys in a binary tree with the binary search tree property:,K,K,K,Example: 5, 3, 1, 10, 12, 7, 9,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Ed
21、ucation, Inc. Upper Saddle River, NJ. All Rights Reserved.,13,Dictionary Operations on Binary Search Trees,Searching straightforward Insertion search for key, insert at leaf where search terminated Deletion 3 cases: deleting key at a leaf deleting key at node with single child deleting key at node w
22、ith two childrenEfficiency depends of the trees height: log2 n h n-1, with height average (random files) be about 3log2 nThus all three operations haveworst case efficiency: (n) average case efficiency: (log n) Bonus: inorder traversal produces sorted list,A. Levitin “Introduction to the Design & An
23、alysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,14,Balanced Search Trees,Attractiveness of binary search tree is marred by the bad (linear) worst-case efficiency. Two ideas to overcome it are: to rebalance binary search tree when a new
24、insertion makes the tree “too unbalanced”AVL treesred-black trees to allow more than one key per node of a search tree2-3 trees2-3-4 treesB-trees,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
25、,15,Balanced trees: AVL trees,Definition An AVL tree is a binary search tree in which, for every node, the difference between the heights of its left and right subtrees, called the balance factor, is at most 1 (with the height of an empty tree defined as -1),Tree (a) is an AVL tree; tree (b) is not
26、an AVL tree,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,16,Rotations,If a key insertion violates the balance requirement at some node, the subtree rooted at that node is transformed via one
27、 of the four rotations. (The rotation is always performed for a subtree rooted at an “unbalanced” node closest to the new leaf.),Single R-rotation,Double LR-rotation,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ.
28、All Rights Reserved.,17,General case: Single R-rotation,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,18,General case: Double LR-rotation,A. Levitin “Introduction to the Design & Analysis of
29、Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,19,AVL tree construction - an example,Construct an AVL tree for the list 5, 6, 8, 3, 2, 4, 7,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education,
30、Inc. Upper Saddle River, NJ. All Rights Reserved.,20,AVL tree construction - an example (cont.),A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,21,Analysis of AVL trees,h 1.4404 log2 (n + 2) -
31、1.3277 average height: 1.01 log2n + 0.1 for large n (found empirically) Search and insertion are O(log n) Deletion is more complicated but is also O(log n) Disadvantages: frequent rotations complexity A similar idea: red-black trees (height of subtrees is allowed to differ by up to a factor of 2),A.
32、 Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,22,Multiway Search Trees,Definition A multiway search tree is a search tree that allows more than one key in the same node of the tree.Definition A
33、 node of a search tree is called an n-node if it contains n-1 ordered keys (which divide the entire key range into n intervals pointed to by the nodes n links to its children): Note: Every node in a classical binary search tree is a 2-node,k1 k2 kn-1, k1,k1, k2 ), kn-1,A. Levitin “Introduction to th
34、e Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,23,2-3 Tree,Definition A 2-3 tree is a search tree thatmay have 2-nodes and 3-nodesheight-balanced (all leaves are on the same level)A 2-3 tree is constructed by successive in
35、sertions of keys given, with a new key always inserted into a leaf of the tree. If the leaf is a 3-node, its split into two with the middle key promoted to the parent.,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ
36、. All Rights Reserved.,24,2-3 tree construction an example,Construct a 2-3 tree the list 9, 5, 8, 3, 2, 4, 7,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,25,Analysis of 2-3 trees,log3 (n + 1
37、) - 1 h log2 (n + 1) - 1Search, insertion, and deletion are in (log n) The idea of 2-3 tree can be generalized by allowing more keys per node 2-3-4 trees B-trees,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All
38、Rights Reserved.,26,Heaps and Heapsort,Definition A heap is a binary tree with keys at its nodes (one key per node) such that: It is essentially complete, i.e., all its levels are full except possibly the last level, where only some rightmost keys may be missingThe key at each node is keys at its ch
39、ildren,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,27,Illustration of the heaps definition,a heap,not a heap,not a heap,Note: Heaps elements are ordered top down (along any path down from i
40、ts root), but they are not ordered left to right,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,28,Some Important Properties of a Heap,Given n, there exists a unique binary tree with n nodes t
41、hatis essentially complete, with h = log2 n The root contains the largest keyThe subtree rooted at any node of a heap is also a heap A heap can be represented as an array,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River,
42、 NJ. All Rights Reserved.,29,Heaps Array Representation,Store heaps elements in an array (whose elements indexed, for convenience, 1 to n) in top-down left-to-right order Example:Left child of node j is at 2j Right child of node j is at 2j+1 Parent of node j is at j/2 Parental nodes are represented
43、in the first n/2 locations,9,1,5,3,4,2,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,30,Step 0: Initialize the structure with keys in the order givenStep 1: Starting with the last (rightmost)
44、 parental node, fix the heap rooted at it, if it doesnt satisfy the heap condition: keep exchanging it with its largest child until the heap condition holds Step 2: Repeat Step 1 for the preceding parental node,Heap Construction (bottom-up),A. Levitin “Introduction to the Design & Analysis of Algori
45、thms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,31,Example of Heap Construction,Construct a heap for the list 2, 9, 7, 6, 5, 8,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle Ri
46、ver, NJ. All Rights Reserved.,32,Pseudopodia of bottom-up heap construction,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,33,Stage 1: Construct a heap for a given list of n keysStage 2: Repea
47、t operation of root removal n-1 times: Exchange keys in the root and in the last (rightmost) leaf Decrease heap size by 1 If necessary, swap new root with larger child until the heap condition holds,Heapsort,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearso
48、n Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,34,Sort the list 2, 9, 7, 6, 5, 8 by heapsortStage 1 (heap construction) Stage 2 (root/max removal)1 9 7 6 5 8 9 6 8 2 5 72 9 8 6 5 7 7 6 8 2 5 | 92 9 8 6 5 7 8 6 7 2 5 | 99 2 8 6 5 7 5 6 7 2 | 8 99 6 8 2 5 7 7 6 5 2 | 8 92 6 5 | 7 8 96 2 5 | 7 8 95 2 | 6 7 8 95 2 | 6 7 8 92 | 5 6 7 8 9,Example of Sorting by Heapsort,A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 6 2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.,
