1、Recursion,Algorithm : Design & Analysis 3,In the last class,Asymptotic growth rate The Sets , and Complexity Class An Example: Searching an Ordered Array Improved Sequential Search Binary Search Binary Search Is Optimal,Recursion,Recursive Procedures Proving Correctness of Recursive Procedures Deriv
2、ing recurrence equations Solution of the Recurrence equations Guess and proving Recursion tree Master theorem Divide-and-conquer,Thinking Recursively,Towers of Hanoi How many moves are need to move all the disks to the third peg by moving only one at a time and never placing a disk on top of a small
3、er one.,Recursion: an Implementation View,Activation Frame Basic unit of storage for an individual procedure invocation at run time Providing a frame of reference in which the procedure executes for this invocation only Space is allocated in a region of storage called the frame stack Activation trac
4、e,Recursion Tree,int fib(int n) int f, f1, f2; if (n2) f=n; else f1=fib(n-1); f2=fib(n-2); f=f1+f2; return f;,fibn: 3f1:1f2:1f: 2,fibn: 0f1:f2:f: 0,fibn: 1f1:f2:f: 1,fibn: 2f1:1f2:0f: 1,fibn: 1f1:f2:f: 1,Activation Tree Activation frame creation is in preorder,The cost for the simplicity: inefficien
5、cy: The number of recursive calls for computing Fib(31): ?,2692537,Loop-free Complexity,In a computation without while or for loops, but possibly with recursive procedure calls, The time that any particular activation frame is on the top of the frame stack is O(L), where L is the number of lines in
6、the procedure that contain either a simple statement or a procedure call. (Note: L is a constant) The total computation time is (C), where C is the total number of procedure calls that occur during the computation. (Note: for an algorithm, maxLi is a constant.),2-Tree,2-Tree,Common Binary Tree,inter
7、nal nodes,external nodes no child any type,Both left and right children of these nodes are empty tree,External Path Length(EPL),The EPL of a 2-tree t is defined as follows: Base case 0 for a single external node Recursion t is non-leaf with sub-trees L and R, then the sum of: the external path lengt
8、h of L; the number of external node of L; the external path length of R; the number of external node of R; In fact, the external path length of t is the sum of the lengths of all the paths from the root of t to any external node in t.,Calculating the External Path Length,EplReturn calcEpl(TwoTree t)
9、 EplReturn ansL, ansR; EplReturn ans=new EplReturn(); 1. if (t is a leaf) 2. ans.epl=0; ans.extNum=1; 3. else 4. ansL=calcEpl(leftSubtree(t); 5. ansR=calcEpl(rightSubtree(t); 6. ans.epl=ansL.epl+ansR.epl+ansL.extNum +ansR.extNum; 7. ans.extNum=ansL.extNum+ansR.extNum+1 8. Return ans;,TwoTree is an A
10、DT defined for 2-tree EplReturn is a organizer class with two field epl and extNum,Correctness of Procedure calcEpl,Let t be any 2-tree. Let epl and m be the values of the fields epl and extNum, respectively, as returned by calcEpl(t). Then: 1. epl is the external path length of t. 2. m is the numbe
11、r of external nodes in t. 3. epl mlg(m) (note: for 2-tree with internal n nodes, m=n+1),Proof on Procedure calcEpl,Induction on t, with the “sub-tree” partial order: Base case: t is a leaf. (line 2) Inductive hypothesis: the 3 statements hold for any proper subtree of t, say s. Inductive case: by in
12、d. hyp., eplL, eplR, mL, mR,are expected results for L and R(both are proper subtrees of t), so: Statement 1 is guranteed by line 6 Statement 2 is guranteed by line 7 (any external node is in either L or R) Statement 3: by ind.hyp. epl=eplL+eplR+m mLlg(mL)+mRlg(mR)+m, note f(x)+f(y)2f(x+y)/2) if f i
13、s convex, and xlgx is convex for x0, so, epl 2(mL+mR)/2)lg(mL+mR)/2)+m = m(lg(m)-1)+m =mlgm.,Binary Search: Revisited,int binarySearch(int E, int first, int last, int K) 1. if (lastfirst) 2. index=-1; 3. else 4. int mid=(first+last)/2 5. if (K=Emid) 6. index=mid; 7. else if (KEmid) 8. index=binarySe
14、arch(E, first, mid-1, K) 9. else if (KEmid) 10. index=binarySearch(E, mid+1, last, K) 11.return index;,Binary Search: Correctness,Correctness of binarySearch: For all n0, if binarySearch(E, first, last, K) is called, and if: the problem size is (last-first+1)=n Efirst, , Elast are in non-decreasing
15、order then: it returns -1 if K does not occur in E within the range first, , last it returns index if K=Eindex,Binary Search: Proof of Correctness,Induction on n, the problem size. Base case: n=0, line 2 executed, returning -1. For n0(that is: firstlast), assume that binarySearch(E,f,l,K) satisfies
16、the correctness on problems of size k(k=0,1,2,n-1), and k=l-f+1. mid=(first+last)/2 must be within the search range, so: if line 5 is true, line 6 executed. otherwise, the induction hypothesis applies for both recursive calls, on line 8 and 10. What we have to do is to verify that the preconditions
17、for the procedure hold, and return value of the recursive call satisfies the post-conditions of the problem.,Proving Binary Search (cont.),Case that line 5 is false, i.e. KEmid Since firstmidlast, we have (mid-1)-first+1n-1 Last-(mid+1)+1n-1 So, the inductive hypothesis applies for both recursive ca
18、lls on line 8 and 10 For both calls, only one actual parameter is changed and decreased, so, line 8 or line 10 will return the appropriate results,Recurrence Equation: Concept,A recurrence equation: defines a function over the natural number n in term of its own value at one or more integers smaller
19、 than n Example: Fibonacci numbers Fn=Fn-1+Fn-2 for n2 F0=0, F1=1 Recurrence equation is used to express the cost of recursive procedures.,Recurrence Equation for Sequential Search,seqSearchRec(E,m,num,K) 1. if (mnum) 2. ans=-1 3. else if (Em=K) 4. ans=m 5. else 6. ans = seqSearchRec(E,m+1,num,K) 7.
20、 return ans,“cost” is defined as the number of comparison of array element. For simple statement, only line 3 costs 1. The result:T(n)=(0+max(0, 1+max(0, T(n-1)+0 = T(n-1)+1,1,2,3,4,6,7,Recurrence Equation for Binary Search,int binarySearch(int E, int first, int last, int K) if (lastEmid) index=bina
21、rySearch(E, mid+1, last, K) return index;,the only nonrecursive cost,the two alternative recursions with the max input size of n/2 or (n-1)/2,The equation: T(n) = T(n/2)+1,Guess the Solutions,Example: T(n)=2T(n/2) +n Guess T(n)O(n)? T(n)cn, to be proved for c large enough T(n)O(n2)? T(n)cn2, to be p
22、roved for c large enough Or maybe, T(n)O(nlogn)? T(n)cnlogn, to be proved for c large enough,Try to prove T(n)cn:T(n)=2T(n/2)+n 2c(n/2)+n 2c(n/2)+n = (c+1)n, Fail!,However: T(n) = 2T(n/2)+n 2cn/2+n 2c(n-1)/2+n = cn+(n-c) cn,T(n) = 2T(n/2)+n 2(cn/2 lg (n/2)+n c lg (n/2)+n= cn lg n cn log 2 +n= cn lg
23、n cn + n cn log n for c1,Note: the proof is invalid for T(1)=1,Decrease for Larger,T(n)=T(n/2)+T(n/2)+1 Its difficult to prove T(n)cn directly. However, we can prove T(n)cn-b for some b0. T(n) = T(n/2)+T(n/2)+1 (cn/2-b)+ (cn/2-b)+1= cn-2b+1which is no larger than cn for any b1,Whats Wrong?,We have p
24、roved that T(n)=2T(n/2)+n has a tight lower bound of O(nlgn). But: if we assume that T(n/2)cn/2 T(n) = 2T(n/2)+n cn+n = O(n) Whats wrong?,Recursion Tree,T(size),nonrecursive cost,The recursion tree for T(n)=T(n/2)+T(n/2)+n,T(n),n,Recursion Tree Rules,Construction of a recursion tree work copy: use a
25、uxiliary variable root node expansion of a node: recursive parts: children nonrecursive parts: nonrecursive cost the node with base-case size,Recursion tree equation,For any subtree of the recursion tree, size field of root = nonrecursive costs of expanded nodes + size fields of incomplete nodes Exa
26、mple: divide-and-conquer: T(n) = bT(n/c) + f(n) After kth expansion:,Evaluation of a Recursion Tree,Computing the sum of the nonrecursive costs of all nodes. Level by level through the tree down. Knowledge of the maximum depth of the recursion tree, that is the depth at which the size parameter redu
27、ce to a base case.,Recursion Tree,T(n),n,Work copy: T(k)=T(k/2)+T(k/2)+k,At this level: T(n)=n+2(n/2)+4T(n/4)=2n+4T(n/4),n/2d,(size 1),T(n)=nlgn,Recursion Tree for,T(n)=3T(n/4)+(n2),cn2,T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),c(1/16)n)2,c(1/16)n)2,c( n)2,c( n)2,c( n)2,log4n,
28、cn2,Total: (n2),Note:,c(1/16)n)2,c(1/16)n)2,c(1/16)n)2,T(1),Verifying “Guess” by Recursive Tree,Inductive hypothesis,Recursion Tree for,T(n)=bT(n/c)+f(n),f(n),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),T(1),f(n/c2),f(n/c2),f(n/c2),f(n/c2),f(n/c2),f(n/c2),f(n/c2),f(n/c2),f(n/c2),f(n/
29、c),f(n/c),f(n/c),logcn,f(n),Note:,b,b,Total ?,Solving the Divide-and-Conquer,The recursion equation for divide-and-conquer, the general case:T(n)=bT(n/c)+f(n) Observations: Let base-cases occur at depth D(leaf), then n/cD=1, that is D=lg(n)/lg(c) Let the number of leaves of the tree be L, then L=bD,
30、 that is L=b(lg(n)/lg(c). By a little algebra: L=nE, where E=lg(b)/lg(c), called critical exponent.,Divide-and-Conquer: the Solution,The recursion tree has depth D=lg(n)/ lg(c), so there are about that many row-sums. The 0th row-sum is f(n), the nonrecursive cost of the root. The Dth row-sum is nE,
31、assuming base cases cost 1, or (nE) in any event. The solution of divide-and-conquer equation is the nonrecursive costs of all nodes in the tree, which is the sum of the row-sums.,Solution by Row-sums,Little Master Theorem Row-sums decide the solution of the equation for divide-and-conquer: Increasi
32、ng geometric series: T(n)(nE) Constant: T(n) (f(n) log n) Decreasing geometric series: T(n) (f(n),This can be generalized to get a result not using explicitly row-sums.,Master Theorem,Loosening the restrictions on f(n) Case 1: f(n)O(nE-), (0), then: T(n)(nE) Case 2: f(n)(nE), as all node depth contr
33、ibute about equally: T(n)(f(n)log(n) case 3: f(n)(nE+), (0), and f(n)(nE+), (), then: T(n)(f(n),The positive is critical, resulting gaps between cases as well,Using Master Theorem,Using Master Theorem,Looking at the Gap,T(n)=2T(n/2)+nlgn a=2, b=2, E=1, f(n)=nlgn We have f(n)=(nE), but no 0 satisfies
34、 f(n)=(nE+), since lgn grows slower that n for any small positive . So, case 3 doesnt apply. However, neither case 2 applies.,Proof of the Master Theorem: Case 3,(Note: in asymptotic analysis, f(n)(nE+) leads to f(n) is about (nE+), ignoring the coefficients.,Decreasing geometric series,Chip and Conquer,The equation: T(n) = bT(n-c)+f(n)As an example: Hanoi Tower, with b=2, c=1,T(n-c),f(n-c),T(n-c),f(n-c),T(n-c),f(n-c),T(n-2c),f(n-2c),b branches,totalling n/c levels,Solution of Chip and Conquer,grows exponentially in n,Home Assignment,pp.143- 3.3-3.4 3.6 3.9-3.10,
