1、Dynamic Programming,Algorithm : Design & Analysis 17,In the last class,Shortest Path and Transitive Closure Washalls Algorithm for Transitive Closure All-Pair Shortest Paths Matrix for Transitive Closure Multiplying Bit Matrices - Kronrods Algorithm,Dynamic Programming,Recursion and Subproblem Graph
2、 Basic Idea of Dynamic Programming Least Cost of Matrix Multiplication Extracting Optimal Multiplication Order,Natural Recursion may be Expensive,0,0,0,0,0,1,1,1,1,1,1,1,1,2,2,2,2,2,4,3,3,3,6,5,4,Fibonacci: Fn= Fn-1+Fn-2,16,13,11,10,9,8,12,3,4,5,6,7,14,15,21,2,1,19,18,17,25,23,24,22,20,The Fn can be
3、 computed in linear time easily, but the cost for recursion may be expotential.,For your reference,The number of activation frames are 2Fn+1-1,1, 1, 2, 3, 5, 8, 13, 21, 35, .,Subproblem Graph,For any known recursive algorithm A for a specific problem, a subproblem graph is defined as: vertex: the in
4、stance of the problem directed edge: the subproblem graph contains a directed edge IJ if and only if when A invoked on I, it makes a recursive call directly on instance J. Portion A(P) of the subproblem graph for Fibonacci function: here is fib(6),Properties of Subproblem Graph,If A always terminate
5、s, the subproblem graph for A is a DAG. For each path in the tree of activation frames of a particular call of A, A(P), there is a corresponding path in the subproblem graph of A connecting vertex P and a base-case vertex. A top-level recursive computation traverse the entire subproblem graph in som
6、e memoryless style. The subproblem graph can be viewed as a dependency graph of subtasks to be solved.,Basic Idea for Dynamic Programming,Computing each subproblem only once Find a reverse topological order for the subproblem graph In most cases, the order can be determined by particular knowledge o
7、f the problem. General method based on DFS is available Scheduling the subproblems according to the reverse topological order Record the subproblem solutions for later use,To backtracking, record the result intothe dictionary (Q, turned black),Dynamic Programming Version DP(A) of a Recursive Algorit
8、hm A,Q,a instance, Q, to be called on,Q is finished (black), only “checking” the edge, retrieve the result from the dictionary,Case 2: Black Q,Note: for DAG, no gray vertex will be met,DP(fib): an Example,fibDPwrap(n)Dict soln=create(n); return fibDP(soln,n),fibDP(soln,k)int fib, f1, f2;if (k2) fib=
9、k;elseif (member(soln, k-1)=false)f1=fibDP(soln, k-1);elsef1= retrieve(soln, k-1);if (member(soln, k-2)=false)f2=fibDP(soln, k-2);elsef2= retrieve(soln, k-2);fib=f1+f2;store(soln, k, fib); return fib,This is the wrapper, which will contain processing existing in original recursive algorithm wrapper.
10、,Matrix Multiplication Order Problem,The task: Find the product: A1A2An-1An Ai is 2-dimentional array of different legal size The issues: Matrix multiplication is associative Different computing order results in great difference in the number of operations The problem: Which is the best computing or
11、der,Cost of Matrix Multiplication,Let C = ApqBqr,There are q multiplication,C has pr elements as ci,j,So, pqr multiplications altogether,Looking for a Greedy Solution,Greedy algorithms are usually simple. Strategy 1: “cheapest multiplication first” Success: A301(A140 A4010)A1025 Fail: (A41A1100)A100
12、5 Strategy 2: “largest dimension first” How about the situation that the largest is not unique, such as A110A1010A102,mmTry1(dim, len, seq)if (len3) bestCost=0elsebestCost=;for (i=1; ilen-1; i+)c=cost of multiplication at position seqi;newSeq=seq with ith element deleted;newDim=updated dim;b=mmTry1(
13、newDim, len-1, newSeq);bestCost=min(bestCost, b+c);return bestCost,Optimum Order by Recursion,Recursion on index sequence: 0, 1, 2, , n with the kth matrix is Ak (i0) of the size dk-1dk , and the kth(kn) multiplication is AkAk+1.,T(n)=(n-1)T(n-1)+n, in (n-1)!),Constructing the Subproblem Graph,The k
14、ey issue is: how can a subproblem be denoted using a concise identifier? For mmTry1, the difficult originates from the varied intervals in each newSeq. If we look at the last (contrast to the first) multiplication, the the two (not one) resulted subproblems are both contiguous subsequences, which ca
15、n be uniquely determined by the pair: ,Best Order by Recursion: Improved,mmTry2(dim, low, high)if (high-low=1) bestCost=0elsebestCost=;for (k=low+1; khigh-1; k+)a=mmTry2(dim, low, k);b=mmTry2(dim, k, high);c=cost of multiplication at position k;bestCost=min(bestCost, a+b+c);return bestCost,with dime
16、nsions: dimlow, dimk, and dimhigh,Still in (2n)!,Only one matrix,Best Order by Dynamic Programming,DFS can traverse the subproblem graph in time O(n3) Atmost n2/2 vertices, as , 0ijn. Atmost 2n edges leaving a vertexmmTry2DP(dim, low, high, cost)for (k=low+1; khigh-1; k+)if (member(low,k)=false) a=m
17、mTry2(dim, low, k);else a=retrieve(cost, low, k);if (member(k,high)=false) b=mmTry2(dim, k, high);else b=retrieve(cost, k, high);store(cost, low, high, bestCost);return bestCost,Corresponding to the recursive procedure of DFS,Simplification Using Ordering Feature,For any subproblems: (low1, high1) d
18、epending on (low2, high2) if and only if low2low1, and high2high1 Computing subproblems according the dependency order,matrixOrder(n, cost, last)for (low=n-1; low1; low-)for (high=low+1; highn; high+)return cost0n,Compute solution of subproblem (low, high) and store it in costlowhigh and lastlowhigh
19、,Matrix Multiplication Order: Algorithm,Input: array dim =(d0, d1, , dn), the dimension of the matrices. Output: array multOrder, of which the ith entry is the index of the ith multiplication in an optimum sequence.,float matrixOrder(int dim, int n, int multOrder)for (low=n-1; low1; low-)for (high=l
20、ow+1; highn; high+)if (high-low=1) else bestcost=;for (k=low+1; khigh-1; k+)a=costlowk;b=costkhighc=multCost(dimlow, dimk, dimhigh);if (a+b+cbestCost)bestCost=a+b+c; bestLast=k;costlowhigh=bestCost;lastlowhigh=bestLast;extrctOrderWrap(n, last, multOrder)return cost0n,Using the stored results,An Exam
21、ple,Input: d0=30, d1=1, d2=40, d3=10, d4=25,cost as finished,First entry filled,Last entry filled,Note: costij is the least cost of Ai+1Ai+2Aj. For each selected k, retrieving:least cost of Ai+1Ak.least cost of Ak+1Aj. and computing:cost of the last multiplication,Array last and the Arithmatic-Expre
22、ssion Tree,Example input: d0=30, d1=1, d2=40, d3=10, d4=25,last as finished,A1,A2,A3,A4,(0,1),(1,2),(3,4),(1,3),(1,4),(0,4),(2,3),As root,Computing in post order,The core procedure is extractOrder, which fills the multiOrder array for subproblem (low,high), using informations in last array.extractOr
23、der(low, high, last, multOrder)int k;if (high-low1)k=lastlowhigh;extractOrder(low, k, last, multOrder);extractOrder(k, high, last, multOrder);multOrdermultOrderNext=k;multOrderNext+;,Extracting the Optimal Order,Just a post-order traversal,Calling Map,float matrixOrder (int dim, int n, int multOrder
24、 )int last; float cost; int low, high, for (low=n-1; low1; low-)for (high=low+1; highn; high+)for (k=low+1; khigh-1; k+)extractOrderWrap(n, last, multOrder)return cost0n;,Output, passed to extractOrder,extractOrder(low, high, last, multOrder) low, call recursively on (low,k) and (k,high) where k=las
25、tlowhigh,Analysis of matrixOrder,Main body: 3 layer of loops Time: the innermost processing costs constant, which is executed (n3) times. Space: extra space for cost and last, both in (n2) Order extracting There are 2n-1 nodes in the arithmatic-expression tree. For each node, extractOrder is called once. Since non-recursive cost for extractOrder is constant, so, the complexity of extractOrder is in (n),Home Assignment,10.1 10.4 10.6 10.7,
