1、Junction Trees: Motivation,Standard algorithms (e.g., variable elimination) are inefficient if the undirected graph underlying the Bayes Net contains cycles. We can avoid cycles if we turn highly-interconnected subsets of the nodes into “supernodes.”,如果贝叶斯网络底层的无向图中包含环，标准算法（如标量消元）是低效的。 如果我们把节点的高度互联的子
2、集转变为“超级节点”，我们可以避开环的存在。,A Running Example for the Steps in Constructing a Junction Tree,Step 1: Make the Graph Moral,Step 2: Remove Directionality,Step 3: Triangulate the Graph,Step 3: Triangulate the Graph,Step 3: Triangulate the Graph,Is it Triangulated Yet?,Triangulation Checking,上述的最大势算法（mcs算法，Ma
3、ximum Cardinality Search），实际上也是求是否存在完美消除序列的方法，存在完美消除序列即为弦图，反之不是，,Is it Triangulated Yet?,Is it Triangulated Yet?,Is it Triangulated Yet?,Is it Triangulated Yet?,Is it Triangulated Yet?,Is it Triangulated Yet?,It is Not Triangulated,Fixing the Faulty Cycle,Continuing our Check.,Continuing our Check.,
4、Fixing this Problem,Continuing our Check.,The Following is Triangulated,Triangulation: Key Points,Previous algorithm is an efficient checker, but not necessarily best way to triangulate. In general, many triangulations may exist. The only efficient algorithms are heuristic. Jensen and Jensen (1994)
5、showed that any scheme for exact inference (belief updating given evidence) must perform triangulation (perhaps hidden as in Draper 1995).,Definitions,Complete graph or node set: all nodes are adjacent. Clique: maximal complete subgraph. Simplicial node: node whose set of neighbors is a complete nod
6、e set.,Step 4: Build Clique Graph,The Clique Graph,Junction Trees,A junction tree is a subgraph of the clique graph that (1) is a tree, (2) contains all the nodes of the clique graph, and (3) satisfies the junction tree property. Junction tree property: For each pair U, V of cliques with intersectio
7、n S, all cliques on the path between U and V contain S.应该是其他文献中所说的变量连通性,Clique Graph to Junction Tree,We can perform exact inference efficiently on a junction tree (although CPTs may be large). But can we always build a junction tree? If so, how?在联合树中，可以高效的进行精确推理（CPT：条件概率分布表） Let the weight of an ed
8、ge in the clique graph be the cardinality of the separator. Than any maximum weight spanning tree（最大生成树） is a junction tree (Jensen & Jensen 1994).,Step 5: Build the Junction Tree,Step 6: Choose a Root,Step 7: Populate Clique Nodes,For each distribution (CPT) in the original Bayes Net, put this dist
9、ribution into one of the clique nodes that contains all the variables referenced by the CPT. (At least one such node must exist because of the moralization step). For each clique node, take the product of the distributions (as in variable elimination).,Better Triangulation Algorithm Specifically for
10、 Bayes Nets, Based on Variable Elimination,Repeat until no nodes remain: If the graph has a simplicial node, eliminate it (consider it “processed” and remove it together with all its edges).去除单纯点 Otherwise, find the node whose elimination would give the smallest potential possible. Eliminate that no
11、de, and note the need for a “fill-in” edge between any two non-adjacent nodes in the resulting potential. Add the “fill-in” edges to the original graph.,Find Cliques while Triangulating (or in triangulated graph),While executing the previous algorithm: for each simplicial node, record that node with
12、 all its neighbors as a possible clique. (Then remove that node and its edges as before.) After recording all possible cliques, throw out any one that is a subset of another. The remaining sets are the cliques in the triangulated graph. O(n3), guaranteed correct only if graph is triangulated.,Choose
13、 Root, Assign CPTs,Junction Tree Inference Algorithm,Incorporate Evidence: For each evidence variable, go to one table that includes that variable. Set to 0 all entries in that table that disagree with the evidence. Upward Step: For each leaf in the junction tree, send a message to its parent. The m
14、essage is the marginal of its table, .,J.T. Inference (Continued),(Upward Step continued) summing out any variable not in the separator. When a parent receives a message from a child, it multiplies its table by the message table to obtain its new table. When a parent receives messages from all its c
15、hildren, it repeats the process (acts as a leaf). This process continues until the root receives messages from all its children.,J.T. Inference (Continued),Downward Step: (Roughly reverses the upward process, starting at the root.) For each child, the root sends a message to that child. More specifi
16、cally, the root divides its current table by the message received from that child, marginalizes the resulting table to the separator, and sends the result of this marginalization to the child. When a .,J.T. Inference (Continued),(Downward Step continued) child receives a message from its parent, mul
17、tiplying this message by the childs current table will yield the joint distribution over the childs variables (if the child does not already have it). The process repeats (the child acts as root) and continues until all leaves receive messages from their parents.,One Catch for Division,At times we m
18、ay find ourselves needing to divide by 0. We can verify that whenever this occurs, we are dividing 0 by 0. We simply adopt the convention that for this special case, the result will be 0 rather than undefined.接受约定，这种特殊情况下，结果将为0，而不是未定义。,Build Junction Tree for BN Below,Inference Example (assume no ev
19、idence): Going Up,Status After Upward Pass,Going Back Down,.194 .231 .260 .315,Status After Downward Pass,Answering Queries: Final Step,Having built the junction tree, we now can ask about any variable. We find the clique node containing that variable and sum out the other variables to obtain our an
20、swer. If given new evidence, we must repeat the Upward-Downward process. A junction tree can be thought of as storing the subjoints computed during elimination.,Significance of Junction Trees,“only well-understood, efficient, provably correct method for concurrently computing multiple queries (AI Ma
21、g99).” As a result, they are the most widely-used and well-known method of inference in Bayes Nets, although Junction trees soon may be overtaken by approximate inference using MCMC.,The Link Between Junction Trees and Variable Elimination,To eliminate a variable at any step, we combine all remainin
22、g distributions (tables) indexed on (involving) that variable. A node in the junction tree corresponds to the variables in one of the tables created during variable elimination (the other variables required to remove a variable). An arc in the junction tree shows the flow of data in the elimination
23、computation.,Junction Trees/Variable Elim.,We can use different orderings in variable elimination - affects efficiency. Each ordering corresponds to a junction tree. Just as some elimination orderings are more efficient than others, some junction trees are better than others. (Recall our mention of heuristics for triangulation.),Recall Variable Elimination Example,First Eliminated Variable A,Next Eliminated Variable B,Next Eliminated Variable C,Left us with P(D,E,F),Corresponding Moralized, Triangulated Graph.,
