1、1,可逆马氏链,Topics,Time-Reversal of Markov ChainsReversibilityTruncating a Reversible Markov ChainBurkes TheoremQueues in Tandem,Time-Reversed Markov Chains,假定Xn: n=0,1, 为遍历的马氏链, 转移概率为 Pi,j , 唯一的平稳分布为 (j 0). 假定过程从开始, 即Xn:n=,n, -1, 0,1, , 则系统在时刻n的状态概率 PrXn=j= 平稳分布j .任意 0，定义 Yn=X-n, 过程Yn是原马氏链的时间逆转过程. 可以证明
2、Yn也是马氏链，转移概率为而且和Xn 有相同的平稳分布 j 通过逆向链可看出正向链的某些性质；,Time-Reversed Markov Chains,Proof of Proposition 1:,Reversibility,马氏链Xn称为可逆的如果正向链和逆向链有相等的转移概率:Pi,j=Pi,j*, 等价于Xn 满足DBE.定理1. 如果能找到一组正数 j, 其和为1,且满足: iPi,j=jPj,i, 则该马氏链是可逆的且以j为平稳分布.,Reversibility Discrete-Time Chains,Example: Discrete-time birth-death proc
3、esses are reversible, since they satisfy the DBE,重要定理,Theorem 1:对转移概率为 Pij.的不可约马氏链，如果存在:一组转移概率 Qij, 满足 j Qij=1, i 0, 和一组整数 j, 其和为1, 并且下式成立则:Qij 为其逆向链的转移概率j 同时既是正向链也是逆向链的平稳分布Remark: 上述定理常用来计算平稳分布：根据马氏链的结构性质猜出两组数Qij,和j,验证是否满足(1)：如果满足，则该马氏链是可逆链且以j为平稳分布，而Qij 为逆链的转移概率.,Continuous-Time Markov Chains,设X(t)
4、: - 0) 为它的唯一的平稳分布:仍假定过程从-开始，则在有穷时间t链处于平稳态: PX(t)=j=pj令Y(t)=X(-t)，则以下命题成立:,Reversed Continuous-Time Markov Chains,Proposition 2:Y(t) 也是连续时间马氏链，转移率为:Y(t) 和正向链有相同的平稳分布: pj Remark: 逆向链离开i 的转移率等于正向链离开i 的转移率：这是GBE的另一表达形式:逆向链的“出”正向链的“入”,Reversibility Continuous-Time Chains,马氏链称为可逆的如果： 或等价的:此即DBETheorem 3:
5、如果有一组正数pj,其和为 1 且满足:则:pj 是唯一的平稳分布该马氏链是可逆的,Birth-Death Process,转移率为满足DBEProof: GBE with S =0,1,n give:M/M/1, M/M/c, M/M/是可逆马氏链,重要的定理,Theorem 4: 对有转移率qij. 的不可约马氏链，如果存在:一组转移率 , 满足 ji ij=ji qij, i 0, 和一组正数pj, 其和为 1, 使得以下方程成立:则:ij 是逆向链的转移率,且pj 是正向和逆向链的相同的平稳分布上述定理用来解马氏链：guess两组数ij和pj，验证上述条件,状态转移率图是树结构的马氏链
6、,Theorem 5:状态转移率图有树结构的马氏链是可逆的.,Burkes 定理,假设N(t)为一生灭过程,有平稳分布pjN(t) 的向上跳跃点为到达点.N(t) 的向下跳跃点为离开点.N(t) 包括了系统的到达和离开过程,Burkes Theorem,如j=, for all,则到达为Poisson. 这时称此过程为 (, j)-过程M/M/1, M/M/c, M/M/为(, j)-过程Poisson arrivals LAA: For any time t, future arrivals are independent of X(s): st(, j)-process at steady
7、 state is reversible: forward and reversed chains are stochastically identicalArrival processes of the forward and reversed chains are stochastically identicalArrival process of the reversed chain is Poisson with rate The arrival epochs of the reversed chain are the departure epochs of the forward c
8、hainDeparture process of the forward chain is Poisson with rate ,Burkes Theorem,Reversed chain: arrivals after time t are independent of the chain history up to time t (LAA)Forward chain: departures prior to time t and future of the chain X(s): st are independent,Burkes Theorem,Theorem 10: Consider
9、an M/M/1, M/M/c, or M/M/ system with arrival rate . Suppose that the system starts at steady-state. Then:The departure process is Poisson with rate At each time t, the number of customers in the system is independent of the departure times prior to tFundamental result for study of networks of M/M/*
10、queues, where output process from one queue is the input process of another,Burkes定理说两件事情:处于平稳态的M/M/排队系统的离开过程是Poisson，而且参数为.处于平稳态的M/M/排队系统内客户数N(t)独立于t以前,即(0,t)时间内的离开过程应用Burkes定理可以推出级联队列的平稳客户数分布有乘积形式: P(n1,n2)=P(n1)P(n2).,Burkes定理,Figure 3.31 (a) Forward system number of arrivals, number of departure
11、s, and occupancy during 0, T.,Figure 3.31 (b) Reversed system number of arrivals, number of departures, and occupancy during 0,T.,根据Burkes定理,不能从客户接连不断的离开推断系统内有大量的客户,因为这两者之间没相关性.(反直觉, counterintuitive),Single-Server Queues in Tandem（级联）,Customers arrive at queue 1 according to Poisson process with ra
12、te . Service times exponential with mean 1/i. Assume service times of a customer in the two queues are independent. Assume i=/i1What is the joint stationary distribution of N1 and N2 number of customers in each queue?Result: in steady state the queues are independent and,Poisson,Station 1,Station 2,
13、Single-Server Queues in Tandem,Q1 is a M/M/1 queue. At steady state its departure process is Poisson with rate . Thus Q2 is also M/M/1.Marginal stationary distributions:To complete the proof: establish independence at steady stateQ1 at steady state: at time t, N1(t) is independent of departures prio
14、r to t, which are arrivals at Q2 up to t. Thus N1(t) and N2(t) independent:Letting t, the joint stationary distribution,Poisson,Station 1,Station 2,如果排队系统组成的网络是有向无环图, 每个系统的客户服务时间是指数分布, 外部到达是Poisson, 则每个内部排队系统的到达过程是Poisson.,Queues in Tandem,Theorem 11: Network consisting of K single-server queues in
15、tandem. Service times at queue i exponential with rate i, independent of service times at any queue ji. Arrivals at the first queue are Poisson with rate . The stationary distribution of the network is:At steady state the queues are independent; the distribution of queue i is that of an isolated M/M
16、/1 queue with arrival and service rates and i,Queues in Tandem: State-Dependent Service Rates,Theorem 12: Network consisting of K queues in tandem. Service times at queue i exponential with rate i(ni) when there are ni customers in the queue independent of service times at any queue ji. Arrivals at
17、the first queue are Poisson with rate . The stationary distribution of the network is:where pi(ni) is the stationary distribution of queue i in isolation with Poisson arrivals with rate Examples: ./M/c and ./M/ queues If queue i is ./M/, then:,Multidimensional Markov Chains,Theorem 8: X1(t), X2(t):
18、independent Markov chainsXi(t): reversibleX(t), with X(t)=(X1(t), X2(t): vector-valued stochastic processX(t) is a Markov chainX(t) is reversibleMultidimensional Chains:Queueing system with two classes of customers, each having its own stochastic properties track the number of customers from each cl
19、assStudy the “joint” evolution of two queueing systems track the number of customers in each system,Example: Two Independent M/M/1 Queues,Two independent M/M/1 queues. The arrival and service rates at queue i are i and i respectively. Assume i= i/i1. (N1(t), N2(t) is a Markov chain. Probability of n
20、1 customers at queue 1, and n2 at queue 2, at steady-state“Product-form” distributionGeneralizes for any number K of independent queues, M/M/1, M/M/c, or M/M/. If pi(ni) is the stationary distribution of queue i:,Stationary distribution:Detailed Balance Equations:Verify that the Markov chain is reve
21、rsible Kolmogorov criterion,Example: Two Independent M/M/1 Queues,Example: Two Independent M/M/1 Queues,Truncation of a Reversible Markov Chain,Theorem 9: X(t) reversible Markov process with state space S, and stationary distribution pj: jS. Truncated to a set ES, such that the resulting chain Y(t)
22、is irreducible. Then, Y(t) is reversible and has stationary distribution: Remark: This is the conditional probability that, in steady-state, the original process is at state j, given that it is somewhere in EProof: Verify that:,Two examples:,例1 M/M/m/m是M/M/的截断,S=0,1,m, G=1+ + m/m!, p(n)=(n/n!)/G, =(
23、/)例2 M/M/1/K是M/M/1的截断(习题3.21) M/M/1有平稳分布n(1- ),截断链的状态集为S=0,1,K, G=n (1- ) =1-K+1,截断链的平稳分布为: p(n)=n(1- )/G= n(1- )/(1-K+1) .,Example: Two Queues with Joint Buffer,The two independent M/M/1 queues of the previous example share a common buffer of size B arrival that finds B customers waiting is blocked
24、State space restricted to E=(n1, n2)| n1+n2=BDistribution of truncated chain:Normalizing:Theorem specifies joint distribution up to the normalization constantCalculation of normalization constant is often tedious,State diagram for B =2,多维马氏链-在电路交换网中的应用,Example：3.12,考虑具有m个独立电路，每个电路都具有相同的传输能力系统中存在两类会话
25、，到达率分别是1 和2的泊松分布当一个会话到达系统时，若发现所有电路都处于繁忙状态，这个会话将被封锁，继而被丢弃。相反，系统将会话分配给任意一个可用的电路。两种会话的持续时间是平均值分别为1/1和1/2的指数分布。求系统在稳态时的封锁概率,例题3.12 （Contd.）,两类sessions 共享一条有m个电路的传输线,类型1 sessions的到达率为1,长度为1/1,类型2 sessions的到达率为2,长度为1/2,我们关心blocking 概率(电路交换系统QoS):需计算传输线上有n1个类型1 session 和 n2个类型2 session的概率, P(n1,n2), n10, n
26、20, n1+n2m 进一步计算blocking 概率=P(n1,n2), 求和在集合 (n1,n2)|n1+n2=m当1= 2 时，系统可以用M/M/m/m队列来模拟，其中到达率为1 +2当 1和 2不相等时需建立多维马氏链.,例题3.12 独立的M/M/的截断,截断的状态集S=(n1,n2),0n1,n2m, n1+n2m.用定理9可以得到截断链的平稳分布和阻塞概率 （式3.39，3.40）,例题3.13,类型1 session 可使用所有m条电路.类型2 session 最多允许使用k条电路,即使session到达时m条电路中还有闲电路. 相当于为类型1 sesson预留m-k条电路(reservation).Blocking 条件: 类型1：m-kn1m & n2=m-n1 类型2： 0n1m & n2=mink, m-n1,例题3.13:状态集为: 0n1m & 0 n2k, n1+n2m.,例题3.13也是两个独立的M/M/的截断平稳分布:计算阻塞概率的公式（见185页），即分别计算,
