1、 *The National Natural Science Foundation of China under Grant No. 60970108, 60970107 (SE1 S ). Received 2011-07, Accepted 2011-09. ISSN 1673-9418 CODEN JKYTA8 E-mail: Journal of Frontiers of Computer Science and Technology http:/www.ceaj.org 1673-9418/2011/05(12)-1114-07 Tel: +86-10-51616056 DOI:
2、10.3778/j.issn.1673-9418.2011.12.006 E I/O KVM EWYZE*丁圣阁1+, 马汝辉2, 梁阿磊1, 管海兵21. 上海交通大学 软件学院, 上海 200001 2. 上海交通大学 计算机科学与工程系, 上海 200001 Optimization for Inter-VMs Communication on KVM with Para-Virtualized I/O Model*DING Shengge1+, MA Ruhui2, LIANG Alei1, GUAN Haibing21. School of Software, Shanghai Ji
3、aotong University, Shanghai 200001, China 2. Department of Computer Science inter-VMs communication; shared memory K1提出了一种半虚拟化网络模型来优化虚拟机域间通信的性能, 通过共享内存建立通信通道来打破虚拟机之前的隔离屏障, 减少在数据传输过程中的拷贝次数。基于内核虚拟机(kernel-based virtual machine, KVM)半虚拟化框架编程接口的实现方法可以简化设备I/O的模拟, 减少特权指令模拟所需的根-非根模式 E I/O KVM EWYZE 1115 的切
4、换, 提供高效的I/O模拟, 并向客户提供标准的网络接口, 使通信通道透明。利用共享内存的通信方法, 其性能接近进程间通信。对系统原型进行测试, 结果表明域间通信的吞吐量和响应延时都得到了较好的优化。 1oM半虚拟化; 域间通信; 共享内存 DS M A ms | TP393 1 E/ $W aE#M ! , T“d iB v , V74 q1b$Bl+ ) = , Z P , + ) = Y ES j (virtual machine monitor hypervisor), Z T“d 1- L= b E/ K5C , |Bt$ s ) b= B , Vh q , v qH1q , 7 E
5、/ 4 ! E#5 ? , # T“d %M ? , Pt TMZL H T“d , VE qY 5 , 9e N 6) kVb Btq %ZF E/ , IBM DS8000 Albeit bFq V$bM E , 4 Zb“ qO , FqWY1 N , s Ti%“dO , v J? 3M E , |Y ?b IBM System S , E/ T“dB1Fq ?4 $ , P System S , G !9 , y aV ? a V Las s T ) , tz)N B“d ? , FqW b E/ WY ?X s 7?1b K ZWY1Xen ES j 2bXen E/ V , ?Z
6、q Ei4 EZb Xen Ps z / LC ! I/O Eb ! , - YV: Z T. , Dom 0 q bZEM1 E! , ?VC , ? WYH1b XenSocket3 XenLoop4Z TLC Z =iWYZE , Unix“d4 Unix Domain Socket Z =iIPC(inter processes communication)bFido5 6B Z =iWYE , LC . V , Jb Xen M1 , = E (kernel- based virtual machine, KVM) B ) = v , q E/ (Intel-VT AMD-V) 7
7、E , Y P QEMU(B EZ T LC E )6 bW ! I/O E , G “d , yNK rWY LCb !9i LC B KVM E ? ZWYb EI gLC ! 4 Z =i Y , hWY H Jb Z , -z4S g P i bK“d s Y kb = # / 2 c KVM E E 3 c ZE!9 LC 4 c “dk , is k 5 c9 , i4/BTb 1116 Journal of Frontiers of Computer Science and Technology 9 S 2011, 5(12) 2 KVM E KVM E Linux0“d , _
8、 Linux9F ES j ? , 4 E/ Zb P KVM V7yi E Zb7 , t Z Y Linux uY , YV E“dbKVM Ba ) Linux E %Z , C$ j Linux?ZV 7b 2.1 KVM E O “ - KVM ? E E/ bLinux E %Z1 cq E/ Z (Intel-VT AMD-V) X86b E Z T“d VV 7b TF , + 7|$ , i MM T) b KVM PFQEMU E 5ai%a9LqFq , Z4 I/O b =iYV Z W LC , 0:V Z:V (extended page table, EPT)b
9、! E/ , QEMU Z4 E ! g , Z E g 5 (virtual network interface card, VNIC)bm 1 B q B E b E g 5TBFq , P Fig.1 The virtual network of KVM m 1 KVM E O Linux 4 TAP/TUN ! (Linux 4BF E q g )# q B LC IP b r 5 , YV q B? TAP/TUN ! , QEMU JM Z = bW , =xp) 8, ES j E Y rb? V# lVMQ , E ! I/O + 7(?JW , V Z TK = TM Tb
10、2.2 E I/O : Virtio KVM 9 ? LC Xen B“E , rz I/O ? , “ -BYr I/O EX$ KVM b Virtio9B a E b m 2 VirtioOb Virtio z$ !9 Y z# T (m 3)b Virtio z Virtio!M , !) - h p , i| a h pq , EMb Fig.2 The framework of Virtio m 2 Virtio E O Xen E9L Xen_bus Xen_storev , Virtio KVM E )+ , P virtio_ring O Linux4I g LC ! T ,
11、 +a bWa ab - virtqueue LCb virtqueue 4u a | T , 3hn E I/O KVM EWYZE 1117 b Virtio E , ! P virtqueue u l?# bW ?b virtio_ring_ i , c Fa Vu Pu sb “ - KVM E , M E9La Ev !a E !XYV Virtio I gLCbV “ - kT A , s 3 4 6 , iA1 Jb Fig.3 The structure of Virtio driver m 3 Virtio zm 2.3 ZWY E ZWY 3B ZWY10by E/ 1 E
12、W , WYY? P g Yb L= t ZW Z a =ia 5 , 4 WY ? V ?b 3 !9 LC KVM O , Z =i T E bWBs 5 b Z T“d :V , 3V Z E L= , nZE0:V EPTbW Y H , ? I/O h p Z|K QEMU T I/O E , Z E !bT , QEMU WY V Z =i IPCLC , 71 q B TAP/TUN !b Virtio I g V LCr - z , O m 4 UbW YH , $ J TUN/TAP E 5 , V4BF T Z =i g Z P , rrWYb Fig.4 Shared-m
13、emory-based inter-VMs communication channel m 4 Z =iWYY Y T“d 1 &q , |WY P Z =i u Z , KVM O/KZE F E !T . Yb Z, M z$ !9i 4 gb 3.1 ! E Virtio I g LC virtio_inter V U E ! , ? virtio_device, | !|+ virtio_bus b - I/O P virtqueue sY3 l? ebWY y , ? | h pF 55 , l V | h pb !B v5s Z =i u , y !Y u b Z7 , PWYY
14、, E ! virtio_inter , P g u , vl FbYV virtio_ring h p. , _ virtqu- eue F |u b? F 1118 Journal of Frontiers of Computer Science and Technology 9 S 2011, 5(12) u Y l, u VYVBQ? u) , “ Vh YQ , 4 ?b llY , V55 |u# b 3.2 Z =iY Z =i IPC r , av Y H PbT , QEMU V7y H P mmap()ZEs B v Z =i , |Bq POSIX(portable op
15、erating system in-terface of Unix) Z =i bWb 6BE WYy H , Z =i u b -z) r q , ZW1WY H , 1v Z =i u , hb V !s “S Z | Z =i u , i y H , N “S ZO M“ T , | v ? 7gbyN -BZE , iYV| ZE LC Z Z =ib - I/O h p) , E !u | s z Z =iuby - P scatter-gather VV U , V_ Z =i , ?| YV Z =i. “S Z , 5| |p =i b “ -%E , PBQ J| 5b Z
16、=ib 6B %E , YV Z V , | Z “S Z , |u ? “S ZbEh JQ , T9 7gb 4 ? k c“d k , is kTbq Linux 2.6.38 = , QEMU 0.15bq Intel Quad CoreTMI5 2.6 GHz ) , 1 MBB)i , 2 GB =ibVs Y H Z 1 virtio_inter Y O/WY ?b NetperfT k , $ k ?Sb L_s Y H k , k (TCP , *3vl ! 16 KBb ! 1 k (1) Z b ZYV g_1 &? b (2) WYYb virtio_inter Z =
17、iWYYb (3) KVM E b P virtio_net! ZW Yb (4) q Bb T“d q B , B TAP/TUN !Yb 4.1 _s hvl f / , _. s vl kT m 5 Ub Z =is 512 MBb Virtio WY HVCi A4 6 q B / , M KVM O/ “ ! E 7g ? Z kMB QEMU E !b B M1 , Z =iWYYh xp) JV , ? v4 6b Fig.5 Test results of unidirectional throughput m 5 _s kT E I/O KVM EWYZE 1119 4.2
18、Y H m 6 V U hvl f / , TCP H kTb q B Z zY HTb Z =iY H ?1 f /WY H ?1zb “hvl9F , J 9F -d T M P H9vb Fig.6 Test results of end-to-end latency m 6 Y H kT 5 9 4 B KVM E Z =iWYZEb Virtio E I/O LC B*_ i E !#Z -z , P Z =iZ TW . b kTV ZE ?4 6WY ?b/ T , | ZMYb References: 1 Meergen M F, Uhlig V, Krieger O, et
19、al. Virtualization for high-performance computingJ. ACM SIGOPS Operating Systems Review, 2006, 40(2): 811. 2 Barham P, Dragovic B, Fraser K, et al. Xen and the art of virtualizationC/Proceedings of the 19th ACM Sympo-sium on Operating Systems Principles (SOSP 03). New York, NY, USA: ACM, 2003: 16417
20、7. 3 Zhang Xiaolan, McIntosh S, Rohatgi P, et al. XenSocket: a high-throughput interdomain transport for virtual ma-chinesC/Proceedings of the ACM/IFIP/USENIX 2007 International Conference on Middleware (Middleware 07). New York, NY, USA: Springer-Verlag New York, Inc, 2007: 184203. 4 Wang Jian, Wri
21、ght K-L, Gopalan K. XenLoop: a trans-parent high performance inter-VM network loopbackC/ Proceedings of the 17th International Symposium on High Performance Distributed Computing (HPDC 08), Boston, Massachussets, June 2008. New York, NY, USA: ACM, 2008: 109118. 5 Burtse A, Srinivasan K, Radhakrishna
22、n P, et al. Fido: fast inter-virtual-machine communication for enterprise appli-ancesC/Proceedings of the 2009 USENIX Annual Tech-nical Conference (USENIX 09), San Diego, June 2009. Berkeley, CA, USA: USENIX Association, 2009: 25. 6 Bellard F. QEMU, a fast and portable dynamic transla-torC/Proceedin
23、gs of the 2005 USENIX Annual Tech-nical Conference (USENIX 05), Marriott Anaheimm, April 2005. Berkeley, CA, USA: USENIX Association, 2005: 41. 7 Kivity A, Kamay Y, Laor D, et al. KVM: the Linux virtual machine monitorC/Proceedings of the Linux Sympo-sium, Ottawa, Ontario, 2007: 225230. 8 Li Shengzh
24、ao, Hao Qinfen, Xiao Limin, et al. Optimizing network virtualization in kernel-based virtual machineC/ Proceedings of the 1st IEEE International Conference on Information Science and Engineering (ICISE 09). Wash-ington, DC, USA: IEEE Computer Society, 2009: 282 285. 9 Russell R. Virtio: towards a de
25、-facto standard for virtual I/O devicesJ. ACM SIGOPS Operating Systems Review, 2008, 42(5): 95103. 10 Diakhate F, Perache M, Namyst R, et al. Efficient shared memory message passing for inter-VM communicationsC/ Proceedings of Euro-Par 2008 WorkshopsParallel Proc-essing. Berlin, Heidelberg: Springer
26、-Verlag, 2009: 5362. 1120 Journal of Frontiers of Computer Science and Technology 9 S 2011, 5(12) DING Shengge was born in 1987. He is a master candidate at Shanghai Jiaotong University. His research interests include system virtualization and network virtualization, etc. (1987i), 3 , C , ZYv V 3 ,
27、1 5“d) E , Eb MA Ruhui was born in 1984. He is a Ph.D. candidate at Shanghai Jiaotong University. His research inter-ests include virtual machines, computer architecture and compiler, etc. (1984i), 3 , ? , ZYvp V 3 , 1 5 E , 9 8“O , Ir b LIANG Alei was born in 1969. He is an associate professor at S
28、hanghai Jiaotong University. His research interests include parallel computing via swarm intelligence and virtualization computing with dynamic binary-translation, etc. (1969i), 3 , ZYv q , 1 5 8 ?i9 , =Jr E/ b GUAN Haibing was born in 1968. He is a professor and doctoral supervisor at Shanghai Jiaotong Univer-sity, and the member of CCF. His research interests include computer architecture, compiler, virtualization and hardware/software co-design, etc. 5Za (
