加载中…
个人资料
felonwan
felonwan
  • 博客等级:
  • 博客积分:0
  • 博客访问:619,608
  • 关注人气:117
  • 获赠金笔:0支
  • 赠出金笔:0支
  • 荣誉徽章:
相关博文
推荐博文
谁看过这篇博文
加载中…
正文 字体大小:

[原创]基于QEMU的多线程虚拟机——COREMU编译

(2014-10-25 16:03:42)
标签:

qemu

多线程

coremu

虚拟机

分类: 程序Linux电脑学习
原创文章,转载声明作者felonwan及出处http://blog.sina.com.cn/s/blog_59cf67260102v28m.html

QEMU开虚拟化速度可以接近原生机,但是只能同种CPU,非同种CPU不行。

在性能比较弱的但是核数众多的CPU上模拟比较强的CPU,比如现在的手机ARM的U模拟x86,虽然4到8核是标配了,但是你也只能干瞪眼,CPU的模拟最多只是单核在运行。

或者即使在性能比较强的CPU上模拟较弱的CPU,比如X86电脑上模拟ARM的CPU,也用不上多核。像安卓官方的SDK里自带的模拟器(基于QEMU),兼容性好,但是运行速度还是很慢,如果能用到X86的多核速度上也能有所提升。

话说回来,多核要在虚拟的系统能很好的利用多核的情况下,才对系统性能有大的提升。最好的情况也就是线性增长的关系吧,最多开2核2倍、开8核最多8倍性能提升。

COREMU是一个基于QEMU的多线程虚拟机,虚拟CPU时会开多个线程,这样就可以利用现在CPU的多核优势。不过目前只支持64位X86系统。

主要参考:[1]http://people.cs.nctu.edu.tw/~chenwj/dokuwiki/doku.php?id=coremu
和coremu官方网站:[2]http://sourceforge.net/p/coremu/home/Home/

先编译libtopology,按[1]编译,$INSTALL变量为自己设定的libtopology目录,没有问题。
然后开始编译COREMU,先是configure
$./configure --target=x86_64 --prefix=/home/node1/Soft/coremu ../qemu
***** [QEMU-git for COREMU build] *****
target:    x86_64-softmmu
...
QEMU_CFLAGS       -m64 -I. -I$(SRC_PATH) -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wendif-labels -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -I/home/node1/Soft/libtopology/include -I/home/node1/Soft/coremu-0.1.2/coremu/incl -g  -fstack-protector-all -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits
LDFLAGS           -Wl,--warn-common -m64 -g -L/home/node1/Soft/libtopology/lib -ltopology -lncurses  
make              make
install           install
host CPU          x86_64
host big endian   no
target list       x86_64-softmmu
...
SDL support       yes
curses support    yes
...
VNC TLS support   yes
VNC SASL support  yes
VNC JPEG support  yes
VNC PNG support   yes
VNC thread        no
xen support       no
...
KVM support       no
...

configure通过,不支持KVM和xen,所以是没有虚拟化支持的。然后就make呗,make出错。
$make
....
  LINK    bin/utils/monitor
obj/utils/monitor.o: In function `print_processor':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:82: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:82: undefined reference to `wattr_on'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:83: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:83: undefined reference to `waddch'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:84: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:84: undefined reference to `wattr_off'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:85: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:92: undefined reference to `waddch'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:93: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:95: undefined reference to `waddch'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:95: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:99: undefined reference to `wattr_on'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:100: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:100: undefined reference to `waddch'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:101: undefined reference to `stdscr'
obj/utils/monitor.o: In function `print_process':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:106: undefined reference to `printw'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:108: undefined reference to `stdscr'
obj/utils/monitor.o: In function `refresh_list':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:150: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:150: undefined reference to `wmove'
obj/utils/monitor.o: In function `init':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:166: undefined reference to `initscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:167: undefined reference to `has_colors'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:168: undefined reference to `start_color'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:169: undefined reference to `init_pair'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:170: undefined reference to `init_pair'
obj/utils/monitor.o: In function `print_processor':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:101: undefined reference to `wattr_off'
obj/utils/monitor.o: In function `print_process':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:108: undefined reference to `waddch'
obj/utils/monitor.o: In function `init':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:171: undefined reference to `init_pair'
obj/utils/monitor.o: In function `main':
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:193: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:193: undefined reference to `werase'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:194: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:194: undefined reference to `wmove'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:196: undefined reference to `printw'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:198: undefined reference to `stdscr'
/home/node1/Soft/coremu-0.1.2/coremu/utils/monitor.c:198: undefined reference to `wrefresh'
collect2: ld 返回 1
make: *** [bin/utils/monitor] 错误 1

网上搜了,也安装了libncurses5的库,还是不行。估计可能编译系统的版本不对,我用的是Ubuntu12.04,而这个项目从10年开始的,或许用的10.04。

放了两天,再用VirtualBox装了个10.04:
sudo apt-get build-dep qemu
然后编译,果然编译成功!

接下来就按官方的启动指南去启动:
$ sudo ./x86_64-linux.sh /home/fu/Downloads/coremu/bin/qemu-system-x86_64 ../../../debian-x86_64.img 12
一执行发现内存不够,最少需要4G?于是关机开了6G,终于能启动,结果出现内核panic了:
Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option.
那就修改内核启动参数吧。想着之前自己的一篇博文修改镜像吧。结果还出错,偏移应该是设对了,但还是提示没有指明文件系统类型。
转念一想,看看它的脚本启动里到底干了啥吧。结果发现很简单,还是-smp参数指定核数,然后bios用修改过的特定bios。于是干脆自己直接调qemu启动,开图形界面,启动时修改grub,加上noapic参数后启动起来了:
sudo ./qemu-system-x86_64 -smp 12 -net none -hda ../../debian-x86_64.img -m 4096 -serial mon:/dev/tty -k en-us -bios seabios.bin
main/core.c(line 80), coremu_init:
TIMERRTSIG: 34
COREMU_TIMER_SIGNAL: 35
COREMU_TIMER_ALARM: 36
COREMU_SIGNAL: 37
COREMU_AIO_SIG: 38

----------------- Dump toplogy[6d] info -----------------
System(5981MB)
  Socket#0
    L3(6144KB)
      L2(512KB)
        L1(64KB)
          Core#0
            P#0
      L2(512KB)
        L1(64KB)
          Core#1
            P#1
      L2(512KB)
        L1(64KB)
          Core#2
            P#2
      L2(512KB)
        L1(64KB)
          Core#3
            P#3
      L2(512KB)
        L1(64KB)
          Core#4
            P#4
      L2(512KB)
        L1(64KB)
          Core#5
            P#5
----------------------------------------------------------
core [1] binds to 1
core [2] binds to 2
core [0] binds to 0
core [3] binds to 3
core [4] binds to 4
core [5] binds to 5
core [6] binds to 0
core [7] binds to 1
core [8] binds to 2
core [9] binds to 3
core [10] binds to 4
core [11] binds to 5
Linux version 2.6.33.1 (coremu@R900) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) #6 SMP PREEMPT Mon Jul 12 10:38:14 EDT 2010
Command line: root=/dev/hda1 ro console=tty0 console=ttyS0,57600n8 noapic
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
 BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dfffd000 (usable)
 BIOS-e820: 00000000dfffd000 - 00000000e0000000 (reserved)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
NX (Execute Disable) protection: active
DMI 2.4 present.
No AGP bridge found
last_pfn = 0x120000 max_arch_pfn = 0x400000000
x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106
last_pfn = 0xdfffd max_arch_pfn = 0x400000000
found SMP MP-table at [ffff8800000f7f20] f7f20
init_memory_mapping: 0000000000000000-00000000dfffd000
init_memory_mapping: 0000000100000000-0000000120000000
(10 early reservations) ==> bootmem [0000000000 - 0120000000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0001000000 - 0001962578]    TEXT DATA BSS ==> [0001000000 - 0001962578]
  #2 [0001963000 - 0001963049]              BRK ==> [0001963000 - 0001963049]
  #3 [000009f400 - 00000f7f20]    BIOS reserved ==> [000009f400 - 00000f7f20]
  #4 [00000f7f20 - 00000f7f30]     MP-table mpf ==> [00000f7f20 - 00000f7f30]
  #5 [00000f80ec - 0000100000]    BIOS reserved ==> [00000f80ec - 0000100000]
  #6 [00000f7f30 - 00000f80ec]     MP-table mpc ==> [00000f7f30 - 00000f80ec]
  #7 [0000001000 - 0000003000]       TRAMPOLINE ==> [0000001000 - 0000003000]
  #8 [0000008000 - 000000c000]          PGTABLE ==> [0000008000 - 000000c000]
  #9 [000000c000 - 000000d000]          PGTABLE ==> [000000c000 - 000000d000]
Zone PFN ranges:
  DMA      0x00000000 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x00120000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
    0: 0x00000000 -> 0x0000009f
    0: 0x00000100 -> 0x000dfffd
    0: 0x00100000 -> 0x00120000
Intel MultiProcessor Specification v1.4
MPTABLE: OEM ID: QEMUCPU
MPTABLE: Product ID: 0.1         
MPTABLE: APIC at: 0xFEE00000
Processor #0 (Bootup-CPU)
Processor #1
Processor #2
Processor #3
Processor #4
Processor #5
Processor #6
Processor #7
Processor #8
Processor #9
Processor #10
Processor #11
I/O APIC #12 Version 17 at 0xFEC00000.
Processors: 12
SMP: Allowing 12 CPUs, 0 hotplug CPUs
Allocating PCI resources starting at e0000000 (gap: e0000000:1ffc0000)
setup_percpu: NR_CPUS:255 nr_cpumask_bits:255 nr_cpu_ids:12 nr_node_ids:1
PERCPU: Embedded 23 pages/cpu @ffff880028200000 s71896 r0 d22312 u131072
pcpu-alloc: s71896 r0 d22312 u131072 alloc=1*2097152
pcpu-alloc: [0] 00 01 02 03 04 05 06 07 08 09 10 11 -- -- -- --
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1032245
Kernel command line: root=/dev/hda1 ro console=tty0 console=ttyS0,57600n8 noapic
PID hash table entries: 4096 (order: 3, 32768 bytes)
Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Checking aperture...
No AGP bridge found
Memory: 4052556k/4718592k available (2278k kernel code, 524688k absent, 140364k reserved, 2324k data, 368k init)
Hierarchical RCU implementation.
NR_IRQS:4352
Console: colour VGA+ 80x25
console [tty0] enabled
console [ttyS0] enabled
Fast TSC calibration failed
TSC: Unable to calibrate against PIT
TSC: No reference (HPET/PMTIMER) available
Marking TSC unstable due to could not calculate TSC khz
Calibrating delay loop...
12.28 BogoMIPS (lpj=24576)
Mount-cache hash table entries: 256
Performance Events: AMD PMU driver.
... version:                0
... bit width:              48
... generic registers:      4
... value mask:             0000ffffffffffff
... max period:             00007fffffffffff
... fixed-purpose events:   0
... event mask:             000000000000000f
Freeing SMP alternatives: 18k freed
Setting APIC routing to physical flat
CPU0: AMD QEMU Virtual CPU version 0.13.50 stepping 03
APIC timer disabled due to verification failure
Booting Node   0, Processors  #1
CPU1: Stuck ??
 #2
CPU2: Stuck ??
 #3
CPU3: Stuck ??
 #4
CPU4: Stuck ??
 #5
CPU5: Stuck ??
 #6
CPU6: Stuck ??
 #7
CPU7: Stuck ??
 #8
CPU8: Stuck ??
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 6
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 1
Spurious LAPIC timer interrupt on cpu 5
 #9
Spurious LAPIC timer interrupt on cpu 6
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 1
CPU9: Stuck ??
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 6
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
 #10
Spurious LAPIC timer interrupt on cpu 6
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 6
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 7
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 6
CPU10: Stuck ??
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 6
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 9
 #11 Ok.
Spurious LAPIC timer interrupt on cpu 6
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 5
Spurious LAPIC timer interrupt on cpu 9
Spurious LAPIC timer interrupt on cpu 10
Spurious LAPIC timer interrupt on cpu 8
Spurious LAPIC timer interrupt on cpu 6
CPU11: Stuck ??
Brought up 8 CPUs
Total of 8 processors activated (12.28 BogoMIPS).

指定了12个核,启动起来8个???实际CPU占用率介于4核到5核间,应该是8个线程占用4核。
好吧,为什么先不管了,算是搞定。

下一步还是试一下x64cpu模拟arm,这要能多核才是真的有用,毕竟同种cpu,虚拟化就可以而且技术很成熟。
这下有经验了,编译起来就很快了:
libtopoloty已经编好
在coremu-0.1.2下的coremu和qemu都make clean
在coremu目录下./configure --tartget=arm --prefix=安装路径 ../qemu
然后make,出错:
...
/home/hpcnl/Documents/coremu/coremu-0.1.2/qemu/cm-features/watch.c:432:22: error: ‘R_EAX’ undeclared (first use in this function)
/home/hpcnl/Documents/coremu/coremu-0.1.2/qemu/cm-features/watch.c:432:22: note: each undeclared identifier is reported only once for each function it appears in
/home/hpcnl/Documents/coremu/coremu-0.1.2/qemu/cm-features/watch.c:433:21: error: ‘R_EDI’ undeclared (first use in this function)
/home/hpcnl/Documents/coremu/coremu-0.1.2/qemu/cm-features/watch.c:434:24: error: ‘R_ESI’ undeclared (first use in this function)
/home/hpcnl/Documents/coremu/coremu-0.1.2/qemu/cm-features/watch.c:435:22: error: ‘R_EDX’ undeclared (first use in this function)
...
搜吧,然后参考http://comments.gmane.org/gmane.comp.emulators.qemu.coremu.devel/47
在qemu/Makefile.target中改两处:
第16行去掉$(SRC_PATH)/cm-features
第100行加注释 #libobj-y += watch.o watch-util.o instrument.o trigger.o memtrace.o logbuffer.o

重新configure,重新make,果然成功。
下载测试arm内核和initrd,运行后进入一个busybox环境。测试时,把虚拟机只分2个核,然后用qemu-system-arm时特意分3个核。
$ sudo ./qemu-system-arm -k en_us -M realview-pbx-a9 -nographic -serial mon:/dev/tty -smp 3 -kernel ../../arm-zImage-2.6.28 -initrd ../../arm-initrd.gz -m 1024
[sudo] password for fu:
main/core.c(line 80), coremu_init:
TIMERRTSIG:        34
COREMU_TIMER_SIGNAL:    35
COREMU_TIMER_ALARM:    36
COREMU_SIGNAL:        37
COREMU_AIO_SIG:        38

----------------- Dump toplogy[2d] info -----------------
System(2009MB)
  Socket#0
    L3(6144KB)
      L2(512KB)
        L1(64KB)
          Core#0
            P#0
      L2(512KB)
        L1(64KB)
          Core#1
            P#1
----------------------------------------------------------
core [0] binds to 0
core [1] binds to 1
core [2] binds to 0
...

可以看到模拟的2号核和0号核都绑定到了实际cpu的0号核上。
测试了一下,2个任务时2个核已经占满。虚拟的arm系统里两个进程还显示只各占约40%。3核齐开当然是都接近33%了。


运行图:
[原创]基于QEMU的多线程虚拟机——COREMU编译


0

阅读 评论 收藏 转载 喜欢 打印举报/Report
  • 评论加载中,请稍候...
发评论

    发评论

    以上网友发言只代表其个人观点,不代表新浪网的观点或立场。

      

    新浪BLOG意见反馈留言板 电话:4000520066 提示音后按1键(按当地市话标准计费) 欢迎批评指正

    新浪简介 | About Sina | 广告服务 | 联系我们 | 招聘信息 | 网站律师 | SINA English | 会员注册 | 产品答疑

    新浪公司 版权所有