NVMM
Created:08-09-2018
Updated:27-10-2019

NetBSD Virtual Machine Monitor


NVMM is a hypervisor that provides hardware-accelerated virtualization support for NetBSD. It is made of an ~MI frontend, to which MD backends can be plugged. A virtualization API is shipped via libnvmm, that allows to easily create and manage virtual machines via NVMM. Two additional components are shipped as demonstrators, toyvirt and smallkern: the former is a toy virtualizer, that executes in a VM the 64bit ELF binary given as argument, the latter is an example of such binary.

Resources


NetBSD.org: From Zero to NVMM.

The kernel code: browse.
The libnvmm API: browse.
The demonstrators: download.
The man page: open.
The regression tests: browse.

Qemu+NVMM


Make sure you have the latest NVMM headers installed. If you don't:     # cp /usr/src/sys/dev/nvmm/nvmm.h         /usr/include/dev/nvmm/
    # cp /usr/src/sys/dev/nvmm/nvmm_ioctl.h   /usr/include/dev/nvmm/
    # cp /usr/src/sys/dev/nvmm/x86/nvmm_x86.h /usr/include/dev/nvmm/x86/
Make sure you have the latest version of libnvmm installed. If you don't:     # cd /usr/src/lib/libnvmm/
    # make
    # make install
Load the NVMM kernel driver:     # cd /usr/src/sys/modules/nvmm/
    # make
    # modload ./nvmm.kmod
You may want to run the NVMM tests to confirm that your setup is correct:     # cd /usr/src/tests/lib/libnvmm/
    # make
    # ./h_io_assist
    # ./h_mem_assist
Install the qemu-nvmm package from pkgsrc-wip. Note that this will conflict with any previous Qemu package that you installed. Configure the permissions:     # chown root:nvmm /usr/pkg/bin/qemu-system-x86_64
    # chmod g+s /usr/pkg/bin/qemu-system-x86_64
Once this is done, you can finally launch Qemu with NVMM acceleration, by adding "-accel nvmm" to your usual Qemu command line. For example:     $ qemu-system-x86_64 -cdrom NetBSD-8.1-amd64.iso -m 256M -accel nvmm
Tested guest OSes:
Guest OS Status Notes
NetBSD 32bit Works
NetBSD 64bit Works
OpenBSD 32bit Works
OpenBSD 64bit Works
FreeBSD 32bit Works
FreeBSD 64bit Works
Win95 32bit Doesn't work
Win98 32bit Doesn't work
WinXP 32bit Works
Win7 32bit Works
Win8 32bit Works
Win8 64bit Works On Intel CPUs only for now
Win10 32bit Works
Win10 64bit Works On Intel CPUs only for now
Fedora 64bit Works Pass "no_timer_check" in grub
ArchLinux 64bit Works Pass "no_timer_check" in grub
Ubuntu 64bit Works

Notes:

Linux performs timer checks at boot time which fail under hypervisors. To work around this, on certain distributions, you must pass "no_timer_check" in grub.


NetBSD on Qemu+NVMM Fedora on Qemu+NVMM
WindowsXP on Qemu+NVMM Windows10 on Qemu+NVMM
NetBSD 8 (top left), Fedora 29 (top right), Windows XP (bottom left), Windows 10 (bottom right),
all running via Qemu+NVMM on NetBSD.

Technical details


NVMM can support up to 128 virtual machines, each having a maximum of 256 VCPUs and 128GB of RAM. It works with both x86 AMD CPUs and x86 Intel CPUs.

NVMM has unique design properties, not currently found in other hypervisor solutions. A detailed analysis is available on this blog post.

NVMM Design
Design overview