Tuesday, September 2, 2008

Nested svm virtualization for kvm

Yesterday I found a nice surprise in my inbox - a post, by Alex Graf, adding support for virtualizing AMD's SVM instruction set when running KVM on AMD SVM.

What does this mean? up until now, when kvm virtualizes a processor, the guest sees a cpu that is similar to the host processor, but does not have virtualization extensions. This means that you cannot run a hypervisor that needs these virtualization extensions within a guest (you can still run hypervisors that do not rely on these extensions, such as VMware, but with lower performance). With the new patches, the virtualized cpu does include the virtualization extensions; this means the guest can run a hypervisor, including kvm, and have its own guests.

There are two uses that immediately spring to mind: debugging hypervisors and embedded hypervisors. Obviously having svm enabled in a guest means that one can debug a hypervisor in a guest, which is a lot easier that debugging on bare metal. The other use is to have a hypervisor running in the firmware at all times; up until now this meant you couldn't run another hypervisor on such a machine. With nested virtualization, you can.

The reason the post surprised me was the relative simplicity in which nested virtualization was implemented: less than a thousand lines of code. This is due to the clever design of the svm instruction set, and the ingenuity of the implementers (Alex Graf and Jörg Rödel) in exploiting the instruction set and meshing the implementation so well with the existing kvm code.


Anonymous said...

It should be a bit more tricky to achieve nested vmx virtualization for Intel's CPU. It is because you can access the VMCB of AMD's CPU by normal read/write, but you cannot do that for Intel's. It means that you should switch VMCS when vmexitting to KVM due to the VMX instructions.

Whatever, does the nexted virtualization really make sense?

Grant McWilliams said...

Another use is to have multiple virtualized networks inside one machine. Each network would have a different configuration of hosts. I've been waiting for this.

Arno-Can Uestuensoez said...

The application of VM-Stacks or Nested VMs with multiple levels is described wit
hin the Manual of the UnifiedSessionsManager.org. This is currently implemented
by multiple instaces of QEMU as a n+1-Emulator, but is related to the software a
rchitecture similar to nested hypervisors.

The document is first forwarded to sourceforge as embedded online documentation
at 02/2008 and available as a pdf document since 08/2008.

Arno-Can Uestuensoez said...

I have forgotten to mention, that KVM and VirtualBox are currently
on the way to be integrated within the management of stacked VMs
by the UnifiedSessionsManager too.
Thus the following hypervisors are supported in stacked
Environments with nesting by QEMU:

Now: QEMU+VMware(WS,Server,Player)+Xen
Next: VirtualBox, KVM, XenServer, VMware iESX

Eventually OpenVZ too.

The trick is given by the correct handling of implicit startup and
shutdown in stacked environments, which is solved by multiple
recursive approaches. The neatless handling of physical and
virtual entities is an additional requirement in actual

The reason I mention this here is, the implementation is already
tested with 3 Stack-Levels and is currently available for
download under GPL3 licence on sourceforge.