KVM Virtualization: Refining the Virtual World with Red Hat Enterprise Linux 7 Beta

Ever since Red Hat Enterprise Linux added KVM Virtualization as a kernel-based hypervisor to run virtual machines (way back in Red Hat Enterprise Linux 5.4), the operating system took on a dual personality.

Red Hat Enterprise Linux became both a Virtualization host for high density virtual data centers / cloud service platforms, and a guest operating system running on third party hypervisors such as VMware vSphere and Microsoft Hyper-V. As the topic is sufficiently broad, I plan to split my discussion of virtualization into two posts.

Today’s post will discuss Red Hat Enterprise Linux 7 beta as a hypervisor using KVM Virtualization technology and it will highlight a few key enhancements that make Red Hat Enterprise Linux the operating system of choice for modern hybrid data centers. While the features that I will review are inherently those that I find to be the most exciting (note: I’m hoping you will find them to be exciting and useful as well), a complete list is available in the Red Hat Enterprise Linux 7 beta release notes.

Red Hat Enterprise Linux 7 beta with KVM Virtualization technology provides enterprise ready virtualization capabilities to our server, workstation and desktop customers. It serves as the platform for the open hybrid cloud, and establishes a supportive ecosystem where various other Red Hat products can add value, products such as: Red Hat Enterprise Linux Open Stack Platform, Red Hat Enterprise Virtualization, and Red Hat Storage.

So… what’s new? Drum roll please!

For starters, we have expanded live migration support – Red Hat Enterprise Linux 7 beta includes support for the live migration of a virtual machine from Red Hat Enterprise Linux 6.5 to Red Hat Enterprise Linux 7 beta. Previously, we only supported live migration between two hosts running Red Hat Enterprise Linux 6. This new functionality allows virtualized data centers and cloud providers to easily migrate their existing virtual machines running on Red Hat Enterprise Linux 6.5 to a brand new Red Hat Enterprise Linux 7 host, without virtual machine downtime. This also works hand in hand with in-place upgrade capabilities offered with Red Hat Enterprise Linux 7 beta.

In the world of virtualization security… security is always on our customers’ minds, whether it’s a credit card theft (has anyone out there been caught up in one of the many payment system hacks?) or the most recent Snapchat debacle. We believe security is a fundamental feature of the operating system and hypervisor. How many of you have depended on sVirt technology in our virtualization stack to protect your virtual machines from malicious users? We have introduced two new security features in Red Hat Enterprise Linux 7 beta to increase guest entropy for cryptography and additional security hardening to reduce the guest attack surface.

The first new (aforementioned) security feature, guest entropy, allows KVM Virtualization to meet new cryptographic security requirements from both the United States and United Kingdom. The para-virtualized random number generator (virtio-rng) driver allows the host to feed entropy to the guest. This allows cryptographic applications running on the guest to be more effective by alleviating entropy starvation in guests. The second new feature in uses a security hardening mechanism with libseccomp that allows applications to define interactions with the kernel using syscall filtering, to reduce the risk of a malicious guest exploiting a kernel vulnerability, thereby reducing the guest attack surface. These two security features add additional new and important layers of security to our KVM virtualization stack, above and beyond the existing SELinux mandatory access controls provided by sVirt, which protects against untrusted guests and misconfigured hosts.

In addition to security, we here at Red Hat know that our customers value application performance. With more and more systems, even at the low end, presenting NUMA topologies, there is a real need to address the performance irregularities that such systems present. Red Hat Enterprise Linux 7 beta has introduced a new kernel-based NUMA affinity mechanism for improved application performance allowing for greater efficiency over the traditional user-space based solution.

Automatic NUMA balancing matches significant resource consumers with available memory and CPU resources in order to reduce cross-node traffic. This results in better NUMA resource alignment for applications and virtual machines, thus improving performance by minimizing the cost of remote memory latencies. Users accrue performance benefits from automatic NUMA balancing without needing to explicitly place and bind process threads, including virtual CPU threads for virtual machines. This improves the out-of-box performance experience on NUMA systems in physical, virtual, and cloud, and positions Red Hat Enterprise Linux 7 beta as the open hybrid cloud operating platform.

Even network performance has been improved. Today’s high-end servers have lots of processors, and virtual machines running on such systems have a large number of vcpus. Red Hat Enterprise Linux 7 beta adds the multi-queue NIC feature in the KVM Virtualization virtio-net networking stack, which removes the single queue NIC bottleneck and allows the virtual NIC to process networking packets in parallel. This increases the throughput for both small virtual machines (2 – 4 vCPUs) and large virtual machines with higher virtual CPU counts, by allowing the virtual machines to transmit and receive packets through multiple queues in the virtio-net networking stack.

A note on scalability: The KVM scalability levels enable customers to more efficiently run large-scale workloads in a virtual guest than on hypervisors with much lower limits. The virtual guest size is 160 virtual CPUs, and the maximum supported memory in a KVM guest is 4 TB, doubling the previously supported virtual memory limit.

Finally, there is an intersection of GPUs with KVM Virtualization in Red Hat Enterprise Linux. Another feature in Red Hat Enterprise Linux 7 beta that is both exciting and paves the way for future enhancements is the KVM graphics device assignment capability — wherein the entire graphics card can be passed through to a single virtual machine. Using Red Hat Enterprise Linux 7 beta, you will be able to assign a GPU directly to a virtual machine and provide 3D graphics acceleration for GPU computing (NVIDIA Tesla) or high density farms (NVIDIA GRID) or local graphics (NVIDIA Quadro). Stay tuned for more updates as we go along this journey.

So what do you think of the KVM features I have outlined here? Are the virtualization enhancements to Red Hat Enterprise Linux 7 beta relevant to your own day-to-day operations? I look forward to reading your feedback, comments and questions.

In my next blog post I will expand on the topic of Red Hat Enterprise Linux as a guest operating system on third party hypervisors such as VMware vSphere and Microsoft Hyper-V.

    1. Tony, thanks for your comment. RHEL 7.0 beta supports PCI Passthrough with select Nvidia cards – 1:1 VM to GPU card access. The vGPU support in KVM (which typically refers to 1:many access) is on our roadmap, and getting attention.

      1. Has this been implemented yet or any definite plans for it? Specifically vGPU support for the Nvidia K2 card. Thanks!

  1. Bhavna Sarathy, give more information. For example, what cards of Nvidia are supported? I didn’t find it in documentation.

    1. Thanks for your questions Alex. Here’s additional information regarding the GPU passthrough capability — I hope this answers your questions.

      Virtual Function I/O (VFIO) is a PCI device assignment architecture that allows for development of
      secure, high-performance user-space drivers. The device assignment with VFIO moves out of the KVM
      module into user space and is compatible with secure boot. It provides user I/O (UIO) interfaces
      allowing those drivers to work with DMA and interrupts while keeping overall control over how
      devices access the system’s resources. VFIO recognizes groups of devices connected to an IOMMU
      and assigns them as a unit, such that this group of devices can access shared memory regions.
      RHEL 7 RC enables graphics PCI passthrough in KVM using VFIO infrastructure / framework.

      RHEL 7 RC adds graphics cards to the list of devices that support PCI passthrough in KVM. Using the
      VFIO framework in KVM, the entire graphics device can be passed through to a single virtual machine.
      At this time RHEL 7 RC works with a limited set of Nvidia GPU professional cards:
      Nvidia GRID
      Nvidia Quadro

      1. These are the Nvidia cards we have tested and will be supported in RHEL 7:

        QUADRO K5000(P2004-B02)
        GRID K2,8GB(P2055-A03)
        GRID K1,16GB(P2401-A02)

  2. Sarathy,
    Could you explain how to access KVM GPU pass-throughed VM Desktop,SPICE or others?
    As far as I know, on current only PCoIP and ICA support remote access GPU Pass-throughed VM while SPICE cann`t.
    Does Redhat has any plan on SPICE to support this ? Or any schedule?
    Thank you very much!

    1. Thanks for your question. At this time GPU passthrough doesn’t involve spice. As noted in another response, we are investigating vGPU in KVM which may involve spice.

  3. Thank you for your response. Us completely arranges that RHEL 7 RC works with professional cards of Nvidia GRID and Nvidia Quadro. Question in other. At the moment RHEL 7 RC supports passed through of a graphic card to the single virtual machine. That’s wonderful. Us two questions interest:
    1) When implementation in RHEL vGPU use (when one graphic card can be used by several guests) is planned? At least approximately.
    2) Whether it is possible to use now some professional cards of Nvidia in passed through option on one server? That is on the server some Nvidia Quadro (for example four Nvidia Quadro 2000) is set and passed through to four virtual guests is executed.

    1. Alex, thank you for your continued interest in the GPU capabilities in KVM.

      We are interested in vGPU (1:N), and hearing customer requests. We are collecting use cases and discussing a design. I’m unable at this point to provide exact dates and releases. I’ll be happy to provide an update or write a GPU focussed blog at a future date.

      K1s have 4 separate GPUs on one card and can be passed to 4 separate VMs. If a server can take 2 K1s or a K1 and K2, RHEL should be able to handle that as well.

      1. Thank you for your response.
        And if to use instead of K1 or K2 of cards, some cheaper Nvidia Quadro 2000? Whether it is possible to passed through two Nvidia Quadro 2000 to two virtual guests?

      2. Nvidia provided K1 and K2 cards only for development and testing, and so our support will be limited to professional graphics cards.

  4. Is there any plan on adding AMD r2xx series support? Or is it possible to use these already for 1:1 mapping in the released version of RHEL7 officially or othwerwise?

    1. Thanks for your question, Albert. The RHEL 7 1:1 mapping is for select Nvidia GPUs – the GPU vendor designs vary considerably and it quite likely that AMD r2xx series will not work as is, joint development with AMD (if market demand exists) would be required.

  5. Is live migration of a virtual GPU supported? Assume two identical servers and VM is actively utilizing the GPU.

    1. Thanks for your question. By definition it is not possible for a VM with an assigned device to live migrate to another server at the same time. This is not related to GPUs.

  6. Bhavna Sarathy, whether is the link to How-To, documentation or the instruction on RHEL 7 on passed through video cards to the virtual guests?

    1. The RHEL 7 Nvidia GPU device assignment implementation is done as just another PCI assigned device, so the standard instructions in the virtualization guide for device assigment is what customers use.

  7. You mean the head “Chapter 20. Guest virtual machine device configuration of” instruction “Virtualization Deployment and Administration Guide”? But in it all examples passthrough PCI-devices are shown on the example of network interface cards. It develops vpechatly that it is possible to forward any PCI device. For example, how to use 4 graphic kernels of the video card Nvidia K2? Passthrough it is carried out based on PCI identifier code. At the video card Nvidia K2, which is inserted into one slot PCI-E, 4 graphic kernels are represented by devices from 4 different PCI identifier code?

    1. Alex, please open a bugzilla against the RHEL product so that we can use the appropriate channels to provide you assistance. Thank you.

    1. Work on vGPU support is continuing to happen upstream. Please make your requirements for vGPU on KVM support known to the GPU vendors.

      -regards
      Subhendu

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s