gpu accelerated virtual desktop infrastructure (vdi) on openstack

GPU-Accelerated VDI on OpenStack

NIMBIS SERVICES, INC.


OpenStack DC MeetUpJune 18, 2013

Nimbis Services, Inc.

http://www.nimbisservices.com

Brian Schott, CTO

[email protected]

http://www.nimbisservices.com/

mailto:[email protected]



About Nimbis Services

Nimbis Services is lowering the barriers to high-performance computing (HPC) and offering a secure

marketplace for technical computing in the cloud.

Nimbis partners with leading HPC & cloud providers, software vendors, and domain experts to configure and broker secure

on-demand pay-as-you-go access to applications.



Challenges for GPU-Accelerated Virtual Desktop Infrastructure (VDI) on OpenStack

Making the GPU available to a virtual machine.(3 basic approaches, ok, maybe 4). Bare Metal / Containers (lxc) PCI Pass-Through Para-virtualization (proxy device drivers) Remote Display Protocols (X, NX, VirtualGL, WebGL)

Remote desktop protocol that supports GPU acceleration. Host and Guest Combinations: Linux or Windows Remote Clients: Linux, Windows, OSX, Web Browser, Tablet Networking: “The Joys of Firewalls”, or “Any Port as Long as it is 443” Licenses: “This is too good to be free.”

Provisioning GPU resources from within OpenStack. Nova compute virtualization driver (plugin). GPU instance types, resource discovery, and scheduling. This is still under development in the community. Full disclosure: Nimbis hand-deployed the GPU instances for testing.



Nimbis Testbed Hardware

“Production” Cisco System (1) UCS C220 M3 – dual quad-core 3.3GHz

Intel Xeon E5-2643, 192GB SDRAM, 2TB SAS-7K, dual channel 10GbE interface.

(2) UCS C220 M3 – dual quad-core 2.4GHz Intel Xeon E5-2609, 192GB SDRAM, 2TB SAS-7K, dual channel 10GbE interface.

(3) UCS C240 M3 – dual quad-core 2.4GHz Intel Xeon E5-2609, 192GB SDRAM, 6TB SAS-7K, dual-channel 10GbE.

• (1) Nvidia GRID K2 Board (2) Nexus 3064 10GbE Switches (2) ASA 5520 Adaptive Security Appliances (2) Cisco Catalyst 2960S 1GbE switches

“Development” Cisco System (3) Cisco UCS C200 M2 (1) Nexus 3064 10GbE Switch (1) ASA 5520 Adaptive Security Appliance (1) Cisco Catalyst 2960S 1GbE switch

Cores: 72Memory: 1440 GBDisk: 36 TBServers: 9*

Switches: 9*

* What do you expect with a Cisco testbed?



Nvidia GRID GPU Virtualization (Kepler Architecture)

Source: http://www.nvidia.com/object/grid-boards.html

Source: 1http://www.nvidia.com/content/PDF/kepler/NV_DS_Tesla_KCompute_Arch_May_2012_LR.pdf

GPU Virtualization¹GRID boards feature the NVIDIA Kepler architecture that, for the first time, allows hardware virtualization of the GPU. This means multiple users can share a single GPU, improving user density while providing true PC performance and compatibility.



GPU-Accelerated VDI Approaches Considered

Customer Requirements Combination of Windows and Linux guest images. Applications using OpenGL on both Windows and Linux. Variety of Windows (95, 7, 8) clients, OSX clients, and web browsers

(IE7, IE8, Firefox, Chrome, Safari). Variety of firewall and IT/FSO software challenges.

Technologies Tested Microsoft Hyper-V and RemoteFX XenServer with VNC / RDP / Guacamole Open Source Xen (libvirt) on Ubuntu Linux with VNC

Possible Alternatives Linux Containers (lxc)

• USC/ISI team has the blueprint and code targeted for Havana.• No Windows support.

XenDesktop with HDX 3D Pro (not tested)• Requires a custom client, Citrix Receiver, OpenGL support unclear.• Does not integrate with OpenStack.



Microsoft Hyper-V and RemoteFX

Technology Windows Host GPU device with para-virtualized driver for windows guests.

Host / Hypervisor Microsoft Windows Server 2012 Datacenter Edition (RemoteFX license). Latest Nvidia driver for Windows.

Guests No GPU pass-through available or driver for Linux guests. Accelerates Windows DirectX calls, does software OpenGL rendering.

Clients Good performance with standard Windows RDC > 8.0. Good performance from FreeRDP* OSX client, not Microsoft client. iOS and Android? Not tested, but FreeRDP clients exist on both.

OpenStack Status Integrates with OpenStack Hyper-V nova compute drivers. Nova scheduler support for Hyper-V extra specs not on blueprint radar. Possible to use today with custom nova scheduler plugin.

* http://www.freerdp.com



Hyper-V and RemoteFX Benchmarks with Nvidia GRID K2

Host: Microsoft Windows Server 2012 Datacenter Edition, Guest: Microsoft Windows Server 2012 Standard Evaluation,Client: FreeRDP on Mac OSX * http://gfxbench.com/



XenServer with VNC / RDP / Guacamole

Technology GPU pass-through of Nvidia GRID K2 to Windows and Linux guests. Two GPU devices available for two separate instances. Physical GPU virtualization drivers promised by end of the year (n=64?).

Host / Hypervisor XenServer 6.1 Platinum Edition (enables GPU license).

Guests Linux: specify Windows 7 as server template then standard CentOS 6.4

install process. Good GPU performance using VNC and VirtualGL. Windows: tried both RDP and TigerVNC do software OpenGL rendering.

• Unable to VNC or RDP to “console” display, possibly no monitor? Clients

Windows and OSX Clients comparable to remote desktop to native boxes. Tried RDP, VNC, and Guacamole (HTML5 VNC client).

OpenStack Status XenServer Integrates with OpenStack using XenServer plugins. Nova scheduler support for XenServer not on blueprint radar.



Open Source Libvirt/Xen with VNC / RDP / Guacamole

Technology GPU pass-through of Nvidia GRID K2 to Windows and Linux guests. Two GPU devices available for two separate instances. Unsure of GPU virtualization drivers status for n > 2.

Host / Hypervisor Ubuntu 12.04, Xen (Xen hypervisor 4.1.0), libvirt (0.98) Xen HVM guest cfgfile in /etc/xen/<guest> to enable VGA passthrough:

gfx_passthru=1 pci=['yy:zz.n’]

Guests Linux: Tested CentOS 6.4 image with latest Nvidia drivers (310.44). Verified

device functioned. Expect similar results to XenServer. Windows: Did not test an image. Expect similar results to XenServer.

OpenStack Status Xen supported within OpenStack using nova libvirt driver. Nova specs support for libvirt exists (i.e. this host has a GPU). Feasible to have support included in H or I release given lxc work.

http://wiki.xen.org/wiki/Xen_VGA_Passthrough



XenServer Linux Virtual Machine Installation Hack



Specify GPU Type



VirtualGL Architecture (VirtualGL.org)



GLXINFO for Linux Instance on XenServer



Guacamole HTML5 Browser VNC



Windows Server 2012 on XenServer



Heterogeneous Architecture Support Blueprints

https://wiki.openstack.org/wiki/HeterogeneousInstanceTypes

https://wiki.openstack.org/wiki/ScheduleHeterogeneousInstances

https://wiki.openstack.org/wiki/HeterogeneousGpuAcceleratorSupport



Summary

GPU-accelerated VDI pools within OpenStack are quite feasible. Code support is very rough around the edges, not production ready. We really need to get open community support organized for this feature. Several vertical vendor solutions out there, but Xen and KVM baselines

should be possible as open-source. Questions?

gpu accelerated virtual desktop infrastructure (vdi) on openstack

Technology

gpuaccelerated vdi

gpu available

single gpu

gpu instances

gpu acceleration

openstacknimbis services

gpu passthrough available

accelerated vdi approaches