xen on arm: status and performance
TRANSCRIPT
18 August 2014
Xen on ARMHow fast is it, really?
Stefano Stabellini
Status
© 2013 Citrix | Confidential – Do Not Distribute
Xen Project 4.4 release: statusFeatures:● 64-bit guest support in ARMv8● stable hypercall ABI● basic lifecycle operations● memory ballooning● scheduler configurations, vcpu pinning, cpu pools
HCL (not inclusive):● ARMv8 FVP● Arndale board● Allwinner Sunxi (Cubieboard 2)● TI OMAP5● Applied Micro X-Gene
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Linux v3.13+: status
● dom0 and domU● 32-bit and 64-bit● SMP● swiotlb-xen● PV drivers
○ disk○ network○ console○ framebuffer○ keyboard○ mouse
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5● irq latency reduction: no maintenance interrupts● support for interrupt priorities● irq migration: physical follow virtual● multi-grants support● SMMU support● Device Assignment on ARM● Xen UEFI host support● GICv3 support● support for up to 1TB of guest RAM● super pages support● support for sparse memory map● support for 48-bit physical addresses● QEMU PV backend on ARM: framebuffer,
mouse, keyboard, multi-console© 2014 Citrix
● MiniOS on ARM● PSCI v0.2
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5● irq latency reduction: no maintenance interrupts● support for interrupt priorities● irq migration: physical follow virtual● multi-grants support● SMMU support● Device Assignment on ARM● Xen UEFI host support● GICv3 support● support for up to 1TB of guest RAM● super pages support● support for sparse memory map● support for 48-bit physical addresses● QEMU PV backend on ARM: framebuffer,
mouse, keyboard, multi-console© 2014 Citrix
● MiniOS on ARM● PSCI v0.2
© 2013 Citrix | Confidential – Do Not Distribute
xen-devel traffic
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Maintainers & reviews
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5: performance improvements
● irq latency reduction: no maintenance interrupts● support for interrupt priorities● irq migration: physical follow virtual● multi-grants support● super pages support● support for up to 1TB of guest RAM● support for sparse memory map● support for 48-bit physical addresses
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
irq latency reduction: no maintenance interrupts
DomU
Xen
irq 109
virq 109
DomU
Xen
EOI
DomU
Xen
Maintenance interrupt
GICH_LRWrite
GICH_LRClear
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
irq latency reduction: no maintenance interrupts
DomU
Xen
irq 109
virq 109
DomU
Xen
EOI
DomU
Xen
Maintenance interrupt
GICH_LRWrite
GICH_LRClear
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5: performance improvements
● irq latency reduction: no maintenance interrupts● support for interrupt priorities● irq migration: physical follow virtual● multi-grants support● super pages support● support for up to 1TB of guest RAM● support for sparse memory map● support for 48-bit physical addresses
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
irq migration: physical follow virtual
vcpu0 vcpu1
pcpu0 pcpu1
irq 109
virq 109
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
irq migration: physical follow virtual
vcpu0 vcpu1
pcpu0 pcpu1
irq 109
virq 109
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
irq migration: physical follow virtual
vcpu0 vcpu1
pcpu0 pcpu1
irq 109
virq 109
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5: performance improvements
● irq latency reduction: no maintenance interrupts● support for interrupt priorities● irq migration: physical follow virtual● multi-grants support● super pages support● support for up to 1TB of guest RAM● support for sparse memory map● support for 48-bit physical addresses
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
multi-grants support
© 2014 Citrix
Backend Frontend
PagePage
Page
Grant
Grant
© 2013 Citrix | Confidential – Do Not Distribute
multi-grants support
© 2014 Citrix
Backend Frontend
PagePage
Page
Grant
Grant
mfn
==
mfn
© 2013 Citrix | Confidential – Do Not Distribute
multi-grants support
© 2014 Citrix
Backend Frontend
PagePage
Page
Grant
Grant
mfn
==
mfn
mfn_to_pfn
??
© 2013 Citrix | Confidential – Do Not Distribute
multi-grants support
© 2014 Citrix
Backend Frontend
PagePage
Page
Grant
Grant
Pagepfn == mfn
mfn
==
mfn
mfn_to_pfn
??
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5: performance improvements
● irq latency reduction: no maintenance interrupts● support for interrupt priorities● irq migration: physical follow virtual● multi-grants support● super pages support● support for up to 1TB of guest RAM● support for sparse memory map● support for 48-bit physical addresses
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5: hardware and interfaces
● SMMU support● Xen UEFI host support● PSCI v0.2● GICv3 support
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Coming in Xen 4.5: features
● (Non-PCI) Device Assignment on ARM● QEMU PV backend on ARM: framebuffer, mouse, keyboard, multi-console● MiniOS on ARM
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Looking ahead
● save/restore and live-migration● PCI device assignment
● IOMMU emulation for remote processors● GICv2m support● SMMUv2 support● ACPI support
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Embedded and Automotive
● Embedded and Automotive Xen Project initiativehttp://xenproject.org/about/events/viewevent/146-webinar-virtualization-in-the-automotive-industry-2.html
● Embedded and Automotive PV drivers projecthttp://xenproject.org/developers/teams/embedded-and-automotive.htmlhttp://wiki.xenproject.org/wiki/Embedded_and_Automotive_PV_Drivers/Roadmap
© 2014 Citrix
Benchmarks
© 2013 Citrix | Confidential – Do Not Distribute
Execution environment
● Linux v3.15-rc8dom0, domU and native kernel
● Xen v4.5-unstable+
● QEMU-KVM v2.0.0
● OpenSUSE 13.1
© 2014 Citrix
Applied Micro X-Gene Intel Xeon CPU X5650
ARMv8 64-bit 8 cores 2.4 Ghz x86_64 2 sockets x 6 cores 2.67 Ghz
© 2013 Citrix | Confidential – Do Not Distribute
Kernbenchvirtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
PBZIP2virtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
FIO rand-rwvirtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
SPECjbb2005virtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
PGBenchvirtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
PGBenchvirtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
PGBenchvirtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
PGBenchvirtualization overhead (percentage) - Lower is better
© 2014 Citrix
120
Benchmarks by
© 2013 Citrix | Confidential – Do Not Distribute
SpecINT 2006Xen on ARM virtualization overhead (percentage) - Lower is better
2%
© 2014 Citrix
Conclusions
© 2013 Citrix | Confidential – Do Not Distribute
● many new features coming● good performance results● still lots of room for improvement
Conclusions
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
● need more reviewers
Conclusions
© 2014 Citrix
Questions?
© 2013 Citrix | Confidential – Do Not Distribute
More information
● http://www.xenproject.org
● Xen on ARM @wiki.xenproject.org http://goo.gl/FKNXe
● Xen on ARM whitepaper http://goo.gl/vWMn7D
● http://lists.xenproject.org/mailman/listinfo/xen-devel
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Work better. Live Better.
Citrix
© 2014 Citrix
Slide parking lot
© 2013 Citrix | Confidential – Do Not Distribute
IPerf
Linerate
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
FIOXen on ARM virtualization overhead (percentage) - Lower is better
© 2014 Citrix
© 2013 Citrix | Confidential – Do Not Distribute
Reading xen-devel
© 2014 Citrix