dpdk summit - 08 sept 2014 - ericsson - a multi-socket ferrari for nfv

Slide title

70 pt

CAPITALS

Slide subtitle

minimum 30 pt

Slide title

44 pt

Text and bullet level 1

minimum 24 pt

Bullets level 2-5

minimum 20 pt

Characters for Embedded font: !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~¡¢£¤¥¦§¨©ª«¬®¯°±²³´¶·¸¹º»¼½ÀÁÂÃÄÅÆÇÈËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĀāĂăąĆćĊċČĎďĐđĒĖėĘęĚěĞğĠġĢģĪīĮįİıĶķĹĺĻļĽľŁłŃńŅņŇňŌŐőŒœŔŕŖŗŘřŚśŞşŠšŢţŤťŪūŮůŰűŲųŴŵŶŷŸŹźŻżŽžƒȘșˆˇ˘˙˚˛˜˝ẀẁẃẄẅỲỳ–—‘’‚“”„†‡•…‰‹›⁄€™ĀĀĂĂĄĄĆĆĊĊČČĎĎĐĐĒĒĖĖĘĘĚĚĞĞĠĠĢĢĪĪĮĮİĶĶĹĹĻĻĽĽŃŃŅŅŇŇŌŌŐŐŔŔŖŖŘŘŚŚŞŞŢŢŤŤŪŪŮŮŰŰŲŲŴŴŶŶŹŹŻŻȘș−≤≥fifl

ΆΈΉΊΌΎΏΐΑΒΓΕΖΗΘΙΚΛΜΝΞΟΠΡΣΤΥΦΧΨΪΫΆΈΉΊΰαβγδεζηθικλνξορςΣΤΥΦΧΨΩΪΫΌΎΏ

ЁЂЃЄЅІЇЈЉЊЋЌЎЏАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯАБВГДЕЖЗИЙКЛМНОПРСТУФХЦЧШЩЪЫЬЭЮЯЁЂЃЄЅІЇЈЉЊЋЌЎЏѢѢѲѲѴѴҐҐәǽẀẁẂẃẄẅỲỳ№

Do not add objects or

text in the footer area DPDK Summit 2014 | Public | © Ericsson AB 2014 | September 8 |

Going virtualized

Environment-independency

Flexible platform

Backward compatibility

High and deterministic performance

High availability

Robustness

Intact DPDK common libraries

GOALS

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






That’s life :)

GOALS Going virtualized

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Running in cloud

Hypervisor-agnostic solution

Running also native or bare metal

Reusing existing blades

GOALS Environment-independency

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Flexible yet simple configuration

Support of multiple application models

Scalability

Performance tuning

(adopting to the environment without hardcoding)

GOALS Flexible platform

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Easy migration of existing applications

Goals Backward compatibility

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






High throughput

Low packet delay variation

same performance after VM instantiations

Equal per-instance performance

Equally distributed resources

GOALS High and deterministic performance

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Telco requirement

Fast recovery

Avoid single point of failure

Redundancy

GOALS High availability

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






protect against accidental memory writes

Multi-process support

Increased debuggability

GOALS robustness

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Easy Integration of new DPDK releases

GOALS Intact DPDK common librarieS

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Our typical applications have large memory footprints and significant amount of random

reads/writes

supporting giant virtual machines

NUMA-awareness in XEN HVM

challenge

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






in reality, we Present a virtual world to

DPDK

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






SPLIT EAL into publisher and consumer

Decoupled ENVIRONMENT DETECTION

Topology publishing (CPU, Memory, device topology)

Memory configuration publishing

Device publishing (registered DPDK ports)

solution OUR NEW DPDK EAL

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Selective memory mapping

Memory access control (avoid mapping everything as R/W)

solution multi process support

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






process1 process2 process3

DPDK memory

process1 process2 process3

DPDK memory

DPDK mapping

Our mapping

R/W

R/O

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Virtual address space allocator

Primary role passing (distributed resource management)

Device (re-)initialization

solution multi process support cont’d

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






SIMPLE MEMORY MANAGEMENT API for APPS (named memory partition pools)

FLEXIBLE CONFIGURATION / RE-CONFIGURATION (WITHOUT CODE CHANGE)

MAKING DPDK NUMA AWARE IN A NUMA FLAT OS

More granular way of placing objects (Control TLB entry usage => zero tlb miss)

solution Improved memory management

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






1G

1G

1G

1G

1G

1G

1G

1G

DPDK memory DPDK memory

process1

process2

memzones allocated/used by process2


1G

1G

1G

1G

1G

1G

1G

1G

solution Improved memory management cont’d

DPDK

socket

0

DPDK

socket

1

DPDK

socket

2

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Partitioning memory (MULTIPLE DPDK NUMA SOCKETS)

Fragmentation support

per-lcore private memory (dedicated dpdk socket id)

while leaving DPDK common libraries intact

solution Improved memory management cont’d

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Keep track of available resources: Lcore, CPU, Memory, port, queue, device

on-demand resources

Reclaim resources

solution resource manager

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Publishing configuration

Memzones (memdomains for app)

Named Packet pools

Named Virtual ports / queues

Solution Ready to use Configurations

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






socket memory memory socket CPUs

HT

HT

HT

HT

HT

HT

memory queues queues queues ports ports ports

HT

HT

HT

HT

HT

HT

HT

HT

misc.

devices

misc.

devices

misc.

devices

misc. devices

CPU CPU

HT

HT

HT

HT

HT

HT

HT

HT

HT

HT

excl

excl

excl

excl

NU

MA

defau

lt

lcores

defau

lt

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






results (sample test app brief)

test app

dedicated memory random data

packet packet

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






results (dual socket - NUMA alignment)

Mixed-NUMA

memory

NUMA-aligned

memory

53% improvement

Performance

Socket1 Socket0 Socket1 Socket0

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






results (single socket - zero tlb miss)

Fragmented allocation

1G

1G

1G

1G

1G

1G

1G

1G



Unfragmented allocation

16% improvement

Performance

1G

1G

1G

1G

1G

1G

1G

1G

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






results (NUMA alignment plus zero tlb miss)

Mixed-NUMA

memory

NUMA-aligned

memory

77% improvement

Performance

Socket1 Socket0 Socket1 Socket0

1G

1G

1G

1G

1G

1G

1G

1G

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






be(a)ware of the numa

hugepage backing (hypervisor control)

TLB misses are costly in VM

proper memory alignment is vital

conclusion

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






Improve memzone allocator

Further improve multi-process support

Per-lcore rte_malloc area

DPDK domains

Generic resource manager for DPDK

Additional configuration layer to EAL

Add cache-Qos to memdomains

ideas

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






backup slides

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






ConfiguratioN EXAMPLE CPUALIAS

cpualias all {

cpumask = "0-31" # all available CPU

}

cpualias foreground {

cpumask = "2-31:2" # even cpus excluding cpu0

}

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






CONFIGURATION EXAMPLE NUMA TYPE

# ----------------------------------------

# Per NUMA node shared memory.

# App instances tied to the same NUMA

# are sharing the same memory partition.

# ----------------------------------------

memdomain App_NUMA_Shared {

type = numa

cpualias = "all"

alloc_memzone = true

size {

is_per_numa = true

huge_2M = 0

huge_1G = 1G

}

}

Slide title

44 pt


minimum 24 pt

Bullets level 2-5

minimum 20 pt






CONFIGURATION EXAMPLE EXCLUSIVE TYPE

# -----------------------------------

# Per App instance private memory.

# Every instance has its own memory

# partition.

# -----------------------------------

memdomain App_Thread_Local {

type = excl

cpualias = "foreground"

alloc_memzone = true

size {

is_per_cpu = true

huge_2M = 0

huge_1G = 1G

}

}

dpdk summit - 08 sept 2014 - ericsson - a multi-socket ferrari for nfv

Software