June 2013

LCE-13 Engineering Review

PMWG WG

Europe 2013 (LCE13)

www.linaro.org

CPUIdle: consolidation status and

discussion about next steps

Daniel Lezcano <daniel.lezcano@linaro.org>

Europe 2013 (LCE13)

www.linaro.org

CPUIdle: status

Dynamic timer irq affinity

Timer broadcast consolidation

A simplified function to register the driver and device

Code fixes and cleanup

Europe 2013 (LCE13)

www.linaro.org

Timer broadcast unoptimized behavior

CPU0

Broadcast timer local timer

Tick framework

CPU1

local timer

Enter broadcast mode

Setup timer

Interrupt

Handle broadcast interrupt

Send IPI timer

Z Z Z Z Z

CPU0 is woken up to wake up CPU1

Z Z Z Z Z

Dynamic timer irq affinity

Europe 2013 (LCE13)

www.linaro.org

Dynamic timer irq affinity

Timer broadcast optimized behaviour

CPU0

Broadcast timer local timer

Tick framework

CPU1

local timer

Enter broadcast mode

Setup timer

Interrupt

Z Z Z Z Z

CPU1 is directly wake up by the timer device

Z Z Z Z Z

Set irq to CPU1

Europe 2013 (LCE13)

www.linaro.org

Dynamic timer irq affinity

without dynamic timer irq affinity

with dynamic timer irq affinity

A specific test case 'usleep 10000' on CPU1

Europe 2013 (LCE13)

www.linaro.org

Dynamic timer irq affinity

Results:

reduced by 40% the number of wake up on the system

reduced by 49% the number of wake up for CPU0

increased by factor two idle time for CPU0

increase by 16% package idle hits + 16% average package idle time

Europe 2013 (LCE13)

www.linaro.org

Dynamic timer irq affinity

Pre-requisite :

timer device must support CPU affinity

Irq field in the clockdevice driver must be initialized

Dynamic irq affinity + timer / work queues migration = cpu

quiescent for a long period

In conjunction with the small task packing, we have a cluster

down for long periods

Add the CLOCK_EVT_FEAT_DYNIRQ flag to the clock

device driver if this feature is worth to add to the SoC

Europe 2013 (LCE13)

www.linaro.org

Timer broadcast consolidation

CPUIdle framework CPUIdle backend driver IDLE(cpu1) Enter idle state

Tick framework

Clockevents notify enter Low level sleep code

CPU0 CPU1

Setup broadcast timer

SMP cross call

Z Z Z Z Z

Old behaviour : the timer broadcast is handled in the

backend driver

Clockevents notify exit

Interrupt

Europe 2013 (LCE13)

www.linaro.org

Timer broadcast consolidation

New behaviour : the timer broadcast is handled by the

cpuidle framework with the idle state flag :

CPUIDLE_FLAG_TIMER_STOP

CPUIdle framework CPUIdle backend driver IDLE(cpu1) Enter idle state

Tick framework

Clockevents notify enter

Low level sleep code

CPU0 CPU1

Setup broadcast timer

SMP cross call

Z Z Z Z Z

Clockevents notify exit

Interrupt

Europe 2013 (LCE13)

www.linaro.org

Timer broadcast consolidation

Duplicated code across back end drivers removed

A flag to tell to use the timer broadcast for a specific state

CPUIdle backend drivers only cares about PM work

Idle state callbacks more and more simple and focused on SoC

specific function

Europe 2013 (LCE13)

www.linaro.org

Simplified register function

Usually the initialization routines are always the same

1. Register the driver : cpuidle_register_driver

2. Register all the cpu devices : cpuidle_register_device

With the cpuidle_device statically declared per cpu in each

driver

Europe 2013 (LCE13)

www.linaro.org

Simplified register function

Introduced a new generic register function

Europe 2013 (LCE13)

www.linaro.org

Simplified register function

Results in a nice code consolidation

Europe 2013 (LCE13)

www.linaro.org

Code fixes and cleanup

Multiple driver support simplified

Cpumask in the cpuidle_driver structure

cpuidle_register_driver / cpuidle_register could be used

Removed API cpuidle_register_driver_cpu

➔ Results in internal framework code simplified

Made governors optional at compile time

menu/tickless – ladder/periodic tick

Removed en_core_tk_irqen flag : all drivers have the same

behavior

Time keeping done and local irq enabled by the framework

Misc code cleanups, fixes and unification of the init functions

Europe 2013 (LCE13)

www.linaro.org

Conclusion

Where are we ?

Very similar init / exit functions

Some PM functions can be now factored out

davinci – kirkwood - at91 DDR self refresh

Platform driver and device tree more and more used

Kirkwood, davinci

PM arch dependant code and driver more and more

separated

ux500 and at91 drivers will be moved to drivers/cpuidle soon

Europe 2013 (LCE13)

www.linaro.org

Next steps

b.L cpuidle driver for TC2 is incoming (planned for 3.12)

A single ARM driver

With the code consolidation, we are close now but still depends on the

points below

Handle cpu hotplug in the cpuidle framework

Continue to split arch specific PM code from the driver

And move the drivers to the drivers/cpuidle directory

More use of the device tree

Latencies

Arch specific check (eg. davinci ddr power down flag)

More about Linaro Connect: www.linaro.org/connect/ More about Linaro: www.linaro.org/about/

More about Linaro engineering: www.linaro.org/engineering/

Europe 2013 (LCE13)