gpu implementations of online track finding algorithms at panda

5/27/2018 GPU Implementations of Online Track Finding Algorithms at PANDA

1/37

MitgliedderHelmholtz-Gemeinschaft

GPU Implementations of

Online Track Finding

Algorithms at PANDA

1

HK 57.2, DPG-Frhjahrstagung 2014, Frankfurt21 March 2014, Andreas Herten (Institut fr Kernphysik, Forschungszentrum Jlich) for the PANDA Collaboration


2/37


Andreas Herten, DPG Frhjahrstagung 2014, HK 57.2

PANDA The Experiment

2


3/37



PANDA The Experiment

2

Magnet

STT

MVD


4/37


PANDA Event Reconstruction

Triggerlessread out

Many benchmark channels Background & signal similar

Event Rate: 2 107/s

3

Raw Data Rate:200 GB/s

Disk Storage Space for

Offline Analysis: 3 PB/y

Reduce by

~1/1000(Reject background events,

save interesting physics events)


5/37


PANDA Event Reconstruction

Triggerlessread out

Many benchmark channels Background & signal similar

Event Rate: 2 107/s

3

Raw Data Rate:200 GB/s

Disk Storage Space for

Offline Analysis: 3 PB/y

Reduce by

~1/1000(Reject background events,

save interesting physics events)

GPUs


6/37


Andreas Herten, DPG Frhjahrstagung 2014, HK 57.2 4

Trigger

Detectorla

yers

PANDA Tracking, OnlineTracking

PANDA: No

hardware-based

trigger

But computational

intensive softwaretrigger

!Online Tracking


7/37



Trigger

Detectorla

yers


PANDA: No

hardware-based

trigger

But computational


!Online Tracking


8/37



Trigger

Detectorla

yers

Usual HEP experiment


PANDA: No

hardware-based

trigger

But computational


!Online Tracking


9/37



Trigger

Detectorla

yers



PANDA: No

hardware-based

trigger

But computational


!Online Tracking


10/37



Trigger

Detectorla

yers



PANDA: No

hardware-based

trigger

But computational


!Online Tracking


11/37



Trigger

Detectorla

yers



PANDA: No

hardware-based

trigger

But computational


!Online Tracking


12/37



Trigger

Detectorla

yers


PANDA


PANDA: No

hardware-based

trigger

But computational


!Online Tracking


13/37



Trigger

Detectorla

yers


PANDA


PANDA: No

hardware-based

trigger

But computational


!Online Tracking


14/37



Trigger

Detectorla

yers


PANDA


PANDA: No

hardware-based

trigger

But computational


!Online Tracking


15/37



Trigger

Detectorla

yers


PANDA


PANDA: No

hardware-based

trigger

But computational


!Online Tracking


16/37



GPUs @PANDA OnlineTracking

Porttracking algorithms to GPU

Serial!parallel C++!CUDA

Investigate suitabilityfor online performance

But also: Find & invent tracking algorithms

Under investigation:

Hough Transformation

Riemann Track Finder

Triplet Finder

5


17/37



Algorithm: Hough Transform

Idea:Transform (x,y)i!(,r)ij, find lines via (,r)space

Solve rijline equation for

Lots of hits (x,y,)iand

Many j![0,360) each

Fill histogram Extract track parameters

6

-


18/37




Idea:Transform (x,y)i!(,r)ij, find lines via (,r)space

Solve rijline equation for

Lots of hits (x,y,)iand

Many j![0,360) each

Fill histogram Extract track parameters

6

rij = cosjxi + sinjyi + i

i: ~100 hits/event (STT)

j: every 0.2rij: 180 000

-


19/37

Angle /0 20 40 60 80 100 120 140 160 180

H

oughtransformed

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6 0Entries 2.2356e+08

Mean x 90

Mean y 0.02905

RMS x 51.96

RMS y 0.1063

0

5

10

15

20

25

0Entries 2.2356e+08

Mean x 90

Mean y 0.02905

RMS x 51.96

RMS y 0.1063

1800 x 1800 GridPANDA STT+MVD


7

68 (x,y) pointsr



20/37

Angle /0 20 40 60 80 100 120 140 160 180

H

oughtransformed

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0.6 0Entries 2.2356e+08

Mean x 90

Mean y 0.02905

RMS x 51.96

RMS y 0.1063

0

5

10

15

20

25

0Entries 2.2356e+08

Mean x 90

Mean y 0.02905

RMS x 51.96

RMS y 0.1063

1800 x 1800 GridPANDA STT+MVD


7

68 (x,y) pointsr



21/37




8

Thrust Plain CUDA

Performance:3 ms/event

Independentof granularity

Reduced to set of standard routines Fast (uses Thrusts optimized algorithms)

Inexible (has its limits, hard to customize)

No peakndingincluded

Even possible?

Adds to time!

Performance:0.5 ms/event

Built completely for this task

Fitting to every problem Customizable

A bit more complicated at parts

Simple peaknderimplemented

(threshold)

Using: Dynamic Parallelism, Shared

Memory

Two Implementations


22/37



Idea:Dont t lines (in 2D), t planes (in 3D)!

Create seeds All possiblethree hit combinations

Growseeds to tracks

Continuously test next hit if it ts

Use mapping to Riemann paraboloid

Summer student project (J. Timcheck)

Algorithm: Riemann Track Finder


23/37

Layerx =1

2

8x+ 1 1

pos(nLayerx) =3p

3243x2 1+27x

32/3 +

1

33

3p

3243x2 1+27x

1


10

Algorithm: Riemann Track Finder

intijk =threadIdx.x +blockIdx.x *blockDim.x;for() {for() {for() {}}}

GPU Optimization: Unfolding loops

!100 faster than CPU version

Time for one event (Tesla K20X): ~0.6 ms


24/37



Algorithm: Triplet Finder

Idea:Use only sub-set of detector as seed

Combine 3 hits to Triplet Calculate circle from 3 Triplets (no t)

Features

Tailored for PANDA

Fast & robust algorithm, no t0

Ported to GPU together with NVIDIA Application Lab
http://localhost/Users/Andi/Downloads/triplet-non-bunched-3.numbershttp://localhost/Users/Andi/Downloads/triplet-non-bunched-3.numbers


25/37



Triplet Finder Time


26/37



Triplet Finder Optimizations

Bunching Wrapper

Hits from one event have similar timestamp Combine hits to sets (bunches) which ll up GPU best

13


27/37




Bunching Wrapper


13

Hit


28/37




Bunching Wrapper


13

Hit Event


29/37




Bunching Wrapper


13

Hit Event


30/37




Bunching Wrapper


13

Hit Event

Bunch


31/37




Bunching Wrapper


13

Hit Event

Bunch

!(N2)!!(N)


32/37



Triplet Finder Bunching

Performance


33/37

DynamicParallelism




Compare kernel launch strategies

15

1 thread/bunchCalling

kernel

Triplet

Finder

1 thread/bunch

Callingkernel

1 block/bunch

Joinedkernel

1 block/bunch

Joinedkernel

TF Stage #1

1 stream/bunch

Combiningstream

1 stream/bunch

Callingstream

JoinedKernel

HostStreams

Triplet

Finder

Triplet

FinderCPU

GPU

TF Stage #1

TF Stage #1


34/37



Triplet Finder Kernel Launches

Preliminary

(in publication)


35/37



Triplet Finder Clock Speed / Chipset

Preliminary

(in publication)

K40 3004 MHz, 745 MHz/ 875 MHz

K20X 2600 MHz, 732 MHz/ 784 MHz

Memory Clock Core Clock GPU Boost


36/37



Summary

Investigated different tracking algorithms

Best performance: 20 s/event!Online Tracking a feasible technique for PANDA

Multi GPU system needed !(100) GPUs

Still much optimization necessary (effi

ciency) Collaboration with NVIDIA Application Lab

18


37/37


Summary

Investigated different tracking algorithms

Best performance: 20 s/event!Online Tracking a feasible technique for PANDA

Multi GPU system needed !(100) GPUs

Still much optimization necessary (effi

ciency) Collaboration with NVIDIA Application Lab

Thankyou!

AndreasHerten

[email protected]
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]

gpu implementations of online track finding algorithms at panda

Documents

onlinetracking panda

dpg frhjahrstagung

107s3raw data rate

reject background events

forschungszentrum jlich