1

Paraver internals and details

Jesus LabartaJesus.Labarta@bsc.es

2

Index

• Overview• Semantic module• Analysis module

2

3

overview

4

Paraver: Performance Data browser

Raw data

Performance index : s(t) (piecewise constant)Identifier of functionHardware countsMiss ratiosPerformance (IPC, Mflops,…)Routine duration• • •

tunable

StatisticsProfilesAverage miss ratio per routineHistogram of routine durationNumber of messages• • •

Seeing is believing

measuring is better

3

5

Performance views

• Few types of views

• Timelines

• Textual

• Statistics

• Profiles

• Histograms

• Configuration files

• Capture knowledge on how to generate view from raw data

• Loading configuration file pops up display or analysis window

• Distribution

• Set of basic view and stats provided with the tool.

• Also useful for

• Checkpointing of studies

• Cooperative work

• Bug reporting

• Recording your own favorite views

• Get used with the tool and to learn on the internals

6

Configuration files

Select directory

Select trace to which to apply

List of configuration files in current directory

Description of view of select fileNavigate through directory tree

4

7

TImelines

• Each window displays one view

• Piecewise constant function of time

• One such function of time per object:

• Thread, process, application, workload, CPU, node

• Types of functions

• Categorical

• State, user function, outlined routine

• Logical

• In specific user function, In MPI call, In long MPI call

• Numerical

• IPC, L2 miss ratio, Duration of MPI call, duration of computation burst

[ ] <⊂∈ nNSi ,n0,

[ )1,,)( +∈= iii ttiStS

RSi ∈

{ }1 ,0 ∈iS

8

Timelines

• Representation

• Function of time

• Color encoding

• Gradient color• Light green →Dark blue

• Not null gradient

• Black for zero value• Light green → Dark blue

5

9

Navigation

Right click

Hide lower panel

Draw icon at send points

Draw icon at receive points

Draw flag icon for events

Display as function or color encoded

Display animation

Draw lines for pt2pt communication

Zoom: select area with:• left click – middle click for zoom within same window• 2 left clicks for cloning zoomed window

select area with 2 left clicks (within same or between different windows)

Copy features from one window to another• Window size• Synchronize region of trace analyzed• Set same color gradient• Filter settings

10

Textual

•Appears when clicking on a timeline

•Textual display of

• Semantic value

• Events

• Communications

6

11

Views: Statistics

• 2D

• Profiles

• Histograms

• Correlations

• Communication Patterns

• 3D

MPI calls profile

Duration - IPC

While communicatingWhile computing

Control window

Data window

12

How to read profiles

Thre

ad

MPI call, user function,…

Value is a statistic (i.e. #occurrences, avg,) computed on Data window

Columns determined by categorical Control window

7

13

How to read histograms

Thre

ad

duration, instructions, BW, IPC, ...

Value is a statistic (i.e. #occurrences, avg,) computed on Data window

Columns correspond to bins of values of a numeric Control window

14

2D analysis module

max

min

• Color encoding

Fit color encodingMin and max to dynamic range of statisticBin definition

• Fit bin sizeTo cover the whole control window

dynamic range and generate at most 20 columns

Hide null columns

Color/not cells

Display whole table / cell text

Translate (.pcf)

Transpose

Show/hide lower panel

Region analyzed

Activate 3D analysis

8

15

2D analysis module

Display whole table colored / text for each cell

Text for cursor

Copy/paste analyzed time interval to from other 2D/timeline

Copy/paste color scale between 2Ds

Generate ASCII file with table data

Save configuration file

Right click

Open Control window Open Data window

Generate a timeline, derived form control window by zeroing values outside range selected by clicking in the table

Repeat the analysis (after changing statistics or data window)

Repeat the analysis for the whole trace

Perform a new analysis

(selecting new control window and

interval)

16

Semantic module internals

9

17

Filter module

• Dismiss filter window

• Show list of event types and values

• Communications that pass through the filter

• events that pass through the filter

• Show list of tasks

• Display window to which applies

18

Basic functions of time

• The filter module presents a subset of the trace to the semantic module. Each thread is described by

• A sequence of events , states and communications

• For each event let be its time and its value

• For each state let be its start time its stop time and its value

• For each Communication let be its time, its size and

• Semantic module builds

[ ) Ν∈∈= + itttiSts ii ,,),()( 1

NiEvi ∈,

)( iEvV)( iEvT

NiSti ∈,

)( is StT )( ie StT )( iStV

NiCi ∈,

)( iCT )( iCSz { }recvsendCDir i ,)( ∈

Function of time Series of values

Filter

Semantic

10

19

From events to functions

• Different possibilities

• Last event value

• Next event value

• Average Next Event Value

• Interval btw. Events

• Incoming bytes

• Bytes btw. events

• Bytes sent/received)()( iEvViS = [ ))(),()(|,)()( 1+∈=∑ iij

jj EvTEvTCTjCSziS

)()( 1+= iEvViS

)()()()(

1

1

ii

i

EvTEvTEvViS−

=+

+

)()()( 11 ++ −= ii EvTEvTiS

sizeInCommSS ii i .1 += −

20

The power of maths: Composition

• S’(t) = f(S(t)) S’ = f ° S

• Sign

• 1-sign

• Select range

• Sign ° Is equal

• Delta

• Stacked value

[ ] 0:)(?,)()(' tSbatStS ∈=

)0:)(?)(()(' tSatSsigntS ==

ii SStS −= + 1)('

))(()(' tSsigntS =

))((1)(' tSsigntS −=

11

21

Semantic module

•Derived windows

• Point wise operation

•S = α * Sa <op> β * Sb

•<op> : + , -, *, /, …

Loads

Stores

Mem Ops

L2 Line Loads

L2 miss ratio

x100

22

Semantic module

•Derived windows

• Point wise operation

•S = α * Sa <op> β * Sb

•<op> : + , -, *, /, …

Interval between MPI events In MPI call

MPI call duration

12

23

• How to build expression

Semantic module: derived windows

• Multiplying factor

24

Semantic module

• Angle:

• Process model

• Thread, task, application, workload

• Resource model

• CPU, node, system

fthreadfthreadfthreadfthreadfthreadfthreadfthread

Process view

Resource view

fprocess

fresource

13

25

• Semantic value: S(t)

• S = fcomp2 o fcomp1 o fWorkload o fApplication o ftask o Sthread

• Semantic functions

•fcomp2, fcomp1: sign, mod, div, in range, select range

•fApplication, fWorkload : add, average, max, select

•ftask : add, average, max, select

•Sthread: in state, useful, given state,

• last event value,

• next event value,

• average next event value

• interval between events, …

Semantic module: process model view

fAppl

ftask

fthread fthread fthreadfthread

ftaskftask

fthread fthread

fWorkload

fcomp1

fcomp2

26

• Sfresource = fcomp2 o fcomp1 o fSystem o fNode o fCPU o Sthread

• Semantic functions

•fSystem : add, average, max, select

•fNode : add, average, max, select

•fCPU : active thread, select

•Sthread: in state, useful, given state, next event value, thread_id

Semantic module: resource view

fNode

fCPU

fthread

fSystem

fthread

fthread

fthread

fthread

fthread

fCPU

fCPU

fNode

14

27

Semantic module: process model view

28

2D/3D internals

15

29

3D/2D analysis module

• Single flexible quantitative analysis mechanism• Let

• cw1 and cw2 two views we will call control views

• dw a view we will call data window

• For each control window we define a set of bins

• And the discriminator functions

• The 3D analysis module computes a cube (or plane in the case of 2D) of statistics

• Where the statistic can represent the average value, the number of intervals,….

)(*)()( 21, ttt cw

kcwjkj δδδ =

[ )cwj

cwj

cwj rangerangebin 1, += cwcw

jcwj deltarangerange +=+1

))(*)((),,( ,, ttSstatistickjthreadM kjthdwth δ=

)0:1?))((()( cwj

cwcwj bintSt ∈=δ

Identify regions with cw’s within the (j,k) bin

[ )wi

wi

wth

wth tttiStS 1,),()( +∈=

For each window w

30

2D analysis module

∫=end

start

t

t kjthdwth dtttSkjthIntegral )()(),,( ,,δ

∫∫

=end

start

end

start

t

t kjth

t

t kjthdwth

dtt

dtttSkjthAverage

)(

)()(),,(

,,

,,

δ

δ

∫=end

start

t

t kjth dttkjthTime )(),,( ,,δ

startend

t

t kjth

tt

dttkjthTime

end

start

−=∫ )(

),,(%,,δ

∑∫∫

=

j

t

t kjtn

t

t kjth

end

sctart

end

start

dtt

dttkjthoTimeNotZer

)(

)(),,(%

,,

,,

δ

δ

[ )endstartkjthdwth tttttSkjthMaximum ,)),()(max(),,( ,, == δ

endiendstartistartikjth

i

idwth ttiittiitiSkjthSumBurst end

start<=>==∑ |)max(,|)min(),()(),,(

,,δ

endiendstartistart

startend

ttiittiiiikjthNumBurst

<=>=+−=

|)max(,|)min(1),,(

16

31

Representation module

Seeing is believing

measuring is better

Functions of time

32

2D analysis

• Perform an analysis

• Select statistic

• View control window

• Analyzed area

• View data window

• Select data window

• Selecting new CW• On whole CW

• On whole trace (same CW)

• On same area

17

33

2D analysis module

max

min

• Color encoding• Fit color encodingMin and max to dynamic range of statistic

• Bin definition

• Fit bin sizeTo cover the whole control window

dynamic range and generate at most 20 columns

• Hide null columns

• Color/not cells

• Whole table / cell text

• Translate (.pcf)

• Transpose

• Show/hide lower panel• Text for cursor

• Region analyzed