process mining chapter_05_process_discovery

Chapter 5Process Discovery: An Introduction

prof.dr.ir. Wil van der Aalstwww.processmining.org

http://www.layoutsparks.com/myspace-layouts/footsteps+in++s_0�

Overview

PAGE 1

Part I: Preliminaries

Chapter 2 Process Modeling and Analysis

Chapter 3Data Mining

Part II: From Event Logs to Process Models

Chapter 4 Getting the Data

Chapter 5 Process Discovery: An Introduction

Chapter 6 Advanced Process Discovery Techniques

Part III: Beyond Process Discovery

Chapter 7 Conformance Checking

Chapter 8 Mining Additional Perspectives

Chapter 9 Operational Support

Part IV: Putting Process Mining to Work

Chapter 10 Tool Support

Chapter 11 Analyzing “Lasagna Processes”

Chapter 12 Analyzing “Spaghetti Processes”

Part V: Reflection

Chapter 13Cartography and Navigation

Chapter 14Epilogue

Chapter 1 Introduction

Process discovery

PAGE 2

software system

(process)model

eventlogs

modelsanalyzes

discovery

records events, e.g., messages,

transactions, etc.

specifies configures implements

analyzes

supports/controls

enhancement

conformance

“world”

people machines

organizationscomponents

businessprocesses

Process discovery = Play-In

PAGE 3

event log process model

Play-In

event logprocess model

Play-Out

event log process model

Replay

• extended model showing times, frequencies, etc.

• diagnostics• predictions• recommendations

Example

PAGE 4

a

b

c

de

p2

end

p4

p3p1

start

Event log contains all possible traces of model and vice versa.

Another example

PAGE 5

a

b

c

df

p2

end

p4

p3p1

start

e

p5

Generalization: event log contains only subset of all possible traces of model.

Notation is less relevant (e.g. BPMN)

PAGE 6

a

b

c

de

p2

end

p4

p3p1

start

a

start end

b

c

e

d

Another BPMN example

PAGE 7

a

b

c

df

p2

end

p4

p3p1

start

e

p5

a

start end

b

c

e

d

f

Challenge

• In general, there is a trade-off between the following four quality criteria:

1.Fitness: the discovered model should allow for the behavior seen in the event log.

2.Precision (avoid underfitting): the discovered model should not allow for behavior completely unrelated to what was seen in the event log.

3.Generalization (avoid overfitting): the discovered model should generalize the example behavior seen in the event log.

4.Simplicity: the discovered model should be as simple as possible.

PAGE 8

Process Discovery: example of algorithm

PAGE 9

α

>,→,||,# relations

• Direct succession: x>y iff for some case x is directly followed by y.

• Causality: x→y iff x>y and not y>x.

• Parallel: x||y iff x>y and y>x

• Choice: x#y iff not x>y and not y>x.

a>ba>ca>eb>cb>dc>bc>de>d

a→ba→ca→eb→dc→de→d

b||cc||b

abcdacbdaed

b#ee#bc#ea#d…

Basic Idea Used by α Algorithm (1)

a b

(a) sequence pattern: a→b


a

b

c

(b) XOR-split pattern:a→b, a→c, and b#c

a

b

c

(c) XOR-join pattern:a→c, b→c, and a#b

a

b

c



a

b

c

(d) AND-split pattern:a→b, a→c, and b||c

a

b

c

(e) AND-join pattern:a→c, b→c, and a||b

a

b

c


Example Revisited

PAGE 14

a

b

c

de

p2

end

p4

p3p1

start

Result produced by α algorithm

a>ba>ca>eb>cb>dc>bc>de>d

a→ba→ca→eb→dc→de→d

b||cc||b

b#ee#bc#ea#d…

Footprint of L1

PAGE 15

a

b

c

de

p2

end

p4

p3p1

start

Footprint of L2

PAGE 16

a

b

c

df

p2

end

p4

p3p1

start

e

p5

Simple patterns

PAGE 17

a b

(a) sequence pattern: a→b

a

b

c


a

b

c

(c) XOR-join pattern:a→c, b→c, and a#b

a

b

c


a

b

c

(e) AND-join pattern:a→c, b→c, and a||b

Algorithm

PAGE 18

Let L be an event log over T. α(L) is defined as follows. 1. TL = { t ∈ T | ∃σ ∈ L t ∈ σ}, 2. TI = { t ∈ T | ∃σ ∈ L t = first(σ) }, 3. TO = { t ∈ T | ∃σ ∈ L t = last(σ) }, 4. XL = { (A,B) | A ⊆ TL ∧ A ≠ ø ∧ B ⊆ TL ∧ B ≠ ø ∧

∀a ∈ A∀b ∈ B a →L b ∧ ∀a1,a2 ∈ A a1#L a2 ∧ ∀b1,b2 ∈ B b1#L b2 }, 5. YL = { (A,B) ∈ XL | ∀(A′,B′) ∈ XL

A ⊆ A′ ∧B ⊆ B′⇒ (A,B) = (A′,B′) }, 6. PL = { p(A,B) | (A,B) ∈ YL } ∪{iL,oL}, 7. FL = { (a,p(A,B)) | (A,B) ∈ YL ∧ a ∈ A } ∪ { (p(A,B),b) | (A,B) ∈

YL ∧ b ∈ B } ∪{ (iL,t) | t ∈ TI} ∪{ (t,oL) | t ∈ TO}, and 8. α(L) = (PL,TL,FL).

Key idea: find places

PAGE 19

a1

...

a2

am

b1

b2

bn

p(A,B) ...

A={a1,a2, … am} B={b1,b2, … bn}

4. XL = { (A,B) | A ⊆ TL ∧ A ≠ ø ∧ B ⊆ TL ∧ B ≠ ø ∧∀a ∈ A∀b ∈ B a →L b ∧ ∀a1,a2 ∈ A a1#L a2 ∧ ∀b1,b2 ∈ B b1#L b2 },

5. YL = { (A,B) ∈ XL | ∀(A′,B′) ∈ XLA ⊆ A′ ∧B ⊆ B′⇒ (A,B) = (A′,B′) },

Places as footprints

PAGE 20

a1

...

a2

am

b1

b2

bn

p(A,B) ...

A={a1,a2, … am} B={b1,b2, … bn}

a

b

c

de

p2

end

p4

p3p1

start

Another event log L3

PAGE 22

Model for L3

PAGE 23

a b

c

d

e

p({a,f},{b})iL

p({b},{c})

p({b},{d})

p({c},{e})

p({d},{e})

g

oLp({e},{f,g})

f

Another event log L4

PAGE 24

b

c

p({a,b},{c}) oL

a

iL e

d

p({c},{d,e})

Event log L5

PAGE 25

Discovered model

PAGE 27

b

p({a},{e})

oLaiL

c

e

f

d

p({e},{f})

p({b},{c,f})p({a,d},{b})

p({c},{d})

Limitation of α algorithm(implicit places)

PAGE 28

g

a

c

d

e

fb

p1

p2

Green places are implicit!

Limitation of α algorithm(loops of length 1)

PAGE 29

a c

b

a c

b

Limitation of α algorithm(loops of length 2)

PAGE 30

a b d

c

a

b

d

c

Limitation of α algorithm(non-local dependencies)

PAGE 31

b

c

a

e

dp1

p2

Green places are not discovered!

Difficult constructs for α algorithm

PAGE 32

a

b

c

Taking the transactional life-cycle into account

PAGE 33

start

assigned

assign

running

complete

a

start

running

complete

b

suspended

suspend

resume

start

assigned

assign

running

complete

c

Rediscovering process models

PAGE 34

The rediscovery problem: Is the discovered model N’ equivalent to the original model N?

discoveredprocessmodel

original process model event

log

simulate discover

N=N’ ?N N’

Equivalence: trace equivalence, bisimilarity, and branching bisimilarity

PAGE 35

s1

birth

s2

s3 s4

hell

curse

curse

curse

birth

curse

hell

s5

s6

s7

birth

curse

s8

s9

s11

curse

curses10

TS1 TS2 TS3

heaven

heaven heaven hell

heaven

hell

?

heaven

Three trace equivalent transition systems: TS1 and TS2are not bisimilar, but TS2 and TS3 are bisimilar

Branching bisimilarity defined for YAWL

PAGE 36

s1

TS1 TS2

start

check

end

reject accept

c1 c2

s2

acceptreject

s3 s4

s5

s6

check

s7

acceptreject

s8

τ τ

start

end

reject accept

c3

check check

?

TS1 and TS2 are not branching bisimilar (although trace equivalent).

Challenge: finding the right representational bias

PAGE 37

a a

start endp

There is no WF-net with unique visible labels that exhibits this behavior.

Another example

PAGE 38

a b

start endp1

c

τ

p1

(a)

a b

start endp1

c

p1

(b)

a

a b

start endp1

c

p1

(c)

There is no WF-net with unique visible labels that exhibits this behavior.

Challenge: noise and incompleteness

• To discover a suitable process model it is assumed that the event log contains a representative sample of behavior.

• Two related phenomena:−Noise: the event log contains rare and

infrequent behavior not representative for the typical behavior of the process.

− Incompleteness: the event log contains too few events to be able to discover some of the underlying control-flow structures.

PAGE 39

More on incompleteness

PAGE 40

See also chapter 3 (cross-validation, precision, recall, etc.)

Challenge: Balancing Between Underfitting and Overfitting

Challenge: four competing quality criteria

PAGE 42

process discovery

fitness

precisiongeneralization

simplicity

“able to replay event log” “Occam’s razor”

“not overfitting the log” “not underfitting the log”

Flower model

PAGE 43

g

ac

d

ef

b

start end

h

What is the best model?

A D

C

EB

A D

C

EB

ACDACEBCEBCD

990850

http://upload.wikimedia.org/wikipedia/commons/0/03/Green_check.svg�


A D

C

EB

A D

C

EB

ACDACEBCEBCD

99888578

http://upload.wikimedia.org/wikipedia/commons/0/03/Green_check.svg�


A D

C

EB

A D

C

EB

ACDACEBCEBCD

992853

Example: one log four models

PAGE 47

astart register

request

bexamine thoroughly

cexamine casually

d checkticket

decide

pay compensation

reject request

reinitiate requeste

g

hfend

astart register

request

cexamine casually

dcheckticket

decide reject request

e hend

N3 : fitness = +, precision = -, generalization = +, simplicity = +

N2 : fitness = -, precision = +, generalization = -, simplicity = +

astart register

request

bexamine

thoroughly

cexamine casually

dcheck ticket

decide

pay compensation

reject request

reinitiate request

e

g

h

f

end

N1 : fitness = +, precision = +, generalization = +, simplicity = +

astart register

request

cexamine casually

dcheckticket


e hend

N4 : fitness = +, precision = +, generalization = -, simplicity = -

aregister request

dexamine casually

ccheckticket


e h

a cexamine casually

dcheckticket

decide

e g

a dexamine casually

ccheckticket

decide

e g

register request

register request

pay compensation

pay compensation

aregister request

b dcheckticket


e h

aregister request

d bcheckticket


e h

a b dcheckticket

decide

e gregister request

pay compensation

examine thoroughly

examine thoroughly

examine thoroughly

… (all 21 variants seen in the log)

acdeh

abdeg

adceh

abdeh

acdeg

adceg

adbeh

acdefdbeh

adbeg

acdefbdeh

acdefbdeg

acdefdbeg

adcefcdeh

adcefdbeh

adcefbdeg

acdefbdefdbeg

adcefdbeg

adcefbdefbdeg

adcefdbefbdeh

adbefbdefdbeg

adcefdbefcdefdbeg

455

191

177

144

111

82

56

47

38

33

14

11

9

8

5

3

2

2

1

1

1

# trace

1391

process discovery

fitness

precisiongeneralization

simplicity

“able to replay event log” “Occam’s razor”

“not overfitting the log” “not underfitting the log”

Model N1

PAGE 48

acdeh

abdeg

adceh

abdeh

acdeg

adceg

adbeh

acdefdbeh

adbeg

acdefbdeh

acdefbdeg

acdefdbeg

adcefcdeh

adcefdbeh

adcefbdeg

acdefbdefdbeg

adcefdbeg

adcefbdefbdeg

adcefdbefbdeh

adbefbdefdbeg

adcefdbefcdefdbeg

455

191

177

144

111

82

56

47

38

33

14

11

9

8

5

3

2

2

1

1

1

# trace

1391

astart register

request

bexamine

thoroughly

cexamine casually

dcheck ticket

decide

pay compensation

reject request

reinitiate request

e

g

h

f

end

N1 : fitness = +, precision = +, generalization = +, simplicity = +

Model N2

PAGE 49

acdeh

abdeg

adceh

abdeh

acdeg

adceg

adbeh

acdefdbeh

adbeg

acdefbdeh

acdefbdeg

acdefdbeg

adcefcdeh

adcefdbeh

adcefbdeg

acdefbdefdbeg

adcefdbeg

adcefbdefbdeg

adcefdbefbdeh

adbefbdefdbeg

adcefdbefcdefdbeg

455

191

177

144

111

82

56

47

38

33

14

11

9

8

5

3

2

2

1

1

1

# trace

1391

astart register

request

cexamine casually

dcheckticket


e hend

N2 : fitness = -, precision = +, generalization = -, simplicity = +

Model N3

PAGE 50

acdeh

abdeg

adceh

abdeh

acdeg

adceg

adbeh

acdefdbeh

adbeg

acdefbdeh

acdefbdeg

acdefdbeg

adcefcdeh

adcefdbeh

adcefbdeg

acdefbdefdbeg

adcefdbeg

adcefbdefbdeg

adcefdbefbdeh

adbefbdefdbeg

adcefdbefcdefdbeg

455

191

177

144

111

82

56

47

38

33

14

11

9

8

5

3

2

2

1

1

1

# trace

1391

astart register

request

bexamine thoroughly

cexamine casually

d checkticket

decide

pay compensation

reject request

reinitiate requeste

g

hfend

N3 : fitness = +, precision = -, generalization = +, simplicity = +

Model N4

PAGE 51

acdeh

abdeg

adceh

abdeh

acdeg

adceg

adbeh

acdefdbeh

adbeg

acdefbdeh

acdefbdeg

acdefdbeg

adcefcdeh

adcefdbeh

adcefbdeg

acdefbdefdbeg

adcefdbeg

adcefbdefbdeg

adcefdbefbdeh

adbefbdefdbeg

adcefdbefcdefdbeg

455

191

177

144

111

82

56

47

38

33

14

11

9

8

5

3

2

2

1

1

1

# trace

1391

astart register

request

cexamine casually

dcheckticket


e hend

N4 : fitness = +, precision = +, generalization = -, simplicity = -

aregister request

dexamine casually

ccheckticket


e h

a cexamine casually

dcheckticket

decide

e g

a dexamine casually

ccheckticket

decide

e g

register request

register request

pay compensation

pay compensation

aregister request

b dcheckticket


e h

aregister request

d bcheckticket


e h

a b dcheckticket

decide

e gregister request

pay compensation

examine thoroughly

examine thoroughly

examine thoroughly

… (all 21 variants seen in the log)

Why is process mining such a difficult problem?

• There are no negative examples (i.e., a log shows what has happened but does not show what could not happen).

• Due to concurrency, loops, and choices the search space has a complex structure and the log typically contains only a fraction of all possible behaviors.

• There is no clear relation between the size of a model and its behavior (i.e., a smaller model may generate more or less behavior although classical analysis and evaluation methods typically assume some monotonicity property).

PAGE 52

Creating a 2-D slice of a 3-D reality

PAGE 53

Creating a 2-D slice of a 3-D reality: the process is viewed from a specific angle, the process is scoped using a frame, and the resolution determines the granularity of the resulting model

process mining chapter_05_process_discovery

Documents