mining constraints for artful processes

Mining Constraints for Artful ProcessesClaudio Di Ciccio and Massimo Mecella

Claudio Di Ciccio ([email protected])15th International Conference on Business Information Systems (BIS 2012)Tuesday, May the 22nd, Vilnius, Lithuania

Mining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

Process Mining

• Process Mining [Aalst2011.book], also referred to as Workflow Mining, is the set of techniques that allow the extraction of process descriptions, stemming from a set of recorded real executions (logs).

• ProM [AalstEtAl2009] is one of the most used plug-in based software environment for implementing workflow mining (and more) techniques.• The new version 6.0 is available for download at

www.processmining.org

Definition

BIS 2012, Vilnius

http://www.processmining.org/


Process Mining

• Process Mining involves:• Process discovery

• Control flow mining, organizational mining, decision mining;

• Conformance checking• Operational support

• We will focus on the control flow mining• Many control flow mining algorithms proposed

• α [AalstEtAl2004] and α++ [WenEtAl2007]• Fuzzy [GüntherAalst2007]• Heuristic [WeijtersEtAl2001]• Genetic [MedeirosEtAl2007]• Two-step [AalstEtAl2010]• …

Definition

BIS 2012, Vilnius


• Artful processes [HillEtAl06] informal processes typically

carried out by those people whose work is mental rather than physical (managers, professors, researchers, engineers, etc.)

• “knowledge workers”[ACTIVE09]

• Knowledge workers create artful processes “on the fly”

• Though artful processes are frequently repeated, they are not exactly reproducible, even by their originators, nor can they be easily shared.

Artful processes and knowledge workersA different context

BIS 2012, Vilnius


A different context

• In collaborative contexts, knowledge workers share their information and outcomes with other knowledge workers E.g., a software development

mgr.• Typically, by means of several

email conversations Email conversations are actual

traces of running processes that knowledge workers adhere to

Email conversations

BIS 2012, Vilnius


• From the collection of email messages, you can extract the processes that lay behind Related e-mail conversations are traces of their runs

• Valuable advantages for users Automated discovery of formal representations

• with no effort for knowledge workers Tidy organization for naïve best practices kept only in mind Opportunity to share and compare the knowledge on methodologies Automated discovery of bottlenecks, delays, structural defects

• from the analysis of previous runs• Email conversations are a kind of semi-structured text

this approach is not tailored to the electronic mail• it can be extended to the analysis of other semi-structured texts

Processes from email conversationsA different context

BIS 2012, Vilnius


MailOfMine

MailOfMine is the approach and the implementation of a collection of techniques, the aim of which is to is to automatically build, on top of a collection of email messages, a set of workflow models that represent the artful processes laying behind the knowledge workers’ activities.

[DiCiccioEtAl11] [DiCiccioMecella/TR12]

What is MailOfMine?

BIS 2012, Vilnius

From the e-mail archive to key partsMail archive Mail Database Conversations Key

Parts

Multi-format mail storageplug-in based crawlers

[ZardettoEtAl10]-basedclustering algorithm

[CarvalhoEtAl04]-based filter

The MailOfMine approach

BIS 2012, Vilnius Mining Constraints for Artful Processes (Di Ciccio, C. - DIAG, SAPIENZA Università di Roma)

From key parts to processes

Activityindicium

Tasks

Key PartsConcatenation

[ZardettoEtAl10]-based

Processes

[CohenEtAl04,SakuraiEtAl05]

-based task extractor


MINERful

Tasks

Processes



MINERful

MINERful is a workflow mining algorithm Its input is a collection of strings T and an alphabet ΣT

Each string t is a trace Each character is an event (enacted task) The collection represents the log

Its output is a declarative process model What is a declarative process model?

The mining algorithm in MailOfMine

BIS 2012, Vilnius


On the visualization of processesThe imperative model

• Represents the whole process at once

• The most used notation is based on a subclass of Petri Nets (namely, the Workflow Nets)

BIS 2012, Vilnius


On the visualization of processes

• Rather than using a procedural language for expressing the allowed sequence of activities, it is based on the description of workflows through the usage of constraints• the idea is that every task can be

performed, except the ones which do not respect such constraints

• this technique fits with processes that are highly flexible and subject to changes, such as artful processes

BIS 2012, Vilnius

The declarative modelIf A is performed,

B must be perfomed,no matter

before or afterwards(responded existence)

Whenever B is performed,C must be performed

afterwardsand B can not be repeated

until C is done(alternate response)

The notation here is based on [AalstEtAl06, MaggiEtAl11] (DecSerFlow, Declare)


On the visualization of processesImperative vS declarative

Imperative

Declarative

Declarative models work better in presence of a partial specification of the process scheme

BIS 2012, Vilnius


A real discovered process model“Spaghetti process” [Aalst2011.book]

BIS 2012, Vilnius


Constraint templates as Regular Expressions (REs)Declare constraint templates

BIS 2012, Vilnius


MINERful by example

A project meeting is scheduled We suppose that a final agenda will be committed

(“confirmAgenda”) after that requests for a new proposal (“requestAgenda”), proposals themselves (“proposeAgenda”) and comments (“commentAgenda”) have been circulated.

Shortcuts for tasks (process alphabet): p (“proposeAgenda”) r (“requestAgenda”) c (“commentAgenda”) n (“confirmAgenda”)

Scenario

BIS 2012, Vilnius


MINERful by example

Existence constraints1. Participation(n)2. Uniqueness(n)3. End(n)

Relation constraints4. Response(r,p)5.

RespondedExistence(c,p)

6. Succession(p,n)

The agenda1. must be confirmed,2. only once:3. it is the last thing to do.

During the compilation:4. the proposal follows a

request;5. if a comment circulates, there

has been / will be a proposal;6. after the proposal, there will

be a confirmation, and there can be no confirmation without a preceding proposal.

Constraints on tasks

BIS 2012, Vilnius


MINERful by example

In order to validate the algorithm We translate constraints into REs

• The overall process is expressed by the intersection of REs We use a RE-driven random string builder [Xeger] for creating a test-

and-validation set We analyze the result and evaluate the performances

In order to see how it works now We follow a run of MINERful over a string built by Xeger:

r r p c r p c r c p c n

Testing by replay

BIS 2012, Vilnius


MINERful by example

p p occurred 3 times in 1 string

γp(3) = 1• For each m ≠ 3

γp(m) = 0 p did not occur as the first nor as the last character

gi(p) = 0gl(p) = 0

n γn(1) = 1 For each m ≠ 1, γn(m) = 0 n occurred as the last character in 1 string

gi(n) = 0gl(n) = 1

Building the “ownplay” of p and n

BIS 2012, Vilnius




MINERful by example

With respect to the occurrence of p, n occurred…i. Never before: 3 times

δp,n(-∞) = 3ii. 2 char’s after: 1 time

δp,n(2) = 1iii. 6 char’s after: 1 time

δp,n(6) = 1iv. 9 char’s after: 1 time

δp,n(9) = 1v. Alternating:

i. Onwards: 2 timesb→

p,n = 2ii. Backwards: never

b←p,n = 0

Looking at the string

i. r r p c r p c r c p c n

ii. r r p c r p c r c p c n

iii. r r p c r p c r c p c n

iv. r r p c r p c r c p c n

v. i. r r p c r p c r c p c n

ii. r r p c r p c r c p c n

Building the “interplay” of p and n

BIS 2012, Vilnius


MINERful by example

-∞ -5 -2 +1 +2 +4 +5 +8 +9

δr,p 2 1 2 2 2 1 2 1 1

Building the “interplay” of r and p

b→r,p = 1

b←r,p = 0


BIS 2012, Vilnius


MINERful by example

Interplay and ownplay constitute the Knowledge Base of MINERful The KB construction is such that each new string adds information The algorithm does not need to read the strings more than once each

Constraints are determined by the evaluation of boolean queries on the KB

This allows the discovery of constraints with a faster procedure on a smaller set than the whole input

MINERful is a two-step algorithm

Workflow discovery by constraints inference

BIS 2012, Vilnius


MINERful by example

RespondedExistence(c,p) ¬(δr,p(0) > 0)

There is no string where p does not occur, if r is read Response(r,p)

RespondedExistence(r,p) ¬(δ∧ r,p(+∞) > 0)RespondedExistence(r,p) holds and there is no string where p does not follow r

Precedence(r,p) RespondedExistence(p,r) ¬(δ∧ r,p(-∞) > 0)

RespondedExistence(p,r) holds and there is no string where p does not precede r

Succession(p,n) Response(p,n) Precedence(p,n)∧

…

Some queries for inferring constraints

BIS 2012, Vilnius


Conclusions

MailOfMine is a system designed for mining artful processes out of email collections

MINERful is the worflow mining algorithm designed for MailOfMine

MINERful is Independent on the formalism used for expressing constraints Modular (two-phase) Capable of eliminating redundancy in the process model

BIS 2012, Vilnius

Recap


Conclusions

Linear w.r.t. the number of strings in the testbed|T|

Quadratic w.r.t. the size of strings in the testbed|tmax|

Quadratic w.r.t. the size of the alphabet|ΣT|

Hence, polynomial in the size of the inputO(|T|·|tmax |2·|ΣT |2)

BIS 2012, Vilnius

On the asymptotic complexity of MINERful


References

• [Aalst2011.book] van der Aalst, W.M.P.: Process Mining: Discovery, Conformance and Enhancement of Business Processes. Springer (2011).

• [AalstEtAl2009] van der Aalst, W.M.P., van Dongen, B.F., Güther, C.W., Rozinat, A., Verbeek, E., Weijters, T.: Prom: The process mining toolkit. In de Medeiros, A.K.A., Weber, B., eds.: BPM (Demos). Volume 489 of CEUR Workshop Proceedings., CEUR-WS.org (2009)

• [AalstEtAl2004] van der Aalst, W.M.P., Weijters, T., Maruster, L.: Workflow mining: Discovering process models from event logs. IEEE Trans. Knowl. Data Eng. 16(9) (2004) 1128–1142.

• [WenEtAl2007] Wen, L., van der Aalst, W.M.P., Wang, J., Sun, J.: Mining process models with non-free-choice constructs. Data Min. Knowl. Discov. 15(2) (2007) 145–180.

• [GüntherEtAl2007] Günther, C.W., van der Aalst, W.M.P.: Fuzzy Mining - Adaptive Process Simplification Based on Multi-perspective Metrics. BPM 2007: 328-343.

• [WeijtersEtAl2001] Weijters, A., van der Aalst, W.: Rediscovering workflow models from event-based data using little thumb. Integrated Computer-Aided Engineering 10 (2001) 2003.

• [MedeirosEtAl2007] Medeiros, A.K., Weijters, A.J., Aalst, W.M.: Genetic process mining: an experi- mental evaluation. Data Min. Knowl. Discov. 14(2) (2007) 245–304.

• [AalstEtAl2010] van der Aalst, W., Rubin, V., Verbeek, H., van Dongen, B., Kindler, E., Gnther, C.: Process mining: a two-step approach to balance between underfitting and overfitting. Software and Systems Modeling 9 (2010) 87–111 10.1007/s10270-008- 0106-z.

Cited articles and resources, in order of appearance

BIS 2012, Vilnius


References

• [HillEtAl06] Hill, C., Yates, R., Jones, C., Kogan, S.L.: Beyond predictable workflows: Enhancing productivity in artful business processes. IBM Systems Journal 45(4), 663–682 (2006)

• [ACTIVE09] Warren, P., Kings, N., et al.: Improving knowledge worker productivity - the active integrated approach. BT Technology Journal 26(2), 165–176 (2009)

• [DiCiccioEtAl11] Di Ciccio, C., Mecella, M., Catarci, T.: Representing and Visualizing Mined Artful Processes in MailOfMine. USAB 2011:83-94

• [DiCiccioMecella12] Di Ciccio, C., Mecella,M.: Mining constraints for artful processes. In W. Abramowicz, D. Kriksciuniene, V.S., ed.: 15th International Conference on Business Information Systems. Volume 117 of Lecture Notes in Business Information Processing., Springer (2012) (to appear).

• [DiCiccioMecella/TR12] Di Ciccio, C., Mecella, M.: MINERful, a mining algorithm for declarative process constraints in MailOfMine. Technical report, Dipartimento di Ingegneria Infor- matica, Automatica e Gestionale “Antonio Ruberti” – SAPIENZA, Universita` di Roma (2012).

• [AalstEtAl06] van der Aalst, W.M.P., Pesic, M.: Decserflow: Towards a truly declarative service flow language. Proc. WS-FM 2006

• [MaggiEtAl11] Maggi, F.M., Mooij, A.J., van der Aalst, W.M.P.: User-guided discovery of declarative process models. In: CIDM, IEEE (2011) 192–199

• [Xeger] http://code.google.com/p/xeger/

Cited articles and resources, in order of appearance

BIS 2012, Vilnius

mining constraints for artful processes

Documents

artful processes di

flythough artful processes

kinds of processes artful

roma4artful processes

specialized processes

workflow mining

powerpoint mining constraints

organizational mining