rick hull [email protected]

IBM Research

© 2008 IBM Corporation

Artifacts in Business Processes: Helping Workflow become Declarative -- or –- New Model New Questions

Rick [email protected]

Drawing on discussions and collaborations with Kumar Bhaskaran, Kamal Bhattacharya, Nathan Caswell, David Cohn, Christian Fritz, Santhosh Kumeran, Rong (Emily) Liu, Anil Nigem, Jianwen Su, Fred Wu, and others

CAISE keynote, 20 June 2008

IBM Research Watson2| Artifact-Centric Business Processes | 20 June 2008

Widely used approach to workflow design

Data and business objects are typically an afterthought Hard to evolve the workflow for new requirements Hard to re-use pieces of workflows to make new ones Hard to create a generic workflow with various specialization

(e.g., for different regions) Hard to manage workflows distributed across organizations

Workflow System(flow mgmt, services, databases, resources,

…)

Data Modeling

System inOperation

Ad hocimplementati

on

BusinessLogic

Process Modeling

We propose to re-think workflow … … at its very foundations


The notion of “business artifact” In practice, most business processes are centered

around key data objects which evolve over time, e.g., Sales invoice, book order, shopping cart Insurance claim Trouble ticket in IT support Monthly sales report Warehouse inventory Log of experiments in search of a new drug

These “business artifacts” have a macro-level life-cycle Shared across all artifacts of a given type Artifacts typically persist across much or all of the

workflow Workflow tasks typically focus on updating one or

two artifacts, possibly reading from othersArtifacts + macro life-cycle provide a natural skeleton for workflows which is relatively stable across evolution


Associations

Artifact-centric Workflow in a nutshell

Workflow Implementation(flow mgmt, services, databases, resources,

…)

Principledphysical realization

MacroLife-cycles

Data Modeling(Business)Artifact

s

Services

We obtain different workflow models by varying the data model underlying artifact schemas how services are specified, and how associations are made

Process Modeling(structured around artifacts,spread across Services and

Associations)


Artifact-centric approach forms basis of a current and growing IBM toolkit and professional services offering

Radically Simplified tools

SignificantRevenue Impact

Concepts and DesignImplementation(~40% efficiency

improvement)

tk-5

Create Itinerary

pending itineraryA-9

Flig

ht itin

era

ryB

-11

Ho

tel itin

era

ryC

-13

i tinerary requestA-15

cu

sto

me

rrec

ordB

-12

2

tk-19

Pending Itinerary

tk-24

Reserve Hotel

Hotel itineraryA-28processed hotelitineraryA-30

up

da

te p

en

din

gitin

era

ryA

-32

cancel hotelitenaryA-34

i

tk-36

Reserve Flight

Flight itineraryA-40processed flightitineraryA-42

up

da

te p

en

din

gitin

era

ryA-4

4

ca

nc

el

flig

ht

ite

na

ryA

-46

tk-49

Processed Hotel

tk-55

Processed Flight

tk-61

Confirm ItineraryconfirmeditineraryA-65

co

nfi

rme

dit

ine

rary

ev

en

tA

-67

processcompleted

itineraryA-69tk-72

CompletedItinerary

tk-76

Rejected Itinerary

Requester

ea-84

itine

rary

req

ue

st

A-8

8

itinerary resultA-86

tk-90

cancel itinerary

ca

nc

el h

ote

litin

era

ryA

-94

ca

nc

el flig

ht

itine

rary

A-9

6

reje

cte

d i

tin

era

ryA

-98

cancel hotelitenaryA-100

ca

nc

el

flig

ht

itin

era

ryA

-10

2

reje

cte

d i

tin

era

ryev

ent

A-1

04process

cancelleditineraryA-106

tk-118

Customer Record

Methodology Artifact-centric

modeling Transformation to

UML Mapping to

procedural representation

Code Generation Toolkit

WBM, RSA, WID WBM2UML transform UML2SOA transform

Applications Insurance Retail Procurement Pharmaceutical Finance . . .

This was achieved using an artifact-centric model with a very procedural way of associating services to tasks

We believe that shifting to declarative associations can bring rich benefits


Outline

Artifact-centric workflow models

Research challenges and directions

Conclusions


Running Example:Distributed Enterprise Services (DES)

IT service providerProviding IT services to a large enterpriseWhich has many “small sites”E.g., fast foods, hotels, car rentals

IT servicesTypical involve several or many stepsSteps might be performed by sub-contractors

(or “vendors”) Challenge: Find a systematic way to

manage the different service offeringsMay have varying number of tasksMay have regional differences (e.g., regulations)May deal with numerous vendors for the same

task


Focus in next few slides

Key artifacts in DES

Configuration Artifacts

ExecutionArtifacts

OfferedService

Generic Task

Schedule (for Service Order)

VendorTask

1

n

1

n

1 n

1 n

Customer

Site

1

n

Background Artifacts

Vendorn m

1 n

These artifacts essentially hold small programs

Artifacts can hold many forms of data . . .


ArchivedExecution (&

minor revision)

Task_ planning

(& refinement

Task_ approvals

Planning

ArchivedExecution (&

minor revision)

Schedule_ planning

(& Refinemen

t)

Schedule_ approvals

ExecutionPlanning

Re-approval

Major_ revision

Schedule

VendorTask

Macro Life-cycles for Schedule and Vendor Task

Conditions permitted on transitions Typically several services applied during each “stage” Hierarchical aspect permits scaling and one form of substitution


Going deeper into Offered Service

ER as well-known, convenient way to represent structure of data Physical implementation can be relational or other Can support different views for different kinds of stake-holders

Can use other models – XML, nested relation, name/value pairs

includes

n

m

OfferedService

Generic Task

precedenceoptional?

offered_serv_ID

description

typical_ duration

n

mk

Although using the ER model, we usually refer to the data values associate to an artifact as “attributes” of the artifact


Going deeper into Schedule

revision_checklist used to keep track of the work needed to finish the planning of this schedule

includes

Schedule

Vendor Task

schedule_ID

planned_ start_date

planned_ end_dateSite serves

Generic Task

optimality_factor

no_vendor_available

exec_status

approved_ for_exec

OfferedService based_

on

revision_ checklist

precedence1

m n

m

n k

1 n

1 n


We think in terms of a “soup” of services

For example: create_schedule ( Offered Service, Cust,

Site ) create_vendor_task ( Schedule, Generic Task) adjust_task_general ( Vendor Task ) adjust_task_dates ( Vendor Task ) request_task_govt_approvals ( Vendor Task ) . . .By starting with a view of services as separate from their sequencing, we

have better chance to understand their intrinsic properties

Data Modeling

Process Modeling(structured around artifacts)


Service specifications 1:à la Semantic Web Services, OWL-S

“IOPE”s Input parameters (artifacts and attributes) Output parameters (artifacts and attributes) Pre-conditions (Conditional) Effects

Allows to focus on the intention of the service Actual implementation considered at a lower

level


adjust_task_dates specified using IOPE (informal) Inputs:

Vendor Task artifact t, information about specific requirements for that customer and site, and about the current status of various steps (govt. approvals, equipment availability, etc.).

Vendor artifact v, and specifically information about v’s availability, about the cost for re-scheduling the task, etc.

Schedule artifact s, and specifically information about immediate predecessors and successors of t.

For each Vendor Task artifact t’ that is a descendant of t in s according to the precedence relationship, values of planned start and end dates.

Outputs: Updates to start and/or end dates of t. Updates to relevant parts of s concerning start/end dates for t. (Possibly) updates to the status fields of each Vendor Task artifact that is a descendent of t

in s. (Possibly) update revision_checklist of s

Pre-condition Task t is assigned to supplier v. stage(s) = schedule_planning or stage(s) = execution or stage(s) = major_revision

Conditional effects If true, then the start and/or end dates of t may be overwritten If the start date of t is overwritten, then it does not conflict with the dates of any

predecessor of t. If the start or end date of t is overwritten and this impacts the timing of any successor t’ of t,

then the dates for t’ are invalidated and s.revision_checklist is updated accordingly.Different logics and logic fragments will yield

different expressive power, different properties


Service specifications 2:Conditions in a more structured, pictorial representation

This is the style supported in IBM’s offering today

Vendor Tasksin stage

Task_Planning

adjust_task_dates

successor_tasks

Schedulesin stage

Schedule_Planning

Sitesin stageStable

preceding_tasks

All preceding tasks have valid

dates

Schedule is owner of primary

task

Schedule is for on this site


Associating Services to Artifacts 1:Procedural

Start with pictorial representation of services Use states for both macro- and micro-level

life-cycle of artifacts Add triggers so that entry into a state can cause

invocation of a service (If done correctly) this will induce a flow of

artifacts through the workflow Claim

AcceptedRejectedReview Needed

Discharged

Closed

Additional Data

Needed

Data Added

Investigation Not Required

Benefit Offered

AnalyzedRecordedCreatedStart Investigation Required

Prepare ClaimDischarge

RejectClaim

OfferBenefit

Review ClaimRejection

ValidateClaim

RecordClaim

AnalyzeClaim

Decideon Claim

NotifyClaim

ProvideAdd’l Data

Record BenefitPayment

This is the style used in IBM’s offering today


Associating Services to Artifacts 2:Event-Condition-Action (ECA) rules

R1: Create new schedule Event: request by performer p to create a schedule instance for

Offered Service artifact o, Customer artifact c, and Site artifact si, where offer_manager in role(p)

Condition: the appropriate non-disclosure agreements (NDAs) are in place for c

Action: invoke create_schedule(o, c, si) By: performer p where qualification(p, o, region: si.region) ≥ 5

R2: Move to schedule approval stage Condition: for Schedule artifact sch, sch is in stage

Schedule_planning, sch.revision_checklist is empty, and for each Generic task artifact g of sch, g has an associated Vendor task artifact t which has t.status = ready_for_execution.

Action: move_to(sch, Schedule_approval) By: automatic

. . . .

Unlike “pure” ECA, the artifacts and macro life-cycles provide a solid structure for the workflows


Associating Services to Artifacts 3:Goals+Constraints

Can we provide business stake-holders with something higher-level and broader than ECA rules

Illustrative examples (diff between goals/constraints is gray) “absolute constraint”: A task cannot start until its

predecessors have ended “absolute goal”: each task in schedule must

have optimality > 75 “preferred goal”: obtain the highest overall

optimality

The absolute constraints and goals might typically be captured using Variations on first-order logic and/or temporal logics, or OMG’s Semantic Business Vocabulary and Rules (SBVR)

We are exploring Various approaches to capture Goals+Constraints How to map from Goals+Constraints to ECA, procedural


Summary: Key options for artifact-centric WF models

Workflow Implementation(flow mgmt, services, databases, resources,

…)

Data Modeling

More systematic

implementation


MacroLife-

cycles

Ariifacts

Associations

Services

Triggers and flows

ECA

Goals + Constraints

Traditional procedural

Pictorial + conditions

IOPE

IBM’s current offering


A vision for a multi-tiered artifact-centric workflow framework

Declarative Specification(Goals+Constraints – SBVR?)

Declarative Specification(ECA)

Conceptual Realization(Procedural, optimized)

Physical Realization(DBs, queues, triggers, …)

•Business managers•Business analysts•IT architects

•Business analysts•IT architects•IT system engineers

•IT architects•IT system engineers•IT developers

•IT system engineers•IT developers•DBAs, …

A rich parallel

Withdatabase

mgmt

RelationalCalculus

SQL

Optimized algebraquery plan

Query planimplementation


Outline



Conclusions


Research Questions: Detailing the models

Workflow Implementation(flow mgmt, services, databases,

resources, …)

Data Modeling

systematicimplementation


MacroLife-

cycles

Artifacts

Associations

Services

Triggers and flows

ECA

Goals + Constraint

s

Traditional

procedural


IOPE

Is ER the “best” data model? Compare ER vs. XML vs. . . . Create preceise syntax/semantics for ECA and Goals+Constraints What is “best” approach to concurrency in ECA? For

Goals+Constraints? What logics are most useful for IOPEs, ECA, Goals+Constraints? How to capture precisely “preferred constraints”?


Research Questions: Analysis


resources, …)

Data Modeling

More systematic

implementation


MacroLife-

cycles

Artifacts

Associations

Services

Triggers and flows

ECA

Goals + Constraint

s

Traditional

procedural


IOPE

Analysis at and across kinds of associations ECA: reachability, termination, deadlock, … Goals + Constraints: same Goals+Constraints ECA: correctness ECA Procedural: correctnessPreliminary work [Bhattacharya et al, BPM 2008] In quite limited settings, reachability, etc, is NP-complete for CA

rules


Research Questions: Synthesis


resources, …)

Data Modeling

More systematic

implementation


MacroLife-

cycles

Artifacts

Associations

Services

Triggers and flows

ECA

Goals + Constraint

s

Traditional

procedural


IOPE

Given a set of Goals+constraints, Can you automatically generate ECA rules that correspond to

G? Can you automatically generate a procedural spec that

corresponds to G? Preliminary work [Fritz+H.+Su, in prep.]

In limited setting, can perform synthesis in 2EXPTIME


Research Questions: Understanding Generic / Specialization


resources, …)

Data Modeling

More systematic

implementation


MacroLife-

cycles

Ariifacts

Associations

Services

Triggers and flows

ECA

Goals + Constraint

s

Traditional

procedural


IOPE

What are design guidelines for different approaches? What is precise relationship – can each simulate the other?

Incrementality: When is incremental analysis more tractable than full analysis? When is incremental synthesis more tractable than full synthesis?

Use hierarchical aspect of state

machines

Use different services

Use different associations


Outline



Conclusions


Related Work (Selected) Artifact-centric and Adaptive Business Objects

[Nigam+Caswell 03, Kumaran+Nandi 04, Bhattacharya et al 07]

Pioneering combo of artifact + life-cycle as basis for workflow Document Engineering

[Glushgo+McGrath book] Like artifacts, but focused on exchange between organizations

Vortex [H. et al 99, H. et al 00] Similar to artifacts, services have declarative “guards”

Evolving Documents in Active XML [Abiteboul+Vianu 08] Services associated with leaves of XML documents; analysis

results Semantic Web Services, OWL-S

[McIlraith+Son+Zeng 01, H.+Su 05] Focus on automatic discovery, composition, invocation,

monitoring of services Workflow may be the “low-hanging fruit” for SWS techniques


An analogy to Relational Databases (à la Jianwen Su)

Before After

Data

base

sW

ork

flow

RelationalData Model

Physical Storage(files, indexes, …)

Declarative (SQL)

QueriesAutomated

Logical

Physical

Artifact Classes

Workflow Implementation

Tasks (Declarative)

Goals

(D

ecl

ara

tiv

e)

AutomatedLogical

Physical

Graph-based

Data Model

COBOL, IMS, …

Navigational

QueriesLogical

PhysicalManual

SequentialProcess

Modeling

Workflow System

Ad hoc Data Mgmt

Logical

PhysicalManual


University/Institute Collaborations << partial list >>

Active UC Santa Barbara University of Zurich FORTH in Crete UC San DiegoEmerging University of Rome – La Sapienza University of Balzano Penn State INRIAThere is lots of research to be done, and we are

eager to collaborate with academia/institutes


Summary Artifact-centric provides a new basis for

designing (and implementing) workflows that is Easy for business stake holders to understand Can enable flexibility in

Evolution Component re-use Generic / specialized

Has already shown its value in the field

Artifact-centric can support a spectrum from procedural to declarative workflow specification A declarative approach could dramatically simplify

workflow design, evolution, specialization, component re-use, . . .

This framework raises many research questions


Backup slides

In artifact-centric workflow, is the challenge of synthesizing procedural workflows from high-level goals+constraints a “low-hanging fruit” for techniques from (and advances to) semantic web services?


OWL-S (Formerly DAML-S)

An important framework to add semantics to web services: An upper ontology for describing properties & capabilities of web

services using OWL

Resource

Service

ServiceProfile

ServiceModel

ServiceGrounding

communication protocol (RPC, HTTP, …)

port number marshalling/serialization

input types output types preconditions effects

process flow composition hierarchy process definitions

providespresents

(what it does)

describedby(how it works) supports

(how to access)


OWL-S Atomic Process

To model impact on real world, this model builds on Situation Calculi (cf. also PSL) Use of “Fluents” to model real world, for pre-conditions, effects Use tree of situations to represent possible execution paths

Acct#

AmountConfirm#Debit_

Account

Account_Balance

Acct# Owner Balance

1234 Mary $500

. .

.

. .

.

. .

.

If balance of ACC# is Amount, then replace record using Balance – Amount and set Confirm# = newIf balance of ACC# is < Amount, then no-op

. . .

“Conditional Effect” “Real World”, or “Fluents”

Describe services using “IOPEs”

Input parameters Output parameters Pre-conditions (Conditional) Effects


Store

Ware-House

Bank

Impact on “real world”

Representative Semantic Web Services model --Colombo: “Signatures” that combine semantics and messaging behaviors [Berardi, Calvanese, De Giacomo, H., Marcella VLDB ’05]

“Real World”

Client(human or machine)

“View” of internal process model – guarded automata

– modeled as keyed relations

Messages between services