Supporting Early Decision-Making in the Presenceof Uncertainty

Jennifer Horkoff∗, Rick Salay†, Marsha Chechik†, and Alessio Di Sandro†∗Department of Information Engineering and Computer Science, University of Trento, Italy

horkoff@disi.unitn.it†Department of Computer Science, University of Toronto, Toronto, Canada

{rsalay,chechik,adisandro}@cs.toronto.edu

Abstract—Requirements Engineering (RE) involves eliciting,understanding, and capturing system requirements, which nat-urally involves much uncertainty. During RE, analysts chooseamong alternative requirements, gradually narrowing down thesystem scope, and it is unlikely that all requirements uncertaintiescan be resolved before such decisions are made. There is a needfor methods to support early requirements decision-making inthe presence of uncertainty. We address this need by describinga novel technique for early decision-making and tradeoff analysisusing goal models with uncertainty. The technique analyzes goalsatisfaction over sets of models that can result from resolvinguncertainty. Users make choices over possible analysis results,allowing our tool to find critical uncertainty reductions whichmust be resolved. An iterative methodology guides the resolutionof uncertainties necessary to achieve desired levels of goalsatisfaction, supporting trade-off analysis in the presence ofuncertainty.

I. INTRODUCTION

During the process of eliciting requirements, it is commonto encounter uncertainties, including gaps in domain knowl-edge, disagreements between stakeholders, or uncertainty overrequirement details. Early requirements elicitation typicallyuncovers a large space of alternative requirements and con-flicting needs, making it necessary to support decision-makingover alternatives in order to find acceptable trade-offs betweenconflicting requirements. Given the wide scope of early systemanalysis, it may not be feasible to resolve all uncertainty beforedecision-making occurs. Ignoring uncertainty forces implicitand premature decisions over the space of early requirements.Thus, methods and tools to (a) support early decision-makingand trade-off analysis and (b) do so in the presence ofuncertainty are needed. To support these needs, we makeuse of existing, established Requirements Engineering (RE)techniques: goal modeling and associated analysis [11], andthe MAVO framework for formally capturing and reasoningover model uncertainty [20].

Goal models (e.g., NFR [3], KAOS [4], i* [24]) have beenwidely studied in RE as a means to explicate system goals andhigh-level requirements. The ability to facilitate the analysis ofalternative requirements makes goal models especially power-ful for RE. For example, “what if?” analysis [3], [7], [11] usesthe effects on and relationships between goals to determinethe (degrees of) goal satisfaction resulting from a selection ofalternative requirements. Such analysis helps modelers select

a viable set of requirements, making acceptable trade-offsbetween user needs.

Current goal model analysis procedures evaluate the effectsof alternative requirements on user goals but do not accountfor model uncertainties discovered at design time. Much focushas been placed on capturing and reasoning over vague goals,using the softgoal concept to capture goals without clear-cutcriteria for success. However, in addition to being vague, goals(and associated relationships) can be uncertain, for example,we might be uncertain about the presence or absence of anelement. Ignoring uncertainty, i.e., treating the element aspresent, may lead us to eliminate viable alternatives or acceptalternatives which do not sufficiently satisfy key goals.

The MAVO framework [20] has been introduced in orderto explicitly and formally capture uncertainty in the contentsof models. MAVO is language-independent, and our previouswork, [18], [19], allowed us to record model changes,including change rationale, checking that these changes reduceuncertainty, and to propagate uncertainty reductions acrosstraceability links between different models. In this work, wetake the semantics of the target language (i*) into account,allowing us to determine the satisfaction of goals, evaluatealternative solutions, make tradeoffs and find satisfactory so-lutions – even in the presence of model uncertainty.

Specifically, we facilitate analysis and decision-making inthe presence of uncertainty by integrating goal model anal-ysis with uncertainty expressed using MAVO. We describea semi-automated method which finds sets of possible goalsatisfaction analysis results, accounting for all of the waysin which model uncertainty may be resolved. Our approachdetermines which uncertainties must be resolved to achievedesired levels of goal satisfaction, allowing for targeted do-main elicitation. We provide a methodology which supports aniterative reduction of uncertainty, moving towards “concrete”models in which uncertainties are resolved and desired levelsof goal satisfaction are achieved. Thus, we allow users to makedecisions over the space of early requirements even in thepresence of uncertainty.

A variety of goal modeling languages and analysis proce-dures have been introduced (e.g., [3], [24], [4], [7]). In orderto make our contribution concrete, we selected a particularlanguage, i* [24], and a particular method for qualitativeanalysis [11]. However, our approach can be generalized to

978-1-4799-3033-3/14 c© 2014 IEEE RE 2014, Karlskrona, Sweden

Accepted for publication by IEEE. c© 2014 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

33

AMeeting

Initiator

(MI)

Meeting

Participant

(MP)

Organize

Meeting

(OM)

Determine

Meeting

Date (DMD)

Low Effort

(LE)

Other Ways to

Organize

Meeting

(OWOM)

Dependencies

(Dep)D D

Unknown

Unknow

n

Attend

Meeting

(AM)

Participate

in Meeting

(PiM)

Provide

Details

(PrD)

Details

(D)

DD

Meeting

Scheduler

(MS)

Meeting be

Scheduled

(MBSD)

D

D

Schedule

Meeting

(SM)

Let Scheduler

Schedule

Meeting

(LSSM)

Meeting Be

Scheduled

(MBS)

HelpH

elp

Low Effort

(LEP)

Agreeable

Meeting

Date (AMD)

Decide

Convenient

Dates (DCD)

Convenient

Meeting Date

(CMD)

Use

Profiles

(UP)

?

Satisfied (FS)

Partially Satisfied (PS)

Conflict (C)

Unknown (U) N

Partially Denied (PD)

Denied (FD)

No Label (N)

?

Actor

Goal

Softgoal

Task

Resource

D

Help

Means-Ends

Decomposition

Contribution

Dependency

i* Legend

Initial

Analysis

Label

Actor

Boundary

Quick

(Q) ?

Fig. 1. A goal model for the Meeting Scheduler Example with analysis of the OWOM alternative.

any goal-oriented language with formal semantics. We do notmodel the intrinsic, run-time uncertainty of the environment,but the design-time uncertainty of the modeler, which can beeventually resolved. This type of uncertainty is best capturedpossibilistically rather than probabilistically as the statisticsneeded for the latter is typically not available at design time.

The rest of this paper is organized as follows: Sec. IIdescribes a motivating scenario and notation background usingthe Meeting Scheduler example. We provide the necessary for-mal background and notation in Sec. III. Sec. IV describes howto compute and interpret analysis labels over uncertain models.Sec. V describes a methodology for analysis of uncertainmodels, including finding suggested uncertainty refinementsgiven analysis choices. Sec. VI describes our tool support andexperiences. We compare our approach with related work inSec. VII. Sec. VIII summarizes the paper and discusses futurework.

II. BACKGROUND AND MOTIVATING SCENARIO

We motivate our approach using a well-known meetingscheduler example1. In this scenario, a Meeting Initiator hasa variety of ways to schedule meetings, including using a(semi)automated Meeting Scheduler, and must chose betweenscheduling alternatives. However, in the early analysis of thisscenario, we are uncertain about several aspects of the domain.What are some other ways for the Meeting Initiator to schedulea meeting? What information is needed from the MeetingParticipant? Who determines possible meeting dates? Despitethese uncertainties, we need to make decisions over the spaceof scheduling alternatives. We ask: Can we evaluate whether ornot an alternative is viable even in the presence of uncertainty?and What uncertainties must be resolved in order to facilitateour decision-making process?

1Parts of this scenario have appeared in previous work, e.g., [24], [18].

i* Language. Fig. 1 depicts an i* model, model A for themeeting scheduler capturing a simplistic view of the goals,actors and dependencies for this system. It does not yetexplicitly capture our uncertainties concerning the scenario.The social aspect of i* is represented by actors, e.g., MeetingInitiator (MI for short, see Fig. 1 for shorthands), and MeetingParticipant (MP). Actors depend upon each other for theaccomplishment of tasks, e.g., Provide Details (PrD), andDetermine Meeting Date (DMD), the provision of resources,e.g., Details (D), the satisfaction of goals, e.g., Meetingbe Scheduled (MBS) and softgoals, goals without clear-cutsatisfaction criteria, e.g., Low Effort (LE). Together, these arecalled intentions. Actor “boundaries” contain the intentionsexplicitly desired by these actors, e.g., MP wants to achieveParticipate in Meeting (PiM).

The interrelationships between the intentions are depictedvia three types of links: decomposition, representing intentionsnecessary to accomplish a task, e.g., to achieve PM the MPactor must satisfy AM, PrD, and satisfy the other three decom-position intentions; means-end, representing alternative taskswhich can accomplish a goal, e.g., Other Ways to OrganizeMeeting (OWOM) or Let Scheduler Schedule Meeting(LSSM) will satisfy MBS; and contribution, showing theeffects of softgoals, goals, and tasks on softgoals, e.g., LSSMhelps Quick (Q). Positive/negative contributions representingevidence sufficient to satisfice/deny a softgoal are representedby Make/Break links, respectively. Weaker contributions arerepresented by Help/Hurt links. Unknown contribution linksrepresent the presence of evidence with yet unknown polarity.

Goal Model Analysis. Model A allows the analyst to cap-ture requirement options, and existing analysis procedures,e.g., [11], allow her to evaluate such alternatives. The analysisprocedure starts with a question of the form “How effective isan alternative w.r.t. the goals in the model?”. In our example,

34

BMI MP

OM

Q LE

(S)

OWOM

(S)

(MS) Dep (MS) D (MS) D

(S) Unknown

(S)

Unknow

nAM

PiM

(MS) PrD

(comp)

(S) D

(MS

) D

(S) D

MS

MBSD

D

DSB

LSSM

MBS

HelpH

elp

LEP

(MS)

(V) AMD

(V)

(V) Date

Determiner

(DD)

(VM)

(MS)

(M) DCD

DMD

(M)

(VM) CMD

(M) UP

(M)

unknown set of other ways

to organize meetings

unknown

types of

contributions

unknown which actor

will perform this task

unknown

dependencies

unknown whether

these are actually

different goals

unknown if we

need this

unknown if

we need this

unknown what or

how many details

there will be

Fig. 2. A goal model for the Meeting Scheduler Example annotated with uncertainties informally (dashed notes) and formally (annotations).

there are two alternative ways to satisfy MBS: LSSM andOWOM. If we want to find the effects of selecting OWOM,we place initial labels, satisfying the task OWOM and denyingLSSM in model A. For completeness, other initial labels, re-flecting a more detailed analysis question, are chosen (shadedintentions in model A).

Initial labels are iteratively propagated through links usingpropagation rules for decomposition, means-ends, dependency,and contribution links, resulting in labels for other modelelements, representing the cumulative level of positive ornegative evidence. The propagation continues until no newlabels can be produced. In our example, initial labels forOWOM and LSSM are propagated through the means-end(OR) link. By this rule, MBS is (fully) satisfied (FS or , seelegend in Fig. 1). Similarly, incoming labels for PiM in MPare propagated through the (AND) decomposition, this timeselecting the minimum label, fully denied (FD).

Since there is a dependency relationship between D andPrD, a denied value for the latter implies a denied valuefor the former. Propagation through contribution links usesrules adopted from [3]. In order to capture partial evidencepropagated through contribution links, goal model analysiscaptures partial positive or negative evidence towards goalsatisfaction, via partially satisfied (PS) and partially denied(PD) labels, respectively. For example, the FD label for LSSMpropagates a PD label through the Help link to Q and LE.Similarly, the Unknown label (U) represents the presence ofevidence which could be positive or negative. For example,the FS value for OWOM propagates U through the Unknownlinks. In this case, U represents a missing piece of evidencethat should be addressed by users.

As softgoals often have multiple incoming contributionlinks, multiple labels might apply at once, e.g., Q is both U

and PD. Such cases can be combined into a single label forsubsequent propagation; possibly making use of the Conflictlabel, (C) indicating the presence of positive and negativeevidence of roughly the same strength. In this paper, wecombine evidence automatically: we select U when present,or C when evidence is both positive and negative, and other-wise select the minimum (most pessimistic) label. When anintention is the target of both a dependency link and anothertype of link (decomposition, means-ends, contribution), thepropagation results from both types of links are combined viaAND (taking the minimum label). The N (no label) symbol isused to explicitly indicate the absence of other labels.

Domain experts and analysts can determine whether theanalysis results are sufficient or viable. In our example, theanalysis indicates that the MI actor is unable to satisfy OM(FD), so OWOM is a non-viable alternative. In this case, themodeler would analyze other model alternatives (LSSM) orperform additional elicitation to find further alternatives.

Model Uncertainty. As mentioned, the scenario involvesseveral points of uncertainty. We summarize this uncertaintyusing free-form notes in model B in Fig. 2. We can use theMAVO method described in [18] to formally express theuncertainties in our model, shown in the same figure.

MAVO uses four types of annotations, each adding supportfor a different type of uncertainty in a model. The Mayannotation allows us to express uncertainty about the presenceof an element in a model. Annotating it with M indicates thatit “may exist”; otherwise, it “must exist”. E.g., in model B,we are unsure if the MP’s task DCD is present in the model.Uncertainty is reduced by removing the May annotation oreliminating the element altogether.

The Abs annotation allows a modeler to express uncertaintyabout the number of elements in the model. An S represents

35

TABLE IANALYSIS QUESTIONS OVER UNCERTAIN GOAL MODELS.

Q1 What are the analysis results given a particular analysis alternative in the goal model, considering model uncertainties?Q2 Can viable choices be made over the set of results from Q1?Q3 Can viable choices be achieved simultaneously? If so, find example uncertainty resolutions that achieve choices.Q4 Given choices, what uncertainty reductions are forced? How can we target elicitation?Q5 Are uncertainty reductions suggested by Q3 reasonable given the domain? If not, find further uncertainty reductions.

TABLE IISELECTED PROPAGATION RULES.

Link Type Original Rule Example MAVO RuleDependency

isidD (v ∈ V) v(is)⇒ v(id) ∀t : Task, g : Goal · (Dep(t, g) ∧ FS(g))⇒ FS(t)

Decomposition

i1 in…

id(∧n

j=1 FS(ij))⇒ FS(id) ∀t1 . . . tn : Task, g : Goal · (Decomp(t1 . . . tn, g) ∧∧n

j=1 FS(tj))⇒ FS(g). . . . . .(∨n

j=1 FD(ij))⇒ FD(id) ∀t1 . . . tn : Task, g : Goal · (Decomp(t1 . . . tn, g) ∧∨n

j=1 FD(tj))⇒ FD(g)

Contributionidis c

(c = Make) FS(is)⇒ FS(id) ∀t : Task, g : Goal · (Make(t, g) ∧ FS(t))⇒ FS(g)(c = Help) FS(is)⇒ PS(id) ∀t : Task, g : Goal · (Help(t, g) ∧ PS(t))⇒ PS(g). . . . . .(c = Unk, v ∈ V) v(is)⇒ U(id) ∀t : Task, g : Goal · (Unknown(t, g) ∧ v(t))⇒ U(g)(c ∈ {Make,Help . . .}) U(is)⇒ U(id) ∀t : Task, g : Goal · (c(t, g) ∧ U(t))⇒ U(g)

a “set”; omitting the annotation means that the element is“particular”. E.g., the S-annotated operation OWOM in actorMI in model B indicates that there are some such operationsbut they are as yet unknown. Uncertainty is reduced byelaborating the content of S elements into a set of (possiblyannotated) elements.

The Var annotation allows a modeler to express uncertaintyabout the distinctness of individual elements in the modelby annotating an element as a “variable” (V); omitting theannotation means that the element is a “constant”. Uncertaintyis reduced by merging variable elements with constants orother variables. E.g., while it is known that task DMD shouldbe in model B, it is not yet clear which actor should perform it.Yet it must be assigned to some actor for i* well-formedness.We thus put the task in a V-annotated actor, allowing it tomerge with other actors and eventually be assigned to a “real”(constant) actor.

The OW annotation allows a modeler to explicitly statewhether her model is incomplete (i.e., can be extended) (INC )or complete (the default). Here the annotation is at the level ofthe entire model rather than individual elements. Our modelB is complete.

Analysis with Uncertainty. We want to be able to make earlydecisions over alternative requirements, even in the presenceof uncertainty. Given our explicit recording of uncertainties inthe modeling process, we want to know how their presenceaffects analysis results (Fig. 1), and how such uncertaintiescan be used to guide analysis and elicitation. Specifically, (Q1)how do the uncertainties in model B affect analysis results inmodel A? For example, if we are not sure about the presenceof PrD (labelled with FD), perhaps the MP actor may be ableto satisfy PiM (i.e., this task could be FS). If the MP actor cansatisfy PiM, then the satisfaction of OM in the MI actor mayactually be unknown (U). We wish to see all of the possiblelabels given possible uncertainty reductions in the model.

Given the answer to Q1, we wish to answer additionalquestions as listed in Table I. The rest of this paper describes a

method which takes uncertainty into account when analyzinggoal models, allowing us to answer these questions.

III. PRELIMINARIES

In this section, we give the necessary formal background ongoal model analysis and uncertainty via the MAVO framework.

A. Goal Model Analysis

The forward propagation algorithm [11] begins by applyinginitial labels corresponding to the analysis question to themodel, queuing the labels. All labels in the queue are prop-agated through outgoing links, with multiple incoming labelsresolved using human judgment as in [11] or, in our case, rulesas described in Sec. II. Propagation ends when the label queueis empty (see [11] for a discussion of termination).

The above procedure uses six predicates over modelelements corresponding to the six analysis labels (V ={FS(),PS(),C(),U(), PD(),FD()}), where the predicateholds if the label applies.

Initial Analysis Labels. A selected subset of intentionswithin a goal model are assigned initial analysis labels, e.g.,FS(OWOM) and FD(LSSM) in Fig. 1.

Propagation Constraints. The procedure provides rules inorder to facilitate a standard propagation of labels through goalmodel relationships (links). The Original Rule column of TableII presents selected propagation rules for the dependency,decomposition, and contribution relationships depicted in theLink Type columns. Dependency links propagate evidenceunchanged, decomposition combines evidence using “AND”(minimum), while means-ends combines evidence using “OR”(maximum). The following order between labels is used:(FS ≥ PS ≥ U ≥ C ≥ PD ≥ FD). Furthermore,FS(i)⇒ PS(i) and FD(i)⇒ PD(i), where i is an element inthe model.

Means-end rules (omitted from Table II) are derived fromdecomposition rules by replacing

∧with

∨and vice versa.

Contribution links can weaken or negate evidence, or makeevidence unknown, depending on the incoming label, (v(is)),

36

and the type of contribution link. For example, the secondto last row of the table says that for an Unknown (Unk)contribution link, regardless of the incoming analysis label(v(is)), the U label is propagated (U(id) holds).

B. Capturing Uncertainty with MAVO

MAVO [20] is an approach for adding uncertainty informa-tion as annotations in models. Although MAVO can be appliedto any type of goal model, this exposition uses i* models. AMAVO i* model contains annotations on the model elementsthat together represent a set of different possible concrete (i.e.,ordinary) i* models that would resolve the uncertainty:

Definition 1 (MAVO i* model): A MAVO i* model Mconsists of an i* base model, denoted bs(M), and a setof annotations. [M ] denotes the set of i* models called theconcretizations of M . M is called consistent iff it allows atleast one concretization, i.e., [M ] 6= ∅.Formalizing MAVO. We now describe how MAVO annotationsformally characterize a set of i* models – it is the foundationof the method for encoding and automatically reasoning withMAVO models. The i* metamodel represents the set of well-formed i* models and can be expressed as a First Order Logic(FOL) theory.

Definition 2 (i* metamodel): The i* metamodel is an FOLtheory T = 〈Σ,Φ〉, where Σ is the signature with sortsrepresenting the entity types (e.g., Actor, Intention, Task) andpredicates representing the relations (e.g., Task decomposesGoal); Φ is a set of sentences representing the i* well-formedness constraints (e.g., multiplicities, dependencies be-tween actors). The models that conform to T are the finiteFO Σ-structures that satisfy Φ according to the usual FOsatisfaction relation. We denote this set of models by Mod(T ).

Like the i* metamodel, a MAVO i* model also representsa set of models and thus can also be expressed as an FOLtheory. Specifically, for a MAVO i* model M , we construct atheory FO(M) s.t. Mod(FO(M)) = [M ]. We illustrate theconstruction of FO(B) for MAVO i* model B in Fig. 2.

(1) Let G = bs(B) be the base model of model B. We definea new MAVO i* model BG with G as its base model and itssole concretization, i.e., bs(BG) = G and [BG] = {G}. Wecall BG the ground model of B.

(2) To construct the FOL encoding of BG, FO(BG), weextend the i* metamodel to include a unary predicate for each

ΣG has unary predicates MP(Actor), AM(Task)), . . . ,and binary predicates AMinMP(Task,Actor), . . .

ΦG contains the following sentences:(Complete) (∀x : Actor ·MP(x) ∨MS(x) ∨ DD(x) ∨ . . .)∧

(∀y : Task, x : Actor · in(y, x)⇒ (AMinMP(y, x) ∨ . . .)) ∧ . . .MP:

(ExistsMP) ∃x : Actor ·MP(x)(UniqueMP) ∀x, x′ : Actor ·MP(x) ∧MP(x′)⇒ x = x′

(DistinctMP−MS)∀x : Actor ·MP(x)⇒ ¬MS(x)(DistinctMP−DD)∀x : Actor ·MP(x)⇒ ¬DD(x)(DistinctMP−MI) ∀x : Actor ·MP(x)⇒ ¬MI(x)

similarly for all other element and relation predicates

Fig. 3. The MAVO predicates and constraints for G.

element in G and a binary predicate for each relation instancebetween the elements in G. Then, we add constraints to ensurethat the only first order structure that satisfies the resultingtheory is G itself:

FO(BG) = 〈Σ ∪ ΣG,Φ ∪ ΦG〉 (see Def. 2)

where ΣG and ΦG are model G-specific predicates and con-straints, defined in Fig. 3. We refer to ΣG and ΦG as the MAVOpredicates and constraints, respectively.

The FO structures that satisfy FO(BG) are those i* modelsthat satisfy the constraint set ΦG in Fig. 3. Assume that K isone such i* model. The MAVO constraint Complete ensuresthat K contains no more elements or relation instances thanG. Now consider the actor MP in G. ExistsMP says that Kcontains at least one actor called MP, UniqueMP – that itcontains no more than one actor called MP, and the clausesDistinctMP−∗ – that the actor called MP is different from allthe other actors. Similar MAVO constraints are given for allother elements and relation instances in G. These constraintsensure that FO(BG) has exactly one concretization and thusK = G.

(3) We construct FO(B) from FO(BG) by removing con-straints corresponding to the annotations in B. This constraintrelaxation allows more concretizations and thus representsincreasing uncertainty. If B were incomplete, we could expressthis fact by removing the Complete clause from ΦG, therebyallowing concretizations to be those i* models that extend G.Similarly, expressing the effect of the M, S and V annotationsfor an element E corresponds to relaxing ΦG by removingExistsE , UniqueE and DistinctE−∗ clauses, respectively. E.g.,removing the DistinctDD−∗ clauses expresses the V annotationon the actor DD (i.e., it may or may not be distinct fromanother actor).

IV. ANALYSIS OF GOAL MODELS WITH MAVOUNCERTAINTY

In this section, we describe our key contribution, thecombination of goal model analysis with MAVO uncertaintyannotations, “lifting” goal model analysis from conventionalto uncertain goal models.

A. Correctness Condition for MAVO Goal Model Analysis

A MAVO goal model represents a set of goal models (i.e.,its concretizations), whereas goal model analysis applies onlyto a single goal model. Thus, correct analysis results over aMAVO goal model should aggregate the analysis results overall of its concretizations:

Definition 3 (Correct Labeling): Let M be a MAVO goalmodel. M is correctly labelled iff for all intentions I in M ,label v is assigned to I iff ∃m ∈ [M ], i ∈ Im · v(i), where Imis the set of intentional elements mapped to I in m.Thus, a label gets applied to an intention when there is aconcretization that has that label on the intention. Model Cin Fig. 4 shows a correct analysis labeling of our examplemodel from Fig. 2. Here we use a shorthand notation forsets of possible labels, e.g., N for {FS, FD, N } in PiM.

37

C

MIMP

OM

Q LE

(S)

OWOM

(S)

(MS) Dep (MS) D(MS) D

Unknown

Un

kn

ow

n

AM

PiM

(MS) PrD

(comp)

(S) D

(MS) D

(S) D

MS

D

DSM

LSSM

MBS

Help

Help

LEP

(MS)

(V) AMD

(V)

(V) DD

(VM)

(M) DCD

DMD

(M)

(VM) CMD

(M) UP

(M)

MBSD

(MS)

N

N

N

N

N

N

N

N

?

??

Forced Uncertainty Reduction

(M)

Fig. 4. Analysis of an alternative considering model uncertainty showing forced uncertainty reductions (result of Q1).

Thus, the FS label in PiM means that there exists at least oneconcretization in which at least one concrete intention mappedto PiM has label FS. An example of such a concretization isone in which PiM has the two sub-intentions AM and AMDwith labels FS (they are initial labels indicated by the shading)and the remaining sub-intentions removed (possible becausethey are M-annotated). In this concretization, PiM gets thepropagated label FS.

B. Constructing the MAVO Goal Analysis Encoding

In Sec. III-B, we described construction of an FOL encodingFO(M) for every MAVO model M , whose satisfying instancesare exactly the concretizations of M (see Fig. 3 for an exam-ple). We can incorporate goal analysis into this formalizationby adding constraints that express the initial conditions andpropagation rules as described in [11]. We define this extendedFOL encoding as follows:

Definition 4 (Extended Encoding): Let M be an i* MAVOmodel, FO(M) = 〈Σ∪ΣM ,Φ∪ΦM 〉 be its FOL encoding andIntention ∈ Σ its sort for intentions. The extended encodingFOe(M) is a tuple FOe(M) = 〈Σ∪ΣM ∪Σlabel,Φ∪ΦM ∪Φl ∪ Φp〉, where Σlabel = {FS(Intention), . . . ,N(Intention)}are unary predicates representing the possible analysis labelson intentions, Φl is the set of constraints encoding initialanalysis values and Φp is the set of constraints encoding theanalysis propagation rules. We define Φe

M = Φ∪ΦM∪Φl∪Φp.The above definition ensures that each instance of FOe(M)

is a concretization of M with the analysis labeling that wouldresult from performing goal analysis on it with the initiallabeling defined by Φl. Thus, it guarantees that M satisfies theCorrect Labeling condition (Def. 3) and we can use FOe(M)to reason about the analysis results on the concretizations ofM . Specifically, we use it to answer the analysis questionsposed over our example model in Sec. II.

Initial Analysis Labels (Φl). Goal model analysis starts with aset of initial labels representing the analysis question (e.g., theFS label applied to OWOM and FD applied to LSSM, i.e., theshaded intentions in model A in Fig. 1). On the MAVO modellevel, we interpret these labels as follows: if an intention i ina MAVO goal model is given an initial label, then all concreteintentions mapped to i must also have this label. A set ofconstraints Φl expresses this condition. For example, to saythat the initial FS label is applied on OWOM, (FS(OWOM)),we add the constraint ∀i : Intention · OWOM(i) ⇒ FS(i) toΦl.

We enforce the constraint that all leaf intentions (intentionswith no incoming links) which are not explicitly assignedan initial label must be assigned an explicit N (none) label.Similarly, we enforce a constraint that each intention in theconcrete model must have only one analysis label. These initialconstraints are added to Φl.

Propagation Constraints (Φp). Φp consists of the FOLencoding of the propagation rules summarized in Sec. III-Aand described in [8]. For example, if an FS-labelled task isconnected to a goal by a Make contribution link, the goalshould also get label FS. This rule is encoded in the originalgoal model analysis using the predicate FS(is) ⇒ FS(id).In order to encode it using MAVO, we add the constraint∀t : Task, g : Goal · (Make(t, g) ∧ FS(t)) ⇒ FS(g), whereTask and Goal are the sorts for tasks and goals, respectively,for the MAVO model being encoded. We call the set ofall such propagation constraints Φp. Similar constraints areadded for the propagation of other labels on other elementtypes through other links (see the third column of Table IIfor more examples). The full set of propagation constraintsin our encoding covers all combinations of element types,relationships, and intentions.

38

C. Determining Possible Analysis Labels

We can use our encoding to find analysis results given a par-ticular alternative (set of initial labels) and model uncertainty,answering Q1. By Def. 3, a label can hold for an intention ina MAVO model iff there exists a concretization containing acorresponding concrete intention with that label. The followingproposition shows we can use the extended encoding FOe(M)to check this condition.

Proposition 1 (Analysis Labeling): Let M be a MAVO i*model with an analysis label set assigned to each intention. Mis correctly labelled according to Def. 3 iff for all intentions Iin M , label v is assigned to I iff Φe

M ∪ (∃i : Intention · I(i)∧v(i)) is satisfiable.For each possible labeling of intention I with label v, thischecks whether the labeling is consistent with Φe

M . If so, theremust exist a concretization with this labeling. Checking thisfor all labellings ensures that condition in Sec. 3 is met. Forexample, for model C , to check whether PiM should havelabel FS, we ask whether Φe

C∪(∃i : Intention ·PiM(i)∧FS(i))is satisfiable. It is satisfiable since there is a concretization (seeSec. 3) in which PiM gets the propagated label FS. Performingthis check for every possible combination of model elementand analysis label allows us to find results across the entiremodel, as we do in model C.

V. REASONING METHODOLOGY

In our motivating scenario, we raised five analysis questions(Table I). We have shown how to answer Q1, finding possibleanalysis labels given uncertainty, in Sec. IV-C. In this section,we describe how the combination of goal model analysis andMAVO uncertainty can be used to answer Q2-Q5, providingan analysis methodology guiding users in asking the questionsand interpreting the answers. We summarize our methodologyin Fig. 5. Steps which require manual judgment are indicatedby the presence of an actor.

Answering Q2: Making Choices. Given the set of analysisresults over the uncertain model, users can select a subset ofthe results that they consider acceptable. We call this subsetchoices. In our example, the fully satisfied ( ) value is themost desired for PiM in Model C, while unknown ( ) valueis the most desired for OM (circled in model C).

Answering Q3: Checking Simultaneous Achievablilityof Choices. Once analysis choices have been selected, wecan use the extended encoding to check if such choices aresimultaneously achievable in one or more concretizations.This is done by encoding the desired label choices asconstraints to require that concretizations simultaneouslyachieve these labels. For example, for model C, our choicesconsist of U for OM and FS for PiM and so we define thefollowing constraints: (1) ∀i : Intention · OM(i) ⇒ U(i) and(2) ∀i : Intention · PiM(i) ⇒ FS(i). We call the set of allsuch constraints Φc and use it to check the existence of aconcretization satisfying these choices.

Pose Analysis Question (Alternative)

Start

(Q1) Apply Analysis

(Q4) Find Forced Uncertainty Reductions

[Acceptable choices found]

[Forced reductions acceptable]

(Q2) Make choices

Viable Alternative Found

(Q5) Evaluate Uncertainty Reductions

[Choices achieved with certainty]

(Q3) Check Simultaneous Achievability

[Choices simultaneously achievable]

[Further analysis/

elicitation]

Fig. 5. Goal model analysis methodology with actors indicating manualtasks.

Proposition 2 (Choice Achievablility Check): Given aMAVO model M and a constraint set Φc encoding a set ofchoices, this set of choices is simultaneously achievable iffΦe

M ∪ Φc is satisfiable.For model C, checking the constraints Φe

C∪Φc shows that ourlabel choices (U for OM and FS for PiM) are simultaneouslyachievable.

Answering Q4: Finding Forced Uncertainty Reductions.Once the users check their choices for simultaneous achiev-ablility, we can use the approach defined in [21] to determinewhich uncertainties are forced, i.e., those which must beresolved in all concretizations in order to achieve analysischoices. In our example, the forced uncertainty reductionsare those related to PrD. In this case, the decomposition linkmust be removed for our choices to be achieved. We indicatethis in model C by making the link and the corresponding Muncertainty annotation red and bold. Such information allowsusers to target elicitation on a subset of uncertainties in themodel. In our example, users judge this forced resolution tobe acceptable, and continue with the analysis process.

Answering Q5: Evaluating Uncertainty Reductions. As anoutcome of checking for simultaneous achievablility in Q3, weproduce a concrete model with uncertainties resolved such thatchoice values are achieved. Model D in Fig. 6 is a particularconcretization for our example. It shows the meeting schedulerexample after a set of uncertainty reductions (in bold) whichallow achievement of the chosen analysis results. For example,the M-annotated task PrD and the uncertainty concerning UPhave been removed when compared to model C.

In our example, users judge some, but not all, of thesuggested uncertainty reductions in D to be consistent withthe domain. For example, it is deemed acceptable that the MP

39

DMI MP

OM

Q LE

OWOM

Dep D D

UnknownU

nkn

ow

nAM

PiM

D

D

MS

MBSD

D

D SM

LSSM

MBS

HelpH

elp

LEP

AMD

DCD

CMD

UP

Removed in concretization

Uncertainty Removed

DD

DMD

N

N

N

N

??

?

Fig. 6. One possible concretization achieving analysis choices in Model C in Fig. 4 (result of Q3).

E

MIMP

OM

Q LE

(S)

OWOM

(S)

(MS) Dep (MS) D(MS) D

Unknown

Un

kn

ow

n

AM

PiM

(comp)

(S) D

(S) D

(S) D

MS

D

DSM

LSSM

MBS

Help

Help

LEP

(V) AMD

(V)

(V) DD

(V)

DCD

DMD

(V) CMD

UP

MBSD

??

?

Fig. 7. Chosen and forced uncertainty reductions achieving analysis choices (results of Q5, reapplying Q1).

actor will not perform the task PrD (removed in D); instead,she can keep track of participant details using the UP task. Inanother example, it is deemed unacceptable for the dependencylink from resource D to be removed from the model altogether.This link should now depend on UP instead of the removedPrD. Further uncertainties are accepted or rejected and themodel is updated to reflect these decisions (uncertaintiesare removed, model elements are changed, analysis (Q1) isreapplied).

When iterating over analysis results, different constraintscan be added in order to determine if concretizations satisfyingthem exist, producing one such concretization. For example,when finding concretizations for model C, we may want tospecify that actors DD and MP cannot be merged together,expressed by the property Ψ. Checking Φe

C∪Φc∪{Ψ} producesa concretization in which Ψ holds.

An iterative process of finding concrete models, constrain-ing the problem and accepting or rejecting suggested uncer-tainty reductions will lead, in the optimal case, to a model inwhich choices are achieved with certainty and a viable set ofalternative requirements is found. Otherwise, the model maynot contain viable alternatives, prompting further elicitation.Alternative outcomes are discussed in Sec. VI.

In our running example, iterative analysis directed us toreduce targeted uncertainty in the model and the domain suchthat we could achieve our analysis choices with certainty. Weshow the final model resulting from this process, model E,in Fig. 7. In the model without uncertainty, A, we rejectedthe alternative that included OWOM. Yet considering modeluncertainty, as we did in model C, made this alternativeviable. We have determined the viability of this alternativewithout the need to remove all model uncertainty. Applying

40

this methodology iteratively can enable exploring the viabilityof other alternatives in the model (e.g., satisfying LMSM),allowing for a final selection between alternative requirements.

VI. EXPERIENCE AND DISCUSSION

Implementation. We have implemented the automatable ques-tions in our methodology (Q1, Q3, Q4), building on topof Model Management Tool Framework (MMTF) [17]. Ourimplementation automatically converts an i* model with un-certainty annotation 〈ΣM,ΦM〉, together with the i* metamodel〈Σ,Φ〉, analysis constraints (ΦM ∪Φp), and initial labels (Φl),into their corresponding FOL encoding.

Our implementation uses an SMT solver, specifically, Z3version 4.3.12. It answers Q1 by iteratively calling the solver,checking each possible label for each intention. We answer Q3by adding the user choices to the encoding (Φc) and checkingfor satisfiability. The result is a suggested concrete model.Q4 is answered using the implementation outlined in [21],integrated into the MMTF Framework.

Experience. Answering questions Q1, Q3, and Q4 for theillustrated alternative in the Meeting Scheduler example took20.72, 0.18, and 7.31 seconds, respectively, using a PC withIntel Core i7-2600 3.40 GHz x 4 cores and 8 GB RAM.We have applied our approach to three additional alternativesfor the Meeting Scheduler and to a larger example, Inflo. Adescription of these models, analysis results, SMT encodings,and running times can be found online3.

Alternative Analysis Paths. Our scheduler example describeda successful analysis path through our methodology, leadingto a viable analysis alternative with certainty.

However, there may be less successful outcomes whichcorrespond to back-edges in Fig. 5. We claim that even thosescenarios that do not directly lead to a viable alternativecan help eliminate alternative sets of requirements, reduceuncertainty, and refine the model.

In one scenario, the set of possible analysis values maynot yield viable choices, (i.e., Q2 is answered negatively).For example, if the analysis in model D in Fig. 6 yieldedonly negative values (PD or FD) for both PiM and OM, userscould evaluate further alternatives in the model (e.g., LSSM),or perform further elicitation to discover other requirementalternatives.

In another scenario, if users are able to find sufficient choicevalues over uncertain results but these values are not simul-taneously achievable (Q3 is answered negatively), users mustrevise their choices, or evaluate further alternatives. Similarly,if critical uncertainty reductions (Q4) are not consistent withthe domain, options include relaxing choice values, finding analternative set of choice values making different trade-offs, orevaluating further alternatives.

If the reductions in uncertainty evaluated as part of Q5 arenot consistent with domain knowledge, users can articulate thereasons for such inconsistencies and either revise the model

2http://z3.codeplex.com/3http://www.cs.utoronto.ca/~jenhork/AnalyzingUncertainGM

and MAVO uncertainties to reflect this understanding, or addmore specific constraints to the SMT encoding (see Sec. V foran example).

Backward Reasoning. In this paper, we have illustrated ourtechnique using forward analysis, i.e., placing labels on leafelements and asking “what if?” questions. It is also possibleto conduct backward analysis by placing analysis labels onroot elements and asking if it is possible to achieve themand if so, how? [8]. Our current implementation can supportbackward analysis by removing the constraint which assignsthe N label to all leaf intentions not explicitly assigned aninitial label. In this case, the set of possible analysis labelspresented when answering Q1 includes possible labels whichsatisfy backward constraints and labels made possible viauncertainty. Future work will focus on making the sets ofpossible labels comprehensible to the user, both in forwardand backward analysis, e.g., understanding how they can beused to visualize consistent solutions over the whole model.

VII. RELATED WORK

Uncertainty in RE. Several approaches consider uncertaintyin requirements, often as part of an overall strategy for manag-ing uncertainty in software development (e.g, [12], [16]). TheMAVO approach adopted in this work takes a different viewon uncertainty in RE, allowing users to capture uncertainty ontop of their preferred RE modeling language [20].

Much of the investigation of uncertainty in RE concernsadaptive systems. Such systems aim to respond to run-timeuncertainty by specifying functional adaptations as part ofRE (see [22] for overview). MAVO is aimed to representuncertainty in the content, or structure, of requirements modelsarising as part of elicitation and design, ideally resolved as partof the software development process, and does not explicitlyhandle run-time or environmental uncertainty.

Uncertainties in software development are often consideredas part of risk management, (e.g, [13]). Although certain riskfactors (e.g., an unknown budget) may motivate the presenceof model uncertainty, MAVO is not explicitly intended tocapture risks.

Partial Modeling. May uncertainty in MAVO is related tovarious modal extensions to behavioural modeling formalisms.For example, Modal Transition Systems (MTSs) [14] allowintroduction of uncertainty about transitions on a given event,whereas Disjunctive Modal Transition Systems (DMTSs) [15]add a constraint that at least one of the possible transitionsmust be taken in the refinement. Concretizations of these mod-els are Labelled Transition Systems (LTSs). MTSs and DMTSshave been used to capture some forms of uncertainty in earlydesign models [23]. The MAVO approach allows specificationof more uncertainty types, although it is applicable only tostructural models.

Goal Model Analysis. Several goal model analysis procedureshave been proposed, facilitating decision-making in RE (see[10] for a survey). In this paper, we have focused on qualitativesatisfaction analysis. Quantitative approaches (e.g., [7]) are

41

appropriate for later, detailed requirements analysis, wherereliable metrics can be found, and are less applicable topossibilistic, early uncertainties. Other approaches consideruncertainty in goal models through varying contexts (e.g.,[1]) or by analyzing risk (e.g., [2]). These approaches captureuncertainty inherent in the domain, for example, a change inuser context or failure of a component, while our approachfocuses on evolving uncertainty arising as part of requirementselicitation, ideally resolved before implementation.

Further approaches have applied SAT solving to goal mod-els, using interpolation to automatically refine operationalgoals [5]. This technique is not aimed for decision-making ortrade-off analysis but to automatically find goal operational-izations which satisfy safety and liveness goals.

Related work has explored the application of search based(SB) techniques to the problem of requirement selection,finding non-dominated sets of optimal requirements [25].Yet application of SB techniques requires quantitative utilityfunctions mapping all alternatives to all objectives, which aredifficult to elicit with accuracy in early RE. Our approach fo-cuses on qualitative analysis over goal models with uncertaintywhich are more feasible for early decision making.

VIII. CONCLUSION AND FUTURE WORK

In this paper, we have motivated the need for support ofearly decision-making in the presence of uncertainty. We pro-vide such support via an explicit consideration of uncertaintyas part of goal model analysis. We have provided a tool-supported methodology, allowing users to answer five analysisquestions using uncertain goal models. The result of thisprocess is a reduction of necessary uncertainties, acceptingor rejecting alternative requirements with certainty. This workis the first example application of the MAVO uncertaintyframework [20] taking semantics of a target language intoaccount (in our case, such semantics has been provided in[11], [8]).

While we chose i* as a particular goal modeling frameworkand [11] as a particular analysis algorithm, our approach canbe applied more generally to work with other goal modelingframeworks (e.g., NFR [3] or KAOS [4]) or other qualitativegoal model analysis approaches (e.g., [3], [7]), especiallyif such approaches come with their own formal semantics,translatable into FOL. We are currently working to improvethe usability of MAVO labels (e.g., representing uncertaintyover link types [6]).

We plan to further evaluate our technique by applying it toindustrial RE case studies. The visual presentation of multipleanalysis labels should be evaluated and improved throughuser studies. This work should be further integrated with goalmodel visualization techniques, helping users understand whythe tooling produces no solution for a model [9]. Finally, weare currently considering the consequences of analysis overincomplete models, in a open world.

ACKNOWLEDGMENTS

This work was supported in part by ERC advanced grant267856, “Lucretius: Foundations for Software Evolution”,www.lucretius.eu.

REFERENCES

[1] R. Ali, F. Dalpiaz, and P. Giorgini. A Goal-Based Framework for Con-textual Requirements Modeling and Analysis. Requir. Eng., 15(4):439–458, Nov. 2010.

[2] Y. Asnar, V. Bryl, and P. Giorgini. Using Risk Analysis to EvaluateDesign Alternatives. In Proc. of AOSE’07, pages 140–155, 2007.

[3] L. Chung, B. Nixon, E. Yu, and J. Mylopoulos. Non-FunctionalRequirements in Software Engineering. Kluwer Academic Publishers,2000.

[4] A. Dardenne, A. V. Lamsweerde, and S. Fickas. Goal-Directed Require-ments Acquisition. Sci. of Comp. Prof., 20(1-2):3–50, 1993.

[5] R. Degiovanni, D. Alrajehy, N. Aguirre, and S. Uchitely. Automatedgoal operationalisation based on interpolation and sat solving. In Proc.of ICSE’14, 2014.

[6] M. Famelis and S. Santosa. MAV-Vis: A Notation for Model Uncertainty.In Proc. of MiSE’13, pages 7–12, 2013.

[7] P. Giorgini, J. Mylopoulos, E. Nicchiarelli, and R. Sebastiani. Reasoningwith Goal Models. In Proc. of ER’02, volume 3084 of LNCS, pages167–181, 2002.

[8] J. Horkoff and E. Yu. Finding Solutions in Goal Models: An InteractiveBackward Reasoning Approach. In Proc. of ER’10, volume 6412 ofLNCS, page 59, 2010.

[9] J. Horkoff and E. Yu. Visualizations to support interactive goal modelanalysis. In Proc. of REV’10, pages 1–10, 2010.

[10] J. Horkoff and E. Yu. Analyzing Goal Models: Different Approachesand How to Choose among Them. In Proc. of SAC’11, pages 675–682,2011.

[11] J. Horkoff and E. Yu. Interactive goal model analysis for earlyrequirements engineering. Requirements Engineering (accepted withminor revisions), 2014.

[12] H. Ibrahim, B. H. Far, A. Eberlein, and Y. Daradkeh. UncertaintyManagement in Software Engineering: Past, Present, and Future. InProc. of CCECE’09, 2009.

[13] S. Islam and S. H. Houmb. Integrating Risk Management Activities intoRequirements Engineering. In Proc. of RCIS’10, pages 299–310, 2010.

[14] K. G. Larsen and B. Thomsen. A Modal Process Logic. In Proc. ofLICS’88, 1988.

[15] P. Larsen. The Expressive Power of Implicit Specifications. In Proc. ofICALP’91, volume 510 of LNCS, pages 204–216, 1991.

[16] J. Noppen, P. van den Broek, and M. Aksit. Software Development withImperfect Information. J. Soft Computing, 12(1):3–28, 2008.

[17] R. Salay, M. Chechik, S. Easterbrook, Z. Diskin, P. McCormick,S. Nejati, M. Sabetzadeh, and P. Viriyakattiyaporn. An Eclipse-BasedTool Framework for Software Model Management. In Proc. of OOPSLAEclipse Wrksp., pages 55–59, 2007.

[18] R. Salay, M. Chechik, and J. Horkoff. Managing Requirements Uncer-tainty with Partial Models. In Proc. of RE’12, pages 1–10, 2012.

[19] R. Salay, M. Chechik, J. Horkoff, and A. Sandro. Managing re-quirements uncertainty with partial models. Requirements Engineering,18(2):107–128, 2013.

[20] R. Salay, M. Famelis, and M. Chechik. Language Independent Refine-ment Using Partial Modeling. In Proc. of FASE’12, volume 7212 ofLNCS, 2012.

[21] R. Salay, J. Gorzny, and M. Chechik. Change Propagation Due toUncertainty Change. In Proc. of FASE’13, pages 21–36, 2013.

[22] P. Sawyer, N. Bencomo, J. Whittle, E. Letier, and A. Finkelstein.Requirements-Aware Systems: A Research Agenda for RE for Self-adaptive Systems. In Proc. of RE’10, pages 95–103, 2010.

[23] S. Uchitel and M. Chechik. Merging Partial Behavioural Models. InProc. of SIGSOFT FSE’04, pages 43–52, 2004.

[24] E. Yu. Towards Modelling and Reasoning Support for Early-PhaseRequirements Engineering. In Proc. of RE’97, pages 226–235, 1997.

[25] Y. Zhang, A. Finkelstein, and M. Harman. Search based requirementsoptimisation: Existing work and challenges. In Proc. of REFSQ’08,pages 88–94, 2008.

42