re tutorial user stories

User story best practices requirements in agile context

Garm Lucassen and Sjaak Brinkkemper, 12 September 2016Utrecht University, The Netherlands

Lucassen & Brinkkemper 12 September 2016

Today

• User story basics

• Improving user story quality

• Embedding user stories in agile practice

• Advanced analysis

2


Garm Lucassen• PhD @ Utrecht University

- Software product management • students • professionals

- User stories

• Contact: [email protected]

• http://garmlucassen.nl • http://softwareproductmanagement.org

3

mailto:[email protected]

http://garmlucassen.nl

http://softwareproductmanagement.org


Sjaak Brinkkemper• Professor Software Production @ Utrecht University

- Leads research group of 35 staff and PhDs

- Product Software: Methodology of Development, Implementation and Entrepreneurship

• Contact: [email protected]

• http://www.uu.nl/staff/SBrinkkemper/0

4


http://www.uu.nl/staff/SBrinkkemper/0


How about you?• Name

• Company

• Role

• Experience with user stories

• What do you hope to learn today?

5

6

Schedule Time1 Introduction 8.302 Familiarize with user story concept 9.003 Exercise 1: creating user stories 9.154 Varieties in industry 9.455 Coffee break 10.006 Industry survey 10.307 User story quality basics 11.308 Lunch 12.009 Quality User Story Framework 13.3010 Exercise II: Grimm Tool 14.3011 Coffee break 15.0012 User stories in agile practice 15.3013 Advances in user story analysis 16.1514 Exercise 3: Visual Narrator 16.4515 Contribute to ongoing research and closing 17.00


What is a user story?• “As a Visitor, I want to purchase an event ticket”

• “As a visitor, I want to search for new events by favorited organizers so that I am the first to know of new events”

• “As a Visitor, I want to be notified when an event is close to becoming sold out, so that I do not miss the event”


What is a user story?

8

• User stories represent customer requirements in a card, leading to conversation and confirmation (Jeffries, 2001)

• User stories only capture the essential elements of a requirement: - who it is for - what it expects from the system - why it is important (optional?)

• Simple format used by 70% of practitioners

who what why

(Connextra)As a role I want to action (so that benefit, , )

(Lucassen et al., 2016)


What is a user story?• “As a Visitor, I want to purchase an event ticket”

• “As a visitor, I want to search for new events by favorited organizers so that I am the first to know of new events”


“As a Visitor I want to search for new events by favorited organizers

so that I am the first to know of new events”

, ,


History• First mention in Kent Beck’s 1999 book

Extreme Programming Explained - Unstructured text - Similar to use cases - Restricted in size

• Jeffries 2001: card, conversation, confirmation

• Widespread popularity after Mike Cohn’s User Stories Applied in 2004

10


Popularity• 45% of practitioners employ user stories (Kassab, 2015)

• In agile: 90%! (Wang, 2015)

11


Exercise 1Creating user stories

• We are going to create user stories for an event ticketing service: TicketExpert

• The fundamental stories are: - “As an Organizer, I want to distribute tickets, so that visitors can attend my event”

- “As a Visitor, I want to attend an event, so I can see an artist perform”

- “As a Visitor, I want to one click purchase tickets, so that I do not need to supply my personal information with every purchase”

- “As a TicketExpert Employee, I want to manage events”

• Expand upon this list by creating 6 user stories: - Create 3 simple user stories

- Create 2 more advanced user stories

- How about a quality attribute?

12

Functional architecture

On-lineEventTicketing

Productscope

ticketingcontract

eventrequestbooking

tickettypesbankingdetails

payeeinteraction

optionstermsofservices

customerdetails

eventdetails

Contractmanagement

TicketsalesPay-ment

Acqui-sition

Module

Informationflow

FA on module level: Ticket sales

Modulescope

Event registration

Event reporting

templates

booking

reports

tickettypesfinalticket

payment details

Visitor management eventdetails

Ticketselection

Pre-payment arrangement

Ticketsales

Typemanagement

packagestructures

ticketoverview

visitordetails

Sub-module


Exercise 1Creating user stories

• We are going to create user stories for an event ticketing service: TicketExpert

• The fundamental stories are: - “As an Organizer, I want to distribute tickets, so that visitors can attend my event”

- “As a Visitor, I want to attend an event, so I can see an artist perform”

- “As a Visitor, I want to one click purchase tickets, so that I do not need to supply my personal information with every purchase”

- “As a TicketExpert Employee, I want to manage events”

• Expand upon this list by creating 6 user stories: - Create 3 simple user stories

- Create 2 more advanced user stories

- How about a quality attribute?

15


Exercise evaluation• How did it go?

• Things to consider - Why part is very important! - Don’t force a story into its format when its unnatural - Remember: business/domain/application language, - No technical details!

• What did you observe for your own stories?

16


Exercise evaluation• How did it go?

• Things to consider - Is your role the actual role? - Why part is very important! - Don’t force a story into its format when its unnatural - Remember: business/domain/application language, - No technical details!

• What did you observe for your own stories?

• Save stories as input for exercise 2!

17

18



Industry survey• Survey w/182 responses & 21 follow-up interviews

• Use of user stories - Development Methods - Templates

• Perception of user story effectiveness - Impact on productivity? - Impact on work deliverable quality?

19

(Lucassen et al. 2016a)http://bit.ly/us_effective

http://bit.ly/us_effective


Usage

20

As a role I want to action (so that benefit, , )

User Story Quality

Pragmatic

Complete Independent Uniform UniqueEstimatable Full sentence

Semantic

Conflict-free UnambiguousProblem-oriented Conceptually sound

SyntacticMinimal Atomic Well-formed

IndependentNegotiable Valuable Estimable Simple Testable

• 70% use the Connextra template (10% w/o optional)

• Minority uses pre-defined quality guidelines like INVEST (23,5%) or QUS


How are user stories applied?

• Huge diversity

• User story is the most granular representation of a requirement developers use to build new features

• Strong connection to Scrum (94%)

21

“For me, user stories and Scrum are interconnected”

22

- Requirements engineer

23

Schedule Time1 Introduction 8.302 Familiarize with user story concept 9.003 Exercise 1: creating user stories 9.154 Industry survey introduction 9.455 Coffee break 10.006 Industry survey continued 10.307 User story quality basics 11.308 Lunch 12.009 Quality User Story Framework 13.3010 Exercise II: Grimm Tool 14.3011 Coffee break 15.0012 User stories in agile practice 15.3013 Advances in user story analysis 16.1514 Exercise 3: Visual Narrator 16.4515 Contribute to ongoing research and closing 17.00

24


User Story Effectiveness- Perception of all practitioners - The role of using a template - The impact of quality guidelines - Technical vs. non-technical roles

25

Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S. (2016a). The use and effectiveness of user stories in practice. In REFSQ (pp. 205-222).



Garm Lucassen12 September 2016

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%

S6. Quality − guidelines

S5. Productivity − guidelines

S4. Quality − template

S3. Productivity − template

S2. Quality

S1. Productivity

Percentage

Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree

source_data_role_r

Perception of all practitioners

26

11%

15%

15%

33%

22%

37%

52%

52%

33%

30%

30%

22%

37%

33%

52%

37%

48%

41%

6. Quality − guidelines

5. Productivity − guidelines

4. Quality − template

3. Productivity − template

2. Quality

1. Productivity

Percentage


non_template

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%


Perception of all practitioners11%

15%

15%

33%

22%

37%

52%

52%

33%

30%

30%

22%

37%

33%

52%

37%

48%

41%





2. Quality

1. Productivity

Percentage


non_template

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%





S2. Quality

S1. Productivity

Percentage


source_data_role_r

27

Practitioners agree, productivity and work deliverable quality increases when using: - user stories - templates



15%

15%

33%

22%

37%

52%

52%

33%

30%

30%

22%

37%

33%

52%

37%

48%

41%





2. Quality

1. Productivity

Percentage


non_template

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%





S2. Quality

S1. Productivity

Percentage


source_data_role_r

28

Respondents are ambivalent about quality guidelines



15%

15%

33%

22%

37%

52%

52%

33%

30%

30%

22%

37%

33%

52%

37%

48%

41%





2. Quality

1. Productivity

Percentage


non_template

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%

9%

8%

13%

14%

7%

10%

68%

61%

54%

53%

48%

40%

23%

31%

32%

33%

45%

51%





S2. Quality

S1. Productivity

Percentage


source_data_role_r

29

Very few practitioners are negative (7 - 14%)


“Why do you think user stories improve productivity or quality?”

30

In follow-up interviews:

“The right software”

31

- 10 interviewees

“User stories optimize for happiness”

32


33


Template12%

8%

52%

62%

36%

30%

12%

8%

52%

71%

36%

21%

36%

11%

32%

56%

32%

33%

36%

10%

24%

59%

40%

31%

24%

8%

32%

41%

44%

52%

16%

6%

36%

50%

48%

44%

S1. Productivity

S2. Quality





Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

34

12%

8%

52%

62%

36%

30%

12%

8%

52%

71%

36%

21%

36%

11%

32%

56%

32%

33%

36%

10%

24%

59%

40%

31%

24%

8%

32%

41%

44%

52%

16%

6%

36%

50%

48%

44%

S1. Productivity

S2. Quality





Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

n = 155 n = 27

35

Those who use templates are more positive about templates

12%

8%

52%

62%

36%

30%

12%

8%

52%

71%

36%

21%

36%

11%

32%

56%

32%

33%

36%

10%

24%

59%

40%

31%

24%

8%

32%

41%

44%

52%

16%

6%

36%

50%

48%

44%

S1. Productivity

S2. Quality





Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

Non−Template

Template

n = 155 n = 27

36

More positive, but no statistical confirmation.

37

In follow-up interviews:“Why do you use this template?”

“A template not

the template”

38

- 3 interviewees

“A template not

the template”“It’s not the template that improves quality, it’s what we’re

doing - we’re sharing requirements and a template makes that easy to do and more likely that we’ll do it”.

39 - VP Engineering

“The why is essential”

40

“The why is essential”

“Typically, the why question is correctly answered if after the initial answer, you ask ‘why?’ again for three more times”

41- Agile coach

User Story Effectiveness

42

- Perception of all practitioners - The role of using a template - The impact of quality guidelines - Technical vs. non-technical roles


8%

7%

10%

49%

74%

67%

43%

19%

23%

12%

5%

8%

62%

77%

68%

25%

19%

23%

17%

14%

12%

49%

58%

55%

35%

28%

33%

19%

7%

10%

47%

72%

53%

33%

21%

37%

11%

7%

10%

24%

67%

38%

65%

26%

52%

6%

5%

10%

25%

84%

52%

69%

12%

38%

S1. Productivity

S2. Quality





Yes, self−defined guidelines

Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No

Quality guidelines

43


8%

7%

10%

49%

74%

67%

43%

19%

23%

12%

5%

8%

62%

77%

68%

25%

19%

23%

17%

14%

12%

49%

58%

55%

35%

28%

33%

19%

7%

10%

47%

72%

53%

33%

21%

37%

11%

7%

10%

24%

67%

38%

65%

26%

52%

6%

5%

10%

25%

84%

52%

69%

12%

38%

S1. Productivity

S2. Quality






Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No


Yes, INVEST

No

Quality guidelinesn=72n=43 n=60

44

Respondents that follow quality guidelines are more positive. Respondents applying INVEST are even more positive.

45

In follow-up interviews:“Why do you think quality guidelines improve work deliverable quality and productivity?”

“INVEST is not a checklist”

46


- 3 interviewees

“INVEST is useful for inexperienced teams”

47


- 2 interviewees

User Story Effectiveness

48

- Perception of all practitioners - The role of using a template - The impact of quality guidelines - Technical vs. non-technical roles


Technical vs. non-technical8%

9%

61%

60%

31%

31%

6%

15%

74%

55%

20%

31%

13%

16%

56%

45%

31%

38%

9%

22%

62%

36%

28%

42%

10%

9%

48%

20%

42%

71%

6%

11%

56%

31%

39%

58%

S1. Productivity

S2. Quality





Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

49


8%

9%

61%

60%

31%

31%

6%

15%

74%

55%

20%

31%

13%

16%

56%

45%

31%

38%

9%

22%

62%

36%

28%

42%

10%

9%

48%

20%

42%

71%

6%

11%

56%

31%

39%

58%

S1. Productivity

S2. Quality





Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

Non−Technical

Technical

50

Non-technical stakeholders are: - On average 25% more positive - Strongly in favor of quality guidelines


51


Key take awaysa. The simple structure of user stories enables

developing the right software, for it facilitates creating a common understanding concerning the requirement

b. Specifying the why part of a user story is essential for requirements quality.

c. Practitioners who use the INVEST quality guidelines are significantly more positive about the impact of user stories on productivity and the impact of templates on work deliverable quality.

52

Garm Lucassen12 September 2016 53

We call for an increase in the diffusion of knowledge concerning quality guidelines

54



User story quality

55

Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S. (2016b) Improving agile requirements: the Quality User Story framework and tool. Requirements Engineering Journal, 1-21.

http://bit.ly/improving_us

http://bit.ly/improving_us


User story quality• Conceptual model of user stories

• INVEST

• Quality User Stories Framework

• Prevalence of errors in real user story sets

• Exercise 2: Grimm Tool

56


Meet Anne• Project manager web dev team

• Creates high quality user stories

• Developers do not

• Confused clients

57


In this tutorial section

Foundations for:

• Solving Anne’s problem

• Helping practitioners write high quality user stories

• Conducting advanced analyses in final section

58


User story template

59

As a role I want to action (so that benefit, , )


Conceptual model

60

Subject

User Story

EndMeans

Role 1 1..*

1 0..*

Format0..1

1..*has_parent

Action Verb Direct Object

Indirect objectAdjective

1

Epic

1..*has

QualityClarification

0..* 0..*

11

1 1 10..* 0..*

Dependency

0..*


“As a User, I want to search for new events by favorited organizers,


Conceptual model

61

User Story

EndMeans

Role 1 1..*

1 0..*

Format0..1

1..*has_parent

1

11


Conceptual model

62

User Story

EndMeans

Role 1 1..*

1 0..*

Format1

11

0..1

1..*has_parent

“As a User I want to search for new events by favorited organizers


, ,


Means concepts

63

Subject

Means



1 1 10..* 0..*

“I want to search for new events by favorited organizers”I search eventsnew

Subject Action Verb Direct Object

Adjective Indirect object

favorited organizers


Ends concepts“so that I am the first to know of new events”

64

End


0..* 0..*

Dependency

0..*

“so that I am the first to know of new events”new events

DependencyClarification Quality

the first


Conceptual model

65

Subject

User Story

EndMeans

Role 1 1..*

1 0..*

Format0..1

1..*has_parent



1

Epic

1..*has


0..* 0..*

11

1 1 10..* 0..*

Dependency

0..*

66


67



Quality problems in practice• Model captures correct

stories

• Stories from practitioners: - Too long - Unnecessary information - Too little information - Inconsistent - Irrelevant - Ambiguous

68

Subject

User Story

EndMeans

Role 1 1..*

1 0..*

Format0..1

1..*has_parent



1

Epic

1..*has


0..* 0..*

11

1 1 10..* 0..*

Dependency

0..*


INVEST?• Most popular and simple reminder for

characteristics of a good user story: - Independent: - Negotiable - Valuable - Estimable - Simple - Testable

• Drawbacks: non-specific, generic, qualitative

69


Slides available online

70

http://bit.ly/re16tut

http://bit.ly/re16tut

Quality User Story Framework

71

Based on critical analysis of: - Hundreds of user stories - Existing quality frameworks - Problems in practice

72

User Story Quality

Pragmatic


Semantic



RQ1

RQ1

RQ2

RQ3

RQ4

RQ5

RQ6

RQ7

RQ8

RQ9

RQ10

RQ11

RQ12

RQ13

73

IndividualRQ1 Well-formed A user story includes at least a role and an actionRQ2 Atomic A user story expresses a requirement for exactly one featureRQ3 Minimal A user story contains nothing more than role, action and benefit RQ4 Conceptually sound The action expresses a feature and the benefit expresses a rationaleRQ5 Problem-oriented A user story only specifies the problem, not the solution to itRQ6 Unambiguous A user story avoids terms or abstractions that lead to multiple interpretationsRQ8 Full sentence A user story is a well-formed full sentenceRQ9 Estimatable A story does not denote an unrefined requirement that is hard to plan and prioritize

SetRQ7 Conflict-free A user story should not be inconsistent with any other user story RQ10 Unique Every user story is unique, duplicates are avoidedRQ11 Uniform All user stories in a specification employ the same templateRQ12 Independent The user story is self-contained and has no inherent dependencies on other storiesRQ13 Complete Implementing a set of user stories creates a feature-complete application, no steps are missing

Quality User Story Framework


Creating user stories• Not all quality criteria immediately critical

• Focus on: RQ1. Well-formed RQ2. Atomic RQ3. Minimal RQ4. Conceptually sound RQ5. Problem oriented RQ8. Full sentence RQ11. Uniform

• Avoid premature optimization!

74


A user story includes at least a role and an action

I want to revoke access for problematic event organizers

⬇ add role

As a TicketExpert Employee, I want to revoke access for problematic event organizers

RQ1 - well-formed


A user story expresses a requirement for exactly one feature/problem

As a Visitor, I want to register for an event and create a personal account, so that I can quickly register for more events in the future

⬇ split

1. As a Visitor, I want to register for an event, so that I am admitted to the event 2. As a Visitor, I want to create a personal account during event registration, so that I

can quickly register for more events in the future

RQ2 - atomic


RQ3 - minimal

A user story contains nothing more than role, action and benefit

As an Event Organizer, I want to see the personal information of attendees (split into price levels). See: Mockup by Alice NOTE: - First create the overview screen

⬇ (re)move unnecessary information

As an Event Organizer, I want to see the personal information of attendees


RQ4 - conceptually sound

The action expresses a feature and the benefit expresses a rationale

As an Event Organizer, I want to open the event page, so that I can see the personal information of attendees

⬇


RQ4 - conceptually sound

The action expresses a feature and the benefit expresses a rationale

As an Event Organizer, I want to open the event page, so that I can see the personal information of attendees

⬇ ends becomes separate means

1. As an Event Organizer, I want to open the event page, so that I can review event related information

2. As a User, I want to see personal information of attendees, so that I know the demographical distribution of the event


RQ5 - problem oriented

A user story only specifies the problem, not the solution to it

As a Visitor, I want to download an event ticket. - Add download button on top right (never grayed out)

⬇


RQ5 - problem oriented

A user story only specifies the problem, not the solution to it

As a Visitor, I want to download an event ticket. - Add download button on top right (never grayed out)

⬇ remove solution

As a Visitor, I want to download an event ticket


RQ8 - full sentence

A user story is a well-formed full sentence

update profile

⬇ add ‘want to’

As a Visitor, I want to update my profile


RQ 11 - uniform

All user stories follow roughly the same template

1. As a Visitor, I want to create an account 2. As a Visitor, I want to reset my password

3. As a TicketExpert Manager, I receive an email notification when a new user is registered

⬇ add ‘want to’

As an TicketExpert Manager, I want to receive an email notification when a new user is registered

84


Grimm Tool

85


Grimm Tool Technology: Automatic Quality User Story Artisan

• Automatically assess user story quality • Restrict ourselves to criteria with potential for 100% recall

• The Berry Recall Criterion

• Syntactic: Well-formed Atomic Minimal

• Semantic 100% recall unachievable • Selection of pragmatic:

Explicit dependencies Uniform Unique

86

(Daniel Berry et al., 2012)

(Ryan, 1993)

User stories

AQUSA

Linguistic parser

Enhancer

Analyzer

Synonyms Homonyms Ontologies

Error report

Atomic Independent UniqueMinimal Uniform

Report generator

User story base

Corrections


Grimm Tool

87

Well-formedAtomic Minimal Uniform Unique

AQUSA

√only one feature no unnecessary text follows the template has a role and meansno duplicates

(with my name)“As a Visitor, I want to supply my personal details, so that the ticket is personalized ”

√√√⤫

“As a Visitor, I want to supply my personal details, so that the ticket is personalized (with my name)”

error!

none found

√Explicit dependencies


Grimm Tool

88

Well-formedAtomic Minimal Uniform Unique

AQUSA

√only one feature no unnecessary text follows the template has a role and meansno duplicates

√√√

“As a Visitor, I want to supply my personal details, so that the ticket is personalized”

none found

√Explicit dependencies

√ “As a Visitor, I want to supply my personal details, so that the ticket is personalized”

√

perfect story


Jira integration automatically posts comments

89


Grimm Tool

90


Grimm Tool evaluation

• Measure: Precision Recall Most common violations

• Do we achieve the Berry Recall Criterion?

• Apply mechanics to seventeen datasets

91

Agile

Requirem

entsQ

uality:The

Quality

User

StoryFram

ework

andTool

13

Table 3 Detailed results split per data sets, showing number of defects correctly detected (Def), false positives (FP), and false negatives (FN)

1: ResearchComp 2: ExpenseComp 3: EnterpriseComp 4: DataComp 5: RealProd# Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN

Atomic 5 2 1 10 4 0 1 1 0 6 0 1 6 3 2Minimal 6 3 0 3 1 0 25 5 0 4 2 0 16 6 0Well-formed 6 4 0 1 1 0 33 21 0 2 0 0 0 0 0Uniform 17 8 0 27 9 0 38 17 0 7 0 0 9 0 1Unique 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0SUM 36 17 1 41 15 0 97 45 0 19 2 1 33 9 3N, precision, recall 50 53% 95% 50 63% 100% 50 55% 100% 23 89% 94% 51 73% 89%

6: E-ComComp 7: EmailComp 8: ContentComp 9: CMSComp 10: HealthComp# Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN


11: AccountancyComp 12: PharmacyComp 13: SupplyComp 14: IntegrationComp 15: HRComp# Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN


16: FilterComp 17: FinanceComp Public 1: Duke University# Def # FP # FN # Def # FP # FN # Def # FP # FN

Atomic 42 39 0 13 4 0 10 1 2Minimal 5 0 0 25 5 0 4 3 0Well-formed 0 0 0 0 0 0 0 0 0Uniform 38 0 0 29 0 0 18 0 0Unique 2 0 0 6 0 0 0 0 0SUM 87 39 0 73 9 0 32 4 2N, precision, recall 51 55% 100% 55 88% 100% 48 88% 93%


Grimm Tool evaluation• 56% of selected stories violate one or more quality

criteria

• These data sets: 93.8% recall and 72.2% precision

93

14

Table 4 Overall recall and precision of AQUSA v1, com-puted using both the micro- and the macro average of thedata sets

Recall PrecisionMacro 92.1% 77.4%Micro 93.8% 72.2%

Table 5 Number of defects, false positives, false negatives,recall and precision per quality criterion

n = 1023 Totals# Def # FP # FN Rec Prec

Atomic 170 83 18 82.9% 51.18%Minimal 187 57 6 95.5% 69.52%Well-formed 104 60 2 95.7% 42.31%Uniform 426 55 18 95.4% 87.09%Unique 30 0 0 100% 100%SUM 917 255 44 93.8% 72.2%

From this source data we can extract a numberof interesting findings. At first glance, the results arepromising, indicating high potential for successful fur-ther development. The average number of user storieswith at least one defect as detected by AQUSA is 56%.Furthermore, 23% of all defects are either secondary,tertiary or further defects of one user story.

The average recall and precision of AQUSA for allthe company sets is shown in Table 4. Note the differ-ences between the macro-average and weighted micro-average for recall and precision [47]. This highlights theimpact of outliers like #15 SupplyComp, having only2 violations, 0 false positives and 1 false negative outof 50 user stories. For the micro-average the number ofviolations of each set is taken into account, while themacro-average considers each set equally. This meansthat #15 supplycomp its macro-average 67% recall and100% precision weighs as much as all other results,while for the micro-average calculations its impact isnegligible.

In total, AQUSA fulfills the desired Berry RecallCondition for 5 cases, obtains between 90-100% of de-fects for 6 sets and manages to get between 55% and89% for the remaining 6. AQUSA’s results for preci-sion are not as strong, but this is expected because ofour focus on the Berry Recall Condition. For just 2 setsAQUSA manages to get 100% precision, for 5 sets preci-sion is between 90-100%, 3 sets are only just below thisnumber with 88-90%. In 7 cases, however, precision israther low with a range of 50-73%. While AQUSA isunable to achieve 100% recall and precision for any ofthe sets, some do come close: for companies 7, 8, 9,11 and 14, AQUSA v1 achieves 90%+ recall and preci-sion. We investigate how to improve this performancein Section 6.

Looking at the distribution of violations in Table 3and the total number of violations, false positives andfalse negatives in Table 5, a number of things standout. With the exception of the quality criteria unique,the absolute number of false positives lie close to oneanother. Relatively speaking, however, well-formed andatomic stand out. Approximately 50-60% of violationsas detected by AQUSA are false positives. Similarly,the number of false negatives is particularly large foratomic, minimal and uniform. In the remainder of thissection, we investigate the causes for these errors.

Atomic. Throughout the user story sets, the most fre-quently occurring false positive is caused by the symbol‘&’ within a role such as: “As an Product Owner W&O”and “As an R&D Manager” (n=38). As we show in Sec-tion 6, this can be easily improved upon. The other twomain types of false positives, however, are more difficultto resolve: nouns incorrectly tagged as nouns trigger-ing the AtomicAnalyzer (n=18) and multiple conditionswith verbs interspersed (n=14).

Tallying the number of false negatives, we find adiversity of causes. The biggest contributor is that for-ward or backward slashes are not recognized as a con-junction and thus do not trigger the atomic checker(n=5). A more significant issue, however, is that ourstrategy of checking whether a verb is present on bothsides of the conjunction backfired in 2 cases. Specifi-cally, the words ‘select’ and ‘support’ were not recog-nized as a verb by the CoreNLP part-of-speech tagger,which employs a probabilistic maximum entropy algo-rithm that miscategorized these words as nouns.

Minimal. The primary cause for minimality false pos-itives is the idiosyncratic use of a symbol at the startof a user story such as the asterisk symbol (n=24). Al-though a fairly easy false positive to prevent from oc-curring, the fix will introduce false negatives because insome cases a symbol at the start is an indication of aminimality error. Because our priority is to avoid falsenegatives, we have to accept these false positives as anunavoidable byproduct of the AQUSA tool. Anotherfrequently occurring error is abbreviations or transla-tions between brackets (n=14). It might be possible toreduce this number with custom methods.

The 7 false negatives for minimality primarily con-cern idiosyncratic, very specific textual constructs thatare unsupported by AQUSA v1. For example, dataset11 (AccountancyComp) delivered 2 user stories with su-perfluous examples preceded by the word ‘like’. Health-Comp (dataset 12) has 3 very large user stories withmany different if clauses and additional roles includedin the means and one user story with an unnecessarypre-condition interspersed between the role and means.


Exercise 2: Grimm Tool• Who brought user stories?

• Upload to: aqusa.nl - Create an account - Example CSV files: bit.ly/us_csvs - Put your stories in a CSV, UTF-8 encoding

- One story per row - Add “ and ” to beginning and end of each row

• Pre-submitted user stories

94

http://aqusa.nl

http://bit.ly/us_csvs


1. Create an account

95


2. Create CSV of your stories

96


3. Upload a CSV

97


4. Review output

98


4. Review output

99

Sometimes necessary!


Example output

100


Exercise 2: Grimm Tool• Who brought user stories?

• Upload to: aqusa.nl - Create an account - Example CSV files: http://tinyurl.com/us-csv - Put your stories in a CSV, UTF-8 encoding

- One story per row - Add “ and ” to beginning and end of each row

• Pre-submitted user stories

101

http://aqusa.nl

http://tinyurl.com/us-csv


Output• What kind of errors does the tool detect?

• How would you fix the user stories?

• Things to ask yourself: - Well-formed -> are these quality requirements? - Atomic -> impact on estimation? - Minimal -> too little detail? - Uniform -> how many styles do you use? - Unique -> where do these duplicates come from?

102


Estimating and developing• Remaining criteria gradually become relevant

• Focus on: RQ6. Unambiguous RQ7. Conflict-free RQ9. Estimatable RQ12. Independent RQ10. Unique RQ13. Complete

• Keep iterating!

103


RQ6 - unambiguous

A user story avoids terms or abstractions that lead to multiple interpretations

As an Event Organizer, I want to edit the content that I added to an event’s page

⬇


RQ6 - unambiguous

A user story avoids terms or abstractions that lead to multiple interpretations

As an Event Organizer, I want to edit the content that I added to an event’s page

⬇ clarify the content

As an Event Organizer, I want to edit video and text content that I added to an event’s page


RQ7 - conflict-free

A user story should not be inconsistent with any other user story

1. As an Event Organizer, I’m able to edit any event 2. As an Event Organizer, I’m able to delete only the events that I added

⬇


RQ7 - conflict-free

A user story should not be inconsistent with any other user story

1. As an Event Organizer, I’m able to edit any event 2. As an Event Organizer, I’m able to delete only the events that I added

⬇ change 1

1. As an Event Organizer, I’m able to edit events that I added


RQ9 - estimatable

A story does not denote a coarse-grained requirement that is difficult to plan and prioritize

As an Event Organizer, I want to see my task list during the event, so that I can prepare myself (for example I can see at what time I should start traveling)

⬇ split1. As an Event Employee, I want to see my task list during the event, so that I can

prepare myself 2. As an Event Organizer, I want to upload a task list for event employees


RQ10 - unique

Every user story is unique, duplicates are avoided

1. As a Visitor, I’m able to see a list of new events, so that I stay up to date 2. As a Visitor, I’m able to see a list of new events, so that I stay up to date

⬇ remove one1. As a Visitor, I’m able to see a list of news items, so that I stay up to date


???

RQ12 - independent

The user story is self-contained and has no inherent dependencies on other user stories

1. As an Event Organizer, I am able to add a new event2. As a Visitor, I am able to view an event page

⬇


• Try to avoid dependencies as much as possible

• Impossible to be fully independent

• But… always remain flexible!

111

RQ12 - independentThe user story is self-contained and has no inherent dependencies on

other user stories1. As an Event Organizer, I am able to add a new event

2. As a Visitor, I am able to view an event page

⬇???


RQ13 - complete

112

Implementing a set of user stories creates a feature-complete application, no steps are missing

1. As an Event Organizer, I want to update an event 2. As an Event Organizer, I want to delete an event

⬇add story

As an Event Organizer, I want to create an event

113


114



User stories in agile practice

• The context of user stories in systems development

• Epics for high-level planning

• Creating user stories as a team

• Industry experiences

115


The context of user stories in systems development

116


User storyDefinition:

A User story describes a functionality valuable to a user of a software system

• Not just a description

• Assist in discussion on details of the story

• No attributes, conditions, special cases are included in the description

• Determines tests to state when the story is completed.

• Reference: Sonja Dimitrijevic, Jelena Jovanovic, Vladan Devedzic (2015). A Comparative Study of Software Tools for User Story Management. Information and Software Technology, vol 57, pp. 352-368.

117


User Story processes• There is no unique US process.

• Applied in any systems development process (waterfall, agile, iterative)

• Standard in agile processes: Scrum, XP

• Key US processes 1. Gathering from relevant sources 2. Role modeling 3. Acceptance testing 4. Estimating for releases and sprints 5. Planning in releases and sprints 6. Tracking and communicating

118


Waterfall, Scrum, or …?

119


US in Waterfall

User stories are written during Requirements stage as refinements of high level specification. User stories are input for:

• Design: design of functionality; stakeholder interaction; quality • Code: assignment of coding tasks • Test: acceptance tests for user functionality

120

Requirements

Design

Code

Integration

Test

Deploy


US in Waterfall



121

Requirements

Design

Code

Integration

Test

Deploy

User Stories


US in Waterfall



122

Requirements

Design

Code

Integration

Test

Deploy

User Stories


US in Scrum

User stories are assigned to sprints during Release planning stage as refinements of high level specification, called epics. User stories are input for:

• Sprint planning: Refinement into Backlog items • Test: acceptance criteria

123

Sprint 1 Sprint 2 Sprint 3 Sprint 4

refine

design code

test

review deploy


US in Scrum



124


refine

design code

test

review deploy

User Stories


US in Scrum



125


refine

design code

test

review deploy

User Stories


Gathering US• All kinds of sources

• Active involvement of users and customers for the whole duration of the project/release in Scrum

• Customer team can write user stories; and perform the prioritization

• Language of application domain must be used

• Non-intrusive lightweight techniques: interviews, questionnaires, observation, and story-writing workshops

• High level US, so-called epics, are detailed in other USs.

126


User Role modelingDefinition A user role represents a collection of attributes the characterize a population of users and their intended interactions with the system • Examples: Event organizer, event visitor, payment bank, entertainer, ticket

fulfillment officer

• Identification of user roles should be done before writing of user stories • Stories without a user role are apparently not essential

• Technique: Persona as used in User experience design, (introduced by Alan Cooper: The Inmates Are Running the Asylum (1998)). A persona is a play-acted fictitious characters in order to explain design questions. Personas are more figurative, whereas user role is more formally structured.

127


Acceptance testing• Verification of complete development of user

stories

• Criteria: expectations of customer team

• Provide details for design and developments

• Written on the back of the story card

• Test in 2 ways

1. Manually by customer representative

2. Automatically by test tool

128


Estimating User stories• Estimation is done collaboratively by team: measured in story points • Using mixture of expert opinion, analogy, decomposition

• Simple process for estimating US 1. Identify base stories Identify base stories for relative sizing of the backlog.

Story done earlier. Understanding this story is same among everyone on the team. 2. Walk through the requirements of the story Product owner explains story. 3. Discuss any details ScrumMaster adds details to story. 4. Sample questions the team should ask:

1. Design: What to learn before start work on this story? 2. Coding: What will be coding effort? 3. Unit testing: Special setup required for unit test? 4. Acceptance testing: Help customer automate acceptance tests? 5. Integration points: Does the story have any external dependencies? 6. Expertise: Does anyone have prior experience on similar story?

5. Reach a consensus Team to come up with the best estimation that everyone accepts. 6. Validation of estimated story points Estimates are internally consistent among : the

1's are about the same, all of the 2's match, etc.. Also, agree 4-point story is twice a 2-point story.

129

Source: scrumalliance.org: A Practical Guide: Story Points-Based Estimation


Planning releases and sprints

Velocity: number of story points per sprint • Experience data: total number of story points of project/ sprints per project

• Assign USs over different sprints: Planning Game • USs of sprint do not exceed velocity

• Prioritization: maximize value delivered to customer • Technique: MoSCoW, cost/value, relative weigths • Value: financial, development cost, amount of learning, amount of risk

• Planning USs in sprint: decomposition in tasks • Changes due to corrections in plan

130


Tracking and communicating

• Monitoring and discussion of progress

• Refine plan according to observations

131

Task board Burndown chart


Refinement of User Stories• User stories need to be fit into the Sprint planning

• User Stories are formulated as Product Backlog Items, and refined in Sprint tasks

• Some companies define Sprint tasks also in User Stories

132


Epics▪ It is common to formulate Product Backlog Items (PBI) in User Story form ▪ Oversized PBIs are called epics. ▪ Traditional planning breaks features into several tasks that are hard to

prioritize as they lack business value

▪ Most customers don’t use most features of most products ▪ Wise to split epics to deliver the most valuable stories first. ▪ Delivering lower-value features later is likely to involve some rework. ▪ Backlog Refinement Meeting: also called “Backlog Grooming” or

“Backlog Maintenance”.

133


Epic splitting▪ Large User Stories are called epics. ▪ Agility requires to split large epics into user stories representing small

product features. ▪ Example: “As a theatre visitor I want to display the available seats in a theatre so

I can make a good decision on which seat to select” ▪ Example ticketing application:

▪ Epic “display the available seating in a theatre” ▪ US1 “display available seats per row” ▪ US2 “display left, middle or right with empty seats” ▪ US3 “display theatre seating plan“ ▪ US4 “display available seats on seating plan”

▪ Visualizing the theatre chairs in various forms is technical challenging,where the number of seats and the distance to the stage are most important to know for booking.

▪ So US4 can be left to a later release with some rework.

134


Creating user storiesas a team

• Many approaches to creation process

• Quality should concern everyone

• Poker planning

• Team Estimation Game

• Different quality criteria apply

135

Poker planning

136

Garm Lucassen25 April 2016 137

Estimating user stories using poker planning

• Consensus-based and extremely simple: 1. Project manager reads user story 2. Developers discuss user story (change if necessary) 3. Each developer scores using story points

0, 1, 2, 3, 5, 8, 13, 20, 40 4. Debate score differences 5. Repeat 3 and 4 until developers reach consensus

Garm Lucassen25 April 2016 138

Poker planning impact

• Bring a QUS cheat sheet to poker planning

• Use question mark to indicate you don’t know

• If applicable, use QUS terminology to convey the problem

• Discuss, change story if necessary, repeat

1. Unambiguous 2. Estimatable 3. Conflict-free 4. Independent 5. Unique


Industry experiences

• Many different approaches to working with user stories. Currently exploring types!

• Want to contribute? Contact us!

139


Introducing QUS and Grimm Tool

First evaluation with three companies

- 40% reduction in user story defects

- Unexpected benefits

140

“As a Visitor, I want to get messages from the system (tricks etc)”

“I think the conversation effectiveness did change positively, but the Grimm Method training made me more critical of what to expect from user story conversation in terms of effectiveness”


Topics• Introduction

• User story basics

• User story quality

• User stories in agile practice

• User story analysis

141


Topics• Extracting models from user stories

• Advanced analysis through visualization: - Conflict detection - Duplicate prevention - Ambiguity resolution - Incompleteness mitigation

• Exercise 3: visualize your user stories

142

143


Automated Extraction of Conceptual Models from User Stories via NLP Marcel Robeer, Garm Lucassen, Jan Martijn E.M. van der Werf,

Fabiano Dalpiaz and Sjaak Brinkkemper September 16th 2016c @ RE’16

ticket

visitor

account

system

create, rename

log in, log out

event type

filter on

event

see

keep track of

account password

has

password

change purchase


• “As a Visitor, I want to buy an event ticket”


“As a Visitor I want to search for new events by favorited organizers


, ,

145

Going from…


.. to

146


C. Heuristics

148

Noun Subject

Verb (including preposition)

Concept

Relationship

Verb ‘to be’ Hierarchical relationship


C. Heuristics

149

Noun Subject

Verb (including preposition)

Concept

Relationship

Verb ‘to be’ Hierarchical relationship

Noun-noun compound


As a visitor

2. Functional role

150

I want to choose an event

so that I can book a ticket for that event

Role

Means

End


As a visitor

3. Simplify the means

151



Role

Means

End


As a visitor

3. Simplify the means

152



Role

Means

End



As a visitor

4/5. Main verb & main object

153


Role

Means

End



As a visitor

6. Main relationship

154


Role

Means

End

visitor eventchoose


As a visitor

7. Remaining information

155


Role

Means

End

visitor eventchoose


As a visitor

156


Role

Means

End

visitor eventchoose


ticketbook


As a visitor

157


Role

Means

End

visitor eventchoose


ticketbookhas



However…

Models can get BIG


Just 32 stories!


Currently exploring

zoom and filtertechniques


First step: role filter

161

hasInstalla...

canRun

canMigrate

hasAdminis...

hasNode

System

Installation

System Admini...

Neurohub

Administrator

Script

Neurohub Insta...

Data

Neurohub Node

Node


Conceptual: colors

162


Practitioner perception

163

1. Training new employees 2. Active documentation, finding inconsistent

terminology 3. Discussing with stakeholders to detect

incompleteness 4. Requirements prioritization

In their current state, the models can have the following practical applications:


Advanced analysis

• Possible inconsistencies - Conflict detection - Duplicate prevention - Ambiguity resolution

• Incompleteness mitigation

164


Conflict detection

165

hasMachine

hasDepen...

canInstall

hasAdminis...

Neurohub De...

LTS

Neurohub

Administrator

System

LTS Machine

Machine

Dependency

System Admini...

hasInstalla...

canRun

canMigrate

hasAdminis...

hasNode

System

Installation

System Admini...

Neurohub

Administrator

Script

Neurohub Insta...

Data

Neurohub Node

Node

Systems administrator System administrator


Duplicate prevention

166

Separate stories for: - find flight - search flight number - look flight name

canKnow5

canSee6

canKnow4

canMeasure

canSee7

canKnow7

canKnow6

canAlertTo

canKnow9

canSee2

canSearc...

canKnow8

canSee3

canSee4

canSearc...

canSee5

canKnow1

canKnow3

canKnow2

hasNumber

canLookAt

hasCondition

hasName

canKnow...

canKnow10

canPreview

canLookT...

canRead

canLook

hasRecord

hasJourney

canAlert

canFind

canSee1

Weather Condi...

Gate

Flight Name

Weather

Turbulence

Amount

Route

Direction

Explanation

Air

Number

Library

Pressure

Temperature

Flight Number

Name

Cause

Condition

Record

Journey

Longer

Sound

Flight

Plane

Time

Graph

User

World

What

Expectation

Turbulence Re...

Plane Journey


Ambiguity resolution

167

canDelete

canEdit

canUse

canGet

canBe

hasLocation2canSeehasAddresscanClick

canFind

canAdd4

canAdd5

canView2

canRequest

canAdd2

canView1

canAdd3

canAdd6

canOpen

canAdd1

hasReset

canSet

canRemove

User

Map

Video Description

Plot Location

Text

Event

Fragment

Address

Location

Password Reset

Email Event Location

Access

Information

Email Address

Content

Password

Image


Incompleteness mitigation

168

hasFile

hasType

canSearchBy

canAccess

canUpload

canAttach1

canSearch...

canKeep

canBulk

canShare

canForm

hasData

canDownload

canLocate

canIndicat...

canCollect

canTag

canAttach2

canAttach3

Write Ups

File

Directory

Type

File Type

Data File

Meta Data

Link

Researcher

DataCan search for data by type. But researcher cannot

search?

169



Exercise 3• Go to WebVowl or OWLGrED

• Upload .omn or paste link to .omn on the web

• Can you identify inconsistencies or incompleteness?

• Your user stories available in email

• Example visualizations tinyurl.com/us-vis

170

http://vowl.visualdataweb.org/webvowl/index.html

http://owlgred.lumii.lv/online_visualization

http://tinyurl.com/us-vis

174



Contribute to research!• Supply user stories

• Integrate aqusa.nl with Jira

• Beta test future visualization tools

• Participate in a survey on user story practice

• To indicate interest: bit.ly/us-research or contact: [email protected]

175

http://aqusa.nl

http://bit.ly/us-research



Did we succeed?

• Did you learn about: ➡ User story basics ➡ Improving user story quality ➡ Embedding user stories in agile practice ➡ Advanced analysis

176

User Story Quality

Pragmatic


Semantic



User Story Research Utrecht University

• Why are user stories effective?

• How to create better user stories?

• Create RE support tools using NLP

• Less is more! - QUS Framework - Grimm Method - Visual/Interactive Narrator - Next: connecting code

• Contribute: bit.ly/us-research!

http://bit.ly/us-research


Thank you!

Thank you!

178


References

- Cohn, M. (2004). User stories applied: For agile software development. Addison-Wesley Professional. - Jeffries, R. (2001). Essential XP: Card, conversation, confirmation. XP Magazine, 30. - Wake, B. (2003). INVEST in good stories, and SMART tasks. h ttp://xp123. com/articles/invest-in-good-stories-and-smart-tasks.

- Beck K (1999) Extreme programming explained: embrace change. Addison-Wesley, Boston - Berry D, Gacitua R, Sawyer P, Tjong S (2012) The case for dumb requirements engineering tools. In: Proceedings of international conference on requirements engineering: foundation for software quality (REFSQ), LNCS, vol 7195. Springer, pp 211–217

179

- Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S. (2016a). The use and effectiveness of user stories in practice. In REFSQ (pp. 205-222). Springer International Publishing.

- Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S. (2016b) Improving agile requirements: the Quality User Story framework and tool. Requirements Engineering Journal, 1-21.

- Lucassen, G., Dalpiaz, F., van der Werf, J. M. E. M., & Brinkkemper, S. (2016c). Visualizing User Story Requirements at Multiple Granularity Levels via Semantic Relatedness. Proc. of ER 2016 (forthcoming)

- Robeer, M., Lucassen, G., Van der Werf, J. M. E. M., Dalpiaz, F., & Brinkkemper, S. (2016). Automated Extraction of Conceptual Models from User Stories via NLP. Proc. of RE 2016 Friday: 11.30 @ Modeling and Simulations track

re tutorial user stories

Education