chapter 3 · chapter 3 agile software development slide 14 “manifesto for agile software...

Chapter 3 Agile Software Development Slide 1

Chapter 3

Agile Software Development


Topics covered

Rapid software development and the emergence of

agile methods

Agile methods

Agile development techniques

Extreme programming (XP)

Agile project management

Scaling agile methods


Rapid software development

and the emergence of agile

methods


Rapid software development

Rapid software development and delivery is the most

critical requirement for most business systems.

• Businesses now operate in a global, rapidly changing

environment.

• They must be responsive to new opportunities, markets,

changing economic conditions, and competition.

• New software has to be developed quickly to facilitate this

responsiveness.

Businesses may be willing to accept lower quality

and compromise on requirements if they can deploy

essential new software quickly.

(cont’d)


Rapid software development (cont’d)

Plan-driven development processes that completely

specify requirements and then design, build, and test a

system are essential for some types of systems,

but are not geared to the kind of rapid development

needed in fast moving business environments.

Agile development methods emerged in the late 1990s

to radically reduce the delivery time for software

systems used in such environments.


Common characteristics of all agile

development methods

1. Processes of specification, design and imple-

mentation are interleaved.

• No detailed system specification – “user requirements” is

an outline definition of important system characteristics

• Design documentation is minimal and/or automatically

generated

2. System is developed in a series of increments with

end-users and other stakeholders involved in their

specification and evaluation, during which hanges

and new requirements may be proposed.

(cont’d)


Common characteristics of all agile

development methods (cont’d)

3. Extensive tool support is used to support the devel-

opment process.

• Automated testing tools

• Configuration management and system integration tools

• User interface generation automation tools

• etc.


Essential distinctions between plan-

driven and agile approaches

Agile approaches…

• Use incremental development methods in which

increments are small; iteration occurs across activities

• New releases made available every two or three weeks

• Involve customers in development to get rapid feedback

on changing requirements

• Minimize documentation by using informal communication

• Consider design and implementation to be the central

development activities

• Incorporate other activities (e.g., RE and testing) into

design and implementation

(cont’d)


Essential distinctions between plan-

driven and agile approaches (cont’d)

Plan-driven approaches…

• Separates development activity into distinct stages (i.e.,

requirements definition, design, implementation, inte-

gration and system test) with outputs (formal documents)

associated with each stage planned in advance

• Iteration normally occurs within activity stages only

• Do not normally involve customers in development after

requirements are defined, except for acceptance testing

• Formal documentation is used to communicate between

development stages

(cont’d)


Essential distinctions…caveats

But these distinctions are not invariant.

Software development often combines elements of

plan-driven and agile approaches. For example…

• A plan-driven process may involve allocating requirements

and then planning design, implementation, and test stages

as a series of increments (“Mill’s Incremental Delivery”)

• An agile process is not inevitably code-focused; developers

may decide than an iteration should produce system models

and documentation instead of new code.


Agile methods


Agile methods background and aim

Dissatisfaction with the overhead of heavyweight,

plan-driven development approaches of the 1980s

and 1990s led to the creation of agile methods.

• Careful project planning, formalized QA, CASE-

supported analysis and design, and a controlled,

rigorous process are appropriate for large, long-

lifetime systems developed by large, geographically

dispersed teams over many years, but...

• When applied to small and medium-sized business

systems, the overhead dominates and interferes

with rapid development and delivery.

(cont’d)


Agile methods background and aim

The newly proposed agile methods allowed:

• A focus on the software itself rather than its

design and documentation.

• A reliance on iterative development.

• An ability to deliver working software quickly

which can evolve quickly to meet rapidly

changing requirements.

• A reduction in bureaucracy by avoiding

unnecessary work and overhead.

See: www.agilealliance.org

http://www.agilealliance.org/


“Manifesto for Agile Software Developers”

We are uncovering better ways of developing software

by doing it and helping others do it. Through this

work we have come to value:

• Individuals and interactions over processes and tools

• Working software over comprehensive documentation

• Customer collaboration over contract negotiation

• Responding to change over following a plan

That is, while there is value in the items on the right,

we value the items on the left more.


“12 Principles Behind the Agile

Manifesto”

1. Our highest priority is to satisfy the customer through

early and continuous delivery of valuable software.

2. Welcome changing requirements, even late in

development. Agile processes harness change for

the customer’s competitive advantage.

3. Deliver working software frequently, from a couple of

weeks to a couple of months, with a preference to

the shorter timescale.

4. Business people and developers must work together

daily throughout the project.

(cont’d)



Manifesto” (cont’d)

5. Build projects around motivated individuals. Give them

the environment and support they need, and trust them

to get the job done.

6. The most efficient and effective method of conveying

information to and within a development team is face-

to-face conversation.

7. Working software is the primary measure of progress.

8. Agile processes promote sustainable development.

The sponsors developers, and users should be able to

maintain a constant pace indefinitely.

(cont’d)



Manifesto” (cont’d)

9. Continuous attention to technical excellence and

good design enhances agility.

10. Simplicity – the art of maximizing the amount of work

not done – is essential.

11. The best architectures, requirements, and designs

emerge from self-organizing teams.

12. At regular intervals, the team reflects on how to

become more effective, then tunes and adjusts its

behavior accordingly.


Agile methods applicability

Agile methods have been particularly successful for

two kinds of system development:

• Software companies developing small or medium-

sized (generic) products for sale. (Virtually all such

software products and apps are now developed using

an agile approach.)

• Custom system development within an organization

where there is clear commitment from the customer to

be involved in the process and where there are few

external stakeholders and regulations that affect the

software.


Agile development techniques


Extreme programming (XP)

A very influential agile method, developed in the late 1990s, that introduced a range of agile development techniques.

Takes an “extreme” approach to iterative devel-

opment:

• New versions may be built several times per day.

• Increments are delivered to customers every 2 weeks.

• All tests must run successfully for every build.

“XP = iterative development on steroids”


The XP release cycle


XP practices

XP was controversial as it introduced a number of

agile practices that were quite different from the

development practice of that time.

These practices reflect the principles behind the agile

manifesto…

(cont’d)


XP practices (cont’d)

In practice, application of XP practices as originally

proposed has proved to be difficult.

XP cannot be readily integrated with the manage-

ment practices and culture of most businesses.

Those adopting agile methods pick and choose

those practices that are appropriate – either into their

own processes, or more commonly, in conjunction

with a management focused agile method such as

Scrum.


Sommerville’s “most important

agile development practices

introduced by XP”:

1. User stories for specification

2. Refactoring

3. Test-first development

4. Pair programming


User stories for requirements

A customer or user is part of the XP team and is

responsible for making decisions on requirements.

Requirements are expressed as scenarios or

user stories written on index cards.

The development team breaks them down into

implementation tasks, which are the basis of

schedule and cost estimates.

The customer chooses stories for inclusion in the

next release based on priorities and schedule

estimates.


A “prescribing medication” story


Examples of task cards for

prescribing medication


Refactoring

Conventional SE wisdom is design for change

(e.g., via “information hiding”) to reduce main-

tenance costs – i.e., “it is worth spending time

and effort anticipating changes as this reduces

costs later in the life cycle.”

XP maintains that this is NOT worthwhile since

changes cannot be reliably anticipated –

instead, it proposes constant code improve-

ment (“refactoring”) to make changes easier

when they have to be implemented.


Refactoring (cont’d)

Team looks for possible software improvements

and makes them even if there is no immediate

need for this.

Improves understandability of software and so

reduces the need for documentation.

Changes are easier to make because the code is

well-structured and clear.

However, some changes requires architecture

refactoring which is much more expensive. (An

obvious disadvantage of: “Simple design – enough design is

carried out to meet the current requirements and no more.”)


Examples of refactoring

Reorganizing class hierarchies to remove

duplicate code.

Tidying-up and renaming attributes and

methods to make them easier to understand.

Replacing in-line code with calls to methods

that have been included in a program library. (= information hiding!)


Test-first development (TFD)

XP required an approach for program testing

without a formal specification.

XP testing features:

• Test-first development:* tests are written before

coding to clarify requirements.

• Incremental test development from scenarios

• User involvement in test development and validation

• Automated test harnesses are used to run all com-

ponent tests each time that a new release is built.

* Test-Driven Development is a generalization of this idea that will

be considered later in the course.


Customer involvement in TFD

The role of the customer is to help develop

acceptance tests for the stories that are to be

implemented in the next release.

In theory, then, all new code is validated to

ensure that it does what the customer requires.

However, customers may be reluctant to accept

this role – either because of limited time avail-

ability or because they feel that providing the

requirements was a sufficient contribution to the

process.


Test case description for dose

checking


Test automation

Tests are written as executable components

before the task is implemented.

• Test components simulate the submission of input to

be tested and check that the result meets the output

required.

• An automated test framework (e.g. Junit) makes it

easier to write executable tests and submit them for

execution.

• Whenever any functionality is added to the system,

the tests can be run and problems that the new code

has introduced can be caught immediately.

“automated regression testing”


Problems with test-first

development

1. Programmers prefer implementing tasks over writing

test scripts for which they therefore may take short

cuts – e.g., they may not check for all possible

exceptions that may occur.

2. Some tests can be difficult to write incrementally –

e.g., for code that implements “display logic” and

workflow between screens when a user interface is

complex.

Thus, although a large set of frequently run tests

may suggest the system is complete and correct, this

may not, in fact, be the case.


Pair programming

Programmers work in pairs, sitting together at the

same workstation to develop software.

Pairs are created dynamically so that all team

members work with each other during the develop-

ment process.

Supports idea of collective ownership and responsi-

bility for the system (“egoless programming”).

Resultant sharing of knowledge is very important as it

reduces the overall risks to a project when team

members leave.

(cont’d)


Pair Programming in XP (cont'd)

Serves as an informal, continuous review

process, as each line of code is looked at by at

least two people.

This, in turn, encourages refactoring, from which

the whole team benefits.

Some (but not all) evidence suggests that a pair

working together may be more productive than

two programmers working separately.


Agile project management


Plan-driven and agile project

management

The principal responsibility of software project

management is to ensure (quality) software is

delivered on time and within the planned budget.

The traditional approach is plan-driven. Managers

determine what should be delivered, when it should

be delivered, and who will work on the deliverables.

Agile project management requires a different

approach – one that is adapted to incremental

development and the particular strengths of agile

methods.


The Scrum* management approach

Scrum is a general agile method that focuses on

managing iterative development rather than specific

agile practices.

There are three phases in Scrum...

• Planning phase: establish general objectives for the

project and design the software architecture.

• Sprint or development phase: a series of “sprint cycles,”

where each cycle yields an increment of the system.

• Closure phase: wraps-up the project, completes required

documentation such as help frames and user manuals,

and assesses the lessons learned from the project.

* Scrum is a rugby term for the close-knit, shoulder-to-shoulder formation a rugby

team forms to jointly move the ball forward.


Scrum terminology

Scrum term Definition

Development team A self-organizing group of software developers, which should be no more than

7 people. They are responsible for developing the software and other

essential project documents.

Potentially shippable

product increment

The software increment that is delivered from a sprint. The idea is that this

should be ‘potentially shippable’ which means that it is in a finished state and

no further work, such as testing, is needed to incorporate it into the final

product. In practice, this is not always achievable.

Product backlog This is a list of ‘to do’ items which the Scrum team must tackle. They may be

feature definitions for the software, software requirements, user stories or

descriptions of supplementary tasks that are needed, such as architecture

definition or user documentation.

Product owner An individual (or possibly a small group) whose job is to identify product

features or requirements, prioritize these for development and continuously

review the product backlog to ensure that the project continues to meet critical

business needs. The Product Owner can be a customer but might also be a

product manager in a software company or other stakeholder representative.


Scrum terminology (cont’d)

Scrum term Definition Scrum A daily meeting of the Scrum team that reviews progress and prioritizes

work to be done that day. Ideally, this should be a short face-to-face

meeting that includes the whole team.

ScrumMaster The ScrumMaster is responsible for ensuring that the Scrum process is

followed and guides the team in the effective use of Scrum. He or she is

responsible for interfacing with the rest of the company and for ensuring

that the Scrum team is not diverted by outside interference. The Scrum

developers are adamant that the ScrumMaster should not be thought of

as a project manager. Others, however, may not always find it easy to

see the difference.

Sprint A development iteration. Sprints are usually 2-4 weeks long.

Velocity An estimate of how much product backlog effort that a team can cover in

a single sprint. Understanding a team’s velocity helps them estimate

what can be covered in a sprint and provides a basis for measuring

improving performance.


The Sprint cycle

Sprints are fixed length (normally 2-4 weeks) and

correspond to the development of a new system

release in XP.

The starting point for each sprint is the product

backlog, which is the list of work to be done on

the project.

The project team works with the customer to

select the (stable) features and functionality to

be developed during the sprint.

(cont’d)


The Sprint cycle (cont’d)

Once these are agreed upon, the team is isola-

ted from the customer, with all communications

channelled through the so-called “Scrum

master.”

The role of the Scrum master is to protect the

development team from external distractions.

At the end of the sprint, the development work is

reviewed and presented to stakeholders. The

next sprint cycle then begins.


The Sprint cycle (cont’d)


Teamwork in Scrum

The Scrum master is a facilitator who arranges daily

meetings, tracks the backlog of work to be done,

records decisions, measures progress, and commu-

nicates with customers and management.

The whole team attends short daily meetings* to

share information, describe their progress and prob-

lems, and plan their work for the following day.

Thus, everyone on the team knows what is going on

and, if problems arise, can re-plan their short-term

work to cope with them.

* See, for example, “The Daily Scrum Meeting.”

http://www.mountaingoatsoftware.com/scrum/daily-scrum


Scrum benefits

1. The product is broken down into a set of manageable and

understandable pieces.

2. Unstable requirements do not, in theory, hold-up progress.

3. The whole team has visibility of everything, so team

communication is improved.

4. Customers see regular, on-time delivery of increments and

learn how the product works.

5. Trust between customers and developers is established

and a positive culture is created in which everyone expects

the project to succeed.


Distributed scrum

As originally designed, scrum was intended for co-

located teams with all team members physically

attending daily meetings.

Much software development now involves distributed

teams, allowing lower costs, access to more special-

ized skills, and even 24 hour work schedules.

Typically, the product owner is in a different country

than the development team(s).


Distributed scrum (cont’d)



Agile methods work well for small and medium sized

projects that can be developed by a small, co-located

team.

Some argue this is because of the improved comm-

unications that naturally take place when small

groups work together in one location.

To scale effectively, agile methods must be modified

to cope with larger, longer projects utilizing multiple

development teams, perhaps working in different

locations.


Two perspectives on “scaling”

agile methods

“Scaling-up:” using agile methods for developing large

software systems that cannot be developed by a small

team.

“Scaling-out:” introducing agile methods across a large

organization with years of (traditional) software

development experience.

In either case, it is essential to maintain agile funda-

mentals (i.e., flexible planning, frequent system releases,

continuous integration, test-driven development, and good

team communications).

classic “technology transfer”

problem


Practical problems with agile

methods

The informality of agile development is incompatible

with the legal approach to contract definition that is

traditionally used in large companies.

Agile methods are most appropriate for new software

development rather than software maintenance. Yet

the majority of software costs in large companies

come from maintaining their existing systems.

Agile methods are designed for small, co-located

teams, yet much software development now involves

worldwide distributed teams.


Contractual issues

Most software contracts for custom systems are

based around a specification, which sets out what

has to be implemented by the developer for the

customer.

But this precludes interleaving specification and

development as is the norm in agile development.

A contract that pays for developer time rather than

functionality is required.

• However, this is seen as high risk my many legal depart-

ments because what needs to be delivered cannot be

guaranteed.


Agile methods and software

maintenance

Since most organizations spend more on main-

taining existing software than they do on new

software development, if agile methods are to be

successful, they must support maintenance as

well as original development.

Issues that can arise:

• Lack of product documentation

• Keeping customers involved in the development

process

• Maintaining the continuity of the development team

(cont’d)


Agile methods and software

maintenance (cont’d)

Maintenance requires assessing the impact of

proposed system changes. But the lack of a

requirements document may make this difficult.

While a customer may be able to justify the full-time

involvement of a representative during system

development, this is less likely during maintenance

where changes are not continuous.

Agile methods rely on team members understanding

aspects of the system without having to consult

documentation, so if the development team is broken

up, this implicit knowledge may be lost.


Agile and plan-driven methods

A fundamental requirement of scaling agile methods

is to integrate them with plan-driven approaches.

Early adopters of agile methods were enthusiasts

and deeply committed to the agile manifesto.

But the principles underling agile methods are

sometimes difficult to realize in practice.

Thus, most large “agile” projects combine practices

from plan-drive and agile approaches.

To decide on the balance you have to answer a

range of technical, human, organizational questions.


Factors influencing the balance


System issues

1. How large is the system being developed?

• Agile methods are most effective when a

relatively small co-located team can

communicate informally.

2. What type of system is being developed?

• Systems that require a lot of analysis before

implementation need a fairly detailed design to

carry out this analysis.

(cont’d)


System issues (cont’d)

3. What is the expected system lifetime?

• Long-lifetime systems require documentation to

communicate the intentions of the system

developers to the support team.

4. Is the system subject to external regulation?

• If a system is regulated you will probably be

required to produce detailed documentation as

part of the system safety case.


People and team issues

1. How good are the designers and programmers

in the development team?

• Agile methods may require higher skill levels than

plan-based approaches in which programmers simply

translate a detailed design into code.

2. How is the development team organized?

• Design documents may be required if the team is

distributed.

3. What support technologies are available?

• IDE support for visualisation and program analysis is

essential if design documentation is not available.


Organizational issues

1. Is it important to have a detailed specification

and design before moving to implementation? • It may be standard organizational practice to develop a

detailed system specification.

2. Is an incremental delivery strategy realistic? • Customer representatives may or may not be available to

provide feedback of system increments.

3. Are there cultural issues that may affect system

development? • Informal agile development may not fit into an

organizational culture of detailed documentation.


Which label?

“In reality, the issue of whether a project can be

labeled as plan-driven or agile is not very

important. Ultimately, the primary concern of

buyers of a software system is whether or not they

have an executable software system that meets

their needs…”

“Software developers should be pragmatic and

should choose those methods that are most

effective for the type of system being developed,

whether or not these are labeled agile or plan-

driven.”


Agile methods for large systems

Agile methods must evolve to be used for

large-scale software development, because

such systems are :

• much more complex,

• difficult to understand, and

• difficult to manage

Six principal factors contribute to this complexity…


Factors in large systems


IBM’s agility at scale model


Tenets common to all approaches

to scaling agile methods

1. A completely incremental approach to requirements

engineering is impossible.

2. There cannot be a single product owner or customer

representative.

3. It is not possible to focus only on the code of the

system.

4. Cross-team communication mechanisms have to be

designed and used.

5. Continuous integration is practically impossible. (But

frequent system builds and regular releases is

essential. )


Adaptation of Scrum to large-

scale development

Role replication -- each team has a Product Owner

for their work component and ScrumMaster.

Product architects – each team chooses a product

architect and these architects collaborate to design and

evolve the overall system architecture.

Release alignment -- the dates of product releases

from each team are aligned so that a demonstrable and

complete system is produced.

Scrum of Scrums – there is a daily Scrum of Scrums

where representatives from each team meet to discuss

progress and plan work to be done.


Potential difficulties of “scaling out”

agile methods across organizations

1. Project managers who lack agile methods experience

may be reluctant to accept the risk of a new approach.

2. Large organizations often have quality procedures and

standards that all projects are expected to follow which

are likely to be incompatible with agile methods.

3. Agile methods seem to work best when team members

have a relatively high skill level. But skills and abilities

within large organizations are likely to vary widely.

4. There may be cultural resistance to agile methods,

especially in those organizations that have a long history

of using conventional systems engineering processes.


Key points

Agile methods are incremental development methods that focus on rapid development, frequent software releases, reducing process overhead, and producing high-quality code.

Agile development practices include

• User stories for system specification

• Frequent releases of the software

• Continuous software improvement

• Test-first development

• Customer participation in the development team

(cont’d)


Key points (cont’d)

Extreme programming is a well-known agile

method that integrated a range of new program-

ming practices such as frequent releases of the

software, continuous software improvement, and

customer participation in the development team.

A particular strength of XP is the development of

automated tests before code is written. All tests

must successfully execute when an increment is

integrated into a system.

(cont’d)


Key points (cont’d)

Scrum is an agile method that provides a project

management framework. Its focus is a set of sprints,

which are fixed time periods during which a system

increment is developed.

Many practical development methods are a mixture

of plan-driven and agile development.

Scaling agile methods for large systems is difficult.

Large systems require up-front design and some

documentation. Organizational practice may conflict

with the informality of agile approaches.


Chapter 3

Agile Software Development

chapter 3 · chapter 3 agile software development slide 14 “manifesto for agile software...

Documents