agile development process -...

E-Guide

Agile Development Process

More than ever organizations are looking to identify ways to reduce

application development costs and improve productivity. Many

companies are transitioning to the agile process as a way to achieve

both while reducing software defects and improving the time to

market.

The SearchSoftwareQuality.com E-Guide: Agile Development Process

examines organizational and team challenges of implementing the

agile process. This E-Guide details how to scale agile software

development to large organizations by scaling agile practices and agile

work; how to transition small and large development teams to Scrum;

and how to ensure the communication and workflow of remote and co-

located agile teams.

Sponsored By:

http://www.ibm.com/us

SearchSoftwareQuality.com E-Guide


Sponsored By: of 16

E-Guide


Table of Contents

Scaling Agile software development: Challenges and solutions

Large-scale Agile: Making the transition to Scrum

One hundred percent distributed Agile teams: Communication and workflow

Resources from IBM



Sponsored By: of 16

Scaling Agile software development: Challenges

and solutions

By Nari Kannan

The ability to scale agile software development to large organizations has always had

skeptics. Typical arguments are that agile works for small software development groups but

not for large ones. Or, that they use outsourcing providers with fixed price contracts for

software development and an agile methodology does not provide the discipline for them to

fulfill contracts without a great deal of specification and design upfront.

Scaling agile software development to large organizations is still possible if enough attention

is paid to:

• Scaling agile practices – Understanding agile practices and making sure that the rest

of the organization also does the same.

• Scaling agile work – Organizing work and people appropriately for scaling agile

properly.

Scaling agile practices to a large organization

Lean thinking guides agile practices significantly. The sources of many ideas in lean thinking

are the Toyota production system (TPS) and the House of Quality that many lean companies

practicing lean thinking use. The main principle in lean thinking is the idea that people are

inherently responsible, capable of self-organization and learning, and do not need active

management from supervisors. The other main idea in lean thinking is continuous

improvement. Continuous improvement is best practiced by software development people

that actually do the work. The Japanese technique of Gembutsu or "Go See" is applicable

here. The principle is that each software development in each product or project

environment is different and that methodologies and practices need to be tailored by the

people who do the work after observing what is happening with the project closely for a

while.



Sponsored By: of 16

Reduction of waste is another strong agile practice that needs to be understood clearly and

scaled in a large organization. Duplication of code in two different software projects is pretty

common and well-known. Teams waiting for requirements documents to be complete and

approved, waiting for design documents for coding to start, waiting for completed code for

testing to start are all well-known wastes due to delays. Many processes like the stage-gate

and other product management practices introduce their own delays. Software development

teams waste time twiddling their thumbs while they are waiting.

For success, misconceptions about scaling with agile in large organizations need to be

addressed. Agile does not mean there should be no documentation. Agile does not mean

you are not disciplined. Agile does not mean no planning. The Agile Manifesto lays out a

continuum of emphasis – individuals and interactions over processes and documentation. It

just means that individuals and interactions are more important than any one process or

extensive documentation, but not unimportant. Removing misconceptions is very important

for agile to scale because such misconceptions have the potential to derail adoption.

Scaling agile work to a large organization

Organizing agile work to a large organization consists of two major areas that need to be

addressed. Tackling one without the other is ineffective and counterproductive. These are

organizing the work to be done and organizing people.

Organizing work traditionally has been done along some internal divisions like product

divisions (personal tax preparation products, corporate tax preparation software products,

for example) or functional divisions (user interface group, database management group,

middleware group, etc.) or based on platforms (Windows, Windows Mobile, Unix, etc).

All of these ways of organizing work waste enormous amounts of time in unused talents and

waste of time in waiting. In practice, there are almost always delays in handoffs and people

are waiting for someone to give them something so that they can continue their own work.

The UI group may be waiting for the middleware group to finish their designs. There could

be enormous duplication of code – the two product divisions could be writing the same code

to do the same thing without realizing it. There could be very good programmers that are



Sponsored By: of 16

good in UI design, coding and database design and implementation. The silo method of

organizing work leaves a lot of talent untapped and unused.

It is better to organize work around requirements or features. Requirement areas will have

their own requirement area owners that report to the product owner. Requirement areas

could be IP protocols or performance or device support in the case of a telecommunication

software product, for example.

Or, they could be organized around features such as downloading device data or batch

download of data in the case of an embedded hardware/software product. In both cases,

you will see that teams address entire set of coding functions - coding, UI design and

development, and database design and development also.

Organizing people for scaling agile requires a lot of organizational change. It needs to be

reflected in the policies and procedures of the company and needs to be adopted and used

on a daily basis diligently for agile to be effective. Just adopting the superficial ways or

organizing work without addressing these will be ineffective. Organizing people needs to

follow the principles of empowerment of people, self-organization and self-management.

Reporting hierarchies need to be flattened first and the reporting spans should be larger. If

people are empowered and self-managed, you need fewer managers to oversee their work.

Managers need to be coaches or subject matter experts. Multi-skilling and job rotation

needs to be built into the system. Software engineers may need to be experts at coding,

architecture, design, database design and development and testing. Job titles prevent

people from utilizing their full potential and contribute their best to the organization. Since

now teamwork needs to be emphasized, reward structures need to be modified and job

titles get in the way of team work. Job titles need to become generic and pay is tied to

seniority and experience and automatic. These are pretty radical changes but without them

re-organizing work alone may not help agile scale. These changes enable employees to be

more proactive in taking on responsibilities and self-management and contribution.



Sponsored By: of 16

Agile scaling, distributed and offshore software development

Agile scaling is really difficult with distributed and offshore software development. Many

ideas that work when software development is centralized break down when the teams are

distributed or done offshore.

Cultural differences, time zone differences do not pose big problems when software

development is centralized. However, they become big problems in scaling agile

development when teams are distributed, and some are offshore. The key here is to adapt

and modify agile practices appropriately so that they work properly. A Daily Standup is

possible if the entire team is in the same building or campus. If they are distributed across

the globe only a weekly standup may be more practical and advisable. Clients or product

owners may not be available for a daily standup at odd hours (because of time zone

differences) and a weekly standup may be the only feasible solution.

Another way to address this is to use the distributed or offshore team as a self-contained

requirement area group or a feature group. Communication is the #1 problem with

distributed or offshore teams. There are no easy answers there except to use many

communication mechanisms as possible – Skype or daily video conference, weekly team

meetings, personal visits onsite by offshore teams, and personal visits by the client,

onshore teams to the offshore location, at least every quarter or so.

Conclusion

Agile software development works in the small and can also work in the large, if approached

carefully and many organizational changes and approaches are diligently made, and

followed. Understanding and infusing the principles behind agile practices goes a long way

in making scaling agile to large organizations successful. The keys are in not adopting only

the superficial rituals but really adapt agile practices to the situation at hand, one

organization at a time. Every organization and every software development project has its

own unique aspects and a single magic bullet may not work in all cases. The underlying

principle in agile is this flexibility and adaptation rather than blindly following a single set of

prescriptions!



Sponsored By: of 16

About the author:

Nari Kannan is currently the Chief Delivery Officer of V-Soft Consulting, Inc., a Louisville,

Kentucky-based software consulting firm. Nari has 20 years of experience in information

technology and started out as a senior software engineer at Digital Equipment Corp. He has

since served variously as vice president of engineering or CTO of six Silicon Valley startup

companies, working in the areas of business process improvement, IT consulting,

automotive claims processing, human resources and logistics applications.



Sponsored By: of 16

Large-scale Agile: Making the transition to Scrum

By Matt Heusser

If you've got a technical staff of fifteen people, a Scrum conversion isn't that hard. You get

everyone in one room, lock the door, give them a problem and say "solve it."

Ok, ok, it's a little more work than that. Still, in that environment, switching to Scrum is

mostly a matter of mindset and culture.

With fifty people, though, it's not so easy. Five hundred or more?

Forget about it.

Large organizations have policies, organizational structures, politics, and just plain inertia

that will hold them back from an overnight overhaul. In other words: "big bang" Scrum

projects are expensive, cause the team to slow to a crawl for weeks or months before

improving performance, and are likely to fail.

Let's look at an alternative to "Everybody comes in on Monday and does Scrum."

A typical Scrum scenario

As an example, let's look at a large software development organization. Specifically let‟s

look at a software company that has grown through acquisition, off-shored some of its

development and test operations and also needs to integrate with some third-party

applications. Changes in the software will now require “ripples” to other teams and the

communication costs for a single change skyrocket.

Yet on every large team I have ever worked with there was a small team trying to get out.

This small team did the core of the work -- the analysts, developers, and testers who were

going to work on the core components. In many cases, everyone else's job was to translate

the core work, push it to include their sub-products, or integrate with the new core engine.

Ken Schwaber, co-founder of modern Scrum, claims that, over time, the core work tends to



Sponsored By: of 16

be considered „legacy,‟ or at least less interesting. This means that the staff capable of

doing the work tends to leave or move on to other roles within the organization. Over time,

the core engine becomes the bottleneck.

Suddenly performance falters and a large-scale Scrum conversion looks appealing.

Start with a tiger team

Before we jump to a big conversion, let‟s consider a series of small refinement, starting with

the core development team.

It‟s likely this „team‟ is actually a half-dozen to a dozen people, scattered throughout the

organization. Instead of re-organizing six departments, we temporarily borrow the people

we need and create one tiger team. To fund the tiger team, all we need is a single war

room, a little consulting time, and a little management attention. Perhaps we send the team

to Scrum training; if we do, this costs us twenty person-days instead of two hundred. We

allow the team to self-organize and self-direct while insulating the team from politics and

providing it some sort of interface to work with other, more traditional organizations.

Another way to do this is to respond to an emergency, crisis, or big piece of software that is

needed in a hurry, taking volunteers.

In either case, we have now created a single multi-disciplinary product team. At the same

time, we have found the bottleneck holding up development and massively decreased its

communications delays. So this new big project, be it an upgrade, a migration, or some

change to comply with some law, is going get done faster than usual.

By the end of the project we now have a high-functioning, multi-skilled team. Instead of

breaking the team up, we formalize it, creating a sort of special-missions team that

operates differently than the old functional teams.

If you can get this far, pat yourselves on the back. You‟ve won. You can now sit back,

resting on your laurels, certain that you have improved the organization in a real and

specific way.



Sponsored By: of 16

Expand

Sure, you could stop at one Scrum team. You might consider the Scrum team some sort of

elite unit, and the other teams „mere humans‟ or „doing it the old way.‟

If the new Scrum team appears happy, if they appear successful, if they get the promotions

and recognition, well... it‟s likely the other teams will start to feel ignored, or maybe a bit

jealous.

So take volunteers to form Scrum team number two. Once this second team completes its

first project, we give it a formal manager and transfer the team members, thus creating

another independent unit. Lather, rinse and repeat.

Now it‟s unlikely that every single team member will want to move over to a Scrum team,

and that‟s okay. In addition to new development, the team will have maintenance requests,

ongoing operations and support, or what some call MOOSE work. The people that stay back

are likely the ones who take pride in straightforward, repeatable work, and there will likely

be plenty of that kind of work to “keep the lights on.” Meanwhile, over time, the Scrum

teams eventually become the primary drivers of new feature development.

How do I staff and manage this transition?

The newly-formed Scrum teams will need Scrum-masters, coaches, some sort of product

manager, and possibly a director or two to report to. So there will be plenty of opportunity

for traditional managers to transition over time to the Scrum side of the house.

At the same time, the groups those people manage will be shrinking. That means that as

leaders transfer over to the Agile “side of the house,” few managers are needed to control

the MOOSE work. As the MOOSE work becomes more administrative and easy to measure,

it will make sense to merge teams and have fewer managers of those teams.

Eventually you‟ll end up with two organizations: The development staff and the systems

support staff. Or perhaps not; you can always stop somewhere in the middle with a few



Sponsored By: of 16

active Scrum teams, and other organizations largely untouched. In a heavily outsourced or

distributed organization, that might be the best win possible, at least for the time being.

Conclusions

This friction between those who want a new way to do things, and folks who prefer the old

way, is the cause of a fair amount of conflict, strife, and failure in Scrum adoption.

Some people want to have explicit goals and defined process, and won‟t be excited about an

edict to “Start Scrum on Monday; figure out what needs to be done and do it.” On top of

that, groups organized by function will have a number of functional managers whose role

may be disrupted by a Scrum transition. My advice for Agile transition is not “big bang” but

instead incremental, done in a way that respects people and gives them options.

This kind of focusing on people isn‟t limited to Scrum adoption. It turns out that every time

you change the process, you get to decide if you want to focus on people sitting in the same

room and talking or churning out documentation. You get to decide if you want to manage

to plan or to outcome -- and if you‟d like to plan every action of the team up front, or

enable them to make good decisions in the moment.

Choose wisely.

About the author:

Matt Heusser is a member of the technical staff at Socialtext, focusing on testing Web and

mobile applications. The initial lead organizer of the Great Lakes Software Excellence

Conference (now in it's 5th year), and lead editor of the "How to Reduce the Cost of

Software Testing" project, Matt has been a working developer, tester, QA lead, and project

manager since 1998. In addition to his day job, Matt did a two-year stint as a part-time

instructor in Information Systems at Calvin College, and teaches and speaks at software

development events. You can learn more about Matt on his blog, Creative Chaos, or follow

his tweets at mheusser.



Sponsored By: of 16

One hundred percent distributed Agile teams: Communication and workflow

By Chris McMahon

For most of the last five years, I have had the pleasure and privilege of working as a

software tester on two different highly successful agile teams whose members all work

remotely. The team members are distributed around the US, and in a couple of cases,

around the world. There are a number of advantages to such an arrangement. For one

thing, it is possible to hire the most talented and skilled people available, since geography is

not a factor in the decision of whom to hire. Distributed teams do not have to settle for

whomever is available locally. For another thing, everyone is comfortable all the time, and

this is important to our happiness.

But there are critical differences between a remote agile team and a co-located agile team

that must be addressed if the team is to succeed. These differences fall into two main

categories: communication and workflow. A remote team necessarily has much reduced

communication bandwidth compared to a co-located team, and thus must communicate

among themselves with great efficiency. And having a well-designed workflow is critical. A

co-located team can change their development process in an instant. Doing that on a

remote team is much more difficult. Using the appropriate ALM tools for a fully distributed

team is critical for success.

Remote communication

The most important difference in communication between remote agile teams and co-

located agile teams is that most of the communication on a remote team is written and not

verbal. It is critical that everyone on a remote team has excellent writing skills and be able

to express themselves clearly and unambiguously in writing. For a co-located team, that

which is unclear can be worked out right in the room. For a distributed team, that activity

becomes very expensive.



Sponsored By: of 16

Given good writing skills, a distributed team needs many communication channels. And

these channels need to be as public as possible, so that everyone is aware of what the other

team members are doing at every moment. Here are some examples of the most important

communication channels for remote teams, arranged in their preferred order of use from

most public to least public:

IRC

A dedicated multi-channel real-time chat application that provides information on presence

is critical. Internet Relay Chat (IRC) is a mature and inexpensive way to achieve this. Any

number of IRC servers are available for free, as are applications (generally called "bots" or

"chatbots") to improve the experience of IRC users. IRC is where remote team members

"sign in" for work. It is where we work out our day-to-day issues, report ongoing status, ask

questions. Everyone can see the channels they want to see. For instance, there might be

channels named "#dev", "#qa", and "#ops." Where I work today, we have a channel called

"#party" that is just for socializing. I join and leave #party depending on how much

concentration my current work requires.

Wiki

The wiki is where the team keeps all of its institutional and historical knowledge. It is the

single point of reference for documentation on the system itself and on the team's process.

An intraweb with documents is not an acceptable substitute for a full-featured wiki, nor is a

document-management system like Sharepoint.

VOIP

A voice over IP system is important for daily standups and for conversations that are too

complex for IRC. For small teams, Skype may be adequate. My team is using GoToMeeting

right now, which also provides the ability for "presenters" to share their desktop. This is

where we review our burndown charts, for instance.



Sponsored By: of 16

IM

Instant messaging is important for one-to-one chats. The actual client is immaterial. I tend

to use GTalk because it is convenient, but IRC also provides this ability.

Email

Email is not the place for documentation, nor is it the place to hold a conversation. Email is

for broadband announcements, overall status updates on systems, questions for the whole

company, and other bits of general information. Conversations on the work itself should

take place in other channels.

Remote workflow

On a co-located agile team, stories, tasks, obstacles, bugs, problems, etc. are generally

tracked as physical index cards on a board. Obviously, this is not possible for a remote

team. Remote teams need tools to do this, and those tools must be very flexible.

One of the hallmarks of a successful agile team is that the team is constantly adjusting their

own software development process. Any workflow tool a remote agile team uses must be

able to accommodate those constant adjustments. I have used two such tools with good

success.

On one team we used a commercial wiki to track stories and tasks. This wiki was very well

designed and provided an enormous number of API calls that allowed us to tweak the

behavior of the system at a moment's notice. Each large story and task was described on a

wiki page containing links to smaller stories on other wiki pages. We managed the status of

each story and task by tagging each wiki page in particular ways. Using the tags, we could

extract overall status information for burndown charts and other such reporting.

On that team we had a separate defect tracking system, but that system also had a highly

hackable API, and our wiki and our defect tracking system talked to each other in a way

that made managing defects a seamless experience.



Sponsored By: of 16

On my current team we manage everything in a highly-customized system of ALM tools that

integrate tightly. Stories, tasks, obstacles, problems, defects, status, everything has equal

status and is managed in the same common dashboard available to everyone on the team.

This is a critical point: every bit of information the team has is always available to everyone

on the team. To put this information in silos and say for example "this information is only

available to developers" or "this information is only available to QA" is a recipe for failure. If

people on the team lack information about the overall work of the whole team, those people

will make bad decisions about their own work and the work of others.

The future of remote agile development

I think we will see more remote agile teams in the near future. Working this way can be

rewarding, efficient, and even environmentally responsible. I think we will see more and

more of the most talented people in the industry shift to this way of working as the cost of

being co-located increases.

But I do perceive a higher risk of failure for remote teams. For one thing, only very skilled

and talented people can negotiate the reduced bandwidth and succeed at such work. For

another thing, excellent communication and a well-considered, manageable agile workflow

is critical. This article is for those teams.

About the author:

Chris McMahon is a software tester and former professional bass player. His background in

software testing is both deep and wide, having tested systems from mainframes to web

apps, from the deepest telecom layers and life-critical software to the frothiest eye candy.

Chris has been part of the greater public software testing community since about 2004,

both writing about the industry and contributing to open source projects like Watir,

Selenium, and FreeBSD. His recent work has been to start the process of prying software

development from the cold, dead hands of manufacturing and engineering into the warm

light of artistic performance. A dedicated agile telecommuter on distributed teams, Chris

lives deep in the remote Four Corners area of the U.S.



Sponsored By: of 16

Resources from IBM

Scott Ambler's eBook on scaling Agile development

One's enough for Agile Application Development Management: Ride the agile wave

with IBM!

Organizing global Agile teams - Take the free course from IBM

About IBM

At IBM, we strive to lead in the creation, development and manufacture of the industry's

most advanced information technologies, including computer systems, software, networking

systems, storage devices and microelectronics. We translate these advanced technologies

into value for our customers through our professional solutions and services businesses.

worldwide.

https://www.ibm.com/services/forms/signup.do?source=swg-rtl-sd-wp&S_PKG=eb_agile

https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=dw-c-wcsdpr&S_PKG=080210A

https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?lang=en_US&source=dw-c-wcsdpr&S_PKG=080210A

https://www14.software.ibm.com/webapp/iwm/web/signup.do?lang=en_US&source=dw-c-wcsdpr&S_PKG=112410A&S_TACT=100EW0ZW

http://www.ibm.com/us

agile development process -...

Documents