a real-life overview of agile and scrum
TRANSCRIPT
Boston Php 9/21/2016
A real-life overview of Scrum
@mtoppa Michael Toppa www.toppa.com
www.toppa.com @mtoppa
Mike Toppaweb engineer
Ruby, Rails, PHP, Wordpress, JS, HTML, CSS SQL, NoSQL, AWS, TDD, Scrum, Kanban
20 years of experience in web development, project management, and functional management. I’ve worked in a variety of environments: dot coms, universities, start-ups, consultancies. One thing I’ve learned is that organizations are like families - they’re all dysfunctional - it’s just a question of how badly
Overview
✤ Tell me your problems…
✤ How traditional software development workflows can go wrong
✤ Tell me what you want…
✤ What makes software development enjoyable and rewarding?
✤ How do we get there?
✤ Agile principles
✤ The Scrum approach
✤ Additional best practices
✤ Scrum adoption anti-patterns and common difficulties
Tell me your problems…
Features
Cost Schedule
1. The iron triangle
Client can pick two
Quality
I’ve explained the triangle to dozens of clients over the years.Programming is not magic. If the client tries to squeeze all 3 sides of the triangle, quality suffers.
Misalignment of authority and responsibility
Cartoon by Mike Lynch Used with permission
- Following this advise lets you cover yourself politically, and is a great way to make everyone who works for you miserable- I've found that misalignment of authority and responsibility can explain a lot of dysfunction that happens in organizations- When you have responsibility for your work but not enough authority over it, you will feel like a cog in machine
“If you go to the store with a huge shopping list and twenty dollars, you need the authority to go to the money machine for more cash, or the authority to make changes to the list.”
Ron Jeffries, Making the Date
What’s happening is that the client is trying to retain authority on the project while giving you the responsibility. But ultimately, for the project to be successful and for both you and the client to be happy, responsibility and authority need to be brought into alignment.
2. Multiple projects and multitasking
Source
Context switching between two projects eats about 20% of a full-time worker’s schedule. The sense of progress with multitasking is an illusion, compared to not multitasking
My experience at the U Penn School of Medicine
Before Scrum: web team setup
With Scrum: web team setup
Too much work
SWAG chart9 developers, 2 product owners, and me supporting- 22 clients with 124 applications3 designers and 1 product owner supporting- about 200 static content web sitesTaking inventory itself was a huge undertaking
Source
To have any chance of success in the long run, you have to claim authority you may not have had previously. You may have to fight for it…
Source
…but you have to always be professional. Think of how doctors behave in an ER. When the pressure is on is when you want them to be at their most professional.
3. The cone of uncertainty
Source
Traditional “Waterfall” Approach
Source
Features determine the cost and schedule Define all requirements up frontLogically break down the workEstimate the effort / durationsPlan out all the work And only then begin the development…while trying to limit any change that will threaten the plan.
“I find your lack of faith disturbing”
5. No opportunity for feedback
Source
In combination with the cone of uncertainty, this is deadly
Result: we build the wrong features
Source
Ask customers what they want at the beginning, when they really don’t knowPenalize them for adding things laterTCL example
“The main thing that pushed Agile and Scrum was that the success rate on traditional projects was terrible; it was 45%. If that was a car-manufacturing place, that would mean you’d throw out every other car you built.”
Ken Schwaber, co-creator of Scrum, 6/21/2011
Overall result: low success rate
Source
Tell me what you want…
For yourself, and for your customers
What makes a job enjoyable?
✤ Autonomy
✤ Reward for effort
✤ Challenging/complex work
“Work that fulfills these three criteria is meaningful.”
– Malcolm Gladwell, “Outliers: The Story of Success”
“Novices believe that quality and velocity are inverse. They think that hacking is fast.
They haven’t yet recognized what professional developers know all to well:
…the higher the quality, the faster you go”
Bob Martin, Vehement Mediocrity
How do we get there?
Add more people?
Brooks’ law: ”Adding manpower to a late software project makes it later”
- Fred Brooks, The Mythical Man-Month
Hold the teams feet to the fire?
Source
This is the death march. Analogy that software development is like a washing machine.
What’s the alternative?
image source
* In 2001, a group of 17 very experienced software developers got together and came up with the Agile Manifesto. They had spent their careers experiencing all the difficulties of the traditional approach, and were motivated to come up with something better
“Agile” consists of a set of principles and ideals
Scrum is the most popular Agile methodology
The original ideas for Scrum came from an article published in Japan in the mid 1980s, which reviewed case studies of early implementations of the ideas in Japanese manufacturing work. In the 1990s Ken Schwaber and Jeff Sutherland gradually formalized Scrum into a set of specific practices for software development.
Agile solution: flip the triangle
Source
Agile takes into account the “cone of uncertainty” - things will change
Agile: frequent feedback is key
Source
Rather than fight the “cone of uncertainty” we embrace it. We are always checking in to make sure what we’re delivering is what the client wants, and we’re ready to adjust priorities based on feedback. At some point we will run out of time or money, and when that time comes, we want to make sure we have delivered the most important features.
Source
Develop systems in small portions at a time (incremental), through repeated cycles (iterative), and take advantage of what was learned in each cycle (for both features and implementation)
Incremental development: slice vertically, not horizontally
Source
This is where developers unfamiliar with Agile freak out
How do you develop a UI or a database in pieces? This seems like it would lead to a giant mess. Remember the iterative part - we can sketch out the overall design, but we build incrementally, fleshing out the details of what’s needed soon
It is possible, it is practical, and there are a lot of people doing it.
It's actually the opposite of messy hacking. Doing it well requires a very disciplined development process, and strong application design skills, as you are trying to maintain a solid application design while always being ready to adapt to change.
The Agile workflow
✤ Deliver business value incrementally
✤ Deliver in order of business priority
✤ Deliver in small, frequent batches
✤ Solicit and respond to business feedback at each delivery
✤ Measure what is left to do
✤ Make decisions based on track record
✤ Stop when it makes senseSource
Why?
✤ The pace of business keeps getting faster
✤ Feedback is essential
✤ Time is scarce
✤ Things will change
Agile: “inspect and adapt”
Source
Single loop learning is “how can we do better”?Double loop learning is “Why do we believe that?”Double loop learning means challenging fundamental assumptions
Applying Agile with Scrum
Scrum: overview
✤ Scrum consists of these core elements:
✤ 3 roles
✤ 3 artifacts
✤ 5 events
✤ That’s it!
✤ But there are also several recommended best practices
The scrum book we’ve been reading presents doesn’t really distinguish the core elements from the recommended best practices.
Scrum has 3 roles
Highly collaborative approach - minimize handoffs and maximize advantages of bringing together people with different perspectives and backgrounds
Scrum role: Product Owner
Responsible for what the team will work on,
and setting priorities,
but not how the work is done
* Responsible for what the team will work on, but not how the work is done* Works closely with clients to understand their needs* Gathers and writes business requirements in small pieces, called “user stories”* Based on client needs, sets priorities for the team* Does not have authority over technical design decisions* Cannot tell an individual team member what to do* A good Product Owner is: available, business-savvy, communicative, decisive, empowered
Scrum role: Scrum Master
A “servant-leader” for the team
* A “servant-leader” for the team - analogous to a physical trainer* Can coach and evangelize, but not issue commands* But does have authority over the Scrum process* Removes obstacles for the team* A good Scrum Master is: responsible, humble, collaborative, committed, influential, and knowledgeable
The team: self-organizing and cross-functional
Source
* Cross-functional team takes collective responsibility for estimating the work, and doing the work* Doing it in the priority order they are asked to follow* Keeping quality high by working together (inspecting each other's code, discussing best technical solutions, etc)
So who’s the boss?
Source
Personnel management exists completely outside this structure.Works best in relatively “flat” organizations where people are given autonomy and achievable goalsAntithetical to top-down, command and control hierarchies
Scrum workflow: events & artifacts
Source
Artifact #1: the product backlog
Source
* The product backlog is the prioritized list of features, usually written in the form of user stories.* It is a living document, subject to re-prioritization between sprints.* The product owner manages it
Event #1: the sprint (aka increment)
Source
* A regular, repeating work cycle* Sprints should have a consistent length, no longer than 1 month* Good to start with 2 weeks, and then adjust if needed* Purpose:
* Establishing a regular cadence: a sprint should never be extended* Get away from slow periods at the start of a project and death marches at the end* Delivering working software at the end of each sprint becomes routine
* As you get better at scrum, you’ll want to move to shorter sprints, but usually no shorter than 1 week
Artifact #2: the sprint backlog & Event #2: sprint planning
Source
* In the sprint planning meeting, the team reviews the highest priority stories from the product backlog and breaks them down into tasks.* They estimate how many of the stories will fit into the sprint.* Key point: the team decides how much can fit in the sprint, not the Product Owner. The team has authority over what they are responsible for.* Ideally there can be a “sprint goal” where the set of stories come together as a feature set, but this isn’t always possible.
Artifact #3: the “product increment”
Source
To complete a campground project, we can deliver these features, one at a time
etc…
* To deliver a holiday park, we can deliver these complete features, one at a time:* Campsite for tents* Reception area, for check-in and check-out* Toilets and showers* Hook ups for RVs* Laundry facilities* A shop* Etc…
Event #3: the daily standup (aka daily scrum)
Source
* You literally stand* Timeboxed to 15 minutes* The goal is to answer the 3 questions: "What did you do yesterday?", "What will you do today?", and "Are there any impediments preventing you from
completing your work?* This meeting is by and for the team, to speak candidly about the status of the work* The goal is to make sure everyone knows what’s going on, and to deal with any impediments quickly* Management is ideally not there. If they need to be there, they do not speak and definitely do not run the meeting. They should stand behind the team,
so it’s clear the attention is not on them
Remote standups at ElectNext
* Works for remote teams too* And in small organizations with good working relationships, everyone can join, not just developers
Event #4: The sprint retrospective
Source
* Post-mortem meeting at the end of each sprint.* “Inspect & Adapt” is the focus* Length rule of thumb: 1 hour per week in the sprint (2 weeks = 2 hours)* The team generally focuses on these questions:* What went well, and can we learn from it?* What didn’t go well, and what can we learn from it?* In general, how can we do better?* I’ve found it’s also good to review progress on goals from previous retrospectives* Management is absolutely not present, unless there’s a special reason
Event #5: The sprint review
Source
* The product owner runs the sprint review* Length rule of thumb: same as retrospective* Demo the work of the sprint to all stakeholders: customers, CEO, etc* Get feedback: inspect and adapt the software
Additional best practices
User stories
User story with “conditions of acceptance” from a U Penn project
…and a mockup to go with the story
Splitting big stories
✤ “A job seeker can post and manage her resume…”
✤ One way to split it is along CRUD lines:
✤ create a resume…
✤ delete a resume…
✤ etc.
✤ Another way is data boundaries:
✤ add and edit education information…
✤ add and edit job history…
✤ etc.
Good stories: INVEST
✤ Independent
✤ Negotiable
✤ Valuable to customers
✤ Estimatable
✤ Small
✤ Testable
Estimating: story points
Source
* People are bad at estimating time, but we're good at estimating relative size or difficulty* As the points go up, the range of uncertainty also goes up
Estimating: planning poker
Photo by Kelly HiranoUsed with permission
* The teams give “story points” to the work by playing planning poker* After the PO describes the story and answers any questions from the team, each team member shows their card at once. This prevents them influencing
each other.* Team based estimates are more accurate than estimates by individuals
Velocity enables scheduling and “sustainable pace”
Source
* After a few sprints, teams have velocities, which allows for making time estimates for projects.* And this is key to the Agile goal of “sustainable pace”
Many more I don’t have time to describe….✤ Handling non-functional
requirements
✤ Handling back-end system stories
✤ Release planning
✤ Burn down charts
✤ Impact mapping
✤ Specification by example
✤ Pair programming
✤ Automated unit testing and feature testing
✤ Continuous integration
✤ “Walking skeletons”
✤ Etc
Scrum adoption anti-patterns and common difficulties
#1: the top-down, lip-service approach
Scrum master hiring story
#2. Misunderstood and/or misapplied
Source
A Scrum coach can help
Source
* A skilled external coach is often key for driving change - they bring a wide range of experience and can see things objectively* If you’ve never led an Agile transition before, it’s surprisingly easy to do it wrong
* At Penn, I misunderstood the roles: the clients were acting as their own product owners, The scrum masters were our former project managers, and continued doing traditional project management
* You need at least enough management support to pay the coach* You need to make sure you’re bringing in someone good
#3: Not realizing what you’re getting into
The Scrum Promise
“In my Scrum classes I warn attendees of what I call the Scrum Promise: If you adopt Scrum, there will be a day you come into the office nearly in tears over how hard the change can be. This is because Scrum doesn’t solve problems, it uncovers them and puts them in our face. Then, through hard work we address them.”
– Mike Cohn, Agile Trainer
A common reaction when this happens is to blame Scrum rather than actually deal with the issues.
Starting with a pilot project helps
#3: teams doing both development & operational support
Reality…
Source
Team didn’t like it, and clients didn’t like it. It felt like just adding a new set of work and processes on top of the existing work
Handling “emergencies”
✤ Triage… and reject if not really a software issue
✤ Triage… and defer if important but not truly an emergency
✤ Reserve a buffer in the sprint for unplanned work… but beware!
✤ Deal with root causes… prevent ongoing emergencies
Source
#4: Stuck with multiple projects
Additional references
✤ “Succeeding with Agile: Software Development Using Scrum” and “Agile Estimating and Planning” by Mike Cohn
✤ “Specification by Example” and “Impact Mapping” by Gojko Adzic
✤ “Agile Retrospectives: Making Good Teams Great” by Esther Derby
✤ Angry Dinosaurs: Accelerating Change and Institutional Incompetence presentation by Cory Ondrejka, Wharton Web Conference, 2010
✤ “The Nature of Software Development” by Ron Jeffries
References: technical practices
✤ “Clean Code” and “Agile Software Development, Principles, Patterns, and Practices” by Bob Martin
✤ “Refactoring: Improving the Design of Existing Code” by Martin Fowler
✤ “Agile Database Techniques” by Scott Ambler
✤ “Working Effectively with Legacy Code” by Michael Feathers
