jeffrey murray test manager powerpoint microsoft silicon valley
TRANSCRIPT
Goals for today
The parts of the Office Product Cycle
Measurements and quality
Who Microsoft hires
Stories
Microsoft Silicon Valley • Over 1800 employees, plus 400 vendors/contractors. Approx. 450 employees in San Francisco.
The Office Product CycleAbout every 3 years a major release of Office comes out with features aimed at increasing productivity and ease of use for our customers.
What is Office 2010?
Word Excel Outlook PowerPoint
SharePoint
Publisher Access OneNote
Groove InfoPath VisioServer
and Tools
Online
Online
Online
Online
How many people does it take to ship core Office?
800 Software Design Engineers (Developers)
800 Software Test Engineers (Testers)
400 Program Managers (Feature designers)
200 Localizers, Lab Managers, etc.
200 Planners, Recruiters, Sales and Marketing
2400 Total
Project Management
Schedule
• How much time do we have?
Resources
• Who are they and how many?
Features• What are you
going to do and how risky is it?
Plan new features1. What is the vision for the
product?2. Make lists of features
you want to do, listen to customers, and see what is possible
3. Estimate and prioritize what is you can do in the time you have
4. Triage these until you can fit the schedule
5. Then write page 1 specs and go to step 3
New ribbonBetter graphicsAnimation painterOn line editingProjector setupSave to videoNew animation timelineSingle Document interfaceSlideShow broadcastCo-authoringNew transitionsBetter animationsSectionsNative video supportSplit videoCamera integrationHardware supportODF support
Code the features!
• Feature team makes the decisions• Must fit into allowed development time• Must be fully resourced• Responsible for getting it done
• Management will approve features via • Adds/Cuts• Product priorities and opportunities• Manage risk• 8 questions
1 Developer 1 Tester 1 Program Manager
Feature team
Feature
crew
Feature crew
Feature crew
Feature crew
Feature
crew
Feature
crew Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature
crew
Feature crew
Feature crew
Feature crew
Feature
crew
Feature
crew Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature
crew
Feature crew
Feature crew
Feature crew
Feature
crew
Feature
crew Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature
crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
Feature crew
12 monthsPlanned testing phase, validation, user
scenarios, international, stress, security, configuration, accessibility, compatibility
etc.
Typical 28 to 32 month schedule
Plan
Code
Test
Beta
RTM
Planning Phase 4-6 monthsCreate lists and 1 page specs
Development and test estimate and risk assessmentAdds/cuts
4 MonthsFeatures ready to go
Fix last remaining important bugs
8 months: Design and implement
Unit test and validate8 questions
4 Months Beta 1 about 10,000 usersBeta 2 about 1,000,000 users
Important checkpoints
Plan
Code
Test
Beta
RTM
No coding without
dev/test/pm resource
Feature demo and
8 questions answered
Code complete: no changes without a
bug entered
Product must be internally
dogfoodable
Triage teams in
place
All metrics and goals must be
met
Metrics and Quality• Good planning is the key to good quality • “If you fail to plan you plan to fail”• Features added or changed late are always more buggy
and risky• Proper design, test, automation support produces better
code
• Bug rates are good way to track quality all else being equal
• Customer feedback through Watson
What makes it work!• Checks and balances• Testing signs off on specs• Dev signs of on test plans• PM charged with overseeing progress of dev/test
What makes it work? (continued)• Constant and never ending improvement• Test involved earlier• Automation• Technical innovations • Auto code review• Automation validation before release to testers
• Listening to customers and competition
1 5 9 13 17 21 25 29 33 37 41 45 490
50,000100,000150,000200,000250,000300,000350,000400,000450,000
Example Watson Curve
Bucket number
Hit
s
Watson
We don’t have user steps or data
We know what line of code caused the crash and can often guard against it
Career Tips• Companies are better at identifying talent within you than
you are at bluffing your way through an interview. Make sure you are there for the right reasons and don’t hold back.
• Don’t plan your whole career all at once, you will miss out on interesting opportunities
• High tech companies need fresh idea, and that is a great open door for you
• You are a professional, act like it• When you screw up (and you will) what you do next is
critical• Ask yourself each week, what do I like about my job?• Interview the company beyond the job, a good part of your
life will be spent there.
Stories• How I got my Job at Microsoft• Copy protection• Steve’s laptop• OneNote• Office pranks• Elevator• Beach• Peanuts• Disco• balloons