doing analytics right - building the analytics environment
TRANSCRIPT
Building the Analytics Environment
Look Whose Talking
@tasktop
• Nicole Bryan, VP of Product Management, Tasktop– Passionate about improving the
experience of how software is delivered– Former Director at Borland Software– [email protected] |
@nicolebryan
• Dr Murray Cantor – Senior Consultant, Cutter Consortium – Working to improve our industry with
metrics– Former IBM Distinguished Engineer– [email protected] | @murraycantor
What we’ve learned so far….
• Webinar 1: There is no “one size fits all” metric nirvana
• Webinar 2: Use GQM to design the metrics that are right for your mix of development
Today…
It’s all about the execution! Let’s get practical!
©2015 Murray Cantor
Choosing metrics big picture
Agree on goals
- Depends on the levels and mixture of work
Agree on the how they fit into the loop
1. “How would we know we are achieving the goal”
2.” What response we should take?”
Determine the measures needed to answer the questions
- Apply the Einstein test (as simple as possible, but no simpler)
Specify the data needed to answer the questions
Automate collection and staging of the data
4
Today
©2015 Murray Cantor
From Goals to Measures to Data (GQM-ish)
1. Identify a set of corporate, division and project business goals and associated measurement goals.
2. Specify a sense-and-respond loop to steer to the goal.
3. Generate questions based on the goal that if answered:
• Let you know have achieved, are trending to \ the goal?
• Provide the level of detail necessary to take action
– Where is the problem, bottleneck?
• Communicate progress to stakeholders
– Summaries, rollups
4. Select or specify data needed to answer the questions in terms of state transitions of the relevant artifacts
5. Study the data to specify the data set and statistic needed to be collected to answer those questions and track process and
product conformance to the goals.
6. Develop automated mechanisms for data collection.
7. Collect, validate and analyze the data in real identify patterns to diagnose organization situation and provide suggestions for
corrective actions.
8. Analyze the data in a post mortem fashion to assess conformance to the goals and to make recommendations for future
improvements.
5
The “Last Mile Problem”
A phrase used in the telecommunications and technology industries to describe the
technologies and processes used to connect the end customer to a communications
network. The last mile is often stated in terms of the "last-mile problem", because
the end link between consumers and connectivity has proved to be
disproportionately expensive to solve.
Read more: http://www.investopedia.com/terms/l/lastmile.asp#ixzz3dAdJpzAQ
The Last Mile Problem
Aspiration without execution is useless!
No wait … It’s actually worse than useless…
– If execution for your analytics solution is difficult it can quickly leads to “The Light is Brighter Here” anti-pattern
Danger!!!!
How Do I Unlock All This Goodness?
Po
rtfo
lio M
gm
tAgile PM
Requirements
TestDev
Op
eratio
ns
Why So Difficult?
– Tool Reality
• You have lots of them! So it’s not one ETL, its many ETLs! That gets very hard to maintain.
• You’ve got disparate tools but your GQM needs single source fed by variety of tools
– I’ve got defects in HP QC, Rally and JIRA – how do I calculate cycle time!!!
• Yes, tool vendors have analytics solutions…. and these solutions are focused on their particular areas of specialization
Why So Difficult?
– Logistics problems
• SaaS problem – sometimes data only available for limited time
• Transaction based data vs. reporting based data
• Many of the smaller more purpose built tools have no thought that the transactional data they are producing needs to participate in a larger analytics strategy
• You say tomato, I say tomato
Remember – you want your point tools to stay focused on their domain expertise
What is the solution?
– Collated data across tools
– Abstraction away from specific tool representations of artifacts
– Near real time access
– Mix of simplicity so that you can just “get going” combined with the ability to “get sophisticated” when you need to/are ready to
Powering software lifecycle analytics
0
2
4
6
8
May June July Aug
0
2
4
6
ETL
Customer Data
Warehouse
“Raw” Data Storage in customer Database
(etc.)
Remember what Murray taught us?
©2015 Murray Cantor
Kinds of Development Efforts: What is your mix?
18
1. Low innovation/high
certainty
• Detailed understanding
of the requirements
• Well understood code
2. Some innovation/
some uncertainty
• Architecture/Design in
place
• Some discovery required
to have confidence in
requirements
• Some
refactoring/evolution of
design might be required
3. High innovation/Low
Uncertainty
• Requirements not fully
understood, some
experimentation might be
required
• May be alternatives in choice
of technology
• No initial design/architecture
©2015 Murray Cantor
Descriptive example: Cycle times
19
Let’s Bring Cycle Time to Life!!!!!!!!!!!
First, some key concepts of Tasktop Data
Defects
Requirements
Test CasesTimesheets
A tangible by-product produced during the development of software.
Artifacts CollectionsA set of artifacts from your repository
Collection #1
Collection #2
JIRA Defects Collection
Priority• High• Medium• Low• Trivial
Summary
Fix Version
DescriptionPriority• High• Medium• LowReleased InTags
M O D E L
Project #1
Project #2
Project #3
HP Defects Collection
Priority• 1• 2• 3• 4
Description
Release
DescriptionPriority• High• Medium• LowReleased InTags
M O D E L
Project #A
Project #B
Project #C
Event Collection
Priority• High• Medium• Low
Description
Released In
DescriptionPriority• High• Medium• LowReleased InTags
M O D E L
* Raw database collections are a little bit special
Reporting Integration
Flow Specification
And it will results in a database table like below
Another way of looking at it…Use this Model feeding defects from JIRA, HP, etc
Artifact ID
Project Type Created Modified Severity Priority Status Release Assignee
DEF-1 Project A Defect 1/1/15 1/1/15 1 High Open
DEF-1 Project A Defect 1/1/15 1/2/15 1 High In Progress
John
DEF-1 Project A Defect 1/1/15 1/5/15 1 Med In Progress
John
DEF-1 Project A Defect 1/1/15 1/7/15 1 Med Shipped 1.0.0.1 John
1 Artifact, 4 Rows in Database
Event Log Concept
And once you’ve got that, you can easily get things like this….
Demo
(2) Create or reuse a model
(3) Create collections(And Map the Collection to the model)
(4) Create an integration
Four Easy Steps
(1) Connect to your system
1234
(1) Connect To Your System
(2) Create or reuse a model
• Identify the fields to flow• Configure to Normalize the Data
(3) Create Collections (and map them)
(3) Create Collections (and map them)
One Core Artifact Type
Sourced from One Repository
Many Projects
Mapped to One Model
• Configure fields and field values to conform to the normalized model values
• Transform values
Mapping Artifact to Model
(4) Create an Integration
Solves the Last Mile Problem
– Collated data across tools
– Abstraction away from specific tool representations of artifacts
– Near real time access
– Mix of simplicity so that you can just “get going” combined with the ability to “get sophisticated” when you need to/are ready to
@tasktop@cuttertweets