agile business intelligence

Agile Business Intelligence

Applying Best Practices in Software Development

to BI, Analytics, and Data Driven Decisions

● Agile Background● Agile and BI● Trends and applications

Overview

● Started in 2001● Origins in DARPA and Toyota

○ Waterfall Software Development○ Lean Manufacturing

● Agile Manifesto written in 2001

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

What Is Agile?

Source: http://en.wikipedia.org/wiki/Waterfall_software_development

Waterfall Software Development

Waterfall's Failures

● Iterations - months to years● Customer lock in● Requirements rarely met● There's just one little secret...

Waterfall's Failures

● Iterations - months to years● Customer lock in● Requirements rarely met● There's just one little secret...

“Agile” was mentioned in the original paper (back in 1970)!

Why Agile Wasn't Feasible Then

● Testing was expensive○ Needed more hardware○ Tooling?

● Automation was costly○ Needed more hardware ○ Software would be custom

● Hardware and software were prohibitive○ Mainframes cost $4.6 million ○ Little to no integration

Cost source: http://mjperry.blogspot.com/2007/08/ibm-mainframe-computer-in-1970-pictured.html

Source: http://en.wikipedia.org/wiki/Agile_software_development

Why Agile?

● Customer satisfaction ● Business/IT Interaction● Fail early, fail often● Working software = progress● Team ownership - self organized● Faster adaptation

Source: http://en.wikipedia.org/wiki/Agile_software_development

Who to Involve?

● BI team (5-6 people)● Scrum Master - guide of the team● Project Manager - guide of the business● Business Stakeholder - Requestor of work

What Type of Agile?

Source: http://www.leadingagile.com/agile-training/

Where to Start?

● Project (Scrum)○ Great for piloting agile○ Team is consistent○ Find what works in your team

● Support (Kanban)○ Great for mature agile teams○ Team is consistent, but roles will change○ Less structure than Scrum

Implementation Pitfalls

Pitfall 1: The Balancing Act

“We need the time to improve, streamline, etc.” “Get it out the door

and with the customer!”

Pitfall 1: The Balancing Act

“We need the time to improve, streamline, etc.” “Get it out the door

and with the customer!”

Pay off debt, find ways to prevent it.Release as fast as possible, without too much compromise to quality

Pitfall 2: Working with the same tools

● Most BI Tools are not able to support:○ Continuous Integration○ Test Driven Development○ True Version Control○ Collaboration and fast iterations

● agile <> Agile

Pitfall 3: In Teams We Don’t Trust

● Business Team wants full control● BI not treated as software development● BI Team doesn’t involve Business Team

Implementation Tips

Where to Start?

● Find a Stakeholder (the bridge builder)● Find a Scrummaster (the negotiator)● Pick agile tools● Pilot Agile in a small group

Where to Grow

● Test Driven Development● Automation/Continuous Integration● Agile Data Warehouse Modeling

○ Hyper-normalized■ Anchor modeling

○ Hyper-generalized■ Data Vault