data governance in an agile scrum lean mvp world

Karen Lopez @datachick #HeartData

Heart of Data ModelingData Governance in an Agile SCRUM Lean MVP World

@datachick

#heartdata

Please share/tweet

Karen López

Karen has 20+ years of data and information architecture experience on large, multi-project programs.

She is a frequent speaker on data modeling, data-driven methodologies and pattern data models.

She wants you to love your data.

Tamera Clark, T & K Creative Solutions GroupTamera Clark has been involved in the IT industry for greater than ten years, with experiences ranging from systems analysis/engineering to SQL Server and SSRS administration/development.

She is an active member of the SQL Server community, participating in the Women in Technology Virtual Chapter, Co-leading the Nashville BI Chapter, assisting the Nashville PASS Chapter and serving as a Regional Mentor.

Tamera is also the event chair of SQLSaturday Nashville.

She loves data, too.Twitter - @tameraclarkLinkedin - /tameraclarkEmail – [email protected] - Tameraclark.com

You are the panelist...so let’s get to know you….

POLL: Who Are You?

DG at your company?

Aug 2014

Agile/SCRUM/Lean/MVP at your company?

Aug 2014

Be engaging….

Use Q&A for formal questions

Use chat to discuss with each

other

Data GovernanceAn Overview

Data Governance: Bob Seiner

Data governance is the formalexecution and enforcement of authority over the management of data and data related assets.

Data Governance: Gwen Thomas, DGI

Data Governance is a system of decision rightsand accountabilities for information-related processes, executed according to agreed-upon models which describe who can take what actions with what information, and when, under what circumstances, using what methods.

http://www.datagovernance.com/wp-content/uploads/2014/11/dgi_framework.pdf

Data Governance: DGPO

A discipline that provides clear-cut policies; procedures; standards; roles; responsibilities; and accountabilities to ensure that data is well-managed as an enterprise resource.

http://dgpo.org/uploads/2015_DGPO_Overview.pdf

I have no idea why we have to give this stuff another name. Governance is sort of the definition of

“professional practice”.

We don’t have Engineering or Architecture Governance. It’s just baked into those professions.

It’s good we have Data

Governance now.

Why Data Governance is Important

Massively complex architectures•Tools•Vendor Applications•The CLOUD

Complex Data•Feeds, external data•Redundant internal

data•Conflicting data•Poor data quality•Missing data•Unused data

Methods• Infrastructure•DevOps•Development•Data•Financial

Many facets of data governance..

Data Quality Data Stewardship

Compliance Infrastructure and architecture

Business Standards Monitoring and Correction

Where Data Governance Helps

Saving money

Satisfying customers

Retaining customers ROI

Keeping CEO/CIO out of jail

Innovating

Data & Business analytics Reducing costs

Responsiveness

Establishing Data GovernanceHow do we do this?

Where are you now?

Development processes

Staffing

Tools Models

Business and Customer pain

points

IT pain points

Regulatory findings/penalties Audit findings

Key Data Governance Deliverables

Strategy

Policies

Tools/Processes

Roles & responsibilities

Data Quality rules & methods

Data Modeling, including extended metadata

Monitoring, reporting and analysis of results

Establishing a Data Governance Program

http://www.datagovernance.com/wp-content/uploads/2014/11/dgi_framework.pdf

Agile MethodsWhat are they? What else are they?Plus at little bit of SCRUM

I love working on Agile projects

It’s FRAGILEprojects I hate.

Principles Behind the Agile Manifesto1. Our highest priority is to satisfy the

customer through early and continuous delivery of valuable software.

2. Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

3. Deliver working software frequently, from a couple of weeks to a couple of months, with preference to the shorter timescale.

4. Business people and developers must work together daily throughout the project.

5. Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

6. The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

Principles Behind the Agile Manifesto

7. Working software is the primary measure of progress.

8. Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

9. Continuous attention to technical excellence and good design enhances agility.

10. Simplicity--the art of maximizing the amount of work not done--is essential.

11. The best architectures, requirements, and designs emerge from self-organizing teams.

12. At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Manifesto for Agile Software Development

Agile/Scrum development project

28

Typical Sprint

Sprint Planning

Backlog Stories

START

READING DEVELOPMENT DELIVER

END

Where the hell is our database?

Managing Data Wrong - One

Expecting data modeling & database design to be completed in an instant at the

beginning of a sprint

Managing Data – Fix it

Sprint Planning

Backlog Stories

START


END

Managing Data – Fix it Better

Sprint Planning

Backlog Stories

SART

READING DEVELOPMENT

START


END

Sprint Planning

kl S i

START

READING

https://www.scrumalliance.org/why-scrum

Scrum Values

Focus• Because we focus on only a few things at a

time, we work well together and produce excellent work. We deliver valuable items sooner.

Courage• Because we work as a team, we feel

supported and have more resources at our disposal. This gives us the courage to undertake greater challenges.

Openness• As we work together, we express how we're

doing, what's in our way, and our concerns so they can be addressed.

Commitment

• Because we have great control over our own destiny, we are more committed to success.

Respect

• As we work together, sharing successes and failures, we come to respect each other and to help each other become worthy of respect.

- https://www.scrumalliance.org/why-scrum/core-scrum-values-roles#sthash.RgaO3uIK.dpuf

All work performed in Scrum needs a set of values as the foundation for the team's processes and interactions. And by embracing these five values, the team makes them even more instrumental to its health and success.

Lean Software MethodsWhat are they? What else are they?

Lean Software Methods

Eliminate waste

Amplify learning

Decide as late as possible

Deliver as fast as possible

Empower the team

Build integrity in

See the whole

What is waste? Partially done work

Extra processes

Extra features

Task switching

Waiting

Motion

Defects

Management activities

Lean ProcessesWhat are they? What else are they?

Lean Software Methods

Eliminate waste

Amplify learning

Decide as late as possible

Deliver as fast as possible

Empower the team

Build integrity in

See the whole

What is waste? Partially done work

Extra processes

Extra features

Task switching

Waiting

Motion

Defects

Management activities

Minimum Viable ProductWhat is it?

Minimum Viable ProductThe most pared down version of a product that can still be released. An MVP has three key characteristics:

• It has enough value that people are willing to use it or buy it initially

• It demonstrates enough future benefit to retain early adopters

• It provides a feedback loop to guide future development

The catch to this development technique is that it assumes that early adopters can see the vision or promise the final product and provide the valuable feedback needed to guide developers forward.

This suggests that technically orientated products used by technical users may be most appropriate for this type of development technique.

https://www.techopedia.com/definition/27809/minimum-viable-product-mvp

Focus of MVPEarly Feedback

Feedback

And… Feedback

Where Data Governance and Development Methods Intersect…Integrate…Collaborate…build a wall…find peace…

Data Governance and Agile/SCRUM

Program Planning and Design Governing Making Happier

Where DG and Agile Intersect

Sprint planning MUST take into account data governance

But no Agile pro is going to like those words

So we need the right context and the right vocabulary

Where DG and Agile Intersect

“Working software is the primary measure of progress.”

“Continuous attention to technical excellence and good design enhances agility.”

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

You will find

obstacles along

the way….

Agile “Extensions”…

Everyone is a generalist

Agile Blocking

Excluded titles•Administrators•Architects•Managers

Test Driven Development

No BMUF/BDUF

Paired programming

Did I say Blocking?

Agile Blocking & Data Modeling

The blockers effectively implement a “process façade” around your team that makes it appear to the rest of the organization that your team is following their existing procedures. This satisfies the bureaucrats, yet prevents them from meddling with the people that are doing the real work. Although it sounds like a wasted overhead, and it is because it would be far more effective to divert both the blockers and bureaucrats to efforts that produce something of value, the advantage is that it enables the rest of the team to get the job done. The role of blocker is often taken on by your team’s project manager or coach, although in the past I have let this be a revolving role on the project so as to spread out the pain of dealing with the paper pushers.

http://www.agiledata.org/essays/adopting.html#sthash.gvFL7Hd4.dpuf

Data Governance and Lean

Bringing models to the table is Lean

Bringing metadata to the project is Lean

Forcing Lean projects to implement the entire data model may not be Lean

The goal of Lean fast and efficient, much like agile.

Data Governance and Lean

The Data Governance program Deploymentcould follow a Lean Process itself

Finding the right metrics are key

Lean isn’t just an excuse for sloppiness or lack of compliance

Data Governance and MVP

Depends upon what minimal ends up being

Some MVP projects have only a handful of data items

It might be best to have a consulting role on the project for compliance monitoring

MVP is not typically an enterprise method.

10 Tips for Data Modelers

1. Learn about these methods – don’t avoid them

2. Get Agile/Scrum/Lean/MVP training. Get certified even

3. Learn the lingo.

4. Use the lingo

5. Be able to describe data modeling and data governance to the context of these methods


6. Get data models and DDL tasks moved sprints ahead

7. Bring data models (and other models) to the team.

8. Don’t back off from Agile/SCRUM/Lean teams, even if they are hostile.

9. Don’t be a roadbock. Get ahead of the sprints


10. Practice Agile techniques on your own deliverables• Policies, procedures• Test driven development• Backlogging• Parkinglotting• Continuous delivery• Lean• MVP

http://edw2016.dataversity.nethttp://nosql2016.dataversity.net

Half Day: 7 Databases in 170 Minutes

SIG: ER/Studio and Data Modeling Special Interest Group

Panel: Data Modeling & NoSQL Moderator

Session: The Tricky Part of Doing Tricky Things in your Data Model

…and likely some other fun things!

Thank You! www.datamodel.com

[email protected]