taldor data quality einat shimoni - stki

Einat Shimoni

Enterprise

applications

Data management & data governance trends

http://www.google.co.il/url?sa=i&rct=j&q=many changes ahead&source=images&cd=&cad=rja&docid=WIEwzNyRsEMFnM&tbnid=Ddkj6HWGpDH-hM:&ved=0CAUQjRw&url=http://www.belikewater.me/change-management/be-like-water-weathering-the-changing-demands-of-the-market/&ei=3DA3UeeeMNSGswa0_oC4AQ&psig=AFQjCNFHfr2IUWu_PwS7hqIgbJ344VPVmQ&ust=1362657872273273

Einat Shimoni’s work

Copyright@2013 Do not remove source or attribution from any slide, graph or portion of

graph

Data governance: the elephant in the room

2



graph

Quality of data decreasing each year by 10% Number of data sources and data type increasing Data perceived as a by-product of transactions, not as an asset (what is

the cost of inaccurate data?) Mature technological tools. Israeli market is picking up but still not

mature in all areas: Regulations in financial/insurance market -> data cleansing MDM is NOT yet mature enough in Israel! CDI was the main MDM focus but lately also PIM - Financial products

management (banking / insurance) Data quality as part of a migration process (usually one-time, not continuous)

3



graph

The Technological (and political) Problems

Legacy data sets are modeled with vertical applications in mind, which leads to the duplication of the same information across multiple data sets

Creating one “single version of the truth” (source of information) isn’t enough, you have to control the way end users extract and use it

Organizations with vertically structured IT organizations may not be "politically" ready for the move toward a centralized representation of customer information



graph

How do BI trends impact EIM?

BI is becoming easier, data management is becoming harder Data explosion will drive the need for data quality

Self service BI will drive the need for data governance

Loss of central control. The BI user will be “the boss”

Big data = bigger data quality problems

IT should establish a central COE and data governance

BICC will return as best practice

Data management is not a project, it’s an ongoing program

5



graph

Analytics & BI Generations

Gen. 3: Active Analytics End user is boss Classic DW model

DW updated frequently

Proactive BI

DW updated once a day

Static Reports

Gen. 1: Passive BI IT is the boss

Real time analysis of data “on the move”

BI insights linked to operational processes

Gen. 2: Active BI IT is the boss

Usage of data mining tools to create new insights

We are here

Gen. 4: Big data analytics End user is boss Distributed data model

Predictive analytics

Structured data Unstructured data

Passive BI

Advanced visualization

Self service

Use of in-memory

Structured data Structured data

DW updated frequently

Central data approach Central data approach

Interactive analysis

http://www.google.co.il/url?sa=i&rct=j&q=new&source=images&cd=&cad=rja&docid=G3cYeGYMk7j3uM&tbnid=6GfcKlk3PfJRiM:&ved=0CAUQjRw&url=http://theoracle.cc/2011/05/23/god-makes-all-things-new/&ei=MDA3UZyyJoWktAbk-YGYCQ&psig=AFQjCNGfaSyZttDsAHkM2kE3PX-JDJBVOA&ust=1362657664107716

BI State of the Market: Major changes ahead

7

One of the most adopted technologies (after ERP) - 68% of large organizations (Source: Computer Economics)

But still one of the most innovative areas

Next few years will focus on analytics, self service, visualization

What about big data? Big data will “meet” these trends and empower them

Will be an enabler for new type of analytic solutions

Data explosion – too much data!



graph



graph

The natural evolution

8

The top performers (“high digital IQ”) will lead the way into big data, and they are preparing for it

Source: http://www.forbes.com/sites/davefeinleib/2012/07/24/big-data-trends/

Source: PWC Digital IQ survey



graph

Big Data in Israel?

9

Yes 23%

No 77%

My organization will enter into a big data project

Source: STKI Survey 2013

http://www.google.co.il/url?sa=i&rct=j&q=note&source=images&cd=&cad=rja&docid=ZaVmZqP9rpAPrM&tbnid=tnNGIVnQkA44jM:&ved=0CAUQjRw&url=http://allthingshendrick.blogspot.com/2010/06/post-it-note-post.html&ei=hjFGUf71L4XbtAbR9oGAAQ&psig=AFQjCNGtXU894fq4thnvkrd2MNPBfmsdVQ&ust=1363641066746710



graph

We now create as much information every two days as we did from the dawn of civilization to 2003 (Source: IBM CMO Study)

10

Top 3 concerns: • Data explosion • Social media • Growth of channel & device options Source: IBM CMO study



graph 11

To small data

From Big Data

Small data = the new big data

http://www.google.co.il/url?sa=i&rct=j&q=small+and+big&source=images&cd=&cad=rja&docid=XMn4RB0HmUIhyM&tbnid=mVNjAAWRvmJAoM:&ved=0CAUQjRw&url=http://activatedstrategies.com/big-is-small-and-small-is-big/&ei=XW5AUeLVFMrktQafqoGQCw&psig=AFQjCNGYteIkB7_InwkDAWwP9ufDqkY75A&ust=1363262354029951

Too much focus on “big”

12

Big data is less relevant, right data is most important: how to get the right data in real time?

It’s what you do with the data that makes the difference

The challenge :convert data into actionable info.

Data Scientists will play the most important role



graph

http://www.google.co.il/url?sa=i&rct=j&q=hadoop&source=images&cd=&cad=rja&docid=Yrct9MBK3uBQ6M&tbnid=UnHdLegfhhXgVM:&ved=0CAUQjRw&url=http://gigaom.com/2011/12/19/my-hadoop-is-bigger-than-yours/&ei=WMtGUYjUIMfZtQa62IDgDg&psig=AFQjCNHAmE_lNbKHaIfiWjJUXZMsVYi4gQ&ust=1363680423798160

MEGA Trend – BI ownership is shifting

13

IT will focus on data quality and access + effective channels to BI

Business users will be the owners of BI and analytics

By 2014, 40% of BI purchasing will be business-led (Gartner)

Benefits: operational efficiency for IT (reporting and analysis done by LOBs), agility, usability, relevance, fast deployment

The price: consistency, integration, central control



graph

http://www.google.co.il/url?sa=i&rct=j&q=keys&source=images&cd=&cad=rja&docid=HYoAWDe_ABIrfM&tbnid=MR7vljQEQQKqjM:&ved=0CAUQjRw&url=http://college.monster.com/training/articles/1327-3-keys-to-a-successful-job-search&ei=pIZAUaTcNMnMtAadhoGYAQ&psig=AFQjCNFP1kkaHsKRqdBnBLF5PKppVP0O1w&ust=1363269650005307

Roles and organization of the BI department will change

14

Less people creating reports at the BI department More BI will be done in LOBs by analysts / key users and hopefully new types

of users – knowledge workers (self service) BI department will focus on:

Data governance, central definitions and models Data quality issues Center of Excellence for guiding users Creating effective channels to access the data

Search based BI portal Visualization tools Self service Data discovery



graph

http://www.google.co.il/url?sa=i&rct=j&q=BI+organization&source=images&cd=&cad=rja&docid=GoH4RY8EoU3NjM&tbnid=9oxNskj9BAn2yM:&ved=0CAUQjRw&url=http://www.asugnews.com/2011/07/13/what-makes-a-great-bi-organizational-structure/&ei=uTRGUc3ZI8TOtAbXrID4Dg&psig=AFQjCNHVEi1-VoGpwDpoHSC8zdCC4pGZgg&ust=1363641911399999



graph

Data governance maturity model

15

By-product of transactions

No synch Data siloes

Tactical IT driven

ODS

Process-focus

Business involvement

Data = asset Business leads

Source: http://blog.kalido.com/road-data-governance-maturity/

Data management Data governance

http://www.google.co.il/url?sa=i&rct=j&q=data governance maturity model&source=images&cd=&docid=tOWebbPS3FJECM&tbnid=BDJaKYbC-r0jZM:&ved=0CAUQjRw&url=http://blog.kalido.com/road-data-governance-maturity/&ei=zXZiUaelM4GhtAaFo4DQDg&psig=AFQjCNHhMZBeQc38TBVstnfL8zN5faw4fA&ust=1365493641107449



graph

Worldwide maturity level

16

Source: http://blog.kalido.com/road-data-governance-maturity/

64%

0.5%

13%

22.5%

http://www.google.co.il/url?sa=i&rct=j&q=data governance maturity model&source=images&cd=&docid=tOWebbPS3FJECM&tbnid=BDJaKYbC-r0jZM:&ved=0CAUQjRw&url=http://blog.kalido.com/road-data-governance-maturity/&ei=zXZiUaelM4GhtAaFo4DQDg&psig=AFQjCNHhMZBeQc38TBVstnfL8zN5faw4fA&ust=1365493641107449



graph

Perception gap

17



graph 18

Thanks and hope you enjoyed

taldor data quality einat shimoni - stki

Documents