corvelle drives concepts to completion taming big data with visual analytics 1
TRANSCRIPT
Corvelle Drives Concepts to Completion
Taming Big Datawith Visual Analytics
1
Corvelle Drives Concepts to Completion
Yogi SchulzBiography
Partner in Corvelle Consulting Information technology related management
consulting Microsoft Canada columnist & CBC Radio guest PPDM Association board member Industry presenter:– Project World - 6 years– PMI – SAC - 3 years– PMI - Information Systems SIG - 2 years– PPDM Association - several years
2
Corvelle Drives Concepts to Completion
PresentationOutline
• Presentation objectives• Big data• Visual analytics • Conclusions• Recommendations• Questions & Answers
3
• Definition• Trends• Value• Definition• Trends• Value• Software
Corvelle Drives Concepts to Completion
PresentationObjectives
• Increase our understanding of big data• Increase our understanding of the value of
visual analytics
4
Corvelle Drives Concepts to Completion
5
BigData
Corvelle Drives Concepts to Completion
6
Pointy-hair BossExplains Big Data
Corvelle Drives Concepts to Completion
7
Big DataDefinition
“Big data” is high-volume, -velocity and -variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making
Corvelle Drives Concepts to Completion
8
Big Data Defined
Data Velocity
DataVolume
Data Variety
Corvelle Drives Concepts to Completion
9
Digital Storage in Place
Exabytes
Corvelle Drives Concepts to Completion
10
Storage Cost Trends
Corvelle Drives Concepts to Completion
11
Big Data Skills in Demand
Corvelle Drives Concepts to Completion
12
Big Data Issues
Technology– Multiple incompatible data silos– Inadequate development and management tools
People– Shortage of data scientists– Lack of communication between data scientists
and business users Processes– Insufficient attention to data quality
Corvelle Drives Concepts to Completion
13
Value of Big Data
Making data openly available Supporting experimental analysis Assisting in defining market segmentation Supporting real-time analysis and decisions Facilitating computer-assisted innovation
Corvelle Drives Concepts to Completion
14
Big Data Market Forecast
Corvelle Drives Concepts to Completion
15
Visual Analytics
Corvelle Drives Concepts to Completion
16
Visual AnalyticsDefinition
Visual analytics combines automated analysis techniques with interactive visualizations to enable:– Effective understanding– Reproducible reasoning – Defensible decision-making
in the context of large and complex data sets
Corvelle Drives Concepts to Completion
17
Visual AnalyticsGoal
Synthesize information and derive insight from massive, dynamic, ambiguous, and often conflicting data
Detect the expected and discover the unexpected
Provide timely, defensible, and understandable assessments
Communicate assessments effectively for action
Corvelle Drives Concepts to Completion
18
Visual AnalyticsExample
Corvelle Drives Concepts to Completion
19
Visual AnalyticsLeading Software Tools
SmarterAnalytics
Corvelle Drives Concepts to Completion
20
Magic Quadrant for Business Intelligence and Analytics Platforms
Niche Players Visionaries
Challengers Leaders
Corvelle Drives Concepts to Completion
21
Oil & Gas AnalyticsCalgary Software Vendors
Corvelle Drives Concepts to Completion
22
Visual AnalyticsTrends
Machine learning Data discovery platforms Chief Analytics Officers Data products Hadoop datastores
Corvelle Drives Concepts to Completion
23
Value of Visual Analytics
Make data-driven decisions “very frequently”
Make decisions “much faster” than market peers
Execute decisions as intended “most of the time”
Corvelle Drives Concepts to Completion
24
Integrated Analysis: Comparison of Actuals Sales to Estimates
Corvelle Drives Concepts to Completion
25
Recommendations
• Improve your data management processes• Identify operational problem• Select visual analytics software package• Pilot software package for problem• Build on pilot success
Corvelle Drives Concepts to Completion
Questions &Discussion
26
Can you help us
implementvisual
analytics?
Pleasefill out
evaluationform
Corvelle Drives Concepts to Completion
Taming Big Datawith Visual Analytics
Corvelle Consulting300, 400 - 5 Ave. S. W.Calgary, Alberta T2P 0L6Phone: (403) 249-5255E-mail: [email protected]: www.corvelle.com
Yogi SchulzPartner of Corvelle ConsultingInformation technology related
management consultingMicrosoft Canada columnist & CBC
Radio hostIndustry presenterPPDM Association board member
27
Corvelle Drives Concepts to Completion
28
Visual Analytics Software PackagesSelection Criteria
Visual exploration Augmentation of human perception Visual expressiveness Automatic visualization Visual perspective shifting Visual perspective linking Collaborative visualization
Corvelle Drives Concepts to Completion
29
Bibliography – 1
Analytics Trends – 2014 by Deloitte– http://
www.deloitte.com/view/en_US/us/Services/additional-services/deloitte-analytics-service/analytics-trends/index.htm
Big Data - Is your Data Warehouse a Dinosaur?– http://wikibon.org/wiki/v/Big_Data_-_
Is_your_Data_Warehouse_a_Dinosaur%3F
Big Data: Issues and Challenges Moving Forward– http://www.computer.org/csdl/proceedings/hicss/2013/4892/00/4892a995.pdf
Big Data – The 4 V’s: The Simple Truth– http://makingdatameaningful.com/2012/12/10/big-data-the-4-vs-the-simple-tr
uth/
Big data and the E&P organization– http://www.etlsolutions.com/big-data-and-the-ep-organization/
Corvelle Drives Concepts to Completion
30
Bibliography – 2
Big Data Focus on Value, Not Hype, in 2014– http://
www.enterpriseappstoday.com/business-intelligence/big-data-focus-on-value-not-hype-in-2014.html
Big Data Market Size and Vendor Revenues– http
://wikibon.org/wiki/v/Big_Data_Market_Size_and_Vendor_Revenues Big Data Vendor Revenue and Market Forecast 2012-2017
– http://wikibon.org/wiki/v/Big_Data_Vendor_Revenue_and_Market_Forecast_2012-2017
BIG DATA use cases – are there any “killer apps”?– http://jameskaskade.com/?p=2088
Big Data - What it is and why it matters– http://www.sas.com/en_us/insights/big-data/what-is-big-data.html
Corvelle Drives Concepts to Completion
31
Bibliography – 3
Enterprise Business Intelligence Platforms, Q4 2013– http://explore.tibco.com/rs/tibcospotfire/images/Forrester%
20Wave%20for%20Ent%20BI%20Platforms%2012%2018%2013.pdf
Extracting Value from Chaos– http://www.emc.com/collateral/analyst-reports/idc-extractin
g-value-from-chaos-ar.pdf 5 Big Business Intelligence Trends For 2014
– http://www.informationweek.com/software/information-management/5-big-business-intelligence-trends-for-2014/d/d-id/1113468
Four Big Data Challenges– http://tdwi.org/Blogs/Fern-Halper/2013/10/Four-Big-Data-Ch
allenges.aspx Four Ways to Illustrate the Value of Predictive Analytics
– http://tdwi.org/Blogs/Fern-Halper/2013/11/Predictive-Analytics.aspx?j=293380&[email protected]&l=50_HTML&u=5870497&mid=1060748&jb=46
Corvelle Drives Concepts to Completion
32
Bibliography – 4
Gartner's Big Data Definition Consists of Three Parts, Not to Be Confused with Three "V"s– http://www.forbes.com/sites/gartnergroup/2013/03/27/gartners-big-data-defi
nition-consists-of-three-parts-not-to-be-confused-with-three-vs/ Graph Analytics 101
– http://www.forbes.com/sites/emc/2014/02/28/graph-analytics-101/ How Can Graph Analytics Uncover Valuable Insights About Data?
– http://www.forbes.com/sites/emc/2014/03/14/how-can-graph-analytics-uncover-valuable-insights-about-data/
IDC Worldwide Business Analytics Software 2013– http://idcdocserv.com/241689e_sas
The knowledge society: The impact of surfing its tsunamis in data storage, communication and processing– http://
www.wcu.edu/ceap/houghton/readings/tech-trend_information-explosion.html
Corvelle Drives Concepts to Completion
33
Bibliography – 5
Magic Quadrant for Business Intelligence and Analytics Platforms– http://www.gartner.com/technology/reprints.do?id=1-1QYUTPG&ct=140
220&st=sb Selecting a Visual Analytics Application
– http://www.tableausoftware.com/sites/default/files/whitepapers/whitepaper_selecting-visual-analytics-application.pdf
Solving Problems with Visual Analytics– http://www.vismaster.eu/wp-content/uploads/2010/11/VisMaster-book-l
owres.pdf A Survey of Visual Analytics Techniques and Applications: State-of-
the-Art Research and Future Challenges– http://
research.microsoft.com/en-us/um/people/ycwu/Files/va_survey.pdf Ten Benefits of Business Intelligence Software
– http://www.enterpriseappstoday.com/business-intelligence/ten-benefits-of-business-intelligence-software-1.html
Corvelle Drives Concepts to Completion
34
Bibliography – 6
The 3 big problems in big data (hint: They all involve people)– http://venturebeat.com/2013/12/04/the-3-big-problems-in-big-data-hin
t-theyre-all-about-people/ 3 Tips for Getting More Value From Your Data
– http://www.enterpriseappstoday.com/business-intelligence/3-tips-for-getting-more-value-from-your-data.html
Top Business Intelligence Trends For 2014– http://www.enterpriseappstoday.com/business-intelligence/top-busines
s-intelligence-trends-for-2014.html TIBCO Spotfire® Ranked Highest “Current Offering” in Forrester
Wave for Agile BI 2014– http://spotfire.tibco.com/forresterwave?mkt_tok=3RkMMJWWfF9wsRol
sq7MZKXonjHpfsX56O4qULHr08Yy0EZ5VunJEUWy3IMISNQ%2FcOedCQkZHblFnVgBT62%2BWLgNqKUE#sthash.E3zApQRz.dpuf
Upstream Tech 2014...holding a tin cup below a Niagara Falls of data!– http://
www.findingpetroleum.com/event/Upstream_Tech_2014/49cb9.aspx#ixzz2vwOOeQOJ
Corvelle Drives Concepts to Completion
35
Bibliography – 7
The value of Big Data: How analytics differentiates winners– http://www.bain.com/publications/articles/the-value-of-big-data.aspx
Value, Velocity, Volume and Variety: Analyzing Big Data– http://www.iansclarke.com/value-velocity-volume-variety-analyzing-big-d
ata/ Views from the front lines of the data-analytics revolution
– http://www.mckinsey.com/Insights/Business_Technology/Views_from_the_front_lines_of_the_data_analytics_revolution?cid=other-eml-alt-mkq-mck-oth-1403
Visual Analytics: Definition, Process, and Challenges– http://hal-lirmm.ccsd.cnrs.fr/docs/00/27/27/79/PDF/VAChapter_final.pdf
Visual Display of Quantitative Information– Edward Tufte
Wisdom of Crowds® Small and Mid-Sized Enterprise Business Intelligence Market Study– http://explore.tibco.com/rs/tibcospotfire/images/Wisdom_of_Crowds_S
ME_BI_Report-Licensed_to_TIBCO_Software-Copyright_2013.pdf
Corvelle Drives Concepts to Completion
36
Videos
Big Ideas: How Big is Big Data?– http://
www.youtube.com/watch?v=eEpxN0htRKI&list=PLLQoHDLZBTROuXYEn0-CjjziCik9CB_QH
Big Ideas: Why Big Data Matters– http://www.youtube.com/watch?v=rTAn1bvy8vU
Oil & Gas IQ - Understanding Big Data– http://www.youtube.com/playlist?list=PLLQoHDLZBTROuXYEn0-CjjziCik9CB_QH
Power Your Performance! Big Data in the Energy Industry– http://
www.youtube.com/watch?v=F4sIWhIigmo&list=PLLQoHDLZBTROuXYEn0-CjjziCik9CB_QH
What is Big Data?– http://www.youtube.com/watch?v=PlaJsseTgk4
What is Big Data? Part 1 & 2– http://
www.youtube.com/watch?v=B27SpLOOhWw&list=PLLQoHDLZBTROuXYEn0-CjjziCik9CB_QH
Corvelle Drives Concepts to Completion
37
Corvelle Drives Concepts to CompletionCorvelle Drives Concepts to Completion
Corvelle Drives Concepts to Completion
38
Visual AnalyticsTool vs. Application
Characteristic Tool ApplicationPre-built integrations None YesPre-built analytical functions None YesRequired customer developer skills
Significant None
Control over application development direction
Complete High
Analytical functionality limitations
None Some
Corvelle Drives Concepts to Completion
39
Visual AnalyticsTool vs. Application
Characteristic Tool ApplicationDevelopment elapsed time Variable ShortDevelopment risk Significant LowProduction quality application
Feasible but rare
Yes
Corvelle Drives Concepts to Completion
40
Visual AnalyticsTool vs. Application
Characteristic Tool ApplicationElapsed time to initial value Variable ShortEnd-user business knowledge
High Low
Ongoing dependence on vendor
Low Some
Influence on vendor software direction
Low Significant
Cross-industry Yes NoVendor stability risk Modest Modest
Corvelle Drives Concepts to Completion
41
Global Digital InformationCreated
Zettabytes per year
Corvelle Drives Concepts to Completion
42
Sample Big Data Applications
Corvelle Drives Concepts to Completion
43
Enterprise BIVendor Landscape
Corvelle Drives Concepts to Completion
44
Value of Visual Analytics
Eliminate guesswork Answer business questions better & faster Produce key business metrics consistently Build insight into customers & problems Learn how to streamline operations Improve efficiency Learn what your true costs are See where your business has been, where it is
now and where it is going
Corvelle Drives Concepts to Completion
Oil & Gas Data WarehouseContext Diagram
WellViewProprietarywell data
gDCPublic
well data
Data warehouse
45
AvocetProduction
data
Qbyte FMFinancial
data
WCFDFrac’ing
data
ValNavCAPEX
forecastdata
Corvelle Drives Concepts to Completion
46
VISAGE Context Diagram
Data warehouse
VISAGE
Graphs Tables ExportsReports
Summarydata
Config.data
Update
Corvelle Drives Concepts to Completion
47
Any chance I could get better, faster, cheapervisual analytics instead?