making data science accessible to a wider audience

28
This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information. Making Data Science Accessible to a Wider Audience Lou Bajuk-Yorgan, Sr. Director, Product Management Streaming and Advanced Analytics TIBCO Software

Upload: lou-bajuk

Post on 28-Jan-2018

233 views

Category:

Data & Analytics


0 download

TRANSCRIPT

Page 1: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Making Data Science Accessible to a Wider Audience

Lou Bajuk-Yorgan, Sr. Director, Product Management

Streaming and Advanced Analytics

TIBCO Software

Page 2: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

DISCLAIMERDuring the course of this presentation, TIBCO or its representatives may make forward-looking statements regarding future events,

TIBCO’s future results or our future financial performance. Although we believe that the expectations reflected in the forward-looking

statements contained in this presentation are reasonable, these expectations or any of the forward-looking statements could prove to

be incorrect and actual results or financial performance could differ materially from those stated herein.

TIBCO could experience factors that could cause actual results or financial performance to differ materially from those contained in

any forward-looking statement made in connection with this presentation. TIBCO does not undertake to update any forward-looking

statements that may be made from time to time or on its behalf.

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing,

release and availability dates for TIBCO products and services. This document is provided for informational purposes only and its

contents are subject to change without notice. TIBCO makes no warranties, express or implied, in or relating to this document or any

information in it, including, without limitation, that the information is error-free or meets any conditions of merchantability or fitness for a

particular purpose. This document may not be reproduced or transmitted in any form or by any means without our prior written

permission.

The material provided is for informational purposes only, and should not be relied on in making a purchasing decision. The

information is not a commitment, promise or legal obligation to deliver any material, code, or functionality. The development, release,

and timing of any features or functionality described for our products remains at our sole discretion.

Page 3: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Multiple paths from Data to Decision and Action

Data Science helps deliver better

decisions faster

Page 4: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Scarcity of Data Science Skills

Analytical complexity of task and capability of user

Nu

mb

er o

f u

sers

General Population

Citizen Data Scientists (Analysts, Engineers, Scientists)

Data Scientists

Page 5: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

• Citizen Data Scientist: aspire beyond pretty pictures and simplistic dashboards

• By 2019, citizen data scientists will surpass data scientists in the amount of advanced analysis produced.

• By 2020, more than 40% of data science tasks will be automated, resulting in increased productivity and broader usage by citizen data scientists.

• This is the trend. How do we make sure people have the theright tools, to get the right answers?

5

Skeptical about Citizen Data Scientists?

http://www.gartner.com/newsroom/id/3570917

Page 6: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

• Pros• Easy prototyping of new models and analysis

• Huge array of analytic methods available

• The “best” method to solve a given problem is likely available

• Lots of people learning R in university

• Cons• Performance: Not designed for real time or Big Data applications

• Hard for non-Data Scientist to use directly—exacerbates the Data Science skills scarcity, by requiring both coding and Data Science knowledge

• Challenging to integrate and manage in enterprise applications

• Performance, commercial support and Intellectual Property concerns

• Result: Compromises which impact Agility• Recode in a new, less agile environment

• Rewrite, use specialized R packages to solve one problem better

6

Where does R fit in?

Page 7: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

TIBCO Analytics Platform

Numerical Models

Analytic Apps MODEL

ACTION

INSIGHT

Business User and Citizen Data Scientists

• Data Discovery - Insight

• TIBCO Spotfire

Data Scientist

• Analytics - Model

• TIBCO Statistica and TIBCO Enterprise Runtime for R

Developer

• Real time - Action

• TIBCO Streambase

© Copyright 2000-2017 TIBCO Software Inc.

Page 8: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

#1. Smart Visual Analytics Recommendation driven insights

Visual analytics is like a bicycle for your business mind.

Page 9: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

TIBCO Spotfire Mission & Vision

9

S

Help people explore data faster, take action sooner, and move business further…

A smart, unified, secure analytics experienceto take you & your business into the future

Page 10: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Current Capabilities & Differentiation

10

SIn-built TIBCO® Enterprise Runtime for R (TERR™)Live R computationsData function modulesvs. No statsStatisticians only

Instant linked viewsSmart RecommendationsGeoanalyticsAnalytic applicationsvs. Single chart + dashboardManual only analysisCustom development

Analyze & wrangle togetherAuto Self-documenting flowGovernablevs. Separate toolsMust know transforms aheadMessy, unwieldy models

Linked alerts & actionsLow-code applicationsvs. No alerts, no real-time, no actions

Simple to deploy, admin & scale | Smart distributed memory optimizationvs. Complex deploy, ETL scripting, vertical only scaling

Page 11: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

TIBCO Spotfire Visual Analytics

11

• Smart Recommendation-driven insights

• Multiple dynamic perspectives – no “old school” single page

• Fastest in and out of memory data engine for data big and small

• Rich, multilayer, accurate maps

• Threaded, searchable conversations with annotations and bookmarks

• Easy configured process specific analytic applications

• Over 40 relational, big data, cloud & proprietary sources

Page 12: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

#2. Numerical Models Analytic Apps

Page 13: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Point and Click Data ScienceS

• Contextual, “one click” calculations make powerful methods easy to use: descriptive stats, similarity, clustering, correlations, fitting, forecast

• Unique commercial engine for R language

• TIBCO Enterprise Runtime for R (TERR)

• Any statistic can be part of Spotfire visual aggregations or expression language

• Easily leverage the work of your Data Scientists from R, Statistica, SAS, Matlab, Python

• Access to Machine Learning, Deep Learning platforms

• TIBCO Community shares data science components

Page 14: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Embedded TERR in Spotfire®

Write R code directly in Spotfire;

TERR executes locally or on server

Manage TERR analytics locally or

in Server to reuse across

community

Deploy TERR-powered

applications to the web

Page 15: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Spotfire TERR Data Function

contourLines(x,y)

Draw layers on Spotfire Maps with R/TERR scripts

Page 16: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Power of Embedded Advanced Analytics

Page 17: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

TIBCO Spotfire® with H2O Integration

Example: Predictive Analytics for Manufacturing (“scrap parts as early as possible”)

Page 18: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

• Comprehensive Stats and Predictive Analytics – Simple UX

• 1000’s of stats, machine and deep learning, Bayesian methods

• Algorithm marketplaces – Azure ML, Algorithmia, Apervita, H2O

• Open source – R, Python, C#, Spark, H2O, CNTK Deep NN

• Data Blending – any data, anywhere

• Model & Rule Lifecycle Management

• Create workspace, manage, version control, deploy, embed

• Citizen Data Scientists – scale best practices with Web UI

• IoT Analytics – device and gateway publish, scoring

• Security & Governance

• Repeatable, auditable; GXP validation : audit logs, version control

• Non-traditional data – image & audio; text mining; Network Analytics with OrientDB, in-database analytics

18

TIBCO Statistica Analytic Apps

© Copyright 2000-2017 TIBCO Software Inc.

Page 19: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Simple UX for Data Scientist

• Drag-and-drop UI for model + rule creation and deployment

• Simplified data preparation, mash-up, and ETL

• Comprehensive palette of math and analytics

• Machine learning, deep learning, Bayesian methods

• Image, audio, text, Graph-db

• In-db and In-memory algorithms

• Flexible integration with R, Python, Scala, SAS, C++, C#, Java

Model & Rule Management and Deployment

• Metadata repository for model & rule version control, governance, security and audit trail

• Model version and rule lineage; champion/challenger

• Model & rule publish and embed everywhere

• Publish to TIBCO Streambase for streaming analytics on live data feed

• IoT applications - publish to edge

TIBCO Statistica: Highlights

Business User

Data Scientist

Developer

© Copyright 2000-2017 TIBCO Software Inc.

Page 20: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

#3. Streaming Analytics Continuous algorithmic awareness & automation

Page 21: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Streaming Analytics with Spotfire and TERR

LiveView Dashboard

Spotfire Visualization for context, drill down for root cause analysis

Real Time Visualizations

Alerting

Page 22: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Streaming Analytics

22

Visual | Powerful | Scalable | Fast | Extensible

Low/no code workflows for accessing, transforming and acting on Real Time

Data

Score R models in Real Time applications using

native TERR node (+PMML, SparkML, H2O,

etc.)

Page 23: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

community.tibco.com

© Copyright 2000-2017 TIBCO Software Inc.

Page 24: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Spotfire Wiki

© Copyright 2000-2017 TIBCO Software Inc.

community.tibco.com

Page 25: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Spotfire Machine Learning Community

Spotfire (R) Data Functions

• Machine Learning / Deep Learning

• Gradient Boosting

• Random Forests

• Anomaly Detection: Autoencoder

• Segmentation

• Propensity

• Affinity

• Non-Linear Regression; Decline Curves

• Modeling & Simulation

• Genetic Algorithms

• Optimization

Page 26: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information. © Copyright 2000-2017 TIBCO Software Inc.

Page 27: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.

Summary

• More demand than ever for Data Science, with too few skilled Data Scientists

• Rise of “Citizen Data Scientists”, who need the right tools, guidance and frameworks

• Importance of leveraging the work of Data Scientists

• TIBCO Analytics:• Easy to embed/leverage/deploy the work of Data Scientists, from R and

beyond• In Spotfire Visual Applications, used by business users and Citizen Data Scientists• In real time applications, to automate decision making

• Easier for Data Scientists to create and reuse predictive analytics in Statistica

• While leveraging the best of open source R, Python, etc.

• Rich community with examples, reusable assets, etc. • While maintaining necessary analytic governance and model management

Page 28: Making Data Science accessible to a wider audience

This document (including, without limitation, any product roadmap or statement of direction data) illustrates the planned testing, release and availability dates for TIBCO products and services. It is for informational purposes only

and its contents are subject to change without notice. © Copyright 2000-2017 TIBCO Software Inc. All rights reserved. TIBCO Proprietary Information.