How to Build a Successful Data Team ?

Data ScienceSoftware Platform

The Solitude Of The Data Team ManagerData Driven NYC 04/11/16

My nerdy background

Type SystemsAutomated ProvingAbstract Program InterpretationFunctional ProgrammingGarbage Collection and VmsGraph AnalyticsChess IA Natural Language Processing80% Emacs / 20% VIM

So to sum it up 3I (USED TO) HATEGUIs

Well, to sum it up

I used to hate GUIS ..

I mean. I think I actually started using eclipse in 2011. Then I switched to Sublime Text in 2012

and our software is

A supercharged Visual IDE for Data TeamsDeployment

Meet HAL

Hal AlowneBI ManagerDims Private Showroom

Meet Hals Boss, DIM

Hey Hal ! We need a big data platform like the big guys.Just do what theyre doing!

Big DataCopy Cat Project

TECHNOLOGYDisconnectWhat technologies should I use ?

Welcome to Technoslavia !8

TOY PLATFORM ANTI-PATTERN9

Test and Invest in Infrastructure == Skilled Peopleor Go For Cloud / Packaged InfrastructureYour Brand New Hadoop Clusteris perceived as slow, not so used and not reliable

TECHNO MISMATCH ANTI-PATTERN10Assume Being PolyglotorBe a Dictator

VSVSThe PythonClanThe RTribeThe Old ElephantFraternityThe New ElephantClub

PREDICTIVE ANALYTICS DEPLOYMENT STRATEGY11Website 2000 winners Companies that were able to release fast "Artificial Intelligence with Data for Internet of Things" 2010 winners Companies able to put intelligence in production?

Design a way to put PREDITICTIVE MODELS IN PRODUCTION

PEOPLEDisconnect

Who should I hire ?

Classic Business Intelligence Team Organization

Business Leader Data ConsumerLine-of-business Data ConsumerBusiness ProjectSponsorBI Solution ArchitectModel Designer ETL DeveloperDashboard / Report Designer

Specs

DimBig Boss

Data Science Team Organization

Business Leader Data ConsumerLine-of-business Data ConsumerBusiness ProjectSponsorData EngineerData AnalystSystem Engineer / Data ArchitectBusiness NeedsData Scientist

ITConstraintsI.T.

Manage Expectations

15

Data PlumbererDataEngineerData Scientist

Data Waiter

DataCleanerDataAnalyst

REALJOBDREAMJOB

Managing Extreme Personalities16

Data ScientistHighly CreativePassionateHard to hire? Hard to manage?

Want to take Hals job? AmbitiousHard to retain?

Paired for Data17Data AnalystDiscover PatternsData EngineerMake things workFightdata entropyFighttechentropy

What do you prefer? 18

One AnalystOne EngineerOne Data Scientist Four data scientists

OR

Two Mindsets Can CoexistCLICKERSCODERS

DATADisconnect

What about data ?

What is the main reason for data project to fail ? 21> DATA NOT AVAILABLE

BUT FOR ONLY INCREMENTAL GAINBusiness Goal Definition and DataFeature EngineeringAlgorithm

How to Get Data if you dont have it23

THE CICADATHE SPIDERTHE FOXBe Optimistic ! Wait for Open Data InitiativesAnd data available in the Enterprise Hub / Data Lake !Create Network !Create a set of trackers or Addictive Data Collection internallyTo get Data on your side !Hunt for Big Problem!Convince the CEO that you can Solve a Business Critical problem And use it as an excuse to get allThe data you want !

PRODUCTDisconnect

What is Big Data about ? 25

The Age Of Distributed Intelligence26

Global, Personalised and Real Time Data Driven Services

Data to Visualize or Data to Automate ?Moving to a world of automated decision making27

DATA FOR MORE INSIGHTSDATAFOR AUTOMATED DECISIONS

Involve Product Team 28Product FeaturePersonalised Item RankingProduct FeatureNotify User Only when NeededProduct Feature:Historical Data For Path Optimisation

Have Product Management Deeply Involved In the Data Team

Focus on your added value 29Build by the Data TeamIs the problem at the Core of my Business Process? Is it a common problem / with share data?Can i solve it on my own?Really?Hire Consultant and LearnBuild bythe Data TeamGo for Best of Breed SaaS SolutionBuild bythe Data Team?YesNoNoYesNoYesNoYes

Create an API culture

Do not shareRandom Piece of CodeFlat FileEmailDo shareReproductible documented workflowsClean, documented APIs

Did Hal found his solutions ? Technology DataPeopleProductPolyglot on top of open source Find a way to make clickers and coders work together Create an API culture and involve the product teams Hunt for Big Problems and Convince the CEO

Is this the end ?Hal AlowneBI ManagerDims Private Showroom

That was the (romanced) story

data scientists and engineers252locations1software

by the numbers

For a simple software for clickers and coders

32

Car Sharing Worldwide LeaderFlash Sales Worldwide Leader

One Mission : Never leave Hal ALONE

3,700 Hotels Worldwide 2500lovelyusers

by the numbers70customers

33

Food for thoughts www.dataiku.com/blogTHANK YOU !

FREE (as in Beer) Software www.dataiku.com/dss