the future of microsoft data strategy : opensource, iot, analytics and power bi ; global azure...

Global Azure Bootcamp #GlobalAzure #MUGLyon @AZUGFR

LYON - FRANCE

Vincent THAVONEKHAMThe Future of Microsoft Data Strategy : OpenSource, IoT, Analytics and Power BI

1


LYON - FRANCE

GAB 2016 LYON ORAGANIZER

LOCAL SPONSORS

THANK YOU ALL !!

www.iot-world.fr Prochaine édition

21 et 22 mars 2017 à Paris

2

GLOBAL SPONSORS

http://www.bluesyntaxconsulting.com/

http://t.sidekickopen44.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJN7t5XZsRJd4KW653klz8qC9zTW8qlQyH56dPVjf6FcF1v02?t=http://cloudmonix.com/?utm_source%3Dgab2016%26utm_medium%3Dweb%26utm_campaign%3Dgab2016&si=5852753167646720&pi=39f766c2-487e-4caa-8017-da8234ca83ea

http://www.cloudberrylab.com/

http://www.azuredockit.com/

http://global.azurebootcamp.net/wp-content/uploads/2013/02/image8.png


http://global.azurebootcamp.net/wp-content/uploads/2016/03/CODEMagazine.png

https://www.opsgility.com/





LYON - FRANCE

• Description of the Data, traditional vs. new

• What are the problems ?• Solutions, with a focus on Power BI• Conclusion

CONTENT

3


LYON - FRANCE

Microsoft Practice Manager @VISEO

MVP Azure, P-Seller ALM/DevOps

www.thavo.com @vThavo

Microsoft User Group LyonAzure User Group France

Author & co-author DevOps et Azure(France & international)In 2015 :- White paper: Zend /

Programmez!- Round table: live streaming

for Electric Cloud, with Gene Kim (The Phoenix project)

Speaker : Vincent Thavonekham,


LYON - FRANCE

CONTENT

• Description of the Data (traditional vs. new)

• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion


LYON - FRANCECredits : http://blogs.technet.com/b/dataplatforminsider/archive/2013/06/03/sql-server-2014-unlocking-real-time-insights.aspx

INTRODUCTIONThe Data is an “old” topic addressed by SQL Server


LYON - FRANCE

VALU

E

DIFFICULTY

INTRODUCTIONKey types of analysis in the Big DataTraditional BI has evolved. Microsoft has grouped them into 2 categories:

Descriptive and Advanced Analytics (part of the « Data platform »

Source :Gartner

Data Scientists (Senior Doctor in

Maths, Astrophysics, …)


LYON - FRANCE

Data Platform leaps in innovation in 2015 !2012

SQL Server 2012

2013

HDInsight

“Our industry does not respect tradition – it only respects innovation.” – Satya Nadella

2015

Azure Machine Learning

Azure Document DB

HDInsight on Linux, HBase, Storm

Azure SQL DB with In-memory OLTP

Cortana Analytics Preview

IoT Services

2014

SQL Server 2014

APS

Azure SQL Database

Power BI

+ BI DataZen

+ Revolution R

Note : OLAP : On Line Analytic Processing OLTP : On Line Transaction Processing

Source : Microsoft


LYON - FRANCE

INTRODUCTIONKey actors in 2015

2015 2015


LYON - FRANCE

Traditional data warehouse approach

BI & analyticsETL

Staging

+ Sharepoint + Power BI + O365


LYON - FRANCE

Breaking points of the traditional approach

Staging

Increasing data volumes1

Real-time data2

New data types3

Cloud-born data4 $100

B spend on cloud

50% large orgs have hybrid by 2017

40% CRM sold are SaaS

50x Data growth 2010-2020

40ZB Digital Universe 2020

1Trillion Web pages

Source : Microsoft Convergence


LYON - FRANCE

INTRODUCTIONKey actors in 2016

2015 2016

Spectacular progress of the TOP 3

Leaders !


LYON - FRANCE

CONTENT


• What are the problems ?• Solutions (with a focus on Power BI)• Conclusion


LYON - FRANCE

What are the problems ? And how to address them ?

• Many innovations in 2015• Too many data• Too many data types• Cold and hot data (i.e. traditional and real-

time)• Need for Agility (don’t know yet how to use

data)• Need really powerful compute to process


LYON - FRANCE

(Continued) What are the problems ? And how to address them ?

• EXCEL is no more the key peace of Microsoft Agile BI,

• Reporting Service is less and less used,

• OLAP cubes are no more the automatic solution


LYON - FRANCE

Everyone

3rd waveEnd user BI

A new generation of BI Today, BI extends to everyone

Analyst to end user

2nd waveSelf-service BI

IT to end user1st waveTechnical BI


LYON - FRANCE

FRUSTRATIONS : The old Power BI and its « WOW » effect• Beyond the “WOW effect”, you’ll find the « Ouch » effect !

Ouch : EXCEL 250 Mb

limitations!

Ouch : what ?!?PowerQuery, PowerPivot, PowerView, PowerMap

Ouch : Sharepoint ?

Ouch : Silverlight ?

Ouch : Excel Plugins !

Ouch: Office licences

2010/2013/365 Pro

Ouch : Need admin rights ?

Ouch : … « and what else ?? »


LYON - FRANCE

Event Queuing System

Collection Presentation and action

Event producers

Transformation

Long-term storage

Event Hubs

Storage adapters

Stream processi

ngCloud gateways(web APIs)

Field gateways

Applications

Search and query

Data analytics (Excel)

Web/thick client dashboards

Live Dashboards

Apache Storm on

HDInsight

Devices to take action

Kafka /RabbitMQ /ActiveMQ

Web and Social

Devices

Sensors / IoT

Azure Stream

Analytics

HDFS

Azure DBs

Azure storage

HBase

CURRENT BIG DATA ECOSYSTEM !

Source : Microsoft


LYON - FRANCE

IoT AS AN INPUT TO THE BIG DATAZOO

M


LYON - FRANCE

CONTENT


• What are the problems ?• Solutions (with a focus on

Power BI)• Conclusion


LYON - FRANCE

SOLUTIONS (some of them!)• Microsoft is Opening even more to the

world,• Reinventing Microsoft• Many Open Source solutions supported• Following standards

• SaaS + VERY quick Onboarding• Start small with Free offers• “Suites” to simplify usage: Azure IoT Suites (with pre-packaged

solutions), Cortona Analytics Suites,• Focus on PowerBI.com and PowerBI Desktop

MICROSOFT

ANSWERS


LYON - FRANCE

Robust Architecture : LambdaStandards + Generic architecture = OPEN to non Microsoft technologies

Storage Cold Path Analysis

Apache Spark

Apache StormHot Path AnalysisPresentation

ApacheKafka

IoT Hub

Apache Storm on HDInsight

Apache HBASEon HDInsight

Hybrid : MS + Open Source

“Lambda Architecture”

MICROSOFT

ANSWERS


LYON - FRANCE

Azure IoT SuiteZOOM

« Think big, start small » : Microsoft referenced architecture

Pre-packaged solutions:• Asset management• Remote monitoring• Preventive

maintenanceMICROSOFT

ANSWERS


LYON - FRANCE

Cortana Analytics Suite

http://pocavenet-azure-iot-dashboard.azurewebsites.net/

ZOOMMICROSOFT

ANSWERS

http://pocavenet-azure-iot-dashboard.azurewebsites.net/


LYON - FRANCE

Global Vision of the Microsoft IoTMICROSOFT

ANSWERS


LYON - FRANCE

Event Queuing System

Collection Presentation and action

Event producers

Transformation

Long-term storage

IoT Hubs

Storage adapters

Stream processi

ngCloud gateways(web APIs)

Field gateways

Applications

Search and query

Data analytics (Excel)

Web/thick client dashboards

Live Dashboards

Apache Storm on

HDInsight

Devices to take action

Kafka /RabbitMQ /ActiveMQ

Web and Social

Devices

Sensors / IoT

Azure Stream

Analytics

HDFS

Azure DBs

Azure storage

HBase

CURRENT BIG DATA ECOSYSTEM !

Source : Microsoft


LYON - FRANCE

Tendency with reporting / BI• Reporting Services are “brittle” • and needs IT experts to modify

• No more Reporting Services within TFS/VSTS• Reports not based on OLAP cube anymore• Replaced by a lightweight real-time reports and Power BI.com (1)

• No more Power BI inside EXCEL,• Replaced by a lightweight standalone Power

BI.com(1) https://blogs.msdn.microsoft.com/visualstudioalm/2016/01/11/announcing-reporting-capabilities-for-visual-studio-team-services

https://blogs.msdn.microsoft.com/visualstudioalm/2016/01/11/announcing-reporting-capabilities-for-visual-studio-team-services


LYON - FRANCE

• The new SQL Server Reporting Services

https://blogs.msdn.microsoft.com/sqlrsteamblog/2016/03/18/sql-server-2016-rc1-whats-new-in-reporting-services

Tendency with reporting / BI

https://blogs.msdn.microsoft.com/sqlrsteamblog/2016/03/18/sql-server-2016-rc1-whats-new-in-reporting-services


LYON - FRANCE

Zoom on Power BIStarting with a definition

“Power BI is a suite of business analytics tools to analyze data and share insights.

Monitor your business and get answers quickly with rich dashboards available on every device.”

MICROSOFT

ANSWERS


LYON - FRANCE

Zoom on Power BIStarting with a new Portal

===== December 31th 2015 =====

https://powerbi.com


LYON - FRANCE

New and easy prices

Version pro

Version gratuite et pro===== December 31th 2015 =====


LYON - FRANCE

IT challenges

Management of Data in the cloud Security, private data, refresh and data updates.

Need to ease the usage

Need an easy tool for both the IT and the business users, to decrease time spent to create reports and visuals.

Keep data up to date Complex to refresh data from various sources, and manage increasing data volumes due to real time.

Integrating with the existing Time and cost.

IT Administration tasks Time spent for high SLA. Cost of the servers and maintenance.

Various Data sources Complex to connect to both On Premise and Cloud (SaaS) Data.


LYON - FRANCE

Power BI – Key benefits and differentiatorsPre-built content packs, consisting of

dashboards and reports, for popular SaaS solutions

Integration with familiar Microsoft products, and commitment for scale and availability in Azure

Intuitive data exploration by using natural language query

Real-time dashboard updates

Fast deployment, hybrid configuration, secure, and integrated with existing IT systems

Secure, live connection to your data sources, on-premises and in the cloud


LYON - FRANCE

How does Power BI addresses both the IT challenges and the business needs ?


LYON - FRANCE

Data connections Version gratuite et pro


LYON - FRANCE

Connectorsto popular SaaS applications

Today:

Soon:

Version gratuite et pro


LYON - FRANCE

Quick start samples



LYON - FRANCE

Create reports

Desktop



LYON - FRANCE

Power BI DesktopPower BI Desktop

Desktop application to explore data and create interactive reports.Easy usage, with Drag & Drop experience.Publish directly to PowerBI.com.



LYON - FRANCE

Reports


Azure Stream Analytics

Azure SQL/DW

HDInsight

AS Tabular

PBI Desktop

Excel


LYON - FRANCE

New custom visuals: Gallery (subset)Funnel

Combo Chart

Gauge

Tree Map

Fill Map



LYON - FRANCE

Dashboard

Azure Stream Analytics

Azure SQL/DW

HDInsight

AS Tabular

PBI Desktop

Excel



LYON - FRANCE

« Ready to go » content packs :to be used against popular SaaS applications



LYON - FRANCE

« Ready to go » content packs :to be used against your data sources

Version pro


LYON - FRANCE

Live data sources Version pro

Consume live data sources with full interactivity


LYON - FRANCE

Power BI Personnal Gateway Version pro

Dataset

Access your on-premise data using the Data Connectivity Gataways

(Personal and Data Management)


LYON - FRANCE

Data updates : Refresh strategies

Real time data streaming in your dashboard- Using Power BI REST API- Using Azure Stream Analytics

Live connectivity to Microsoft SSAS - Auto-refresh- The Data are updated when the user accesses a dashboard or a report

Planned or automatic refresh- SaaS connectors (updates every 24h or planned or on-demand)

- Personal Gateway : planned or on demand refresh

Data refresh architecture

Live query

Popular SaaS Solutions

Live dashboards and reports

Dynamics Marketing

Power BI

SQL Server Analysis Services (SSAS)

Secure Credential

Store

PRO Version1 million rows/hour

FREE Version10 000 rows/hour

PRO VersionEvery hour

FREE VersionEvery day

https://support.powerbi.com/knowledgebase/articles/474669-refresh-data-in-power-bi

https://support.powerbi.com/knowledgebase/articles/474669-refresh-data-in-power-bi


LYON - FRANCE

Sharing reports and personal Dashboards


Share dashboards with others within your orgianization. • Shared dashboards (and related reports) are Read-Only. • They cannot be saved volontarily• The users cannot access the underlaying datasource• The users always have an up to date view


LYON - FRANCE

Team working collaboration

Version pro

Team working with Office 365 groups within Power BI

Create, publish and display contents packs of the company

Manage access control and sharing via Active Directory groupes


LYON - FRANCE

Stay connected from any mobile devices

• Support for iPad, iPhone, and Windows

• Get alerts when your data changes

•

Share and collaborate with your team. Take collaborative décisions



LYON - FRANCE

Query your data by asking a question in Natural language



LYON - FRANCE

Push data

Push your data directly into Power BI from your apps.

Integration

Add Power BI to the workflows of your apps.

Develop a real-time connection from your data sources, applications, IoT devices, …

Real-time

{ name":{String}, "tables":[ ..." ]}

0110010100001010

“name”:[string],“columns”:[String]“daType”:[String]

{ "name":{String}, "columns":[ { "name":{String}, "dataType":{String} },

Visuals

Create your own visuals for Power BI

Power BI pour les developpeurs

Version gratuite et proDéveloppements spécifiques


LYON - FRANCE

Only ONE access point : Powerbi.com(for both Desktop and Cloud Service)

Sign up for a free Preview account

Take the Power BI Tour

http://www.powerbi.com/


LYON - FRANCE

EXAMPLE 1 : Integration with Azure servicesUse Azure Stream Analytics to push live ‘near-real time’, streaming data to Power BI

– Enables real-time dashboards at scale, over data from devices and applications

– Can stream millions of events per second– Can perform aggregation over time windows


LYON - FRANCE

EXAMPLE 2 : Integration with Azure servicesAdvanced analytics

Coalesce Azure services together to drive advanced analytics:

– Azure HDInsight: Big Data processing

– Azure Machine Learning: Predictive analytics

– Azure Data Factory: Orchestration at scale


LYON - FRANCE

CONCLUSION• STOP thinking Data the traditional way

• « Mobile First, Cloud First » (Nadella Satya)

• “Microsoft: opening the Windows to the world” ( Edouard Danel, 2016)

From the inside to the outside” & “From the outside to the inside”

• Microsoft is facilitating (again) the adoption of the new technologies• The NEW Microsoft Power BI (Power BI .com and Power BI Desktop)• Azure Machine Learning, Cortona Analytics, …


LYON - FRANCE

GAB 2016 LYON ORAGANIZER

LOCAL SPONSORS

THANK YOU ALL !!

www.iot-world.fr Prochaine édition

21 et 22 mars 2017 à Paris

57

GLOBAL SPONSORS

http://www.bluesyntaxconsulting.com/

http://t.sidekickopen44.com/e1t/c/5/f18dQhb0S7lC8dDMPbW2n0x6l2B9nMJN7t5XZsRJd4KW653klz8qC9zTW8qlQyH56dPVjf6FcF1v02?t=http://cloudmonix.com/?utm_source%3Dgab2016%26utm_medium%3Dweb%26utm_campaign%3Dgab2016&si=5852753167646720&pi=39f766c2-487e-4caa-8017-da8234ca83ea

http://www.cloudberrylab.com/

http://www.azuredockit.com/



http://global.azurebootcamp.net/wp-content/uploads/2016/03/CODEMagazine.png





the future of microsoft data strategy : opensource, iot, analytics and power bi ; global azure...

Technology