a dive into microsoft strategy on machine learning, chat bot, and artificial intelligence by seokjin...
TRANSCRIPT
A Dive into Microsoft Strategy on Machine
Learning, Chat Bot, and Artificial Intelligence
SeokJin Han
Microsoft
Businesses will require ROI
from AI
• Investment increased 10 times recent 5
years (2011-2016), but commercial cases are
limited
• Drastic changes of views last 2 years
(AI: from enemies to partners)
Faster development on
Conversational Interface
• Game-changing innovations
(AI learns human languages)
• Natural language search from Google and
Bing, DeepText from Facebook (Personal
Pattern Recognition), Changes on Chat
Bots/Digital Assistants/Messenger Apps
Designs evolve to increase
Credibility of AI
• Reflects onto AI design the knowledge on
how human earns credibility between
people
• AI NLP integrated with Communication
components such as tone, emotion, timing,
visual perception, and word selection
Begin discussion on how AIs
will talk to each other
• Protocols between AIs
• How to evade collision between AI
systems operating as silos
• Consider collisions between AI systems of
different purposes
Imbedded bias will be a big
blocker for AI dev
• Cases from Google/Microsoft
• Gender, Racial imbalance
• Different sources of bias
• Training data, user interactions, lack
of diversity, conflicting purposes
InteractionsComputer – Computer
Human – Computer
Human – Human
5 predictions for artificial intelligence in 2017, Stuart Frankel, CEO, Narrative Science
AI discussion in “2017”?
Digital Transformation
Microsoft dedication to AI
• AI and Research group – Organizational
change
• Microsoft Research
• Information Platform Group
• Cortana Engineering
• “Democratizing AI”
• “Partnership on AI”(NPO) – founding
member
• Aggressive investments for Cloud based
Machine Learning, Cognitive Services, Bots
• Most diversified AI portfolio in the market
Agent Applications Services Infrastructure
Cortana Office 365
Dynamics 365
Cortana Intelligence
• Bot Framework
• Cognitive Services
• Cognitive Toolkit
• Azure Machine
Learning
Azure N Series
FPGA
Platform
Approach
Microsoft AI Portfolio
Machine Learning at Microsoft
• Clutter in Office 365Spam Filtering, Infer.Net probability model
• Power BIData visualization in Natural Language
• CortanaVoice recognition/synthesis, Intent/entity extraction
• KinectBehavior recognition from infra-red images
• HololensAugmented Reality
• Windows Phone KeyboardsEmphasizes keys to pick using spell correction history
• Windows TabletEnhances touch recognition quality
• OneNoteEnhances handwriting recognition quality
• Windows Boot time reductionReads frequently used apps in advance
• Microsoft BandHigher measurement accuracies using cheaper sensors
• XBox GamesAI, Ranking System
• Bing / SharePointSearch
• OneDriveAutomatic image tagging / categorization
• Skype TranslatorReal time bi-directional translation
• Project AdamImage recognition : Recognize different dog breeds,
identify toxic plants + more
Cosmos/Scope
Big Data Services running Microsoft
Stored data : 3 EB+
Cluster size : 10 thousands+ nodes
# of machines : 100 thousands
Analyzed data : 150 PB+ / day
Internal analysts : thousands
Analytics jobs : 100s thousands / day
SMSG
Live
STB Commerce RiskLCA
Cortana Intelligence Suite
Transform data into intelligent action
Intelligence
Dashboard /
Visualization
Info Mgmt Big Data Store Machine Learning /
Advanced Analytics
CortanaIoT Hub
Event Hub
HDInsight
(Hadoop and
Spark)
Stream
Analytics
Data Intelligence Action
People
Automated Systems
Apps
Web
Mobile
Bots
Bot
FrameworkSQL Data
WarehouseData Catalog
Data Lake
Analytics
Data Factory Machine
Learning
Data Lake
StoreCognitive
Services
Power BI
Data
Sources
Apps
Sensors
and
devices
Data
FUTURE PROOF ARCHITECTURE
Azure
API
Management
Backend Services
Data sources
Apps
Sensors and devices
Event Hub
IoT Hub
Machine Learning
HDInsight(Apache Spark)Storage
Power BIStream Analytics
SQL Data Warehouse
Azure Data Factory & Azure Data Catalog
Data Lake StoreData Lake Analytics
SQL Server Integration Services
R ServicesStreamInsights Analytics Platform
System
Reporting Services, Analysis Services,
Mobile Report
Microsoft R ServerMicrosoft Office
Cognitive Services
Bot Framework
Cortana
PolyBase
Po
lyB
ase Pu
bli
sh &
Co
nsu
me
Demo
• Cortana Intelligence Gallery
Rolls-Royce case studyhttps://customers.microsoft.com/en-US/story/rollsroycestory
Rolls-Royce demohttp://rolls-royce.azurewebsites.net/#/fleetlocation
Solutions – Predictive Maintenance for Aerospacehttps://gallery.cortanaintelligence.com/Solution/Predictive-Maintenance-for-Aerospace-4
Tutorial – Simulating phenotypes from genomic datahttps://gallery.cortanaintelligence.com/Experiment/Simulating-phenotypes-from-genomic-data-2
https://github.com/Azure/Cortana-Intelligence-Gallery-Content/tree/master/Resources/Phenotype-Prediction
Solutions – Vehicle Telemetry (IoT)https://gallery.cortanaintelligence.com/Solution/Vehicle-Telemetry-Analytics-9
https://docs.microsoft.com/en-us/azure/machine-learning/cortana-analytics-playbook-vehicle-telemetry
Advanced Analytics Cycle
Act: Score,
Visualize
Deploy Apps,
Services &
Visualizations
Measure
Preparation Modeling
Feature &
Algorithm
Selection
Model Testing &
Validation
Models
Visualizations
Ingest
Profile
Explore
Visualize
Transform
Cleanse
Denormalize
Prepare Model
OperationalizeModels
Visualizations
Data prep and exploration
Statistical analysis
Predictive models
Evaluating models
Input1 Input2 … Actual Predicted
• Classification example – Confusion Matrix
Classification vs Regression
Azure Machine Learning
Machine Learning
Cloud BI
(Power BI)
On-premise 대시보드(SQL Server 2016
Reporting Services)
1. Data Ingestion 2. Experiment
(Build and
evaluate models)
3. Deploy as web services
다양한 지원 Data set • Plain text (.txt)• Comma-separated values (CSV) • Tab-separated values (TSV) • OData values• SVMLight data (.svmlight)• Attribute Relation File Format (.arff) • Zip file (.zip)• R object or workspace file (.RData)
클라우드 BLOB/테이블 저장소(Azure Blob /Table Storage)
Hive 쿼리(HDInsight)
클라우드 PaaS형 DB
(Azure SQL DB)
1) 데이터 셋 업로드2) 클라우드 데이터 원본에 직접 연결
클라우드 BLOB/테이블 저장소(Azure Blob /Table Storage)
Hive 쿼리(HDInsight)
클라우드 PaaS형 DB
(Azure SQL DB)
실험 결과 데이터 셋 저장
웹 서비스로 배포
4. Consume ML models
잘 만들어진 분석 모델의 API화(타 비즈니스 앱에서 사용하기 위해)
On-premise Excel BI
서비스 API 키를 사용하여어플리케이션에서 API를호출하여 JSON 형태의결과 값 직접 사용
C#, Python 등 다양한언어로 API 호출 가능
2) 시각화
1) 비즈니스 어플리케이션에서 활용
Azure BLOB Storage에API 호출 결과(배치) 데이터 집합 저장
실험 결과 데이터 셋 또는 API 호출 결과 데이터셋을 시각화
[웹 서비스 관리 화면][2) 클라우드 직접 연결 방식 : 쿼리 입력 가능] [실험 수행 화면]
1) 모델 API 활용한 비즈니스 앱 개발2) 결과 데이터를 활용한 시각화
실험에 사용할데이터 전송
Demo
• Azure Machine Learning
Simple example : Linear Regression
Predictive Maintenance examplehttps://gallery.cortanaintelligence.com/Experiment/Predictive-Maintenance-Step-2A-of-3-train-and-evaluate-regression-models-2
Evaluate Model - Metrics Reportedhttps://msdn.microsoft.com/library/azure/927d65ac-3b50-4694-9903-20f6c1672089https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-evaluate-model-performancehttps://blogs.msdn.microsoft.com/andreasderuiter/2015/02/09/using-roc-plots-and-the-auc-measure-in-azure-ml/
JupyterNotebook to explore dataset
Excel Add-in for Azure MLhttps://blogs.technet.microsoft.com/machinelearning/2015/09/01/excel-add-in-for-azure-ml/
Operationalizing R with AML
Machine Learning cases – Mfg/Services
Publish Category Customer Use Case
2015/12
Utility /Electricity production prediction
ServusNetBefore : Previous system offered farm level production prediction using daily weather forecast. Now : Using cloud based end-to-end solution, covers more plant types, supports global scale multiple farm portfolios
2015/10Service/
MarketingOpenField
OpenField is an innovative data mgmt company which provides solutions for elite soccer clubs/sports/concert halls. Now : Contextual marketing solutions provide Ticket Purchase prediction, No-show prediction and maximize profits
2015/09Finance/Predictive
MaintenanceDiebold
Before : Unplanned downtime is a big loss and causes revenue drop, sudden repair cost, customer dissatisfactionNow : With advanced IoT technologies, they now can monitor machines periodically / continuously and predict failures before they happen.
2015/06Utility/
Demand Forecasting
GenscapeGenscape provides data and intelligence services in energy industry. Piloted Demand Forecasting model developments.
2015/03Utility/
Workload prediction
eSmart Systems
eSmarts developed S/W for Smart Grid / Meters for Norwegian utility customers. Predicts energy workload from minimum scale(meter-level) to higher, forecast where will be the bottleneck, use results for optimization algorithms to automatically rebalance workloads.
2014/09Utility/
Smart Building
Carnegie Mellon Univer
sity
Carnegie Mellon Univ uses Azure and PI System™ (by OSIsoft, a global ISV with Microsoft) to maintain buildings and reduce energy cost. Now CMU leverages Azure Machine Learning to improve failure detection, diagnosis, and optimize operations.
2014/08Manufacturing/
Predictive Maintenance
ThyssenKrupp
ThyssenKrupp Elevator focuses on service stability as competitive edge. With IoT and Machine Learning, ThyssenKrupp provides unique premium services including predictive maintenance at its core.
Publish Category Customer Use Cases
2015/12Service/
HRRussell Reynold
s Associates
Before : Hire candidates search is labor-centric and requires manual analysis with query writing on DatabaseNow : Microsoft Big Data and Advanced Analytics technologies enabled Machine Learning based Candidate Recommendations based on structured/unstructured data
2015/11Retail/Deep
LearningCoco-Cola
A case where Coca-Cola Company and Universal McCann used Microsoft Deep Learning technologies to perform cutting edge marketing campaign
2015/08Healthcare/Diagnosis
OptolexiaUsing eye movement tracking data and analytics engine built using cloud based Microsoft Azure Machine Learning, Optolexia offers far faster tool to diagnose Dyslexia(난독증) at school.
2015/06Public/Churn
Analytics
Tacoma Public School
Used Churn Analytics approach to predict students with high probability to quit. A public school in Tacoma, WA, dramatically improved its understanding of student behavior and could act upon the insights discovered using Machine Learning.
2015/05Service(Resea
rch)/Marketing
MendeleySocial document platform provider for researchers, built models to predict key users, performed email target marketing, and expanded its user base.
2015/04Healthcare/
Demand Forecast
Gaffey Healthcare
Used Azure ML to build predictive models and integrate with AlphaCollector, providing hospitals with insights how long it will take for insurance companies to pay claims, and help determine whether a human collector is needed to accelerate the claim payment process. Helped customers improve cash flows and reduce operational costs.
2015/02Healthcare/Diagnosis
Aerocrine
Before : Aerocrine’s monitoring tools are effectively used to diagnose and cure Asthma(천식), but very sensitive to small changes in the environment.Now : Using cloud based analytic solutions to improve diagnostic stability, helping millions of Asthma patients WW.
2014/12Retail/
MarketingJJ Food Service
Customers want products they like to be already in the shopping cart (Personalized recommendation). Customers of JJ Food Service are experiencing this whenever the make orders on web and over phone. Enabled by Azure Machine Learning and Dynamics.
2014/12Retail/
MarketingPier 1 Imports
Pier 1 Imports wanted to be connected to customers with data insights. Evaluated Predictive Analytics solutions and chose cloud based Azure Machine Learning and Power BI.
Look out for the cases outside your industry.
Many approaches are applicable across industries.
1. Which area to drive Digital Transformations through data analytics
2. What procedures/tools/algorithms to take
Machine Learning cases – Others
Why Chatbots are disrupting UX
Potential of Chatbots : “ability to individually and contextually communicate one-to-many”
1. 1 to Many communication
• Emails, Social Media are examples, but they are not personal.
2. Individual communication
• Personalization in 1-to-many communication is a recent consideration.
• Best way today is to use programmatic advertising but this requires efforts and know-how.
3. Contextual communication
• This happens whenever you talk to someone. Most people do this unconsciously.
Why Chatbots are rising as a new type of UX
Most people are consistently using messaging apps
• Average owned apps: 27. Daily using 4-6.Keep using 3% In 30 days
• 2.5 billion people owns 1+ messaging apps.3.6 billion expected (50% of WW population) in years.
Do not try to invent new appsto bring in customers.
Instead, offer your services in already popular messaging apps.
• Bots are UX, Conversations as a Platform (CaaP)
• Contents are important as well: From simple information delivery to actionable insights
1.Microsoft R • Statistical Analysis, Data Preparation, Predictive Modeling
Big Data • Hadoop, Spark, Data Lake Analytics
Machine Learning • Predictive Analysis, Deep Learning
Cognitive Services • Image Recognition, Natural Language Understanding
Bot Framework • Dev Framework, Different service channels
Technologies around Bots
Reference Architecture for Bots
Extended Scenarios
Big Data Analytics
Spark on HDInsight
Data Lake Analytics
Real Time Processing
Stream Analytics
Personalized Offer
Machine Learning
SQL Server R Services
On-premises Integration
SQL Server
Data Management Gateway
Visualization enabled
Power BI Embedded
Developing apps that understand human
• Face, image, emotion recognition
• STT/TTS, voice recognition/identification
• Intent/Entity understanding, sentiment/topic recognition, spell check
• Complex task processing, knowledge exploration
• Bing search functionalities integration(Web, auto-complete, image/video/news search)
Intelligence
Cortana
Bot
Framework
Cognitive
Services
Demo
• Cognitive Services Live, Intelligent Kiosk
Bots – wherever you have conversations
Intelligence
Cortana
Bot
Framework
Cognitive
Services
• Bot Connector : Register your own Bots, configure channels, publish on Bot Directory. Connect your Bots to SMS, Office 365 emails, Skype, Slack, Twitter, Facebook, Telegram and more.
• Bot Development SDK: Open source SDK available at GitHub. Offer every tool required for Bot development based on Node.js and C#.
• Bot Directory : A public place where you can publish your own Bots.
Enterprise Meeting Assistant
ATTEN
DESS
STA
RT T
IME
DU
RA
TIO
N
LOC
ATIO
N
Pls schedule a meeting for my team on the
next Tuesday morning with Yong at 13F
User Input
MY TEAM
IS A
LIST OF
PEOPLE
NEXT TUESDAY
MORNING
IS A
DATE
TIME
Yong
IS A
PEOPLE
NAME
13F
IS A
LOCATION
NAME
BOOK A MEETING
IS AN
INTENION
• Resolve Attendees
Create Active Directory query for “my team”
FIND “PEOPLE REPORT
TO ME” IN
ACTIVE DIRECTORY• Slots for Book Meeting
“Book a meeting” is an intention to book meeting
Yohn C. Jingtian J. Wenhao H. Lei F.
Filter related people by name contains “Yong”
• Link to Entities
Yong Rui Yong Liu
Filter people by relationship to me
Yong Rui
Using AI + HI to Complete Tasks
Conversational Entity Extraction
Response suggestion
AI for Business
Provides customers with more personal and natural ways to interact with businesses
Adds AI to business processes and connect Insights to Actions
Use insights hidden in data from in/out of company, understand customers and develop businesses
Demo
• Skype Bots, [email protected], LUIS, QnA Maker
Democratizing AI
To empower every person and organization to achieve more with AI