big data - big insights - waze @google
TRANSCRIPT
![Page 2: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/2.jpg)
Data is Everything.Apps (and companies) win or lose based on how they use it.
![Page 3: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/3.jpg)
Organize the world’s
information and make it
universally accessible
and useful.Google’s Mission
3
“
![Page 4: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/4.jpg)
Google excels at collecting, storing, and extracting value from
big quantities of data
Google is a (Big) Data Company
![Page 5: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/5.jpg)
Google in just 1 minute:
1000 new devices
3M Searches 100 Hours
1B Activated Devices
100M GB Search
Content
![Page 6: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/6.jpg)
10+ Years of Tackling Big Data Problems
6
Google Papers
20082002 2004 2006 2010 2012 2014 2015
GFSMapReduce
Flume Java
Millwheel
OpenSource
2005
GoogleCloudProducts
BigQuery Pub/Sub Dataflow Bigtable
BigTable Dremel PubSub
Apache Beam
Tensorflow
![Page 7: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/7.jpg)
“Google is living a few years in the future and sending the rest of us
messages”
Doug Cutting, Hadoop Co-Creator
![Page 8: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/8.jpg)
Capture ProcessStore Analyze
Data on your terms
![Page 9: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/9.jpg)
Capture ProcessStore Analyze
Data on your terms
![Page 10: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/10.jpg)
![Page 11: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/11.jpg)
Machine Learning
“Machine Learning is concerned with computer
programs that automatically improve their
performance through experience. “Herbert Simon Turing Award 1975Nobel Prize in Economics 1978
![Page 12: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/12.jpg)
Machine Learning
“A breakthrough in machine learning would be worthten Microsofts” (Bill Gates, Chairman, Microsoft)
“Machine learning is the next Internet”(Tony Tether, Director, DARPA)
“Machine learning is the hot new thing” (John Hennessy, President, Stanford)
![Page 13: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/13.jpg)
Google is no stranger to ML
![Page 14: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/14.jpg)
So what do you actually do?
Gain Actionable Insights!
![Page 15: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/15.jpg)
Trending Locations - Hilton TLV
![Page 16: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/16.jpg)
Trending Locations / Day of Week Breakdown
![Page 17: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/17.jpg)
Opening Hours Inference
![Page 18: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/18.jpg)
Optimising - Ad clicks / Time from drive start
![Page 19: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/19.jpg)
Time to Content (US) - Day of week / Category
![Page 20: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/20.jpg)
Irregular Events / Anomaly Detection
Major events, causing out of the ordinary traffic/road blocks etc’ affecting large
numbers of users.
![Page 21: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/21.jpg)
Dangerous Places - Clustering
Find most dangerous areas / streets, using custom developed clustering algorithms
● Alert authorities / users
● Compare & share with 3rd parties (NYPD)
![Page 22: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/22.jpg)
Server Distribution Optimisation
Calculate the optimal routing servers distribution according to geographical load.
● Better experience - faster response time
● Saves money - no need for redundant elastic scaling of servers
![Page 23: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/23.jpg)
Text Mining - Topic Analysis
Topic 1 - ETA Topic 2 - Unusual Topic 3 - Share info Topic 4 - Reports Topic 5 - Jams Topic 6 -Voice
wazers usual road social still morgan
eta traffic driving drivers will ang
con stay info reporting update freeman
zona today using helped drive kanan
usando times area nearby delay voice
real clear realtime traffic add meter
tiempo slower sharing jam jammed kan
carretera accident soci drive near masuk
![Page 24: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/24.jpg)
Text Mining - New Version Impressions
● Text analysis - stemming / stopword detection etc.
● Topic modeling
● Sentiment analysis
Waze V4 update :
● Good - “redesign”, ”smarter”, “cleaner”, “improved”
● Bad - “stuck”
Overall very positive score!
![Page 25: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/25.jpg)
Text Mining - Store Sentiments
![Page 26: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/26.jpg)
Text Mining - Sentiment by Time & Place
![Page 28: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/28.jpg)
![Page 29: Big Data - Big Insights - Waze @Google](https://reader034.vdocuments.site/reader034/viewer/2022052117/58872fe11a28abc0748b4d0f/html5/thumbnails/29.jpg)
Daniel [email protected]@gmail.com