people as sensors - mining social media for meaningful information
DESCRIPTION
The video of this talk is available at https://www.youtube.com/watch?v=4ZdknOPY_jQ More and more we are all broadcasting information. Geolocation data, “this x sucks” data, weather data, etc. More and more that data is being parsed and analysed in realtime, such that we have now become sensors. How does this work, what does this mean, and what risks/benefits will it bring?TRANSCRIPT
![Page 1: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/1.jpg)
People as Sensors
Tom Raftery
ThingMonk,
London Dec 2013.
1
![Page 2: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/2.jpg)
2
• Lead analyst, energy and sustainability practice, RedMonk
• GreenMonk.net
• twitter.com/tomraftery
• +34 677 695 468
• SlideShare.net/TomRaftery
Tom Raftery
![Page 3: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/3.jpg)
3
Mobile data
http://www.flickr.com/photos/traftery/8551389911/
Mobile phones with GPS, accelerometers, compass, M7, barometers, moisture, etc
![Page 4: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/4.jpg)
4
Mobile data
http://www.zeit.de/datenschutz/malte-spitz-data-retention
Mobile phones with GPS and data retention lawsGreen party sued Deutsche Telecom for dataPut it onlineSource of concern
![Page 5: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/5.jpg)
5
Social Media
Mobile is not the only data we’re broadcasting
![Page 6: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/6.jpg)
6
http://newsroom.fb.com/Key-Facts
1.19 billion monthly active users (MAU) as of September 30, 2013
727 million daily active users (DAU) in Sept 2013
874 million mobile MAUs as of September 30 2013
![Page 7: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/7.jpg)
7
http://techcrunch.com/2013/05/17/facebook-growth/
Most recent usage data is from May 2013 - so well out-of-date
4.75 billion content items shared per day (status updates + wall posts + photos + videos + comments)
4.5 billion Likes per day
![Page 8: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/8.jpg)
8
http://hide.dyndns.info/tweetcounter/index-en.cgi
Between 4,000-8,500 tweets per second per dayAvg over 6,000 tweets per sec, all stored in perpetuityTweets drop off at midnight ET, start picking up again at 6am ET!
![Page 9: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/9.jpg)
9
http://hide.dyndns.info/tweetcounter/index-en.cgi
Increasing tweets per sec over the last year (max value per day used)Two peaks on the right-hand-side
![Page 10: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/10.jpg)
10
http://hide.dyndns.info/tweetcounter/index-en.cgi
Increasing tweets per day over the last year
![Page 11: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/11.jpg)
11
https://blog.twitter.com/2013/new-tweets-per-second-record-and-how
Most recent numbers from Twitter:Twitter avg - >500m TPDTwitter avg - 5,700 TPSAug 2nd had a 143k TPS record (>28x the average) - no blip to service
![Page 12: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/12.jpg)
12
Data for sale…
http://online.wsj.com/news/articles/SB10001424052702304441404579118531954483974
Twitter in its IPO filings disclosed it is making $47.5m from selling access to its data
![Page 13: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/13.jpg)
13
Google+
http://googleblog.blogspot.com.es/2013/10/google-hangouts-and-photos-save-some.html
Google+ 540m MAU300m active in stream1.5bn photos per month
![Page 14: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/14.jpg)
14
Sina Weibo
http://sg.finance.yahoo.com/news/sina-weibo-passes-500-million-151054944.html
China’s Sina Weibo is growing with 74% year on year user growth (http://www.chinadaily.com.cn/bizchina/2013-02/21/content_16243933.htm)
Has 220m ‘active users’ while Twitter has 170m ‘active users’ http://blogs.wsj.com/chinarealtime/2013/03/12/how-many-people-really-use-sina-weibo/
![Page 15: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/15.jpg)
15
Waze
https://www.waze.com
Waze had an estimated 50m users in June 2013
![Page 16: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/16.jpg)
16
Waze
https://www.waze.com
Waze had an estimated 50m users in June 2013
![Page 17: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/17.jpg)
17
Use Cases
Some positive use cases of the data
![Page 18: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/18.jpg)
18
Crowd-Sourcing
http://csce.uark.edu/~tingxiny/courses/5013spring13/readingList/crowdsource.pdf
Academic study on feasibility of using Twitter to crowdsourced data
![Page 19: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/19.jpg)
19
Meteorology
http://uksnowmap.com/#/
UK Snow Map by Ben MarshTweet #uksnow, postcode and x/10 rating
![Page 20: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/20.jpg)
20
Utilities
http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/
GE’s Grid IQ Insight can mine social media for mentions of outagesGives early notifications of an outage in an areaIf geotagged and/or includes images/video can confirm cause of outage and speed up time to resolution
![Page 21: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/21.jpg)
21
Utilities
http://greenmonk.net/2012/11/01/sustainability-social-media-and-big-data/
Utilities are aware of reason for outage, speeds up time to resolution (reduces need for investigatory truck roll)
![Page 22: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/22.jpg)
22
Risk Analysis
http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/
United Nations Development Program & their Recorded Future project
Using publicly sourced data looking for signs of disruption or unrest
![Page 23: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/23.jpg)
23
Risk Analysis
http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/
The same graph turned into media sources- who is talking about Georgia in this period of time
![Page 24: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/24.jpg)
24
Risk Analysis
http://europeandcis.undp.org/blog/2012/11/16/social-media-and-political-risk-analysis/
Mentions turned into social network analysis- who is talking to whom, who is meeting whom
“next phase will focus on conducting a regional political risk analysis and forecasting for South Eastern Europe and Central Asia”
UNDP slides courtesy of Milica Begovich (aka @elim????)
![Page 25: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/25.jpg)
25
Automotive
http://www.magazine.pamplin.vt.edu/fall12/vehicledefects.html
Social Media monitoring tool developed by Pamplin College of BusinessInitial version worked from automotive fora and blogs, now expanding to take in Twitter and Facebook“Robust” way to discover and classify vehicle defects from social media posts across multiple automotive brandsFaster than reporting back up through the dealer chain
![Page 26: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/26.jpg)
26
Finance
http://arxiv.org/PS_cache/arxiv/pdf/1010/1010.3003v1.pdf
Academic paper from University of Manchester and Indiana University shows that Twitter can predict the Dow Jones Industrial Average with 87.6% accuracy
![Page 27: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/27.jpg)
27
Finance
http://www.caymanatlantic.com/investment-management/4574471088
UK Firm Derwent Capital Markets signed an exclusive deal with the authors to create a hedge fund - became Cayman Atlantic
![Page 28: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/28.jpg)
28
Law Enforcement
http://support.sas.com/resources/papers/proceedings12/309-2012.pdf
SAS produced an interesting white paper on this space and bought UK firm Memex - definitely chasing this space
Citing use cases like - finding individuals - analysing their social graph to find accomplices/gang structureAlso identify precursor activity to events like riots
![Page 29: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/29.jpg)
29
Law Enforcement
http://www.policemag.com/blog/technology/story/2012/09/social-media-analytics-in-law-enforcement.aspx
“Social media is a huge network of informants—and one you don't have to pay for.”Law Enforcement use cases (information distribution, fake profile creation, etc.)Helps first responders gain situational awareness prior to having feet on the groundHelps Emergency Operations Centres gain information in the event of natural disasters, for exampleSunday’s train crash in NY, for example
![Page 30: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/30.jpg)
X
Law Enforcement
http://www.huffingtonpost.com/2012/09/04/web-surveillance-social-media_n_1854750.html
Other vendors in this space outlined in this article3i-Mind - http://www.3i-mind.com/HMS Technologies - http://www.hmstech.com/Visible Technologies - http://www.visibletechnologies.com/ Attensity - http://www.attensity.com/home/CrowdControlHQ - http://www.crowdcontrolhq.com/index.phpandAs well as Law Enforcement use cases (information distribution, fake profile creation, etc.)
![Page 31: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/31.jpg)
30
Law Enforcement
http://www.lexisnexis.com/government/investigations/
Good infographic on Law Enforcement use of social media - based on a LexisNexis Risk Solutions survey of 1,200 law enforcement professionalsFull report is available at http://solutions.lexisnexis.com/forms/GV12LEOMPSoMeSurveyforLE9677
![Page 32: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/32.jpg)
31
Law Enforcement
Needs to be approached sensitively - the way some of this is reported often prompts visions of ‘pre-crime’ and Minority Report
![Page 33: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/33.jpg)
32
Smart Cities
https://itunes.apple.com/us/app/boston-citizens-connect/id330894558
Graffiti, Pothole, & LA school district apps
![Page 34: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/34.jpg)
33
Healthcare
http://www.google.org/flutrends/intl/en_us/
Google use frequency of certain search terms as a way to estimate flu activityAlso have one for Dengue FeverSearch data is increasingly mobile
![Page 35: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/35.jpg)
34
Healthcare
http://www.nature.com/nature/journal/v457/n7232/full/nature07634.html
Google wrote this up as an academic paper and it was published in Nature
![Page 36: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/36.jpg)
35
Healthcare
http://www.ajtmh.org/content/86/1/39.abstract
A group led from Harvard Medical School studied viability of using social media for predicting cholera outbreakFound that the data from Twitter closely corresponded with government data, was available up to two weeks earlier
The paper concludes informal media could be used to study the activity of other disease outbreaks around the worldFinancial support was provided by Google.org
![Page 37: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/37.jpg)
36
Healthcare
http://propellerhealth.com
Formerly asthmapolis - wireless asthma puff data - where/whenCan map where asthma outbreaks occur - people sensitive can avoid triggers
![Page 38: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/38.jpg)
37
Healthcare
http://www.huffingtonpost.com/keith-runyon/louisville-chooses-asthma_b_4086297.html
Rolled out in conjunction with city of Louisville KyResidents experience as much as a 13-year gap in life expectancy depending upon where they liveFindings eagerly anticipated - only rolled out in Oct
![Page 39: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/39.jpg)
38
CRM
http://www.ft.com/intl/cms/s/0/bd5a5ce2-aa57-11e1-899d-00144feabdc0.html#axzz2OAlD9lav
T-Mobile in US analysed its 33m customer data records, web logs, billing data and social media information to predict customer defections
It halved customer defections in 3 months
![Page 40: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/40.jpg)
39
Brand Management
http://www.youtube.com/watch?v=VaJjPRwExO8
Nestle were Greenpeace’d because palm oil in Kit Kat came from Sinar Mas - company involved in deforestation
![Page 41: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/41.jpg)
40
http://greenmonk.net/2010/03/19/can-corporate-social-responsibility-affect-your-companys-bottom-line/
Brand Management
In the social media storm which followed Nestlé made every mistake in the bookNestlé received over 200,000 protest emails and their share price was negatively affectedSo they decided to work with Greenpeace to fix their supply chain andTo initiate a social media strategy for the organisation
![Page 42: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/42.jpg)
41http://uk.reuters.com/article/2012/10/26/uk-nestle-online-water-idUKBRE89P07Q20121026
Brand Management
Set up a social media command centre, staffed by their Digital Acceleration Team
![Page 43: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/43.jpg)
42http://www.reputationinstitute.com/thought-leadership/global-reptrak-100
Brand Management
In 2013 Nestle entered the Reputation Institute’s Global top 10 for the first time.
![Page 44: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/44.jpg)
43
Transportation
https://twitter.com/ehn/status/396307684661530624
Waze data now being incorporated in Google Maps
![Page 45: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/45.jpg)
44
Transportation
http://www.bbc.co.uk/news/technology-22357748
2.5bn anonymised call records from 5m Orange phone users in Ivory Coast Looked at patterns of people’s movements in Abidjan - capital city of Ivory CoastRealised they could reduce travel times of ppl by 10%
![Page 46: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/46.jpg)
45
Looking Ahead
http://www.flickr.com/photos/35468133931@N01/8699901706
Google Glass
![Page 47: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/47.jpg)
46
Looking Ahead
http://www.instabeat.me
Instabeat gives swimmers stats in their goggles as they swimAnd subsequent download
![Page 48: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/48.jpg)
47
Looking ahead
http://www.fitbit.com/force
Fitbit force, Nike+ Fuelband, Jawbone UpCan see a situation where sports players are broadcasting vital stats similar to F1
![Page 49: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/49.jpg)
48
Data and data sources are increasing exponentially - go hack that data for good.
Conclusion
![Page 50: People as sensors - mining social media for meaningful information](https://reader036.vdocuments.site/reader036/viewer/2022081400/554eb687b4c905fb7c8b5462/html5/thumbnails/50.jpg)
Thanks!Contact information:
Tom Raftery
Principal Analyst, Energy & Sustainability, RedMonk
[email protected], GreenMonk.net,
Twitter.com/tomraftery
+34 677 695 468
No tweets were hurt in the making of this presentation
49