WhatisBigDataandWhyShouldWeCare?

GianlucaDemar-niLecturerinDataScienceUniversityofSheffieldgianlucademar-ni.net

Dataishuge

•  Banks,citycouncils,governments,shops,etc.

•  Facebookprocesses750TB/dayofdata– 48kiPhoneseveryday– 7PBofphotostorage/month

•  Thisrequirescomputers(alotofthem!)

2

FacebookDataCenter(Sweden)

3

Data

4

Machines

•  Googlehasaround900,000servers(260millionwaUs)0.01%globalenergy

•  CERNLargeHadronCollider180MW•  Theflux-capacitorneeds1.21GW(just5xGoogle)

DatafromJuly2013 5

BigData

•  DefinedasVs– Volume:Justaboutsize,Giga,Tera,Petabytes– Variety:Formats,text,databases,pictures,excel– Velocity:Speed,10000tweetspersecond,2000picturesonInstagrampersecond

6

BigDataResearchinSheffield

•  Lookingatdataintegra-onacrosssources

7

Facebook

InstagramMarkZuckerberg

Shouldwecare?

•  Thisdataisaboutus!•  Data:GMail,Facebook,debitcards,shoppingfidelitycards,transport,mobilephones,…

•  Usage:Mortgageapplica-on,healthinsurance,carinsurance

8

Algorithmsruletheworld

•  Somedatamustnotbeprocessedbypeople!– GMailcontentisprocessedbycomputerstodecidewhichadver-sementyouseeontheWeb

9

Algorithmsruletheworld

•  Uberpricesaredecidedbyasoiwareprograms– ThebossofUberdriversisacomputer–  Itdecideshowtheyworkandhowmuchmoneytheymake

•  Computersknowalotaboutpeoplebutnottheotherwayaround

10

Isitallbad?

•  Duolingo:Data-drivenforeignlanguagelearning– Whatisthebestwaytolearnalanguagedependsonyourna-velanguage

•  Languagetransla-on

11

DataScience•  “DataScien-st:TheSexiestJobofthe21stCentury”,inHarvardBusinessReview

•  Companieswantdata-drivendecisions

•  GraduatesfromtheMScDataScienceinSheffieldgoworkin:–  Telecommunica-ondataanalysis–  Cancerresearch– Housingmarket

12

Isyourdataready?

•  Decisionsaboutyouandyourlifewillbedrivenbydata!

•  Areyouincontrolofwhichdataisbeingcollected?

•  Areyouawareofhowthedataisstoredandprocessed?

•  Doyouknowhowdecisionsaretaken?Doyouwanttoknow?

13