Gooda&ernoonandthanksforhavingmehere.InthistalkIwanttolookatthedesignchallengesofsystemsthatan9cipateusers’needsandthenactonthem.Thatmeansthatitsitsattheintersec9onoftheinternetofthings,userexperiencedesignandmachinelearning,whichtomeisnewterritoryfordesignerswhomayhavedealtwithoneofthosedisciplinesbefore,butrarelyallthreeatonce.Thetalkisdividedintoseveralparts:itstartswithanoverviewofhowIthinkInternetofThingsdevicesareprimarilycomponentsofservices,ratherthanbeingself-containedexperiences,howpredic9veanaly9csenableskeycomponentsofthoseservices,andthenIfinishbytryingtotoiden9fyuseexperienceissuesaroundpredic9vebehaviorandsugges9onsforpaDernstoamelioratethoseissues.Acoupleofcaveats:-Ifocusalmostexclusivelyontheconsumerinternetofthings.Althoughpredic9veanaly9csisanimportantpartoftheIndustrialInternetofThingsforthingslikepredic9vemaintenance,Ifeelit’sREALLYkeytotheconsumerIoTbecauseofwhatexperiencesitcreatesforpeople.-IwanttopointoutthatfewifanyoftheissuesIraisearenew.Thoughtheterm“internetofthings”ishotrightnow,theideashavebeendiscussedinresearchcirclesformorethan20years.Searchfor“ubiquitouscompu9ng,”“ambientintelligence,”and“pervasivecompu9ng”andit’llhelpyoukeepfromreinven9ngthewheel.-Finally,mostofmyslidesdon’thavewordsonthem,soI’llmakethecompletedeckwithatranscriptavailableassoonI’mdone.

0

Let me begin by telling you a bit about my background. I�m a user experience designer. I was one of the first professional Web designers. This is the navigation for a hot sauce shopping site I designed in the spring of 1994.

1

I’vealsoworkedontheuserexperiencedesignofalotofconsumerelectronicsproductsfromcompaniesyou’veprobablyheardof.

2

Iwroteacoupleofbooksbasedonmyexperienceasadesigner.Oneisacookbookofuserresearchmethods,andtheseconddescribeswhatIthinkaresomeofthecoreconcernswhendesigningnetworkedcomputa9onaldevices.

3

Ialsocofoundedacoupleofcompanies.Thefirst,Adap9vePath,you’refamiliarwith,andwiththesecondone,ThingM,Igotdeepintodevelopinghardware.

4

TodayIworkforPARC,thefamoushardware,so&wareandAIresearchlab,asaprincipalinitsInnova9onServicesgroup,whichisPARC’sconsul9ngarm.Wehelpcompaniesreducetheriskofadop9ngnoveltechnologiesusingamixofethnographicresearch,userexperiencedesignandinnova9onstrategy.

5

IwantstartbyfocusingonwhatIfeelisakeyaspectofconsumerIoTthat’so&enmissedwhenpeoplefocusonthehardwareoftheIoT,whichisthatconsumerIoTproductshaveaverydifferentbusinessmodelthantradi9onalconsumerelectronics.Tradi9onally,acompanymadeanelectronicproduct,sayaturntable,theyfoundpeopletosellitforthem,theyadver9seditandpeopleboughtit.Thatwastradi9onallytheendofthecompany’srela9onshipwiththeconsumerun9lthatpersonboughtanotherthing,andallofthevalueoftherela9onshipwasinthedevice.WiththeIoT,thesaleofthedeviceisjustthebeginningoftherela9onshipandholdsalmostnovalueforeitherthecustomerorthemanufacturer.Letmeexplain…

6

As value shifts to services, the devices, software applications and websites used to access it—its avatars—become secondary. A camera becomes a really good appliance for taking photos for Flickr, while a TV becomes a nice Flickr display that you don’t have to log into every time, and a phone becomes a convenient way to take your Flickr pictures on the road.

Hardware becomes simultaneously more specialized and devalued as users see “through” each device to the service it represents. The hardware exists to get better value out of the service.

7

Amazonreallygetsthis.Here�satellingolderadfromAmazonfortheKindle.It’ssaying�Look,usewhateverdeviceyouwant.Wedon�tcare,aslongyoustayloyaltoourservice.Youcanbuyourspecializeddevices,butyoudon�thaveto.�

8

WhenFirewasreleased3yearsago,JeffBezosevencalleditaservice.

9

NetFlixisanothergoodmediaexample.Itfeelsnaturaltopauseamovieononedeviceandcon9nueitonanotherbecausefromyourperspec9vethere’sonlyONENeflix.Dropboxcreatesthisforfiles,Evernotefornotes,andAngryBirdsforscoresynchroniza9on.TheserviceiswhereyouraDen9onis,thedeviceistheretogiveyouaccesstotheservice,butotherthanaconvenientformfactor,thehardwareislargelydisposable.

10

Mostlarge-scaleIoTproductsareserviceavatars.Theyusespecializedsensorsandactuatorstosupportaservice,buthaveliDlevalue—ordon’tworkatall—withoutthesuppor9ngservice.SmartThingsclearlystateditsserviceofferingrightupfrontontheirsite.Thefirstthingtheysayabouttheirproductlineisnotwhatthefunc9onalityis,butwhateffecttheirservicewillachievefortheircustomers.Theirhardwareproducts’func9onality,howtheywilltechnicallysa9sfytheservicepromise,isalmostana&erthought.

11

ComparethattoX10,theirspiritualpredecessorthat’sbeeninthebusinessformorethan20years.AllthatX10tellsisyouiswhatthedevicesare,notwhattheservicewillaccomplishforyou.Idon’tevenknowifthereISaservice.WhyshouldIcarethattheyhave“modules”?Ishouldn’t,andIdon’t.

12

Sowhatdotheseservicesoffer?

13

Simpleconnec9vityhelpswhenyou’retryingtomaximizetheefficiencyofafixedprocess,butthat’snotaproblemthatmostpeoplehave.We’vebeenabletosimplyconnectvariousdevicestoacomputersinceaTandyColorComputerscouldlightsoffandonoverX10in1983.Thatwasn’tveryusefulthen,andit’snotveryusefulnow.YoucanreplacetheTandywithaniPhoneandthelampwithawashingmachineandyougetthevalueproposi9onofmostsimpleconnecteddevices.That’snotinteres9ng.

14

Ithinktherealconsumervalueconnectedservicesofferistheirabilitytomakesenseoftheworldonpeople’sbehalf,toreducepeople’scogni9veload,ratherthanincreasingit,byallowingthemtointeractwithdevicesatahigherlevelthansimpletelemetryandcontrol.Fundamentally,humansaregoodpaDernmatchersatcertainthings,butwe’renotbuilttocollectandmakesenseofhugeamountsofdataortoar9culateourneedsascomplexsystemsofmutuallyinterdependentcomponents.Computersaregreatatit.Theycanmakesta9s9calmodelsfrommanydatasourcesacrossspaceand9meandthentrytomaximizestheprobabilityofadesiredoutcome.Apersonprogrammingadevicecanexpresswhatthey’refamiliarwith,ortrytocreateanabstrac9onbasedontheirpastexperience,some9meswithconsiderableskill,butamodellearnedfromtheoutcomeofthousandsofsitua9onsacrossmanypeopleandlongperiodsof9mecancompensateformuchwidervarietyofsitua9onsinamorenuancedwaythananindividual’sperspec9vewilleverbeableto.

15

Predic9onisattheheartofthevalueproposi9onmanyofthemostcompellingIoTproductsoffering,star9ngwiththeNest.TheNestsaysthatitknowsyou.Howdoesitknowyou?Itpredictswhatyou’regoingtowantbasedonyourpastbehavior.

16

Amazon’sEchospeakersaysit’scon9nuallylearning.Howisthat?Predic9veanaly9cs,predic9vemachinelearning.

17

TheBirdismartsmokealarmsaysitwilllearnover9me,whichisagainthesamething.

18

Jaguarcomesrightoutwithit.Theyevenobliquelyreferencethe40yearsofar9ficialintelligenceresearchthatpowerspredic9veanaly9csbycallingtheircarnotjustlearning,butlearningANDintelligent.

19

TheEdynplantwateringsystemadaptstoeverychange.Whatisthatadapta9on?Predic9veanaly9cs.

20

Canary,ahomesecurityservice.

21

Here’sfoobot,anairqualityservice.

22

Predic9vebehaviorcreatesapreDyseduc9veworldofespressomachinesthatstartbrewingasyou’rethinkingit’sagood9meforcoffee,andreorderyourfavoriteblend;officelightsthatdimwhenit’ssunny,powerischeapandyou’renotdoinganythingthatneedsthem;andfoodtruckcaravansthatshowupjustasthecrowdintheparkisgennghungry.Theproblemisthatalthoughthevalueproposi9onisofabeDeruserexperience,it’sunspecificinthedetails.Exactlyhowwillourexperienceoftheworld,ourabilitytouseallthecollecteddata,becomemoreefficientandmorepleasurable?NowI’dliketooffersomeini9althoughtsonuserexperiencedesignforpredic9veanaly9csfortheinternetofthings.We’res9llearlyinourunderstandingofdesignforpredic9vedevices,sorightnowtheproblemsareworsethansolu9onsandIwanttostartbyar9cula9ngtheissuesI’veobservedinourwork.

23

We’veneverhadmechanicalthingsthatmakesignificantdecisionsontheirown,thefirstmajorissueisaroundexpecta9ons.Asdevicesadapttheirbehavior,howwilltheycommunicatethatthey’redoingso?Dowetreatthemlikeanimals?Dowes9ckasignonthemthatsays“adap9ng”,likethelightonavideocamerasays“recording”?Shouldmychairvibratewhenadjus9ngtomyposture?Howwillusers,orjustpassers-by,knowwhichthingsadaptandwhichmerelybehave?Icouldendupsinnguncomfortableforalong9mebeforerealizingmychairdoesn’tadaptonitsown.Howshouldsmartdevicessettheexpecta9onthattheymaybehavedifferentlyinwhatappearstopeopleasaniden9calsetofcircumstances?ChairbyRaffaelloD'Andrea,MaDDonovanandMaxDean.

24

Theironyinpredic9vesystemsisthatthey’repreDyunpredictable,atleastatfirst.Whenmachinelearningsystemsarenew,they’reo&eninaccurateandunpredictable,whichisnotwhatweexpectfromourdigitaldevices.60%-70%accuracyistypicalforafirstpass,buteven90%accuracyisn’tenoughforapredic9vesystemtofeelright,sinceifit’smakingdecisionsallthe9me,it’sgoingtobemakingmistakesallthe9me,too.It’sfineifyourhouseisacoupleofdegreescoolerthanyou’dlike,butwhatifyourwheelchairrefusestogotoadrinkingfountainnexttoadoorbecauseit’sbeentrainedondoorsanditcan’ttellthat’snotwhatyoumeaninthisoneinstance?Forallthe9mesasystemgetsitright,it’sonthemistakesthatwejudgeitandacouplesuchinstancescanshaDerpeople’sconfidence.AliDledoubtaboutwhetherasystemisgoingtodotherightthingisenoughtoturnaUXthat’srightmostofthe9meintoonethat’smoretroublethanit’sworth.Whenthathappens,you’vemorethanlikelylostyourcustomer.Soonerthanwethink,inaccuratepredic9vebehaviorisn’tgoingtobeanisolatedincident,it’sgoingtobethenorm.Whenthereare100connecteddevicessimultaneouslyac9ngonpredic9onsandeachis99%accurate,thenoneisalwayswrong.Sotheproblemis:Howcanyoudesignauserexperiencetomakeadevices9llfunc9onal,s9llvaluable,s9llfun,evenwhenit’sspewingjunkbehavior?Howcanyoudesignforuncertainty?

25

Thelastissuecomesasaresultoftheprevioustwo:control.Howcanwecontrolthesedevices,whentheirbehaviorisbydefini9onsta9s9calandunpredictablebyhumans?Ontheonehandyoucanmangleyourdevice’spredic9vebehaviorbygivingittoomuchdata.WhenIvisitedNestoncetheytoldmethatnoneoftheNestsintheirofficeworkedwellbecausethey’reconstantlyfiddlingwiththem.Inmachinelearningthisiscalledovertraining.Theotherhand,ifIhavenodirectwaytocontrolitotherthanthroughmyownbehavior,howdoIadjustit?AmazonandNeflix’srecommenda9onsystems,whichisakindofpredic9veanaly9cssystem,giveyousomecontextaboutwhytheyrecommendedsomething,butwhatdoIdowhenmyonlyinterfaceisagardenhose?

26

Hereare4paDernsI’veobservedindevelopingpredic9vesystemsthatIthinkmaptotheIoT.FormostoftheseI’mgoingtobeusingexamplesfromNestandrecommendersystemslikeAmazon’s,Google’sandNeflix’swhichhavebeenusingsimilarpredic9vetechnologiesforyearsandhaveaddressedsomeoftheseissues.

27

MyfirstpaDernisametapaDern.Pleasehaveausermodel,auser-facingstory,foreverystageofthemachinelearningandpredic9onprocess,evenifit’sastepthatisinvisibletousersandcustomers.[Acquire]Howwillyouincen9vizepeopletoadddatatothesystematall?WhyshouldIuploadmycar’sdashcamvideotoyourtrafficpredic9onsystemEVERYDAY?[Extract]Howwillyoucommunicateyou’reextrac9ngfeatures?Googlespeechtotextshowspar9alphrasesasyou’respeakingintoit,andvisiblycorrectsitself.ThatUIthatsimultaneouslytellsusersit’spullinginforma9onoutoftheirspeechandittrainstheminhowtomeetthealgorithmhalfway.[Classify]Howdomachine-generatedclassifica9onscomparetopeople’sorganiza9onofthesamephenomena?EtceteraEtceteraBecausemachinelearningandpredic9onaresonovelandsomanyofthestepsareintertwined,youneedtocareabouttheUXineverysinglestepoftheprocess.

28

Whendealingwithapersonorananimal,unpredictablebehaviorisexpectedandtolerated,buttodayweexpectdigitalsystemswillbehaveconsistentlyandthereasonsfortheirbehaviorwillbeclear.NeitheroftheseistruefortheUXofpredic9vesystems,whichdon’tnecessarilybehaveiden9callyinsimilarcircumstances,whichchangetheirbehaviorover9me,andinwhichthereasonsforthebehaviormaynotbeobvious.Apredic9veUXfirstneedstoexplainthenatureofthedevice,todescribeitistryingtopredict,thatit’stryingtoadapt,thatit’sgoingtosome9mesbewrong,toexplainhowit’slearning,andhowlongit’lltakebeforeitcrossesoverfromcrea9ngmoretroublethanbenefit.GoogleNowdescribeswhyacertainkindofcontentwasselected,whichsetstheexpecta9onthatthesystemwillrecommendotherthingsbasedonotherkindsofcontentyou’verequested.Nest’sFAQexplainsyoushouldn’texpectyourthermostattomakeamodelofwhenyou’rehomeornotun9lit’sbeenopera9ngforaweekorso.

29

Predic9vebehavioraboutsequencesofac9vi9es.Manypredic9veUXissuesaroundexpecta9onsanduncertaintyhave9meastheirbasis:whatwereyouexpec9ngtohappenandwhy.Ifitdidn’thappen,why?Ifsomethingelsehappened,orithappenedatanunexpected9me,whydidthathappen?Tellingthatadevicehasactedonyourbehalf,andthatit’sgoingtoact—andHOWit’sgoingtoact—inthefuturegivespeopleamodelofhowit’sworkingandreducesuncertainty.Nest,forexample,hasacalendarofitsexpectedbehavior,anditshowsthatit’sac9ngonyourbehalftochangethetemperature,andwhenyoucanexpectthattemperaturewillbereached.

30

Youhavetogivepeopleaclearwaytoteachthesystemandtellitwhenitsmodeliswrong.Sta9s9calsystems,bydefini9on,don’thavesimplerulesthatcanbechanged.Therearen’tobvioushandlestoturnordialstoadjust,becauseeverythingisprobabilis9c.Ifthemodelismadefromdatacollectedbyseveraldevices,whichdeviceshouldIinteractwithtogetittochangeitsbehavior?GoogleNowaskswhetherIwantmoreinforma9onfromasiteIvisited,Amazonshowsaexplana9onofwhyitgavemeasugges9on.MappingthistotheconsumerIoTmeanswaymoreexplana9onthanwe’recurrentlygenng,whichiseitherthatathinghashappened,orithasn’t.

31

Finally,don’tautomate.Thesesystem’sshouldn’ttrytoreplacepeople,buttosupportthem,toaugmentandextenttheircapabili9es,nottoreplacethem.MeshfireisasocialmediaengagementtoolthathasamachinelearningassistantcalledEmberthatdoesn’ttrytoreplacethesocialmediamanager.Insteaditmanagesthemanager’stodolist.Itaddsthingsthatitthinksaregoingtobeinteres9ng,deletesoldthings,andrepriori9zesthemanager’slistbasedonwhatitthinksisimportant.Emberaugmentsthecapabili9esofthesocialmediamanager.Ithelpsthatpersonfocusonwhat’simportantsothattheycanbesmarterabouttheirdecisions.Itdoesn’ttrytobesmarterthantheyare.

32

Finally,formetheIoTisnotaboutthethings,buttheexperiencecreatedbytheservicesforwhichthethingsareavatars.

33

Machinelearningalgorithmsusedtobestrictlybehind-the-scenes,butintheIoTtheyareactorsinourlives,soasdesignersit’sourresponsibilitytounderstandthesitua9onswherethealgorithmsandthedevicestheycontrolinteractwithpeople’slives,especiallysincethere’sadeepsymbio9crela9onshipbetweenthedatathatcomprisesthemodels,thebehaviorthosemodelsinduceandthepeoplewhoaretheintendedbeneficiaries.Ul9matelyweareusingthesetoolstoextendourcapabili9es,tousethedigitalworldasanextensionofourminds.Todothatwellwehavetorespectthatasinteres9ngandpowerfulasthesetechnologiesare,theyares9llintheirinfancy,andourjobasentrepreneurs,developersanddesignerswillbetocreatesystems,services,thathelppeople,ratherthanaddingextraworkinthenameofsimplis9cautoma9on.Whatwewanttocreateisasymbio9crela9onshipwherewe,andourpredic9vesystems,worktogethertocreateaworldthatprovidesthemostvalue,fortheleastcost,forthemostpeople,forthelongest9me.

34

Thankyou.

35