cs 61c: great ideas in computer architecture …cs61c/sp16/lec/29/2016sp...back in 2011 • google...
TRANSCRIPT
![Page 1: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/1.jpg)
CS61C:GreatIdeasinComputerArchitecture(MachineStructures)Warehouse-ScaleComputing
Instructors:NicholasWeaver&VladimirStojanovichttp://inst.eecs.berkeley.edu/~cs61c/
![Page 2: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/2.jpg)
CoherencyTrackedbyCacheBlock
• Blockping-pongsbetweentwocacheseventhoughprocessorsareaccessingdisjointvariables
• Effectcalledfalsesharing• Howcanyoupreventit?
2
![Page 3: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/3.jpg)
Review:UnderstandingCacheMisses:The3Cs
• Compulsory(coldstartorprocessmigration,1st reference):– Firstaccesstoblock,impossibletoavoid;smalleffectforlong-running
programs– Solution:increaseblocksize(increasesmisspenalty;verylargeblocks
couldincreasemissrate)• Capacity (notcompulsoryand…)
– Cachecannotcontainallblocksaccessedbytheprogramevenwithperfectreplacementpolicyinfullyassociativecache
– Solution:increasecachesize(mayincreaseaccesstime)• Conflict(notcompulsoryorcapacityand…):
– Multiplememorylocationsmaptothesamecachelocation– Solution1:increasecachesize– Solution2:increaseassociativity(mayincreaseaccesstime)– Solution3:improvereplacementpolicy,e.g..LRU
3
![Page 4: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/4.jpg)
Fourth“C”ofCacheMisses:CoherenceMisses
• Missescausedbycoherencetrafficwithotherprocessor
• Alsoknownascommunicationmissesbecauserepresentsdatamovingbetweenprocessorsworkingtogetheronaparallelprogram
• Forsomeparallelprograms,coherencemissescandominatetotalmisses– Itgetsevenmorecomplicatedwithmultithreadedprocessors:YouwantseparatethreadsonthesameCPUtohavecommonworkingset,otherwiseyougetwhatcouldbedescribedasincoherencemisses
4
![Page 5: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/5.jpg)
New-SchoolMachineStructures(It’sabitmorecomplicated!)
• ParallelRequestsAssigned tocomputere.g.,Search“cats”
• ParallelThreadsAssigned tocoree.g.,Lookup,Ads
• ParallelInstructions>[email protected].,5pipelined instructions
• ParallelData>1dataitem@one timee.g.,DeepLearningfor
imageclassification
• HardwaredescriptionsAllgates@onetime
• ProgrammingLanguages 5
SmartPhone
WarehouseScale
ComputerHarness
Parallelism&AchieveHighPerformance
LogicGates
Core Core…
Memory(Cache)
Input/Output
Computer
CacheMemory
Core
InstructionUnit(s) FunctionalUnit(s)
A3+B3A2+B2A1+B1A0+B0
SoftwareHardware
![Page 6: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/6.jpg)
Backin2011• Googledisclosedthatitcontinuouslyusesenoughelectricitytopower200,000homes,butitsaysthatindoingso,italsomakestheplanetgreener.
• Averageenergyusepertypicaluser permonthissameasrunninga60-wattbulbfor3hours(180watt-hours).
6
Urs Hoelzle,Google SVPCo-authoroftoday’sreading
http://www.nytimes.com/2011/09/09/technology/google-details-and-defends-its-use-of-electricity.html
![Page 7: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/7.jpg)
Google’sWSCs
74/8/16
Ex:InOregon
![Page 8: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/8.jpg)
ContainersinWSCs
8
InsideWSC InsideContainer
![Page 9: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/9.jpg)
Server,Rack,Array
9
![Page 10: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/10.jpg)
GoogleServerInternals
10
GoogleServer
![Page 11: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/11.jpg)
Warehouse-ScaleComputers• Datacenter– Collectionof10,000to100,000servers– Networksconnectingthemtogether
• Singlegiganticmachine• Verylargeapplications(Internetservice):
search,email,videosharing,socialnetworking• Veryhighavailability• “…WSCsarenolessworthyoftheexpertiseofcomputer
systemsarchitectsthananyotherclassofmachines”Barroso andHoelzle,2009
11
![Page 12: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/12.jpg)
UniquetoWSCs• AmpleParallelism
– Request-levelParallelism:ex:Websearch– Data-levelParallelism:ex:Imageclassifier training
• ScaleanditsOpportunities/Problems– Scaleofeconomy:lowper-unitcost– Cloudcomputing:rentcomputingpowerwithlowcosts(ex:AWS)
• OperationCostCount– Longerlifetime(>10years)– Costofequipmentpurchases<<costofownership– Oftensemi-customorcustomhardware
• Butconsortiumsofhardwaredesignstosavecostthere• Designforfailure:
– Transientfailures– Hardfailures– High#offailures
ex:4disks/server,annualfailurerate:4%à WSCof50,000servers:1diskfail/hour
12
![Page 13: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/13.jpg)
WSCArchitecture
13
1UServer:8-16cores,16GBDRAM,4x4TBdisk+diskpods
Rack:40-80severs,LocalEthernet(1-10Gbps)switch(30$/1Gbps/server)
Array(akacluster):16-32racksExpensiveswitch(10Xbandwidthà 100xcost)
![Page 14: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/14.jpg)
WSCStorageHierarchy
14
1UServer:DRAM:16GB,100ns,20GB/sDisk:2TB,10ms,200MB/s
Rack(80severs):DRAM:1TB,300us,100MB/sDisk:160TB,11ms,100MB/s
Array(30racks):DRAM:30TB,500us,10MB/sDisk:4.80PB,12ms,10MB/s
LowerlatencytoDRAMinanotherserverthanlocaldiskHigherbandwidthtolocaldiskthantoDRAMinanotherserver
![Page 15: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/15.jpg)
WorkloadVariation
• Onlineservice:Peakusage2Xoff-peak15
Midnight Noon Midnight
Workload
2X
![Page 16: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/16.jpg)
ImpactonWSCsoftware• Latency,bandwidthà Performance– Independentdatasetwithinanarray– Localityofaccesswithinserverorrack
• Highfailurerateà Reliability,Availability– Preventingfailuresiseffectivelyimpossible atthisscale– Copewithfailuresgracefullybydesigningthesystemasawhole
• Varyingworkloadsà Availability– Scaleupanddowngracefully
• Morechallengingthansoftwareforsinglecomputers!
16
![Page 17: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/17.jpg)
PowerUsageEffectiveness• Energyefficiency– PrimaryconcerninthedesignofWSC– Importantcomponentofthetotalcostofownership
• PowerUsageEffectiveness(PUE):
– ApowerefficiencymeasureforWSC– Notconsideringefficiencyofservers,networking– Perfection=1.0– GoogleWSC’sPUE=1.2
• GettingprettyclosetoAmdahl'slawlimit 17
TotalBuildingPowerITequipmentPower
![Page 18: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/18.jpg)
PUEintheWild(2007)
18
![Page 19: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/19.jpg)
19
![Page 20: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/20.jpg)
LoadProfileofWSCs
• AverageCPUutilizationof5,000Googleservers,6monthperiod• Serversrarelyidleorfullyutilized,operatingmostofthetimeat
10%to50%oftheirmaximumutilization20
![Page 21: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/21.jpg)
Energy-ProportionalComputing:DesignGoalofWSC
• Energy=PowerxTime,Efficiency=Computation/Energy• Desire:
– Consumealmostnopowerwhenidle(“Doingnothingwell”)– Graduallyconsumemorepowerastheactivitylevelincreases
21
![Page 22: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/22.jpg)
CauseofPoorEnergyProportionality
22
• CPU:50%atpeek,30%atidle• DRAM,disks,networking:70%atidle!
– Becausetheyareneverreallyidleunlesstheyarepoweredoff!• Needtoimprovetheenergyefficiencyofperipherals
![Page 23: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/23.jpg)
Clicker/PeerInstruction:WhichStatementisTrue
• A:Idleserversconsumealmostnopower.
• B:Diskswillfailoncein20years,sofailureisnotaproblemofWSC.
• C:Thesearchrequestsofthesamekeywordfromdifferentusersaredependent.
• D:MorethanhalfofthepowerofWSCsgoesintocooling.
• E:WSCscontainmanycopiesofdata.23
![Page 24: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/24.jpg)
Administrivia• ReminderthatProject4isout…
24
![Page 25: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/25.jpg)
Agenda
• WarehouseScaleComputing
• Administrivia &Clickers/PeerInstructions
• Request-levelParallelisme.g.Websearch
25
![Page 26: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/26.jpg)
Request-LevelParallelism(RLP)• Hundredsofthousandsofrequestspersec.– PopularInternetserviceslikewebsearch,socialnetworking,…
– Suchrequestsarelargelyindependent• Ofteninvolveread-mostlydatabases• Rarelyinvolveread-writesharingorsynchronizationacrossrequests
• Computationeasilypartitionedacrossdifferentrequestsandevenwithinarequest
26
![Page 27: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/27.jpg)
GoogleQuery-ServingArchitecture
27
![Page 28: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/28.jpg)
AnatomyofaWebSearch
28
![Page 29: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/29.jpg)
AnatomyofaWebSearch(1/3)• Google“cats”– Directrequestto“closest”GoogleWSC
• HandledbyDNS
– Front-endloadbalancerdirectsrequesttooneofmanyarrays(clusterofservers)withinWSC• Oneofpotentiallymanyloadbalancers
– Withinarray,selectoneofmanyGoggleWebServers(GWS)tohandletherequestandcomposetheresponsepages
– GWScommunicateswithIndexServerstofinddocumentsthatcontainsthesearchword,“cats”• IndexserverskeepindexinRAM,notondisk
– Returndocumentlistwithassociatedrelevancescore 29
![Page 30: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/30.jpg)
AnatomyofaWebSearch(2/3)• Inparallel,– Adsystem:runadauctionforbiddersonsearchterms
• Yes,youarebeingboughtandsoldinarealtime auctionallovertheweb
• Pageadsareworsethansearchads
• Usedocids (DocumentIDs)toaccessindexeddocuments• Composethepage– Resultdocumentextracts(withkeywordincontext)orderedbyrelevancescore
– Sponsoredlinks(alongthetop)andadvertisements(alongthesides)
30
![Page 31: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/31.jpg)
AnatomyofaWebSearch(3/3)• Implementationstrategy– Randomlydistributetheentries
– Makemanycopiesofdata(a.k.a.“replicas”)
– Loadbalancerequestsacrossreplicas
• Redundantcopiesofindicesanddocuments
– Breaksupsearchhotspots,e.g.“TaylorSwift”
– Increasesopportunitiesforrequest-levelparallelism
– Makesthesystemmoretolerantoffailures
31
![Page 32: CS 61C: Great Ideas in Computer Architecture …cs61c/sp16/lec/29/2016Sp...Back in 2011 • Google disclosed that it continuously uses enough electricity to power 200,000 homes, but](https://reader033.vdocuments.site/reader033/viewer/2022053013/5f10643c7e708231d448e18c/html5/thumbnails/32.jpg)
Summary• WarehouseScaleComputers– Newclassofcomputers– Scalability,energyefficiency,highfailurerate
• Request-levelparallelisme.g.WebSearch
• Data-levelparallelismonalargedataset– AgazillionVMsfordifferentpeople
– MapReduce– Hadoop,Spark
32