-
Nutanix Complete ClusterA Technical Whitepaper
-
NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Table of Contents
ExecutiveSummary............................................................................... 3
Introduction........................................................................................... 4
LimitationsofCurrentArchitecture........................................... 4
TheGoogleApproach................................................................. 4
NutanixCompleteClusterArchitecture.............................................. 6
MainPillars.................................................................................. 6
ArchitectureOverview................................................................ 7
NutanixScale-OutConvergedStorageKeyComponents...... 8
KeyFeatures........................................................................................ 11
CapacityOptimization.............................................................. 11
EaseofManagement................................................................ 11
PerformanceandScalability..................................................... 12
HighAvailability........................................................................ 13
Conclusion........................................................................................... 15
-
EXECUTIV
ESUMMARY
3NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Executive SummaryStorageisthebiggestchallengeinvirtualizeddatacenterstoday.Thenetwork
storagearchitecturedesignedfifteenyearsagoforphysicalserversistoo
expensiveandcomplexforvirtualmachines.Withoutanenterprise-class
alternative,organizationsareforcedtousetraditionalsolutions,whichcant
keepupwithvirtualmachinesthataredynamic,growrapidlyinnumberand
continuetodemandnewlevelsofperformanceandcapacity.
NutanixCompleteClusterisascale-outcomputeandstorageinfrastructure
thatallowsorganizationstovirtualizetheirdatacenterswithoutrequiring
networkstorage(SANorNAS).Builtfromthegroundupforvirtualmachines,it
providescompletecomputeandstoragecapabilitiesalongwithenterprise-class
performance,scalability,availabilityanddatamanagementfeatures.Itleverages
industry-standardhardwarecomponents,solid-statedrivesandmarket-leading
hypervisorstoprovideanout-of-the-boxsolutionthatmakesvirtualization
extremelyeasyandcosteffective.
-
INTRODUCTIO
N
4NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
IntroductionLimitations of Current ArchitectureOrganizationsarebuildingtheirvirtualizationinfrastructureusingthe
traditionalservers-connected-to-storage-over-a-networkarchitecture,which
cantadapttotheever-changingdemandsofvirtualization.Inadditiontoslow
performance,networkstoragehasbecomethesinglebiggestsourceofcostand
complexityinvirtualizedenvironments.Thenetworkstorage-basedarchitecture
workedwellforphysicalserversthatservedrelativelystaticworkloads.
Virtualization,andnowCloudComputing,hasmadedatacentersextremely
dynamic;virtualmachinesarecreatedonthefly,movefromservertoserverand
dependheavilyonsharedresources.Thesecharacteristicsmakethemanagement
ofvirtualmachinesandtheirunderlyingphysicalinfrastructureextremely
complex.
Datavolumesaregrowingatarapidpaceinthedatacenter,thankstotheease
ofcreatingnewVMs.Intheenterprise,newinitiativeslikedesktopvirtualization
contributetothistrend.Serviceprovidersdealwithanevenlargernumberof
VMsastheybuilddatacenterstoservecustomerswhocantaffordthecost
andmanagementoverheadthatvirtualizationrequires.Thisgrowingpoolof
VMsisexertingtremendouscost,performanceandmanageabilitypressureon
thetraditionalarchitecturethatconnectscomputetostorageoveramulti-hop
network.
Theriseofsolid-statedrivesisanothertrendthatisrapidlywideningthegap
betweencomputeandstoragetiers.UseofSSDsthatare100X-1000Xfaster
thantraditionalharddiskswillmaketheexistingnetworkbottlenecksand
networkcomplexityevenworse,ifvirtualmachinesneedtoaccessthemovera
network.ManySAN/NASvendorsareaddingSSDstotheirsolutions,charginga
heftypremiumforthesedrivesandrequiringadditionalinvestmentsinnetwork
bandwidthtoaccessanalreadyexpensivetierofstorage.
The Google ApproachGoogleandotherleadingcloud-generationcompaniessuchasAmazon,Yahoo
andMicrosoft(Azure)realizedthatanetwork-storagebasedapproachwould
-
INTRODUCTIO
N
5NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
notworkfortheirdatacenters.Theybuiltsoftwaretechnology(suchasGoogle
FileSystem)thatcouldgluealargenumberofcommodityserverswithlocal
storageintoasinglecluster.ThisapproachallowedGoogletobuildaconverged
computeandstorageinfrastructurethatusedcommodityserverswithlocal
storageasitsbuildingblock.GoogleFileSystemrunsacrossaclusterofservers
andcreatesasinglepooloflocalstoragethatcanbeseamlesslyaccessedby
applicationsrunningonanyserverinthecluster.Itprovideshighavailability
toapplicationsbymaskingfailuresofharddisksandevencompleteservers.
GoogleFileSystemallowedGoogletobuilddatacenterswithmassivelyscalable
computeandstorage,withoutincurringthecostsandperformancelimitations
associatedwithnetworkstorage.
Nutanixhastakenasimilarscale-outapproachtobuildanenterprise-ready
computeandstorageinfrastructurethatisdesignedfromthegroundupfor
virtualmachines.
-
NUTANIX
COMPLETECLUSTERA
RCHITECTURE
6NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Nutanix Complete Cluster Architecture Main PillarsNutanixCompleteClusterwasdesignedfromscratchtosolvestoragechallenges
forvirtualmachinesbybuildingasystemthatleveragesthelatestadvancesin
systemarchitecture,hardwareandsoftwaretechnologies.Therearethreecore
pillarsoftheNutanixarchitecture:
Distributed Computing
TheNutanixarchitectureissimilartoGooglesarchitectureinthatitisascale-
outcomputeandstorageinfrastructurethateliminatestheneedfornetwork
storage.Atthesametime,NutanixbuildsuponGooglesarchitectureand
providesanenterprise-classsolution.WhileGoogleFileSystemisacustom
solutionthatworksforGooglesinternalapplications(search,Gmail,etc.),
Nutanixprovidesageneral-purposesolutionforvirtualizedenvironments.In
additiontoitsscale-outcapabilities,ithasthesameorbetterenterprise-class
datamanagementfeaturesthatarecommonlyprovidedbyadvancednetwork
storagesolutions,includinghighavailability,backup,snapshots,anddisaster
recovery.
Virtualization
TheNutanixarchitecturewasdesignedforvirtualmachinessoitsupportsall
hypervisorfunctionsthataresupportedbythetraditionalnetwork-storage
basedarchitecture,includingliveVMmigrationandhighavailability.In
addition,becausetheNutanixarchitectureisVM-aware,itovercomeslimitations
oftraditionalsolutionsthatwereoptimizedtoworkwithphysicalservers.For
example,whilethingsaremanagedonaperVMbasisoncomputeside,theunit
ofmanagementonstoragehastraditionallybeenaLUN.WhenaLUNisshared
bymanyVMs,itbecomesmoredifficulttoperformstorageoperationssuchas
backup,recovery,andsnapshotsonaper-VMbasis.Itisalsodifficulttoidentify
performancebottlenecksinaheavily-sharedenvironmentduetothechasm
betweencomputingandstoragetiers.TheNutanixarchitectureovercomesthese
limitations.
-
NUTANIX
COMPLETECLUSTERA
RCHITECTURE
7NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Solid-State Drives
TheNutanixarchitecturewasdesignedtotakeadvantageofenterprise-grade
solid-statedrives(SSDs).Itisimportanttonotethatthetraditionalstorage
systemsweredesignedforspinningmediaanditishardforthemtoleverage
SSDsefficientlyduetotheentirelydifferentaccesspatternsthatSSDsprovide.
Whileharddiskshavetodealwiththerotationandseeklatencies,SSDsdo
nothavesuchmechanicallimitations.Thisdifferencebetweenthetwomedia
requiresthesoftwaretobeoptimizeddifferentlyforperformance.Onecannot
simplytakesoftwarewrittenforharddisk-basedsystemsandhopetouseit
efficientlyonsolid-statedrives.TheNutanixarchitectureusesSSDstostorea
varietyoffrequently-accesseddata,fromVMmetadatatoprimarydatastorage,
bothinadistributedcacheforhigh-performanceandinpersistentstoragefor
quickretrieval.TomaximizetheperformancebenefitsofusingSSDs,theNutanix
architecture:
ReservesSSDsforI/O-intensivefunctions
Includesspace-savingtechniquesthatallowlargeamountsoflogicaldatato
bestoredinasmallphysicalspace
Migratescoldorinfrequently-useddatatoharddiskdrivesautomatically,
allowsadministratorstobypassSSDsforlow-priorityVMs
Architecture OverviewNutanixCompleteClusterisascale-outclusterofhigh-performancenodes,
orservers,eachrunningastandardhypervisorandthatcontainsprocessors,
memoryandlocalstorage,includingSSDs)andharddiskdrives.Eachnode
runsvirtualmachinesjustlikeastandardvirtualmachinehost.Inaddition,
localstoragefromallnodesisvirtualizedintoaunifiedpoolbyNutanixScale-
outConvergedStorage(SOCS)(Figure1).Ineffect,SOCSactslikeanadvanced
SANthatuseslocalSSDsanddisksfromallnodestostorevirtualmachinedata.
VirtualmachinesrunningontheclusterwritedatatoSOCSasiftheywere
writingtoaSAN.SOCSisVM-awareandprovidesadvanceddatamanagement
features.Itbringsdataclosertovirtualmachinesbystoringthedatalocallyon
thesystem,resultinginhigherperformanceatalowercost.NutanixComplete
Clustercanhorizontallyscalefromafewnodestoalargenumberofnodes,
enablingorganizationstoscaletheirinfrastructureastheirneedsgrow.
-
NUTANIX
COMPLETECLUSTERA
RCHITECTURE
8NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Server 1 Server 2 ..............
FusionIO HDDs
Virtual Machine / Virtual Disk
Server N
N u t a n i x S c a l e - O u t C o n v e r g e d S t o r a g e ( S O C S )
Figure 1: Nutanix Scale-Out Converged Storage Architecture
WithNutanixCompleteCluster,allvirtualizationfeatures,includinghigh
availabilityandliveVMmigration,continuetoworkseamlessly.Administrators
createvirtualmachinesonNutanixCompleteClusterusingtheirstandard
processes.NutanixSOCSprovidesstorageforthesevirtualmachinesintheform
ofvirtualdisks,orvDisks,whicharestandardiSCSIdevices.
Nutanix Scale-Out Converged Storage Key ComponentsThekeytotheNutanixarchitectureisSOCSapatent-pendingscale-out
convergedstoragelayerthathasthefollowinguniquesetofcapabilities:
Itisconvergedwiththecomputelayer.VMsandSOCSco-existonthesame
cluster.
ItisVM-aware.SOCSprovisionsstorageonaper-VMbasisandcanidentifyI/O
comingfromeachVM.
Itcanscaleoutfromafewnodestoalargenumberofnodes.
Ithasground-upintegrationwithsolid-statedrives.
Itprovideshighavailabilityagainstdiskornodefailures.
ItprovideshighperformancebymakingI/Oaccesslocal,leveragingsolid-state
drivesandemployingaseriesofpatent-pendingperformanceoptimizations.
Itprovidesuniquecapacityoptimizationcapabilities.
SOCSisenabledbythefollowingcomponents:
-
NUTANIX
COMPLETECLUSTERA
RCHITECTURE
9NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
n-Way Controller Cluster
WhiletraditionalSANsolutionstypicallyhave1,2,4or8controllers,ann-node
NutanixCompleteClusterhasncontrollers(Figure2).EverynodeonNutanix
CompleteClusterrunsaspecialvirtualmachine,calledaControllerVM.This
virtualmachineactsasavirtualcontrollerforSOCS.AllControllerVMsin
theclustercommunicatewitheachothertoformasingledistributedsystem.
UnliketraditionalSAN/NASsolutionsthatarelimitedtoasmallnumberoffixed
controllers,thisarchitecturecontinuestoscaleasmorenodesareadded.
Node 1 Node 2 ..............
SSDs HDDs
Virtual Machine / Virtual Disk
Node N
Control ler VM Control ler VM Control ler VM...... . . . . . . . .
.... ............ .... ............ .... ............
Figure 2: Nutanix Scale-Out Controller VM Architecture
Heat-Optimized Tiering Cache (HOTcache)
HOTcacheisahighperformancecachebackedbySSDsfromeachnodeina
cluster.WhenguestVMswritedata,thatdataisfirstwrittentoHOTcacheand
then,inthebackground,itisflushedtoSOCS.HOTcacheusesasequentialdata
layouttoprovidehighperformanceevenifworkloadsfromVMsgetmixedinto
arandomworkload.HOTcachekeepsonedatacopyonalocalSSDandanother
copyonadifferentnodesothatthereisnodataloss,eveninthecaseofanode
failure.
Distributed Metadata Service (Medusa)
Intraditionalstoragesystems,controllersoftenbecomeabottleneckasmore
storageisadded.Oneofthemainreasonsforthisissueisthatthestorage
metadataisstoredonthecontrollers.Traditionalsystemsscaletoasmall
numberofcontrollers,soasthenumberofVMsincreases,sodoestheI/Oload
oneachcontroller.Nutanixsdistributedmetadataservice,Medusa,distributes
-
NUTANIX
COMPLETECLUSTERA
RCHITECTURE
10NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
theclustermetadataacrosstheclusterforscalabilityandreplicatesthedata
onmultiplenodesforfaulttolerance.Theserviceishighlyavailableandcan
toleratemultiplemodulefailures.Incomparison,traditionalsolutionsthat
dependonafewstoragecontrollerscannottoleratemultiplecontrollerfailures.
Distributed Data Maintenance Service (Curator)
Nutanixsdistributeddatamaintenanceservice,Curator,isaMapReduce1
-basedframeworkforexecutingbackgrounddatamanagementoperationsina
massivelyparallelmanner.Suchoperationsinclude:
Migrationofcolddatatolowertiers(forHeat-OptimizedTiering)
Garbagecollectionofdatathathasbeendeleted
Dataconsistencythroughroutinechecksums
Replicationofdataincaseofnodeordiskfailures
Re-balancingofdatawhennodesareaddedorremoved
MigrationofdatatomaximizelocalaccesswhenaVMmovesfromonenode
toanother
FlashStore
FlashStoreisthepersistent,flash-basedstorageprovidedbythepoolingof
SSDsfromallnodesinthecluster.DataisfirstwrittentoFlashStoreandthen
movedofftoDiskStore,asitbecomescold.Ascolddatabecomeshotagain,
itisbroughtbackintoFlashStore.ForvDisksthatservelowpriorityVMs,
administratorshaveanoptiontoskipFlashStore.
DiskStore
DiskStoreisthehigh-capacitySATAstoragetierspreadacrossthecluster.
DiskStoreprovideslargestoragecapacityforcolddata.NutanixsHeat-
OptimizedTiering(HOT)combinesFlashStoreandDiskStoretoprovidehigh
performanceaswellashighcapacityatalowercost.
1 MapReduceisatechnologyoriginallybuiltatGoogleformassivelyparallelanalysisofdatainacluster
-
KEYFEATURES
11NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Key FeaturesNutanixCompleteClusternotonlyeliminatestheneedforaSAN,butitalso
providestopofthelineavailability,performanceanddatamanagement
features.Someofthekeyfeaturesaredescribedbelow.
Capacity OptimizationNutanix QuickClone
Nutanixenablesadministratorstorapidlydeploynewvirtualmachinesbyusing
itsQuickClonefeature.QuickClonesarewriteablesnapshotsthatbehavejust
likestandardvDisks-administratorscanattachthemtoaVM,writedataon
them,andevensnapshotthemfurther.Thisisusefulfordeployingnewvirtual
desktops,creatingtestanddevelopmentcopiesofaproductiondatabaseand
anyotherscenariorequiringcloneswithoutduplicatingthedata.Thesystem
alsosupportsread-onlysnapshotsforbackuppurposes.
Nutanix Thin Provisioning
Storageforvirtualmachinesisthinlyprovisionedinthesystem.Administrators
cansetthecapacityofavDiskbutphysicalstorageisallocatedonlywhen
required.Administratorscanalsosetaminimumreservationparameterthat
guaranteesthespecifiedamountofstorageforacollectionofvDisks.
Nutanix Converged Backup
TheConvergedBackupfeatureprovidesinstantbackupandrecoverycapabilities
forvDisks.Severalmonthsworthofbackupscanbekeptinsidetheappliance
withoutrequiringexternalbackupstorage.Whenrecoveryisnecessary,
administratorscaninstantaneouslyrestoreavDisktoanyofitspastbackups.The
appliancealsosupportsoffsitebackupsusingstandardthird-partytools.
Ease of ManagementEase of Deployment
NutanixCompleteClusterisaplug-and-playsolutionthatincludesallhardware
andsoftwarenecessarytorunalargenumberofvirtualserversorvirtual
desktops.AdministratorscansetitupandstartcreatingVMsinamatterof
minutes.
-
KEYFEATURES
12NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Next-Generation User Interface
NutanixCommandCenterisahighlyintuitiveFlex-baseduserinterface
thatprovidesadministratorcompletevisibilityacrosscomputeandstorage
resourcesinthecluster.Itenablesthemtotroubleshootissuesrelatedtoa
virtualmachineseasilybymappingeachVMtothephysicalresourcesinthe
system.NutanixCompleteClusteralsoprovidesacommand-lineinterfacefor
management.
Nutanix Scale-out Converged Storage (SOCS)
NutanixSOCSeliminatestheneedformanagingacomplexnetwork-basedstorage
infrastructure,makingiteasytomanagevirtualenvironmentsatanyscale.
Conformance to IT Standards
WhileNutanixCompleteClusterenablesaconvergedarchitecture,itcontinues
tosupportstandardtoolsandinterfacesthatITdepartmentsalreadyuse.For
example,thenodesrunanindustry-standardhypervisor(VMwareESXi)andall
ITprocessesandsoftwaretoolsthatworkwiththishypervisorcontinuetowork
withNutanix.Similarly,vDisksarestandardiSCSIdevicesthatareconnected
toVMsusingastandardiSCSIinitiatorinthehypervisor.Byleveragingsuch
standardinterfaces,NutanixCompleteClustercanseamlesslyfitintoanexisting
ITecosystem.
Performance and ScalabilitySolid-State Drives
NutanixCompleteClusterwasdesignedwithSSDsinmind.Theincludedserver-
basedSSDsprovidehigherperformancethanSAN-basedSSDsbecausetheyavoid
thenetworkbottleneck.Traditionalsystemshaveoftenbeenlimitedbythe
amountofmetadatatheycankeepinacontrollerscache.InNutanixComplete
Cluster,SSDsareusednotonlyforVMdata,butalsoforstoringSOCSmetadata
forfastaccess.KeepingrichmetadataonSSDsenablesSOCStoprovide
advanceddatamanagementcapabilities.Giventhescale-outarchitecture,SSD
capacityinthesystemgrowsasmorenodesareaddedtothecluster.
-
KEYFEATURES
13NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
Nutanix Heat-Optimize Tiering (HOT)
IntheNutanixcluster,avDiskcanbeallocatedamixofSSDandHDDcapacity.
Toensurethatonlyhigh-valuedatastaysonSSDs,SOCSmovescolddatatohigh-
capacitySATAdrivesinthebackgroundusingitsHOTfeature.
Scalability
NutanixCompleteClusterisdesignedtoscalefromafewnodestoalarge
numberofnodes.Everyaspectofthesystemwasdesignedwiththescalability
requirementsoftodaysvirtualizeddatacentersinmind.Forexample,there
isnocentralizedmetadatamasterinthesystem.Themetadatalayeritself
isdistributedacrossthecluster,eliminatingacommonbottleneckfoundin
mostscale-outsystems.Also,witheverymodulerunningaSOCSController,
thenumberofcontrollersinthesystemcanbemuchhigherthanatypical
network-basedstoragesolutionwithonlyafewcontrollers.Suchdesign
innovationsenablethesystemtostartsmallandscalemassively.
High AvailabilityNutanix Cluster RAID
NutanixCompleteClusterisahighlyavailablescale-outsystemwithnosingle
pointoffailure.UsingNutanixClusterRAID,dataisstripedacrossdiskswithina
nodeforhighperformanceandreplicatedacrosstheclusterforhighavailability.
Thisprovideshighavailabilityforvirtualmachinesevenifdisksorcomplete
nodesfail.
Nutanix Distributed Metadata Service (Medusa)
Nutanixsdistributedmetadataservice,Medusa,distributestheclustermetadata
acrosstheclusterforscalabilityandreplicatesthedataonmultiplenodesfor
faulttolerance.Theserviceishighlyavailableandcantoleratemultiplenode
failures.Incomparison,traditionalsolutionsthatdependonafewstorage
controllerscannottoleratemultiplefailures.
Nutanix Instant vDisk Motion
TheabilitytomigratealiveVMfromonehosttoanotherisaverypowerful
featureprovidedbytheindustry-standardhypervisors.Sofar,organizations
havebeenabletoleveragesuchlivemigrationcapabilitiesonlywithnetwork
-
KEYFEATURES
14NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
storage.Infact,manyorganizationswereforcedtousenetworkstorageonly
toachievelivemigrationandhighavailability.InNutanixCompleteCluster,live
VMmigrationissupportedevenwiththeconvergedarchitecturethatuseslocal
storage.vDisksintheappliancearelogicalentitiesthatarefullymobile.Thisis
enabledbyNutanixInstantvDiskMotionfeaturethatcanquicklymoveavDisk
fromonenodetoanother,whennecessary.
Backup and Recovery
Theabilitytoperformoff-sitedatabackupandrecoveryiskeytoan
organizationsdataprotectionstrategy.NutanixCompleteClusterisfully
compatiblewiththeVMwarevStorageAPIforDataProtection(VADP),and
providesfullyfunctionalbackupandrecoveryofvirtualmachines,through
integrationwithVADPcompatiblebackupandrecoveryproducts.
Disaster Recovery
DisasterRecoveryiskeytothebusinesscontinuityneedsofanenterprise.
NutanixCompleteClusterprovidesfailoverandfailbackcapabilitiesthrough
integrationwithVADPcompatibleDisasterRecoveryproducts.
-
CONCLUSIO
N
15NUTANIX COMPLETE CLUSTER:ATECHNICALWHITEPAPER
ConclusionNutanixCompleteClusterisaninnovativesystemthateliminatestheneedfor
networkstoragewithoutcompromisingtherequirementsofenterpriseIT.It
isbuiltusinganext-generationscale-outarchitecturethathasbeenproven
atsomeofthemostinnovativecloud-generationcompanies.WithNutanix
CompleteCluster,organizationscanbuildacomputeandstorageinfrastructure
forvirtualmachinesthatishighlyavailable,fast,richindatamanagement
featuresandcangrowastheirneedsgrow.
Executive SummaryIntroductionLimitations of Current ArchitectureThe Google Approach
Nutanix Complete Cluster Architecture Main PillarsArchitecture OverviewNutanix Scale-Out Converged Storage Key Components
Key FeaturesCapacity OptimizationEase of ManagementPerformance and ScalabilityHigh Availability
Conclusion