mysql high availability - percona · mysql high availability and other stuff worth talking about...
TRANSCRIPT
©2017Percona1
PeterZaitsev
MySQLHighAvailabilityAndotherstuffworthtalkingabout
CEOMoscowMySQLUsersGroupMeetupJuly11th,2017
©2017Percona2
FewWordsaboutPercona
©2017Percona3
Percona’s Purpose
ToChampionUnbiasedOpenSourceDatabaseSolutions
©2017Percona4
WeDo
Support,ManagedServicesforMySQLandMongoDB
AlsoConsultingandTraining
HelpingcompaniestomigratetoOpenSourceDatabase
DevelopOpenSourceSoftware
Solutionstomaximizeyour success
©2017Percona5
BroadSoftwareEcosystemSupport
MySQL PerconaServer MariaDB PerconaXtraDBCluster
GaleraClusterforMySQL MariaDBGaleraCluster MongoDB PerconaServerforMongoDB
AmazonRDSforMySQL/MariaDB/Aurora GoogleCloudSQL
©2017Percona6
100%FreeandOpenSourceSoftware
PerconaServerforMySQL
PerconaServerforMongoDB
PerconaXtraDBCluster
PerconaXtrabackup PerconaToolkit
PerconaMonitoringandManagement
6
©2017Percona7
MySQLHighAvailability
©2017Percona8
WhatisHighAvailability
WhatDoyouThink?
©2017Percona9
Ithink…
ServingAlltheClientsCorrectlyalltheTime
©2017Percona10
PerformanceandHighAvailability
Unacceptableperformancecanbeindistinguishablefromdowntime
©2017Percona11
HighAvailabilityisNeverPerfect
•99.9%or“3nines”Canbemeasuredintermsof“time”wheresystemwas
available
•Goodtoconsidercasesofpartialavailability
Canbemeasuredinthetermsof
successfulrequestsserved
©2017Percona12
IsPerfectHighAvailabilitytheGoal?
Booking.comhasthe“downtimebudget”
©2017Percona13
TohaveGoodHighAvailability
Godown“less” Recoverfaster
©2017Percona14
NotaDatabaseProblem
SystemsProblem
ButDatabaseisthekeycomponent
©2017Percona15
AchievingHighAvailability
TechnologyAspects
ProcessAspects
©2017Percona16
TechnologyAspects
DesigningandBuildingHighlyAvailableSystem
©2017Percona17
TechnologyAspects
FocusonTechnologyinthispresentation,itiswhatMySQLSpecific
©2017Percona18
ProcessAspects
ManagingSysteminHighlyAvailableway
©2017Percona19
Process– GeneralBestPracticesQualityAssuranceforCodeandOperationsPractices
TablesandQueryReview
PerformanceTesting
CapacityPlanning
ClearRollBackProcess
Beingabletodisablefeature“live”
“OnCall”fordevsandops
©2017Percona20
HighAvailabilitythroughRedundancy
“Server”Redundancy
NetworkRedundancy
DataCenterRedundancy
©2017Percona21
StatelessvsStateful forHA
• Multipleinstances• HighAvailabilityiseasyStateless
• Needto“Maintainstateinhighlyavailablefashion”
• MuchharderproblemStateful
©2017Percona22
Databaseare…
Stateful
©2017Percona23
HighLevelThoughts
Letsthinkaboutdatabaseasablackbox
©2017Percona24
HAConcernsforDatabases• WhatkindofguaranteesdoIhaveaboutmydatabeingpersistedDurability
• Howmychangesarepropagatedandbecomevisible?ChangeVisibility
• WhichfailuressystemisdesignedtohandleandhowitreactsFailureHandling
• HAisneverfree.HowarewepayingforguaranteesPerformance
©2017Percona25
Replication
WithStatefulSystemsHArequiresmaintainingmultiplecopiesofdata
©2017Percona26
Focus
FocusonDatabaseEnginelevelReplication
asMostCommon
©2017Percona27
ThingsToConsider
HowReplicationisHappening
WhoandHowDetectsFailure
MechanicsofFailoverandRecovery
©2017Percona28
MySQLOptions
©2017Percona29
MySQLOptionstoConsider
MySQLReplication
MySQLGroupReplication
Galera andPerconaXtraDB Cluster
©2017Percona30 www.percona.com
MySQL ReplicationStandardMySQLReplication
©2017Percona31
“Classic”MySQLReplication
FullyAsynchronous
NoFailureDetectionandPromotion
ManualProvisioning
CanrunMulti-Master
NoConflictHandling
©2017Percona32
Semi-Synchronous
IsAsynchronous
OneoftheSlavesgetsacopybeforecommitsucceedsontheMaster
©2017Percona33
GTID
• “Position”identifiedbyBinaryLogfileandOffset• DifferentondifferentnodesinReplicationTopology• Simple
OldStyleMySQL
Replication
• Eachtransactiongetsanidentifier• NodesknowwhattransactionstheyExecuted• GTIDsameintheReplicationTopology• Hard
GTID
©2017Percona34
MySQLReplicationTopologies
MastercanhaveanynumberofSlaves
Canreplicateeverythingorpartofdataset
Youcancascadereplicationwithnolimits
Youcancreatereplicationloops(throughnotrecommended)
StartingMySQL5.7slavecansourcefrommultiplemasters
©2017Percona35
MySQLReplicationPerformance
SingleThreadinMySQL5.5andBelog
ThreadperdatabaseinMySQL5.6
Multi-Threaded(evensingletable)inMySQL5.7
©2017Percona36
MakingMySQLReplicationBetter
MHA
MySQLFailover
Orchestrator
©2017Percona37
OrchestratorScreenshot
©2017Percona38
PMMDashboardforReplication
©2017Percona39
MySQLGroupReplicationAndInnodb Cluster
©2017Percona40
MySQLGroupReplicationOverview
InspiredbyGaleraIdeas(andSuccess)
BuiltontopofstandardMySQLReplication
AvailableasPluginforMySQL5.7
ConsideredGAbutreallyveryearlyproduct
©2017Percona41
DifferencefromMySQLReplicationNoMaster/SlavebutGroupMembership
Transactionsarecommittedwhentheyarecertifiedbymajorityofnodes(Paxos)
DoesnotacceptwritesifthereisnoQuorum
FlowControltopreventunlimitedreplicationlag
Nodesencounteringinconsistencyleavethecluster
ConflictDetectionandResolution(oravoidance)
SimpleFailOver
©2017Percona42
MySQLGroupReplication- Writes
ByDefaultconfiguresitselfasSinglePrimary
Canconfiguretoallowwritestoanynode
©2017Percona43
MySQLGroupReplicationLimitations
NoAutomatednodeProvisioning
ManualRecoveryoffailednodes
Noway(yet)toensurenodedoesnotreadstaledata
©2017Percona44
MySQLInnodb Cluster
©2017Percona45
MySQLInnodb ClusterReally
MySQLGroupReplication
MySQLShell(for
configuration)
MySQLRouterfortraffic
management
©2017Percona46 www.percona.com
Percona XtraDB Cluster and “Galera”
©2017Percona47
Percona XtraDB Cluster
VirtuallySynchronousReplication
WellknownInnodbStorageEngine
AllReadsareLocal
ParallelReplication
Writetoanynodebehavior
BuiltinNodeProvisioningandHA
WorksgreatintheCloud!
©2017Percona48
PXCvsGalera
Galera
• Isreplicationtechnology/library• ComparetoLinuxKernel
PerconaXtraDBCluster
• PerconaServer• EnablingEnhancements• GaleraLibrary• ProvisioningTools• HAandLoadBalancing• IntegratedtogetherandTested• ComparetoLinuxDistribution
©2017Percona49
PXC Data Architecture
©2017Percona50
ArchitectureConcepts
AllNodesHaveFullcopyofData
EveryNodeisEqual
NoCentralManagement
NoSPOF
©2017Percona51
ClusterReads
ReadsarealwaysLocal
StaleReadscanbeallowedordisallowed
©2017Percona52
ClusterWrites
WriteononenodeorWriteAnywhere
CertificationBasedReplication
CommunicationonCommitonly
AsynchronousApplication
ParallelReplication
©2017Percona53
TransactionCommitFlow
©2017Percona54
QuorumLossofconnectivity
54
NetworkProblem
DoesnotacceptReads&Writes
©2017Percona55
AutomaticNodeProvisioning 55
writes
writeswrites
new node joining
©2017Percona56
AutomaticNodeProvisioning 56
writes
writeswrites
new node joiningwhen ready
writes
©2017Percona57
TrafficManagementSolutions• NotImmediate,DoesnothandleexistingconnectionsDNS
• TCP/IPPortlevel.DoesnotunderstandMySQLProtocolHAProxy
• LikeHAProxyintheCloudElasticLoadBalancer(etc)
• CurrentlyverybasicMySQLRouter
• ProprietarySolutionsMaxScale,ScaleArc
• MySQLProtocolAware100%OpenSourceProxySolutionforMySQLProxySQL
©2017Percona58
WeintegrateProxySQL withPXC
HighAvailability
Multiplexing(ConnectionPooling)
Caching
Read/writeSplitting
EasyConfigurationwithproxysql-admin
Canrunindedicatedwriterandloadbalancedmode
©2017Percona59
DedicatedsharedProxySQL 59
application server 1 application server 2 application server 3
PXC node 1 PXC node 2 PXC node 3
ProxySQL
©2017Percona60
ProxySQL onapplicationsideapplication server 1 application server 2 application server 3
PXC node 1 PXC node 2 PXC node 3
60
ProxySQL ProxySQL ProxySQL
©2017Percona61
ProxySQL withPMM
©2017Percona62
PerconaXtraDB Cluster5.7pxc_safe_mode – topreventyoufromvisiting
encrypt_pxc_traffic – EasytouseClusterTrafficEncryption
InstrumentationwithPerformanceSchema
BetterUsability
FasterSST
BettergeneralPerformance
©2017Percona63
PXC5.7PerformanceImprovementshttp://bit.ly/2qGCr0T andhttp://bit.ly/2pzvAIW
©2017Percona64
PXCatPMM
©2017Percona65
ScalingWrites?
ShardoverPXCClusterratherthanMySQLReplicas
©2017Percona66
ThingsToKeepinMind
UseInnodbstorageengine
PrimaryKeyonalltables
AvoidLargewritetransactions(changingmanyrows)
PlanDatasizeforSSTtimecorrectly
HotRows
OptimisticLocking
©2017Percona67
PXCvsMySQLGroupReplication
GreatWebinarbyKennyGryphttps://www.percona.com/resources/webinars/percona-xtradb-cluster-galera-cluster-mysql-group-replication
©2017Percona68
SomeUnrelatedbutWonderfulTopics
©2017Percona69
PerconaMonitoringandManagement
©2017Percona70
PerconaMonitoringandManagement
100%FreeandOpenSource
ComprehensiveDatabasefocusedMonitoring
SupportsMySQLandMongoDB(andvariants)
EasytoInstallandUse
Version1.xfocusesonTrendingandQueryAnalyses
ManagementFeaturestocome
©2017Percona71
WhydidwecreatedPMM
Nogooddatabasefocusedmonitoringsolution
ExistingsolutionsProprietaryorCloudOnly
Doityourselfispossiblebuthard
Wanteveryonetobeabletorunmonitoringineveryenvironment
Tofixproblemsbeforetheyappear
©2017Percona72
WhatQueriesarecausingtheload?
©2017Percona73
Whyaretheycausingthisload?
©2017Percona74
Howtofixthem
©2017Percona75
SystemInformation
©2017Percona76
WhathappensonOSandHardwareLevel
©2017Percona77
AswellasDatabaseLevel
©2017Percona78
BesttocheckoutDemo
http://pmmdemo.percona.com
©2017Percona79
©2017Percona80
HaveaFriend?
Referafriendandget$1000ifoneishiredbyPercona
…andeternalgratitudeforhelpingtogetthemostamazingjob
©2017Percona81
PerconaLiveEuropeCallforPapers&RegistrationareOpen!
ChampioningOpenSourceDatabases▪MySQL,MongoDB,OpenSourceDatabases▪TimeSeriesDatabases,PostgreSQL,RocksDB▪Developers,Business/CaseStudies,Operations▪September25-27th,2017▪RadissonBlu RoyalHotel,Dublin,Ireland
SubmitYourProposalbyJuly17th!www.percona.com/live/e17
DATABASE PERFORMANCEMATTERS
DatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMattersDatabasePerformanceMatters