copyright 2002, jeremy zawodny ([email protected]) mysql backup & recovery o’reilly open...
TRANSCRIPT
Copyright 2002, Jeremy Zawodny ([email protected])
MySQL Backup & RecoveryO’Reilly Open Source Convention
Jeremy ZawodnyYahoo! Finance
July 24th, 2002San Diego, California
http://jeremy.zawodny.com/mysql/
Copyright 2002, Jeremy Zawodny ([email protected])
Speaker Info (Who am I?)I’m an engineer in Yahoo! Finance.
I’ve been using MySQL personally for4 years and professionally for over 2.5 years.
I am also an active participant in the MySQLuser community—primarily on the MySQLmailing list.
I write about LAMP and MySQL for LinuxMagazine. I also write a fair amount of Perl code.
Contact Info…
Personal: [email protected]: [email protected]: http://jeremy.zawodny.com/mysql/
Copyright 2002, Jeremy Zawodny ([email protected])
MySQL at Yahoo!• 20 - 50 servers world-wide
– Mission-critical (we lose money if they stop)
• FreeBSD and Linux• Commodity hardware• Replaces Oracle in some cases• Replaces home-grown storage in others• Data growing rapidly• Used “live” and in batch processing• Replication and load-balancing
Copyright 2002, Jeremy Zawodny ([email protected])
Outline• Who am I?
• Outline
• Goals
• Quick Survey
• Tradeoffs and Considerations
• Tools and Techniques
• Shameless Book Plug
• Questions and Answers
Copyright 2002, Jeremy Zawodny ([email protected])
Goals• Learn the various ways to backup MySQL
• Understand the “right ways” to do it
• Know the dangers of the wrong way
• Select and appropriate tools and techniques– Data integrity– Speed– Flexibility
Copyright 2002, Jeremy Zawodny ([email protected])
Required Knowledge• Basic MySQL administration
• Table types– MyISAM– InnoDB– HEAP
• Scripting in any language
Copyright 2002, Jeremy Zawodny ([email protected])
Quick Survey• What version of MySQL are you using?• Table Types
– MyISAM– InnoDB
• Do you perform backups?– Frequently– Using MySQL-specific tools
• Which operating systems?• Familiarity with other RDBMS servers?• Role? DBA? Developer? SysAdmin?• MySQL dedicated or shared servers?
Copyright 2002, Jeremy Zawodny ([email protected])
Considerations• Dump or Raw Backup
• On-line of Off-line
• Table Types and Consistency
• Storage Requirements
• Replication
Copyright 2002, Jeremy Zawodny ([email protected])
Dump or Raw Backup• Dumps
– Can be performed remotely
– Take a lot of space
– CPU intensive and slower
– Are plain SQL, so easy to use later
– Can do selective restore
– Works for any table type
• Raw Backups– Faster, there is no translation
– Restore “instantly”
– Works for ISAM/MyISAM only
Copyright 2002, Jeremy Zawodny ([email protected])
On-line or Off-line• On-line
– Impacts running applications (can’t write)– Must deal with data consistency issues– Slower than off-line because of I/O contention
• Off-line– Very simple– Speed is not an issue– Can use normal backup software
Copyright 2002, Jeremy Zawodny ([email protected])
Table Types & Consistency• All tables need to flushed to disk
– FLUSH TABLES
• Tables can’t be updated during backup– LOCK TABLES...
• Related tables treated as a group, lock all– FLUSH TABLES WITH READ LOCK
• Transactional tables have logs to backup
Copyright 2002, Jeremy Zawodny ([email protected])
Storage Requirements• How long can you store backups?
– Consider staggering older backups– Never know when you’ll want old data
• Raw backups take less space– You can keep just the .MYI header– Rebuild indexes later if need be
• All forms of backup can be compressed
Copyright 2002, Jeremy Zawodny ([email protected])
Replication• Backup your slave instead of your master
– Master won’t be interrupted– Slave can probably be shut down– Or at least you can hold read locks for a long
time and nobody will care
• It may be worth setting up a slave just for doing backups
• Remember to backup the master.info file
Copyright 2002, Jeremy Zawodny ([email protected])
Tools and Techniques• mysqldump
• mysqlhotcopy
• mysqlsnapshot
• InnoDB on-line backup
• Off-line backups
• Filesystem snapshots
• Roll your own
Copyright 2002, Jeremy Zawodny ([email protected])
mysqldump• Comes with MySQL
• Dumps local or remote tables/databases
• Variety of output formats
• Handles locking if you need it
• Works best for small and medium installations
Copyright 2002, Jeremy Zawodny ([email protected])
mysqlhotcopy• Comes with MySQL
– Written by Tim Bunce of Perl DBI fame
• Raw backups of MyISAM tables
• Handles locking
• Regular expression support– Match certain databases or tables
• Can truncate indexes to save space
Copyright 2002, Jeremy Zawodny ([email protected])
mysqlsnapshot• Separate download
– http://jeremy.zawodny.com/mysql/
• Used for setting up replication snapshots
• Or backing up a running server
• Always backs up everything– Either one big tar file or one per database
• Very quick to restore
Copyright 2002, Jeremy Zawodny ([email protected])
InnoDB on-line Backup• Separate download
– http://www.innodb.com/
• Costs extra $$$
• The only good solution for InnoDB backups without replication
• Only does InnoDB tables
Copyright 2002, Jeremy Zawodny ([email protected])
Off-line Backups• Very easy
• Hard to screw up
• Use any backup tool you like– Just remember to backup all the MySQL stuff– You wouldn’t want compatibility problems
after an upgrade
• Restores can be quick
Copyright 2002, Jeremy Zawodny ([email protected])
Filesytem Snapshot• Most “expensive” solution
• Need hardware/software support– Veritas, Network Appliance, LVM, EMC, etc.
• Fastest option available
• Uses least disk space
• Doesn’t guard against all failures
Copyright 2002, Jeremy Zawodny ([email protected])
Roll your Own• Not hard to do in Perl, Python, whatever
• Be sure to hold connection open or locks will be released
• Error checking is critical
• Use existing scripts as references– mysqlhotcopy– mysqlsnapshot
Copyright 2002, Jeremy Zawodny ([email protected])
For More Info…• MySQL mailing lists
– Visit lists.mysql.com
• Books– MySQL Manual– Advanced MySQL– Managing & Using MySQL– MySQL
• Google, of course
Copyright 2002, Jeremy Zawodny ([email protected])
Shameless Book PlugAdvanced MySQL
– Coming in 2003– O’Reilly & Associates– Unknown Animal– Covers MySQL 4.x
http://advanced-mysql.com/
Copyright 2002, Jeremy Zawodny ([email protected])
Questions and Answers
That’s it. I’m done and probably out of time!