database migrations without down time (v2:php:dpc)
TRANSCRIPT
DATABASE MIGRATIONSWITHOUT DOWN TIME
MICHIEL ROOK
@michieltcs
▸ Developer, consultant, trainer, speaker
▸ @michieltcs
ABOUT DATABASE MIGRATIONS
ABOUT SCHEMA MIGRATIONS
SQL
@michieltcs
UP
@michieltcs
UP DOWN
DOCTRINE
@michieltcs
class Version20170228150619 extends AbstractMigration { public function up(Schema $schema) { $this->addSql('CREATE TABLE addresses (id INT NOT NULL, street VARCHAR(255) NOT NULL, PRIMARY KEY(id))'); } public function down(Schema $schema) { $this->addSql('DROP TABLE addresses'); } }
DOCTRINE
@michieltcs
mysql> select * from migration_versions; +----------------+ | version | +----------------+ | 20170228150619 | +----------------+ 1 row in set (0,00 sec)
FLYWAY
@michieltcs
V20170228150619-create_addresses_table.sqlCREATE TABLE addresses (id INT NOT NULL, street VARCHAR(255) NOT NULL, PRIMARY KEY(id))');
FLYWAY
@michieltcs
mysql> select * from schema_version; +----------------+----------------+------------------------+----------+---------------------------------------------+------------+--------------+---------------------+----------------+---------+ | installed_rank | version | description | type | script | checksum | installed_by | installed_on | execution_time | success | +----------------+----------------+------------------------+----------+---------------------------------------------+------------+--------------+---------------------+----------------+---------+ | 1 | 1 | << Flyway Baseline >> | BASELINE | << Flyway Baseline >> | NULL | migrations_1 | 2017-02-28 16:27:21 | 0 | 1 | | 2 | 20170228150619 | create addresses table | SQL | V20170228150619__create-addresses-table.sql | 1789629568 | migrations_1 | 2017-02-28 16:27:21 | 16 | 1 | +----------------+----------------+------------------------+----------+---------------------------------------------+------------+--------------+---------------------+----------------+---------+ 2 rows in set (0,00 sec)
+------------+--------------+---------------------+----------------+---------+ | checksum | installed_by | installed_on | execution_time | success | +------------+--------------+---------------------+----------------+---------+ | NULL | migrations_1 | 2017-02-28 16:27:21 | 0 | 1 | | 1789629568 | migrations_1 | 2017-02-28 16:27:21 | 16 | 1 | +------------+--------------+---------------------+----------------+---------+
TRANSACTIONS?
ROLLBACKS?
@michieltcs
DESTRUCTIVE CHANGES
1
@michieltcs
DESTRUCTIVE CHANGES
1 LONG-LIVED LOCKS
@michieltcs
DESTRUCTIVE CHANGES
1 LONG-LIVED LOCKS
RENAMES
@michieltcs
DESTRUCTIVE CHANGES
1 LONG-LIVED LOCKS
RENAMES
DELETES
@michieltcs
DESTRUCTIVE CHANGES
IMPLICIT COMMITS
1 2
@michieltcs
DESTRUCTIVE CHANGES
IMPLICIT COMMITS
SHARDING
1 2 3
@michieltcs
DESTRUCTIVE CHANGES
IMPLICIT COMMITS
SHARDING MULTIPLE DATABASES
1 2 3 4
ROLLBACKS
DOWNTIME?
@michieltcs
CREATE PACKAGE
@michieltcs
CREATE PACKAGE
SHUT DOWN APPLICATION
@michieltcs
CREATE PACKAGE
SHUT DOWN APPLICATION
RUN MIGRATION
@michieltcs
CREATE PACKAGE
SHUT DOWN APPLICATION
RUN MIGRATION
DEPLOY PACKAGE
@michieltcs
CREATE PACKAGE
SHUT DOWN APPLICATION
RUN MIGRATION
DEPLOY PACKAGE
RUN APPLICATION
DOWNTIME?
DOWNTIME
NO MIGRATIONS!
DOWNTIME
ZERO DOWNTIME DEPLOYMENT
@michieltcs
LOAD BALANCER
APP V1
@michieltcs
LOAD BALANCER
APP V1 APP V2
ZERO DOWNTIME DEPLOYMENT
@michieltcs
LOAD BALANCER
APP V2
ZERO DOWNTIME DEPLOYMENT
DATABASE STATE?
DATABASE VERSION
@michieltcs
LOAD BALANCER
APP V1
DATABASE V1
expects v1
@michieltcs
LOAD BALANCER
APP V1
DATABASE V2
expects v1
DATABASE VERSION
@michieltcs
LOAD BALANCER
APP V1 APP V2
DATABASE V2
expects v1 expects v2
DATABASE VERSION
OLD APPSHOULD WORK WITH
NEW STATE
DECOUPLE
MIGRATION DECOUPLE
DEPLOYMENT
EXPAND - CONTRACT
BACKWARDS COMPATIBILITY
CLIENT
EXISTING API
@michieltcs
CLIENT
CLIENT
CLIENT
EXISTING API
@michieltcs
CLIENT
CLIENT
NEWAPI
DEPRECATION
CLIENT
EXISTING API
@michieltcs
CLIENT
CLIENT
NEWAPI
CLIENT
EXISTING API
@michieltcs
CLIENT
CLIENT
NEWAPI
CLIENT
EXISTING API
@michieltcs
CLIENT
CLIENT
NEWAPI
CLIENT
@michieltcs
CLIENT
CLIENT
NEWAPI
NON-DESTRUCTIVE CHANGES
@michieltcs
ADDING TABLES
@michieltcs
ADDING TABLES
ADDING COLUMNS
@michieltcs
ADDING TABLES
ADDING COLUMNS
CREATING INDEXES
@michieltcs
ADDING TABLES
ADDING COLUMNS
CREATING INDEXES
sometimes ...
ONE DIFF BACK
EXAMPLE: RENAMING A COLUMN
@michieltcs
CREATE COLUMN WITH
NEW NAME
1
@michieltcs
CREATE COLUMN WITH
NEW NAME
WRITE TO OLD & NEW
COLUMNS
1 2
@michieltcs
CREATE COLUMN WITH
NEW NAME
WRITE TO OLD & NEW
COLUMNS
MIGRATE OLD RECORDS
1 2 3
@michieltcs
CREATE COLUMN WITH
NEW NAME
WRITE TO OLD & NEW
COLUMNS
MIGRATE OLD RECORDS
READ FROM NEW COLUMN
1 2 3 4
@michieltcs
CREATE COLUMN WITH
NEW NAME
WRITE TO OLD & NEW
COLUMNS
MIGRATE OLD RECORDS
READ FROM NEW COLUMN
1 2 3 4
remove old column and
code
FEATURE TOGGLES
@michieltcs
ENABLE/DISABLE FEATURES
@michieltcs
ALTERNATIVE CODE PATHS
ENABLE/DISABLE FEATURES
@michieltcs
ALTERNATIVE CODE PATHS
ENABLE/DISABLE FEATURES
AT RUNTIME
@michieltcs
CREATE COLUMN WITH
NEW NAME
WRITE TO OLD & NEW
COLUMNS
MIGRATE OLD RECORDS
READ FROM NEW COLUMN
1 2 3 4
feature toggle feature toggle
NOSQL
NO SCHEMA
MIGRATE DURING READ
BACKGROUND MIGRATION
NO DOWNTIME
LARGE DATASETS
LONG MIGRATIONS
LONG MIGRATIONS
LONG RUNNING QUERIES
LONG MIGRATIONS
MEMORY USAGELONG RUNNING QUERIES
LONG MIGRATIONS
FAILURES
MEMORY USAGELONG RUNNING QUERIES
LONG MIGRATIONS
FAILURES
MEMORY USAGE
RESTARTING, RESUMING
LONG RUNNING QUERIES
LONG MIGRATIONS
FAILURES
MEMORY USAGE
RESTARTING, RESUMING
LOGGING & PROGRESS
LONG RUNNING QUERIES
QUEUES
@michieltcs
UI
Data Layer
Database
messagesqueries DTOs
@michieltcs
Queue
Backend
messages
@michieltcs
UI
Data Layer
Database
messagesqueries DTOs
@michieltcs
Queue
Backend
messages
CQRS
COMMAND QUERY RESPONSIBILITY SEGREGATION
SEPARATEREADS FROM WRITES
@michieltcs
UI
Event Handlers
CommandsData Layer
Database Database
updates
events
queries DTOs
Repository
@michieltcs
Database
TOOLS
@michieltcs
RECAP
MULTIPLE STEPS
NO BREAKING CHANGES
LITERATURE
databaserefactoring.com
github.com/barsoom/devbook/
