alembic documentation · 2021. 1. 30. · alembic documentation, release 0.7.7 1.3.1dependencies...

Alembic DocumentationRelease 0.7.7

Mike Bayer

July 22, 2015

Contents

1 Front Matter 31.1 Project Homepage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Project Status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.4 Community . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.5 Bugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Tutorial 52.1 The Migration Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 Creating an Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Editing the .ini File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.4 Create a Migration Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.5 Running our First Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102.6 Running our Second Migration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.7 Partial Revision Identifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.8 Relative Migration Identifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.9 Getting Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.10 Downgrading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.11 Next Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

3 Auto Generating Migrations 153.1 What does Autogenerate Detect (and what does it not detect?) . . . . . . . . . . . . . . . . . . . . . 163.2 Comparing and Rendering Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 Generating SQL Scripts (a.k.a. “Offline Mode”) 234.1 Getting the Start Version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Writing Migration Scripts to Support Script Generation . . . . . . . . . . . . . . . . . . . . . . . . 244.3 Customizing the Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

5 The Importance of Naming Constraints 275.1 Integration of Naming Conventions into Operations, Autogenerate . . . . . . . . . . . . . . . . . . . 29

6 Running “Batch” Migrations for SQLite and Other Databases 316.1 Controlling Table Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.2 Dealing with Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326.3 Working in Offline Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.4 Batch mode with Autogenerate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346.5 Batch mode with databases other than SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

i

7 Working with Branches 377.1 Merging Branches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397.2 Working with Explicit Branches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427.3 Working with Multiple Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 457.4 Branch Dependencies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

8 Operation Reference 51

9 Cookbook 679.1 Building an Up to Date Database from Scratch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 679.2 Conditional Migration Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 689.3 Sharing a Connection with a Series of Migration Commands and Environments . . . . . . . . . . . . 70

10 API Details 7310.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7310.2 The Environment Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7410.3 The Migration Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8310.4 The Operations Object . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8510.5 Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8510.6 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8610.7 Script Directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8810.8 Revision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9110.9 Autogeneration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9310.10 DDL Internals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

11 Changelog 9911.1 0.7.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9911.2 0.7.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9911.3 0.7.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10011.4 0.7.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10111.5 0.7.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10211.6 0.7.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10211.7 0.7.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10211.8 0.7.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10311.9 0.6.7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10611.10 0.6.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10711.11 0.6.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10711.12 0.6.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10811.13 0.6.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11011.14 0.6.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11011.15 0.6.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11111.16 0.6.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11211.17 0.5.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11311.18 0.4.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11411.19 0.4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11511.20 0.4.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11511.21 0.3.6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11611.22 0.3.5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11711.23 0.3.4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11711.24 0.3.3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11711.25 0.3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11811.26 0.3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11811.27 0.3.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11811.28 0.2.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11911.29 0.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

ii

11.30 0.2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12011.31 0.1.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12011.32 0.1.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

12 Indices and tables 123

Python Module Index 125

iii

Alembic Documentation, Release 0.7.7

Alembic is a lightweight database migration tool for usage with the SQLAlchemy Database Toolkit for Python.

Contents 1

http://bitbucket.org/zzzeek/alembic

http://www.sqlalchemy.org


2 Contents

CHAPTER 1

Front Matter

Information about the Alembic project.

1.1 Project Homepage

Alembic is hosted on Bitbucket - the lead project page is at https://bitbucket.org/zzzeek/alembic. Source code istracked here using Git.

Changed in version 0.6: The source repository was moved from Mercurial to Git.

Releases and project status are available on Pypi at http://pypi.python.org/pypi/alembic.

The most recent published version of this documentation should be at http://alembic.readthedocs.org/.

1.2 Project Status

Alembic is currently in beta status and is expected to be fairly stable. Users should take care to report bugs and missingfeatures (see Bugs) on an as-needed basis. It should be expected that the development version may be requiredfor proper implementation of recently repaired issues in between releases; the latest master is always available athttps://bitbucket.org/zzzeek/alembic/get/master.tar.gz.

1.3 Installation

Install released versions of Alembic from the Python package index with pip or a similar tool:

pip install alembic

Installation via source distribution is via the setup.py script:

python setup.py install

The install will add the alembic command to the environment. All operations with Alembic then proceed throughthe usage of this command.

3

http://bitbucket.org

https://bitbucket.org/zzzeek/alembic

http://git-scm.com/

http://pypi.python.org/pypi/alembic

http://alembic.readthedocs.org/

https://bitbucket.org/zzzeek/alembic/get/master.tar.gz

http://pypi.python.org/pypi/pip


1.3.1 Dependencies

Alembic’s install process will ensure that SQLAlchemy is installed, in addition to other dependencies. Alembic willwork with SQLAlchemy as of version 0.7.3. The latest version of SQLAlchemy within the 0.7, 0.8, or more recentseries is strongly recommended.

Alembic supports Python versions 2.6 and above.

Changed in version 0.5.0: Support for SQLAlchemy 0.6 has been dropped.

Changed in version 0.6.0: Now supporting Python 2.6 and above.

1.4 Community

Alembic is developed by Mike Bayer, and is loosely associated with the SQLAlchemy and Pylons projects.

User issues, discussion of potential bugs and features should be posted to the Alembic Google Group at sqlalchemy-alembic.

1.5 Bugs

Bugs and feature enhancements to Alembic should be reported on the Bitbucket issue tracker.

4 Chapter 1. Front Matter


http://techspot.zzzeek.org

http://www.sqlalchemy.org/

http://www.pylonsproject.org

https://groups.google.com/group/sqlalchemy-alembic

https://groups.google.com/group/sqlalchemy-alembic

https://bitbucket.org/zzzeek/alembic/issues?status=new&status=open

CHAPTER 2

Tutorial

Alembic provides for the creation, management, and invocation of change management scripts for a relationaldatabase, using SQLAlchemy as the underlying engine. This tutorial will provide a full introduction to the theoryand usage of this tool.

To begin, make sure Alembic is installed as described at Installation.

2.1 The Migration Environment

Usage of Alembic starts with creation of the Migration Environment. This is a directory of scripts that is specific to aparticular application. The migration environment is created just once, and is then maintained along with the applica-tion’s source code itself. The environment is created using the init command of Alembic, and is then customizableto suit the specific needs of the application.

The structure of this environment, including some generated migration scripts, looks like:

yourproject/alembic/

env.pyREADMEscript.py.makoversions/

3512b954651e_add_account.py2b1ae634e5cd_add_order_id.py3adcc9a56557_rename_username_field.py

The directory includes these directories/files:

• yourproject - this is the root of your application’s source code, or some directory within it.

• alembic - this directory lives within your application’s source tree and is the home of the migration environ-ment. It can be named anything, and a project that uses multiple databases may even have more than one.

• env.py - This is a Python script that is run whenever the alembic migration tool is invoked. At the very least,it contains instructions to configure and generate a SQLAlchemy engine, procure a connection from that enginealong with a transaction, and to then invoke the migration engine, using the connection as a source of databaseconnectivity.

The env.py script is part of the generated environment so that the way migrations run is entirely customizable.The exact specifics of how to connect are here, as well as the specifics of how the migration enviroment areinvoked. The script can be modified so that multiple engines can be operated upon, custom arguments can bepassed into the migration environment, application-specific libraries and models can be loaded in and madeavailable.

5

http://bitbucket.org/zzzeek/alembic



Alembic includes a set of initialization templates which feature different varieties of env.py for different usecases.

• README - included with the various enviromnent templates, should have something informative.

• script.py.mako - This is a Mako template file which is used to generate new migration scripts. Whatever ishere is used to generate new files within versions/. This is scriptable so that the structure of each migrationfile can be controlled, including standard imports to be within each, as well as changes to the structure of theupgrade() and downgrade() functions. For example, the multidb environment allows for multiplefunctions to be generated using a naming scheme upgrade_engine1(), upgrade_engine2().

• versions/ - This directory holds the individual version scripts. Users of other migration tools may noticethat the files here don’t use ascending integers, and instead use a partial GUID approach. In Alembic, theordering of version scripts is relative to directives within the scripts themselves, and it is theoretically possibleto “splice” version files in between others, allowing migration sequences from different branches to be merged,albeit carefully by hand.

2.2 Creating an Environment

With a basic understanding of what the environment is, we can create one using alembic init. This will create anenvironment using the “generic” template:

$ cd yourproject$ alembic init alembic

Where above, the init command was called to generate a migrations directory called alembic:

Creating directory /path/to/yourproject/alembic...doneCreating directory /path/to/yourproject/alembic/versions...doneGenerating /path/to/yourproject/alembic.ini...doneGenerating /path/to/yourproject/alembic/env.py...doneGenerating /path/to/yourproject/alembic/README...doneGenerating /path/to/yourproject/alembic/script.py.mako...donePlease edit configuration/connection/logging settings in'/path/to/yourproject/alembic.ini' before proceeding.

Alembic also includes other environment templates. These can be listed out using the list_templates command:

$ alembic list_templatesAvailable templates:

generic - Generic single-database configuration.multidb - Rudimentary multi-database configuration.pylons - Configuration that reads from a Pylons project environment.

Templates are used via the 'init' command, e.g.:

alembic init --template pylons ./scripts

2.3 Editing the .ini File

Alembic placed a file alembic.ini into the current directory. This is a file that the alembic script looks for wheninvoked. This file can be anywhere, either in the same directory from which the alembic script will normally beinvoked, or if in a different directory, can be specified by using the --config option to the alembic runner.

6 Chapter 2. Tutorial

http://www.makotemplates.org


The file generated with the “generic” configuration looks like:

# A generic, single database configuration.

[alembic]# path to migration scriptsscript_location = alembic

# template used to generate migration files# file_template = %%(rev)s_%%(slug)s

# max length of characters to apply to the# "slug" field#truncate_slug_length = 40

# set to 'true' to run the environment during# the 'revision' command, regardless of autogenerate# revision_environment = false

# set to 'true' to allow .pyc and .pyo files without# a source .py file to be detected as revisions in the# versions/ directory# sourceless = false

# version location specification; this defaults# to alembic/versions. When using multiple version# directories, initial revisions must be specified with --version-path# version_locations = %(here)s/bar %(here)s/bat alembic/versions

# the output encoding used when revision files# are written from script.py.mako# output_encoding = utf-8

sqlalchemy.url = driver://user:pass@localhost/dbname

# Logging configuration[loggers]keys = root,sqlalchemy,alembic

[handlers]keys = console

[formatters]keys = generic

[logger_root]level = WARNhandlers = consolequalname =

[logger_sqlalchemy]level = WARNhandlers =qualname = sqlalchemy.engine

[logger_alembic]level = INFOhandlers =qualname = alembic

2.3. Editing the .ini File 7


[handler_console]class = StreamHandlerargs = (sys.stderr,)level = NOTSETformatter = generic

[formatter_generic]format = %(levelname)-5.5s [%(name)s] %(message)sdatefmt = %H:%M:%S

The file is read using Python’s ConfigParser.SafeConfigParser object. The %(here)s variable is pro-vided as a substitution variable, which can be used to produce absolute pathnames to directories and files, as we doabove with the path to the Alembic script location.

This file contains the following features:

• [alembic] - this is the section read by Alembic to determine configuration. Alembic itself does not directlyread any other areas of the file.

• script_location - this is the location of the Alembic environment. It is normally specified as a filesystemlocation, either relative or absolute. If the location is a relative path, it’s interpreted as relative to the currentdirectory.

This is the only key required by Alembic in all cases. The generation of the .ini file by the command alembicinit alembic automatically placed the directory name alembic here. The special variable %(here)scan also be used, as in %(here)s/alembic.

For support of applications that package themselves into .egg files, the value can also be specified as a packageresource, in which case resource_filename() is used to find the file (new in 0.2.2). Any non-absoluteURI which contains colons is interpreted here as a resource name, rather than a straight filename.

• file_template - this is the naming scheme used to generate new migration files. The value present is thedefault, so is commented out. Tokens available include:

– %%(rev)s - revision id

– %%(slug)s - a truncated string derived from the revision message

– %%(year)d, %%(month).2d, %%(day).2d, %%(hour).2d, %%(minute).2d,%%(second).2d - components of the create date as returned by datetime.datetime.now()

• truncate_slug_length - defaults to 40, the max number of characters to include in the “slug” field.

New in version 0.6.1: - added truncate_slug_length configuration

• sqlalchemy.url - A URL to connect to the database via SQLAlchemy. This key is in fact only referencedwithin the env.py file that is specific to the “generic” configuration; a file that can be customized by thedeveloper. A multiple database configuration may respond to multiple keys here, or may reference other sectionsof the file.

• revision_environment - this is a flag which when set to the value ‘true’, will indicate that the migrationenvironment script env.py should be run unconditionally when generating new revision files

• sourceless - when set to ‘true’, revision files that only exist as .pyc or .pyo files in the versions directorywill be used as versions, allowing “sourceless” versioning folders. When left at the default of ‘false’, only .pyfiles are consumed as version files.

New in version 0.6.4.

• version_locations - an optional list of revision file locations, to allow revisions to exist in multipledirectories simultaneously. See Working with Multiple Bases for examples.



https://pythonhosted.org/setuptools/pkg_resources.html

https://pythonhosted.org/setuptools/pkg_resources.html


• output_encoding - the encoding to use when Alembic writes the script.py.mako file into a newmigration file. Defaults to ’utf-8’.


• [loggers], [handlers], [formatters], [logger_*], [handler_*], [formatter_*] - thesesections are all part of Python’s standard logging configuration, the mechanics of which are documented atConfiguration File Format. As is the case with the database connection, these directives are used directly as theresult of the logging.config.fileConfig() call present in the env.py script, which you’re free tomodify.

For starting up with just a single database and the generic configuration, setting up the SQLAlchemy URL is all that’sneeded:

sqlalchemy.url = postgresql://scott:tiger@localhost/test

2.4 Create a Migration Script

With the environment in place we can create a new revision, using alembic revision:

$ alembic revision -m "create account table"Generating /path/to/yourproject/alembic/versions/1975ea83b712_create_account_table.py...done

A new file 1975ea83b712_create_account_table.py is generated. Looking inside the file:

"""create account table

Revision ID: 1975ea83b712Revises:Create Date: 2011-11-08 11:40:27.089406

"""

# revision identifiers, used by Alembic.revision = '1975ea83b712'down_revision = Nonebranch_labels = None

from alembic import opimport sqlalchemy as sa

def upgrade():pass

def downgrade():pass

The file contains some header information, identifiers for the current revision and a “downgrade” revision, an importof basic Alembic directives, and empty upgrade() and downgrade() functions. Our job here is to populate theupgrade() and downgrade() functions with directives that will apply a set of changes to our database. Typically,upgrade() is required while downgrade() is only needed if down-revision capability is desired, though it’sprobably a good idea.

Another thing to notice is the down_revision variable. This is how Alembic knows the correct order in which toapply migrations. When we create the next revision, the new file’s down_revision identifier would point to thisone:

2.4. Create a Migration Script 9

http://docs.python.org/library/logging.config.html#configuration-file-format


# revision identifiers, used by Alembic.revision = 'ae1027a6acf'down_revision = '1975ea83b712'

Every time Alembic runs an operation against the versions/ directory, it reads all the files in, and composes a listbased on how the down_revision identifiers link together, with the down_revision of None representing thefirst file. In theory, if a migration environment had thousands of migrations, this could begin to add some latency tostartup, but in practice a project should probably prune old migrations anyway (see the section Building an Up to DateDatabase from Scratch for a description on how to do this, while maintaining the ability to build the current databasefully).

We can then add some directives to our script, suppose adding a new table account:

def upgrade():op.create_table(

'account',sa.Column('id', sa.Integer, primary_key=True),sa.Column('name', sa.String(50), nullable=False),sa.Column('description', sa.Unicode(200)),

)

def downgrade():op.drop_table('account')

create_table() and drop_table() are Alembic directives. Alembic provides all the basic database migrationoperations via these directives, which are designed to be as simple and minimalistic as possible; there’s no relianceupon existing table metadata for most of these directives. They draw upon a global “context” that indicates how toget at a database connection (if any; migrations can dump SQL/DDL directives to files as well) in order to invoke thecommand. This global context is set up, like everything else, in the env.py script.

An overview of all Alembic directives is at Operation Reference.

2.5 Running our First Migration

We now want to run our migration. Assuming our database is totally clean, it’s as yet unversioned. The alembicupgrade command will run upgrade operations, proceeding from the current database revision, in this exampleNone, to the given target revision. We can specify 1975ea83b712 as the revision we’d like to upgrade to, but it’seasier in most cases just to tell it “the most recent”, in this case head:

$ alembic upgrade headINFO [alembic.context] Context class PostgresqlContext.INFO [alembic.context] Will assume transactional DDL.INFO [alembic.context] Running upgrade None -> 1975ea83b712

Wow that rocked! Note that the information we see on the screen is the result of the logging configuration set up inalembic.ini - logging the alembic stream to the console (standard error, specifically).

The process which occurred here included that Alembic first checked if the database had a table calledalembic_version, and if not, created it. It looks in this table for the current version, if any, and then calcu-lates the path from this version to the version requested, in this case head, which is known to be 1975ea83b712.It then invokes the upgrade() method in each file to get to the target revision.



2.6 Running our Second Migration

Let’s do another one so we have some things to play with. We again create a revision file:

$ alembic revision -m "Add a column"Generating /path/to/yourapp/alembic/versions/ae1027a6acf_add_a_column.py...done

Let’s edit this file and add a new column to the account table:

"""Add a column

Revision ID: ae1027a6acfRevises: 1975ea83b712Create Date: 2011-11-08 12:37:36.714947

"""

# revision identifiers, used by Alembic.revision = 'ae1027a6acf'down_revision = '1975ea83b712'


def upgrade():op.add_column('account', sa.Column('last_transaction_date', sa.DateTime))

def downgrade():op.drop_column('account', 'last_transaction_date')

Running again to head:

$ alembic upgrade headINFO [alembic.context] Context class PostgresqlContext.INFO [alembic.context] Will assume transactional DDL.INFO [alembic.context] Running upgrade 1975ea83b712 -> ae1027a6acf

We’ve now added the last_transaction_date column to the database.

2.7 Partial Revision Identifiers

Any time we need to refer to a revision number explicitly, we have the option to use a partial number. As long asthis number uniquely identifies the version, it may be used in any command in any place that version numbers areaccepted:

$ alembic upgrade ae1

Above, we use ae1 to refer to revision ae1027a6acf. Alembic will stop and let you know if more than one versionstarts with that prefix.

2.8 Relative Migration Identifiers

Relative upgrades/downgrades are also supported. To move two versions from the current, a decimal value “+N” canbe supplied:

2.6. Running our Second Migration 11


$ alembic upgrade +2

Negative values are accepted for downgrades:

$ alembic downgrade -1

Relative identifiers may also be in terms of a specific revision. For example, to upgrade to revision ae1027a6acfplus two additional steps:

$ alembic upgrade ae10+2

New in version 0.7.0: Support for relative migrations in terms of a specific revision.

2.9 Getting Information

With a few revisions present we can get some information about the state of things.

First we can view the current revision:

$ alembic currentINFO [alembic.context] Context class PostgresqlContext.INFO [alembic.context] Will assume transactional DDL.Current revision for postgresql://scott:XXXXX@localhost/test: 1975ea83b712 -> ae1027a6acf (head), Add a column

head is displayed only if the revision identifier for this database matches the head revision.

We can also view history with alembic history; the --verbose option (accepted by several commands, in-cluding history, current, heads and branches) will show us full information about each revision:

$ alembic history --verbose

Rev: ae1027a6acf (head)Parent: 1975ea83b712Path: /path/to/yourproject/alembic/versions/ae1027a6acf_add_a_column.py

add a column


Rev: 1975ea83b712Parent: <base>Path: /path/to/yourproject/alembic/versions/1975ea83b712_add_account_table.py

create account table


2.9.1 Viewing History Ranges

Using the -r option to alembic history, we can also view various slices of history. The -r argument acceptsan argument [start]:[end], where either may be a revision number, symbols like head, heads or base,



current to specify the current revision(s), as well as negative relative ranges for [start] and positive relativeranges for [end]:

$ alembic history -r1975ea:ae1027

A relative range starting from three revs ago up to current migration, which will invoke the migration environmentagainst the database to get the current migration:

$ alembic history -r-3:current

View all revisions from 1975 to the head:

$ alembic history -r1975ea:

New in version 0.6.0: alembic revision now accepts the -r argument to specify specific ranges based onversion numbers, symbols, or relative deltas.

2.10 Downgrading

We can illustrate a downgrade back to nothing, by calling alembic downgrade back to the beginning, which inAlembic is called base:

$ alembic downgrade baseINFO [alembic.context] Context class PostgresqlContext.INFO [alembic.context] Will assume transactional DDL.INFO [alembic.context] Running downgrade ae1027a6acf -> 1975ea83b712INFO [alembic.context] Running downgrade 1975ea83b712 -> None

Back to nothing - and up again:

$ alembic upgrade headINFO [alembic.context] Context class PostgresqlContext.INFO [alembic.context] Will assume transactional DDL.INFO [alembic.context] Running upgrade None -> 1975ea83b712INFO [alembic.context] Running upgrade 1975ea83b712 -> ae1027a6acf

2.11 Next Steps

The vast majority of Alembic environments make heavy use of the “autogenerate” feature. Continue onto the nextsection, Auto Generating Migrations.

2.10. Downgrading 13

CHAPTER 3

Auto Generating Migrations

Alembic can view the status of the database and compare against the table metadata in the application, generating the“obvious” migrations based on a comparison. This is achieved using the --autogenerate option to the alembicrevision command, which places so-called candidate migrations into our new migrations file. We review andmodify these by hand as needed, then proceed normally.

To use autogenerate, we first need to modify our env.py so that it gets access to a table metadata object that containsthe target. Suppose our application has a declarative base in myapp.mymodel. This base contains a MetaDataobject which contains Table objects defining our database. We make sure this is loaded in env.py and then passedto EnvironmentContext.configure() via the target_metadata argument. The env.py sample scriptused in the generic template already has a variable declaration near the top for our convenience, where we replaceNone with our MetaData. Starting with:

# add your model's MetaData object here# for 'autogenerate' support# from myapp import mymodel# target_metadata = mymodel.Base.metadatatarget_metadata = None

we change to:

from myapp.mymodel import Basetarget_metadata = Base.metadata

Note: The above example refers to the generic alembic env.py template, e.g. the one created by default whencalling upon alembic init, and not the special-use templates such as multidb. Please consult the source codeand comments within the env.py script directly for specific guidance on where and how the autogenerate metadatais established.

If we look later in the script, down in run_migrations_online(), we can see the directive passed toEnvironmentContext.configure():

def run_migrations_online():engine = engine_from_config(

config.get_section(config.config_ini_section), prefix='sqlalchemy.')

with engine.connect() as connection:context.configure(

connection=connection,target_metadata=target_metadata)

15

http://www.sqlalchemy.org/docs/orm/extensions/declarative.html#synopsis

http://www.sqlalchemy.org/docs/core/metadata.html#sqlalchemy.schema.MetaData

http://www.sqlalchemy.org/docs/core/metadata.html#sqlalchemy.schema.Table



with context.begin_transaction():context.run_migrations()

We can then use the alembic revision command in conjunction with the --autogenerate option. Supposeour MetaData contained a definition for the account table, and the database did not. We’d get output like:

$ alembic revision --autogenerate -m "Added account table"INFO [alembic.context] Detected added table 'account'Generating /path/to/foo/alembic/versions/27c6a30d7c24.py...done

We can then view our file 27c6a30d7c24.py and see that a rudimentary migration is already present:

"""empty message

Revision ID: 27c6a30d7c24Revises: NoneCreate Date: 2011-11-08 11:40:27.089406

"""

# revision identifiers, used by Alembic.revision = '27c6a30d7c24'down_revision = None


def upgrade():### commands auto generated by Alembic - please adjust! ###op.create_table('account',sa.Column('id', sa.Integer()),sa.Column('name', sa.String(length=50), nullable=False),sa.Column('description', sa.VARCHAR(200)),sa.Column('last_transaction_date', sa.DateTime()),sa.PrimaryKeyConstraint('id'))### end Alembic commands ###

def downgrade():### commands auto generated by Alembic - please adjust! ###op.drop_table("account")### end Alembic commands ###

The migration hasn’t actually run yet, of course. We do that via the usual upgrade command. We should also gointo our migration file and alter it as needed, including adjustments to the directives as well as the addition of otherdirectives which these may be dependent on - specifically data changes in between creates/alters/drops.

3.1 What does Autogenerate Detect (and what does it not detect?)

The vast majority of user issues with Alembic centers on the topic of what kinds of changes autogenerate can and can-not detect reliably, as well as how it renders Python code for what it does detect. it is critical to note that autogenerateis not intended to be perfect. It is always necessary to manually review and correct the candidate migrations thatautogenererate produces. The feature is getting more and more comprehensive and error-free as releases continue, butone should take note of the current limitations.

Autogenerate will detect:

16 Chapter 3. Auto Generating Migrations



• Table additions, removals.

• Column additions, removals.

• Change of nullable status on columns.

• Basic changes in indexes and explcitly-named unique constraints

New in version 0.6.1: Support for autogenerate of indexes and unique constraints.

• Basic changes in foreign key constraints

New in version 0.7.1: Support for autogenerate of foreign key constraints.

Autogenerate can optionally detect:

• Change of column type. This will occur if you set the EnvironmentContext.configure.compare_typeparameter to True, or to a custom callable function. The feature works well in most cases, but is off by defaultso that it can be tested on the target schema first. It can also be customized by passing a callable here; see thesection Comparing Types for details.

• Change of server default. This will occur if you set the EnvironmentContext.configure.compare_server_defaultparameter to True, or to a custom callable function. This feature works well for simple cases but cannotalways produce accurate results. The Postgresql backend will actually invoke the “detected” and “metadata”values against the database to determine equivalence. The feature is off by default so that it can be tested on thetarget schema first. Like type comparison, it can also be customized by passing a callable; see the function’sdocumentation for details.

Autogenerate can not detect:

• Changes of table name. These will come out as an add/drop of two different tables, and should be hand-editedinto a name change instead.

• Changes of column name. Like table name changes, these are detected as a column add/drop pair, which is notat all the same as a name change.

• Anonymously named constraints. Give your constraints a name, e.g. UniqueConstraint(’col1’,’col2’, name="my_name"). See the section The Importance of Naming Constraints for background onhow to configure automatic naming schemes for constraints.

• Special SQLAlchemy types such as Enumwhen generated on a backend which doesn’t support ENUM directly -this because the representation of such a type in the non-supporting database, i.e. a CHAR+ CHECK constraint,could be any kind of CHAR+CHECK. For SQLAlchemy to determine that this is actually an ENUM would onlybe a guess, something that’s generally a bad idea. To implement your own “guessing” function here, use thesqlalchemy.events.DDLEvents.column_reflect() event to detect when a CHAR (or whateverthe target type is) is reflected, and change it to an ENUM (or whatever type is desired) if it is known thatthat’s the intent of the type. The sqlalchemy.events.DDLEvents.after_parent_attach() canbe used within the autogenerate process to intercept and un-attach unwanted CHECK constraints.

Autogenerate can’t currently, but will eventually detect:

• Some free-standing constraint additions and removals, like CHECK, PRIMARY KEY - these are not fullyimplemented.

• Sequence additions, removals - not yet implemented.

3.2 Comparing and Rendering Types

The area of autogenerate’s behavior of comparing and rendering Python-based type objects in migration scriptspresents a challenge, in that there’s a very wide variety of types to be rendered in scripts, including those part ofSQLAlchemy as well as user-defined types. A few options are given to help out with this task.

3.2. Comparing and Rendering Types 17

http://www.sqlalchemy.org/docs/core/type_basics.html#sqlalchemy.types.Enum

http://www.sqlalchemy.org/docs/core/events.html#sqlalchemy.events.DDLEvents.column_reflect

http://www.sqlalchemy.org/docs/core/events.html#sqlalchemy.events.DDLEvents.after_parent_attach


3.2.1 Controlling the Module Prefix

When types are rendered, they are generated with a module prefix, so that they are available based on a relativelysmall number of imports. The rules for what the prefix is is based on the kind of datatype as well as configurationalsettings. For example, when Alembic renders SQLAlchemy types, it will by default prefix the type name with theprefix sa.:

Column("my_column", sa.Integer())

The use of the sa. prefix is controllable by altering the value ofEnvironmentContext.configure.sqlalchemy_module_prefix:

def run_migrations_online():# ...

context.configure(connection=connection,target_metadata=target_metadata,sqlalchemy_module_prefix="sqla.",# ...)

# ...

In either case, the sa. prefix, or whatever prefix is desired, should also be included in the imports section ofscript.py.mako; it also defaults to import sqlalchemy as sa.

For user-defined types, that is, any custom type that is not within the sqlalchemy. module namespace, by defaultAlembic will use the value of __module__ for the custom type:

Column("my_column", myapp.models.utils.types.MyCustomType())

The imports for the above type again must be made present within the migration, either manually, or by adding it toscript.py.mako.

Changed in version 0.7.0: The default module prefix rendering for a user-defined type now makes use of the type’s__module__ attribute to retrieve the prefix, rather than using the value of sqlalchemy_module_prefix.

The above custom type has a long and cumbersome name based on the use of __module__ directly, which alsoimplies that lots of imports would be needed in order to accomodate lots of types. For this reason, it is recom-mended that user-defined types used in migration scripts be made available from a single module. Suppose we call itmyapp.migration_types:

# myapp/migration_types.py

from myapp.models.utils.types import MyCustomType

We can first add an import for migration_types to our script.py.mako:

from alembic import opimport sqlalchemy as saimport myapp.migration_types${imports if imports else ""}

We then override Alembic’s use of __module__ by providing a fixed prefix, using theEnvironmentContext.configure.user_module_prefix option:




context.configure(connection=connection,target_metadata=target_metadata,user_module_prefix="myapp.migration_types.",# ...)

# ...

Above, we now would get a migration like:

Column("my_column", myapp.migration_types.MyCustomType())

Now, when we inevitably refactor our application to move MyCustomType somewhere else, we only need modify themyapp.migration_types module, instead of searching and replacing all instances within our migration scripts.

New in version 0.6.3: Added EnvironmentContext.configure.user_module_prefix.

3.2.2 Affecting the Rendering of Types Themselves

The methodology Alembic uses to generate SQLAlchemy and user-defined type constructs as Python code is plain old__repr__(). SQLAlchemy’s built-in types for the most part have a __repr__() that faithfully renders a Python-compatible constructor call, but there are some exceptions, particularly in those cases when a constructor acceptsarguments that aren’t compatible with __repr__(), such as a pickling function.

When building a custom type that will be rendered into a migration script, it is often necessary to explicitly givethe type a __repr__() that will faithfully reproduce the constructor for that type. This, in combination withEnvironmentContext.configure.user_module_prefix, is usually enough. However, if additional be-haviors are needed, a more comprehensive hook is the EnvironmentContext.configure.render_itemoption. This hook allows one to provide a callable function within env.py that will fully take over how a type isrendered, including its module prefix:

def render_item(type_, obj, autogen_context):"""Apply custom rendering for selected items."""

if type_ == 'type' and isinstance(obj, MySpecialType):return "mypackage.%r" % obj

# default rendering for other objectsreturn False


context.configure(connection=connection,target_metadata=target_metadata,render_item=render_item,# ...)

# ...

In the above example, we’d ensure our MySpecialType includes an appropriate __repr__() method, which isinvoked when we call it against "%r".

The callable we use for EnvironmentContext.configure.render_item can also add imports to our mi-gration script. The autogen_context passed in contains an entry called autogen_context[’imports’],



which is a Python set() for which we can add new imports. For example, if MySpecialType were in a modulecalled mymodel.types, we can add the import for it as we encounter the type:

def render_item(type_, obj, autogen_context):"""Apply custom rendering for selected items."""

if type_ == 'type' and isinstance(obj, MySpecialType):# add import for this typeautogen_context['imports'].add("from mymodel import types")return "types.%r" % obj

# default rendering for other objectsreturn False

The finished migration script will include our imports where the ${imports} expression is used, producing outputsuch as:

from alembic import opimport sqlalchemy as safrom mymodel import types

def upgrade():op.add_column('sometable', Column('mycolumn', types.MySpecialType()))

3.2.3 Comparing Types

The default type comparison logic will work for SQLAlchemy built in types as well as basic user defined types. Thislogic is only enabled if the EnvironmentContext.configure.compare_type parameter is set to True:

context.configure(# ...compare_type = True

)

Alternatively, the EnvironmentContext.configure.compare_type parameter accepts a callable functionwhich may be used to implement custom type comparison logic, for cases such as where special user defined types arebeing used:

def my_compare_type(context, inspected_column,metadata_column, inspected_type, metadata_type):

# return True if the types are different,# False if not, or None to allow the default implementation# to compare these typesreturn None

context.configure(# ...compare_type = my_compare_type

)

Above, inspected_column is a sqlalchemy.schema.Column as returned bysqlalchemy.engine.reflection.Inspector.reflecttable(), whereas metadata_columnis a sqlalchemy.schema.Column from the local model environment. A return value of None indicates thatdefault type comparison to proceed.

Additionally, custom types that are part of imported or third party packages which have special behaviors such asper-dialect behavior should implement a method called compare_against_backend() on their SQLAlchemy


http://www.sqlalchemy.org/docs/core/metadata.html#sqlalchemy.schema.Column

http://www.sqlalchemy.org/docs/core/reflection.html#sqlalchemy.engine.reflection.Inspector.reflecttable



type. If this method is present, it will be called where it can also return True or False to specify the types compare asequivalent or not; if it returns None, default type comparison logic will proceed:

class MySpecialType(TypeDecorator):

# ...

def compare_against_backend(self, dialect, conn_type):# return True if the types are different,# False if not, or None to allow the default implementation# to compare these typesif dialect.name == 'postgresql':

return isinstance(conn_type, postgresql.UUID)else:

return isinstance(conn_type, String)

The order of precedence regarding the EnvironmentContext.configure.compare_typecallable vs. the type itself implementing compare_against_backend is that theEnvironmentContext.configure.compare_type callable is favored first; if it returns None, thenthe compare_against_backend method will be used, if present on the metadata type. If that returns None,then a basic check for type equivalence is run.

New in version 0.7.6: - added support for the compare_against_backend() method.


CHAPTER 4

Generating SQL Scripts (a.k.a. “Offline Mode”)

A major capability of Alembic is to generate migrations as SQL scripts, instead of running them against the database- this is also referred to as offline mode. This is a critical feature when working in large organizations where accessto DDL is restricted, and SQL scripts must be handed off to DBAs. Alembic makes this easy via the --sql optionpassed to any upgrade or downgrade command. We can, for example, generate a script that revises up to revae1027a6acf:

$ alembic upgrade ae1027a6acf --sqlINFO [alembic.context] Context class PostgresqlContext.INFO [alembic.context] Will assume transactional DDL.BEGIN;

CREATE TABLE alembic_version (version_num VARCHAR(32) NOT NULL

);

INFO [alembic.context] Running upgrade None -> 1975ea83b712CREATE TABLE account (

id SERIAL NOT NULL,name VARCHAR(50) NOT NULL,description VARCHAR(200),PRIMARY KEY (id)

);

INFO [alembic.context] Running upgrade 1975ea83b712 -> ae1027a6acfALTER TABLE account ADD COLUMN last_transaction_date TIMESTAMP WITHOUT TIME ZONE;

INSERT INTO alembic_version (version_num) VALUES ('ae1027a6acf');

COMMIT;

While the logging configuration dumped to standard error, the actual script was dumped to standard output - so in theabsence of further configuration (described later in this section), we’d at first be using output redirection to generate ascript:

$ alembic upgrade ae1027a6acf --sql > migration.sql

4.1 Getting the Start Version

Notice that our migration script started at the base - this is the default when using offline mode, as no databaseconnection is present and there’s no alembic_version table to read from.

23


One way to provide a starting version in offline mode is to provide a range to the command line. This is accomplishedby providing the “version” in start:end syntax:

$ alembic upgrade 1975ea83b712:ae1027a6acf --sql > migration.sql

The start:end syntax is only allowed in offline mode; in “online” mode, the alembic_version table is alwaysused to get at the current version.

It’s also possible to have the env.py script retrieve the “last” version from the local environment, such as from alocal file. A scheme like this would basically treat a local file in the same way alembic_version works:

if context.is_offline_mode():version_file = os.path.join(os.path.dirname(config.config_file_name), "version.txt")if os.path.exists(version_file):

current_version = open(version_file).read()else:

current_version = Nonecontext.configure(dialect_name=engine.name, starting_rev=current_version)context.run_migrations()end_version = context.get_revision_argument()if end_version and end_version != current_version:

open(version_file, 'w').write(end_version)

4.2 Writing Migration Scripts to Support Script Generation

The challenge of SQL script generation is that the scripts we generate can’t rely upon any client/server database access.This means a migration script that pulls some rows into memory via a SELECT statement will not work in --sqlmode. It’s also important that the Alembic directives, all of which are designed specifically to work in both “liveexecution” as well as “offline SQL generation” mode, are used.

4.3 Customizing the Environment

Users of the --sql option are encouraged to hack their env.py files to suit their needs. Theenv.py script as provided is broken into two sections: run_migrations_online() andrun_migrations_offline(). Which function is run is determined at the bottom of the script by read-ing EnvironmentContext.is_offline_mode(), which basically determines if the --sql flag wasenabled.

For example, a multiple database configuration may want to run through each database and set the output of themigrations to different named files - the EnvironmentContext.configure() function accepts a parameteroutput_buffer for this purpose. Below we illustrate this within the run_migrations_offline() function:

from alembic import contextimport myappimport sys

db_1 = myapp.db_1db_2 = myapp.db_2

def run_migrations_offline():"""Run migrations *without* a SQL connection."""

for name, engine, file_ in [("db1", db_1, "db1.sql"),

24 Chapter 4. Generating SQL Scripts (a.k.a. “Offline Mode”)


("db2", db_2, "db2.sql"),]:

context.configure(url=engine.url,transactional_ddl=False,output_buffer=open(file_, 'w'))

context.execute("-- running migrations for '%s'" % name)context.run_migrations(name=name)sys.stderr.write("Wrote file '%s'" % file_)

def run_migrations_online():"""Run migrations *with* a SQL connection."""

for name, engine in [("db1", db_1),("db2", db_2),

]:connection = engine.connect()context.configure(connection=connection)try:

context.run_migrations(name=name)session.commit()

except:session.rollback()raise

if context.is_offline_mode():run_migrations_offline()

else:run_migrations_online()

4.3. Customizing the Environment 25


26 Chapter 4. Generating SQL Scripts (a.k.a. “Offline Mode”)

CHAPTER 5

The Importance of Naming Constraints

An important topic worth mentioning is that of constraint naming conventions. As we’ve proceeded here, we’ve talkedabout adding tables and columns, and we’ve also hinted at lots of other operations listed in Operation Referencesuch as those which support adding or dropping constraints like foreign keys and unique constraints. The way theseconstraints are referred to in migration scripts is by name, however these names by default are in most cases generatedby the relational database in use, when the constraint is created. For example, if you emitted two CREATE TABLEstatements like this on Postgresql:

test=> CREATE TABLE user_account (id INTEGER PRIMARY KEY);CREATE TABLEtest=> CREATE TABLE user_order (test(> id INTEGER PRIMARY KEY,test(> user_account_id INTEGER REFERENCES user_account(id));CREATE TABLE

Suppose we wanted to DROP the REFERENCES that we just applied to the user_order.user_account_idcolumn, how do we do that? At the prompt, we’d use ALTER TABLE <tablename> DROP CONSTRAINT<constraint_name>, or if using Alembic we’d be using Operations.drop_constraint(). But bothof those functions need a name - what’s the name of this constraint?

It does have a name, which in this case we can figure out by looking at the Postgresql catalog tables:

test=> SELECT r.conname FROMtest-> pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespacetest-> JOIN pg_catalog.pg_constraint r ON c.oid = r.conrelidtest-> WHERE c.relname='user_order' AND r.contype = 'f'test-> ;

conname---------------------------------user_order_user_account_id_fkey

(1 row)

The name above is not something that Alembic or SQLAlchemy created;user_order_user_account_id_fkey is a naming scheme used internally by Postgresql to name con-straints that are otherwise not named.

This scheme doesn’t seem so complicated, and we might want to just use our knowledge of it so that we know whatname to use for our Operations.drop_constraint() call. But is that a good idea? What if for example weneeded our code to run on Oracle as well. OK, certainly Oracle uses this same scheme, right? Or if not, somethingsimilar. Let’s check:

Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production

SQL> CREATE TABLE user_account (id INTEGER PRIMARY KEY);

27


Table created.

SQL> CREATE TABLE user_order (2 id INTEGER PRIMARY KEY,3 user_account_id INTEGER REFERENCES user_account(id));

Table created.

SQL> SELECT constraint_name FROM all_constraints WHERE2 table_name='USER_ORDER' AND constraint_type in ('R');

CONSTRAINT_NAME-----------------------------------------------------SYS_C0029334

Oh, we can see that is.....much worse. Oracle’s names are entirely unpredictable alphanumeric codes, and this willmake being able to write migrations quite tedious, as we’d need to look up all these names.

The solution to having to look up names is to make your own names. This is an easy, though tedious thing to domanually. For example, to create our model in SQLAlchemy ensuring we use names for foreign key constraints wouldlook like:

from sqlalchemy import MetaData, Table, Column, Integer, ForeignKey

meta = MetaData()

user_account = Table('user_account', meta,Column('id', Integer, primary_key=True)

)

user_order = Table('user_order', meta,Column('id', Integer, primary_key=True),Column('user_order_id', Integer,ForeignKey('user_account.id', name='fk_user_order_id'))

)

Simple enough, though this has some disadvantages. The first is that it’s tedious; we need to remember to use a namefor every ForeignKey object, not to mention every UniqueConstraint, CheckConstraint, Index, andmaybe even PrimaryKeyConstraint as well if we wish to be able to alter those too, and beyond all that, all thenames have to be globally unique. Even with all that effort, if we have a naming scheme in mind, it’s easy to get itwrong when doing it manually each time.

What’s worse is that manually naming constraints (and indexes) gets even more tedious in that we can no longer useconvenience features such as the .unique=True or .index=True flag on Column:

user_account = Table('user_account', meta,Column('id', Integer, primary_key=True),Column('name', String(50), unique=True)

)

Above, the unique=True flag creates a UniqueConstraint, but again, it’s not named. If we want to name it,manually we have to forego the usage of unique=True and type out the whole constraint:

user_account = Table('user_account', meta,Column('id', Integer, primary_key=True),Column('name', String(50)),UniqueConstraint('name', name='uq_user_account_name')

)

28 Chapter 5. The Importance of Naming Constraints

http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.ForeignKey

http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.UniqueConstraint

http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.CheckConstraint

http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.Index

http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.PrimaryKeyConstraint




There’s a solution to all this naming work, which is to use an automated naming convention. For someyears, SQLAlchemy has encourgaged the use of DDL Events in order to create naming schemes. Theafter_parent_attach() event in particular is the best place to intercept when Constraint and Indexobjects are being associated with a parent Table object, and to assign a .name to the constraint while making use ofthe name of the table and associated columns.

But there is also a better way to go, which is to make use of a feature new in SQLAlchemy 0.9.2 which makes use ofthe events behind the scenes known as naming_convention. Here, we can create a new MetaData object whilepassing a dictionary referring to a naming scheme:

convention = {"ix": 'ix_%(column_0_label)s',"uq": "uq_%(table_name)s_%(column_0_name)s","ck": "ck_%(table_name)s_%(constraint_name)s","fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s","pk": "pk_%(table_name)s"

}

metadata = MetaData(naming_convention=convention)

If we define our models using a MetaData as above, the given naming convention dictionary will be used to providenames for all constraints and indexes.

5.1 Integration of Naming Conventions into Operations, Autogener-ate

As of Alembic 0.6.4, the naming convention feature is integrated into the Operations object, so that the conventiontakes effect for any constraint that is otherwise unnamed. The naming convention is passed to Operations using theMigrationsContext.configure.target_metadata parameter in env.py, which is normally configuredwhen autogenerate is used:

# in your application's model:

meta = MetaData(naming_convention={"ix": 'ix_%(column_0_label)s',"uq": "uq_%(table_name)s_%(column_0_name)s","ck": "ck_%(table_name)s_%(constraint_name)s","fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s","pk": "pk_%(table_name)s"

})

# .. in your Alembic env.py:

# add your model's MetaData object here# for 'autogenerate' supportfrom myapp import mymodeltarget_metadata = mymodel.Base.metadata

# ...

def run_migrations_online():

# ...

context.configure(connection=connection,

5.1. Integration of Naming Conventions into Operations, Autogenerate 29

http://www.sqlalchemy.org/docs/core/events.html#sqlalchemy.events.DDLEvents.after_parent_attach

http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.Constraint






target_metadata=target_metadata)

Above, when we render a directive like the following:

op.add_column('sometable', Column('q', Boolean(name='q_bool')))

The Boolean type will render a CHECK constraint with the name "ck_sometable_q_bool", assuming the back-end in use does not support native boolean types.

We can also use op directives with constraints and not give them a name at all, if the naming convention doesn’t requireone. The value of None will be converted into a name that follows the appopriate naming conventions:

def upgrade():op.create_unique_constraint(None, 'some_table', 'x')

When autogenerate renders constraints in a migration script, it renders them typically with their completed name.If using at least Alembic 0.6.4 as well as SQLAlchemy 0.9.4, these will be rendered with a special directiveOperations.f() which denotes that the string has already been tokenized:

def upgrade():op.create_unique_constraint(op.f('uq_const_x'), 'some_table', 'x')

For more detail on the naming convention feature, see Configuring Constraint Naming Conventions.

30 Chapter 5. The Importance of Naming Constraints

http://www.sqlalchemy.org/docs/core/constraints.html#constraint-naming-conventions

CHAPTER 6

Running “Batch” Migrations for SQLite and Other Databases

Note: “Batch mode” for SQLite and other databases is a new and intricate feature within the 0.7.0 series of Alembic,and should be considered as “beta” for the next several releases.


The SQLite database presents a challenge to migration tools in that it has almost no support for the ALTER statementupon which relational schema migrations rely upon. The rationale for this stems from philosophical and architecturalconcerns within SQLite, and they are unlikely to be changed.

Migration tools are instead expected to produce copies of SQLite tables that correspond to the new structure, transferthe data from the existing table to the new one, then drop the old table. For our purposes here we’ll call this “moveand copy” workflow, and in order to accommodate it in a way that is reasonably predictable, while also remainingcompatible with other databases, Alembic provides the batch operations context.

Within this context, a relational table is named, and then a series of mutation operations to that table alone are specifiedwithin the block. When the context is complete, a process begins whereby the “move and copy” procedure begins; theexisting table structure is reflected from the database, a new version of this table is created with the given changes,data is copied from the old table to the new table using “INSERT from SELECT”, and finally the old table is droppedand the new one renamed to the original name.

The Operations.batch_alter_table() method provides the gateway to this process:

with op.batch_alter_table("some_table") as batch_op:batch_op.add_column(Column('foo', Integer))batch_op.drop_column('bar')

When the above directives are invoked within a migration script, on a SQLite backend we would see SQL like:

CREATE TABLE _alembic_batch_temp (id INTEGER NOT NULL,foo INTEGER,PRIMARY KEY (id)

);INSERT INTO _alembic_batch_temp (id) SELECT some_table.id FROM some_table;DROP TABLE some_table;ALTER TABLE _alembic_batch_temp RENAME TO some_table;

On other backends, we’d see the usual ALTER statements done as though there were no batch directive - the batchcontext by default only does the “move and copy” process if SQLite is in use, and if there are migration directives otherthan Operations.add_column() present, which is the one kind of column-level ALTER statement that SQLitesupports. Operations.batch_alter_table() can be configured to run “move and copy” unconditionally inall cases, including on databases other than SQLite; more on this is below.

31


6.1 Controlling Table Reflection

The Table object that is reflected when “move and copy” proceeds is performed using the standardautoload=True approach. This call can be affected using the reflect_args and reflect_kwargs ar-guments. For example, to override a Column within the reflection process such that a Boolean object is reflectedwith the create_constraint flag set to False:

with self.op.batch_alter_table("bar",reflect_args=[Column('flag', Boolean(create_constraint=False))]

) as batch_op:batch_op.alter_column(

'flag', new_column_name='bflag', existing_type=Boolean)

Another use case, add a listener to the Table as it is reflected so that special logic can be applied to columns or types,using the column_reflect() event:

def listen_for_reflect(inspector, table, column_info):"correct an ENUM type"if column_info['name'] == 'my_enum':

column_info['type'] = Enum('a', 'b', 'c')

with self.op.batch_alter_table("bar",reflect_kwargs=dict(

listeners=[('column_reflect', listen_for_reflect)

])

) as batch_op:batch_op.alter_column(

'flag', new_column_name='bflag', existing_type=Boolean)

The reflection process may also be bypassed entirely by sending a pre-fabricated Table object; see Working in OfflineMode for an example.

New in version 0.7.1: added Operations.batch_alter_table.reflect_args andOperations.batch_alter_table.reflect_kwargs options.

6.2 Dealing with Constraints

There are a variety of issues when using “batch” mode with constraints, such as FOREIGN KEY, CHECK andUNIQUE constraints. This section will attempt to detail many of these scenarios.

6.2.1 Dropping Unnamed or Named Foreign Key Constraints

SQLite, unlike any other database, allows constraints to exist in the database that have no identifying name. On allother backends, the target database will always generate some kind of name, if one is not given.

The first challenge this represents is that an unnamed constraint can’t by itself be targeted by theBatchOperations.drop_constraint() method. An unnamed FOREIGN KEY constraint is implicit when-ever the ForeignKey or ForeignKeyConstraint objects are used without passing them a name. Only onSQLite will these constraints remain entirely unnamed when they are created on the target database; an automaticallygenerated name will be assigned in the case of all other database backends.

32 Chapter 6. Running “Batch” Migrations for SQLite and Other Databases



http://www.sqlalchemy.org/docs/core/type_basics.html#sqlalchemy.types.Boolean


http://www.sqlalchemy.org/docs/core/events.html#sqlalchemy.events.DDLEvents.column_reflect



http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.ForeignKeyConstraint


A second issue is that SQLAlchemy itself has inconsistent behavior in dealing with SQLite constraints as far as names.Prior to version 1.0, SQLAlchemy omits the name of foreign key constraints when reflecting them against the SQLitebackend. So even if the target application has gone through the steps to apply names to the constraints as stated in thedatabase, they still aren’t targetable within the batch reflection process prior to SQLAlchemy 1.0.

Within the scope of batch mode, this presents the issue that the BatchOperations.drop_constraint()method requires a constraint name in order to target the correct constraint.

In order to overcome this, the Operations.batch_alter_table() method supports anaming_convention argument, so that all reflected constraints, including foreign keys that are unnamed,or were named but SQLAlchemy isn’t loading this name, may be given a name, as described in Integration of NamingConventions into Operations, Autogenerate. Usage is as follows:

naming_convention = {"fk":"fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",

}with self.op.batch_alter_table(

"bar", naming_convention=naming_convention) as batch_op:batch_op.drop_constraint(

"fk_bar_foo_id_foo", type_="foreignkey")

Note that the naming convention feature requires at least SQLAlchemy 0.9.4 for support.

New in version 0.7.1: added naming_convention to Operations.batch_alter_table().

6.2.2 Including unnamed UNIQUE constraints

A similar, but frustratingly slightly different, issue is that in the case of UNIQUE constraints, we again have the issuethat SQLite allows unnamed UNIQUE constraints to exist on the database, however in this case, SQLAlchemy prior toversion 1.0 doesn’t reflect these constraints at all. It does properly reflect named unique constraints with their names,however.

So in this case, the workaround for foreign key names is still not sufficient prior to SQLAlchemy 1.0. If our tableincludes unnamed unique constraints, and we’d like them to be re-created along with the table, we need to includethem directly, which can be via the table_args argument:

with self.op.batch_alter_table("bar", table_args=(UniqueConstraint('username'),)

):batch_op.add_column(Column('foo', Integer))

6.2.3 Including CHECK constraints

SQLAlchemy currently doesn’t reflect CHECK constraints on any backend. So again these must be stated explicitlyif they are to be included in the recreated table:

with op.batch_alter_table("some_table", table_args=[CheckConstraint('x > 5')

]) as batch_op:batch_op.add_column(Column('foo', Integer))batch_op.drop_column('bar')

6.2. Dealing with Constraints 33


6.2.4 Dealing with Referencing Foreign Keys

If the SQLite database is enforcing referential integrity with PRAGMA FOREIGN KEYS, this pragma may need tobe disabled when the workflow mode proceeds, else remote constraints which refer to this table may prevent it frombeing dropped; additionally, for referential integrity to be re-enabled, it may be necessary to recreate the foreign keyson those remote tables to refer again to the new table (this is definitely the case on other databases, at least). SQLite isnormally used without referential integrity enabled so this won’t be a problem for most users.

6.3 Working in Offline Mode

In the preceding sections, we’ve seen how much of an emphasis the “move and copy” process has on using reflectionin order to know the structure of the table that is to be copied. This means that in the typical case, “online” mode,where a live database connection is present so that Operations.batch_alter_table() can reflect the tablefrom the database, is required; the --sql flag cannot be used without extra steps.

To support offline mode, the system must work without table reflection present, which means the full table as it intendsto be created must be passed to Operations.batch_alter_table() using copy_from:

meta = MetaData()some_table = Table(

'some_table', meta,Column('id', Integer, primary_key=True),Column('bar', String(50))

)

with op.batch_alter_table("some_table", copy_from=some_table) as batch_op:batch_op.add_column(Column('foo', Integer))batch_op.drop_column('bar')

The above use pattern is pretty tedious and quite far off from Alembic’s preferred style of working; however, if oneneeds to do SQLite-compatible “move and copy” migrations and need them to generate flat SQL files in “offline”mode, there’s not much alternative.

New in version 0.7.6: Fully implemented the copy_from parameter.

6.4 Batch mode with Autogenerate

The syntax of batch mode is essentially that Operations.batch_alter_table() is used to enter a batchblock, and the returned BatchOperations context works just like the regular Operations context, except thatthe “table name” and “schema name” arguments are omitted.

To support rendering of migration commands in batch mode for autogenerate, configure theEnvironmentContext.configure.render_as_batch flag in env.py:

context.configure(connection=connection,target_metadata=target_metadata,render_as_batch=True

)

Autogenerate will now generate along the lines of:

def upgrade():### commands auto generated by Alembic - please adjust! ###



with op.batch_alter_table('address', schema=None) as batch_op:batch_op.add_column(sa.Column('street', sa.String(length=50), nullable=True))

This mode is safe to use in all cases, as the Operations.batch_alter_table() directive by default only takesplace for SQLite; other backends will behave just as they normally do in the absense of the batch directives.

Note that autogenerate support does not include “offline” mode, where theOperations.batch_alter_table.copy_from parameter is used. The table definition here wouldneed to be entered into migration files manually if this is needed.

6.5 Batch mode with databases other than SQLite

There’s an odd use case some shops have, where the “move and copy” style of migration is useful in some cases fordatabases that do already support ALTER. There’s some cases where an ALTER operation may block access to thetable for a long time, which might not be acceptable. “move and copy” can be made to work on other backends, thoughwith a few extra caveats.

The batch mode directive will run the “recreate” system regardless of backend if the flag recreate=’always’ ispassed:

with op.batch_alter_table("some_table", recreate='always') as batch_op:batch_op.add_column(Column('foo', Integer))

The issues that arise in this mode are mostly to do with constraints. Databases such as Postgresql and MySQL withInnoDB will enforce referential integrity (e.g. via foreign keys) in all cases. Unlike SQLite, it’s not as simple toturn off referential integrity across the board (nor would it be desirable). Since a new table is replacing the old one,existing foreign key constraints which refer to the target table will need to be unconditionally dropped before the batchoperation, and re-created to refer to the new table afterwards. Batch mode currently does not provide any automationfor this.

The Postgresql database and possibly others also have the behavior such that when the new table is created, a namingconflict occurs with the named constraints of the new table, in that they match those of the old table, and on Postgresql,these names need to be unique across all tables. The Postgresql dialect will therefore emit a “DROP CONSTRAINT”directive for all constraints on the old table before the new one is created; this is “safe” in case of a failed operationbecause Postgresql also supports transactional DDL.

Note that also as is the case with SQLite, CHECK constraints need to be moved over between old and new tablemanually using the Operations.batch_alter_table.table_args parameter.

6.5. Batch mode with databases other than SQLite 35

CHAPTER 7

Working with Branches

Note: Alembic 0.7.0 features an all-new versioning model that fully supports branch points, merge points, and long-lived, labeled branches, including independent branches originating from multiple bases. A great emphasis has beenplaced on there being almost no impact on the existing Alembic workflow, including that all commands work prettymuch the same as they did before, the format of migration files doesn’t require any change (though there are somechanges that are recommended), and even the structure of the alembic_version table does not change at all.However, most alembic commands now offer new features which will break out an Alembic environment into “branchmode”, where things become a lot more intricate. Working in “branch mode” should be considered as a “beta” feature,with many new paradigms and use cases still to be stress tested in the wild. Please tread lightly!


A branch describes a point in a migration stream when two or more versions refer to the same parent migration as theiranscestor. Branches occur naturally when two divergent source trees, both containing Alembic revision files createdindependently within those source trees, are merged together into one. When this occurs, the challenge of a branch isto merge the branches into a single series of changes, so that databases established from either source tree individuallycan be upgraded to reference the merged result equally. Another scenario where branches are present are when wecreate them directly; either at some point in the migration stream we’d like different series of migrations to be managedindependently (e.g. we create a tree), or we’d like separate migration streams for different features starting at the root(e.g. a forest). We’ll illustrate all of these cases, starting with the most common which is a source-merge-originatedbranch that we’ll merge.

Starting with the “account table” example we began in Create a Migration Script, assume we have our basemostversion 1975ea83b712, which leads into the second revision ae1027a6acf, and the migration files for thesetwo revisions are checked into our source repository. Consider if we merged into our source repository anothercode branch which contained a revision for another table called shopping_cart. This revision was made againstour first Alembic revision, the one that generated account. After loading the second source tree in, a new file27c6a30d7c24_add_shopping_cart_table.py exists within our versions directory. Both it, as well asae1027a6acf_add_a_column.py, reference 1975ea83b712_add_account_table.py as the “down-grade” revision. To illustrate:

# main source tree:1975ea83b712 (create account table) -> ae1027a6acf (add a column)

# branched source tree1975ea83b712 (create account table) -> 27c6a30d7c24 (add shopping cart table)

Above, we can see 1975ea83b712 is our branch point; two distinct versions both refer to it as its parent. TheAlembic command branches illustrates this fact:

$ alembic branches --verboseRev: 1975ea83b712 (branchpoint)

37


Parent: <base>Branches into: 27c6a30d7c24, ae1027a6acfPath: foo/versions/1975ea83b712_add_account_table.py

create account table


-> 27c6a30d7c24 (head), add shopping cart table-> ae1027a6acf (head), add a column

History shows it too, illustrating two head entries as well as a branchpoint:

$ alembic history1975ea83b712 -> 27c6a30d7c24 (head), add shopping cart table1975ea83b712 -> ae1027a6acf (head), add a column<base> -> 1975ea83b712 (branchpoint), create account table

We can get a view of just the current heads using alembic heads:

$ alembic heads --verboseRev: 27c6a30d7c24 (head)Parent: 1975ea83b712Path: foo/versions/27c6a30d7c24_add_shopping_cart_table.py

add shopping cart table

Revision ID: 27c6a30d7c24Revises: 1975ea83b712Create Date: 2014-11-20 13:03:11.436407

Rev: ae1027a6acf (head)Parent: 1975ea83b712Path: foo/versions/ae1027a6acf_add_a_column.py

add a column


If we try to run an upgrade to the usual end target of head, Alembic no longer considers this to be an unambiguouscommand. As we have more than one head, the upgrade command wants us to provide more information:

$ alembic upgrade headFAILED: Multiple head revisions are present for given argument 'head'; please specify a specifictarget revision, '<branchname>@head' to narrow to a specific head, or 'heads' for all heads

The upgrade command gives us quite a few options in which we can proceed with our upgrade, either giving itinformation on which head we’d like to upgrade towards, or alternatively stating that we’d like all heads to be upgradedtowards at once. However, in the typical case of two source trees being merged, we will want to pursue a third option,which is that we can merge these branches.

38 Chapter 7. Working with Branches


7.1 Merging Branches

An Alembic merge is a migration file that joins two or more “head” files together. If the two branches we have rightnow can be said to be a “tree” structure, introducing this merge file will turn it into a “diamond” structure:

-- ae1027a6acf -->/ \

<base> --> 1975ea83b712 --> --> mergepoint\ /-- 27c6a30d7c24 -->

We create the merge file using alembic merge; with this command, we can pass to it an argument such as heads,meaning we’d like to merge all heads. Or, we can pass it individual revision numbers sequentally:

$ alembic merge -m "merge ae1 and 27c" ae1027 27c6aGenerating /path/to/foo/versions/53fffde5ad5_merge_ae1_and_27c.py ... done

Looking inside the new file, we see it as a regular migration file, with the only new twist is that down_revisionpoints to both revisions:

"""merge ae1 and 27c

Revision ID: 53fffde5ad5Revises: ae1027a6acf, 27c6a30d7c24Create Date: 2014-11-20 13:31:50.811663

"""

# revision identifiers, used by Alembic.revision = '53fffde5ad5'down_revision = ('ae1027a6acf', '27c6a30d7c24')branch_labels = None


def upgrade():pass

def downgrade():pass

This file is a regular migration file, and if we wish to, we may place Operations directives into the upgrade()and downgrade() functions like any other migration file. Though it is probably best to limit the instructions placedhere only to those that deal with any kind of reconciliation that is needed between the two merged branches, if any.

The heads command now illustrates that the multiple heads in our versions/ directory have been resolved intoour new head:

$ alembic heads --verboseRev: 53fffde5ad5 (head) (mergepoint)Merges: ae1027a6acf, 27c6a30d7c24Path: foo/versions/53fffde5ad5_merge_ae1_and_27c.py

merge ae1 and 27c

Revision ID: 53fffde5ad5

7.1. Merging Branches 39


Revises: ae1027a6acf, 27c6a30d7c24Create Date: 2014-11-20 13:31:50.811663

History shows a similar result, as the mergepoint becomes our head:

$ alembic historyae1027a6acf, 27c6a30d7c24 -> 53fffde5ad5 (head) (mergepoint), merge ae1 and 27c1975ea83b712 -> ae1027a6acf, add a column1975ea83b712 -> 27c6a30d7c24, add shopping cart table<base> -> 1975ea83b712 (branchpoint), create account table

With a single head target, a generic upgrade can proceed:

$ alembic upgrade headINFO [alembic.migration] Context impl PostgresqlImpl.INFO [alembic.migration] Will assume transactional DDL.INFO [alembic.migration] Running upgrade -> 1975ea83b712, create account tableINFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart tableINFO [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a columnINFO [alembic.migration] Running upgrade ae1027a6acf, 27c6a30d7c24 -> 53fffde5ad5, merge ae1 and 27c



merge mechanics

The upgrade process traverses through all of our migration files using a topological sorting algorithm, treatingthe list of migration files not as a linked list, but as a directed acyclic graph. The starting points of this traversalare the current heads within our database, and the end point is the “head” revision or revisions specified.When a migration proceeds across a point at which there are multiple heads, the alembic_version tablewill at that point store multiple rows, one for each head. Our migration process above will emit SQL againstalembic_version along these lines:

-- Running upgrade -> 1975ea83b712, create account tableINSERT INTO alembic_version (version_num) VALUES ('1975ea83b712')

-- Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart tableUPDATE alembic_version SET version_num='27c6a30d7c24' WHERE alembic_version.version_num = '1975ea83b712'

-- Running upgrade 1975ea83b712 -> ae1027a6acf, add a columnINSERT INTO alembic_version (version_num) VALUES ('ae1027a6acf')

-- Running upgrade ae1027a6acf, 27c6a30d7c24 -> 53fffde5ad5, merge ae1 and 27cDELETE FROM alembic_version WHERE alembic_version.version_num = 'ae1027a6acf'UPDATE alembic_version SET version_num='53fffde5ad5' WHERE alembic_version.version_num = '27c6a30d7c24'

At the point at which both 27c6a30d7c24 and ae1027a6acf exist within our database, both values arepresent in alembic_version, which now has two rows. If we upgrade to these two versions alone, then stopand run alembic current, we will see this:

$ alembic current --verboseCurrent revision(s) for postgresql://scott:XXXXX@localhost/test:Rev: ae1027a6acfParent: 1975ea83b712Path: foo/versions/ae1027a6acf_add_a_column.py

add a column


Rev: 27c6a30d7c24Parent: 1975ea83b712Path: foo/versions/27c6a30d7c24_add_shopping_cart_table.py



A key advantage to the merge process is that it will run equally well on databases that were present on ver-sion ae1027a6acf alone, versus databases that were present on version 27c6a30d7c24 alone; whicheverversion was not yet applied, will be applied before the merge point can be crossed. This brings forth a way ofthinking about a merge file, as well as about any Alembic revision file. As they are considered to be “nodes”within a set that is subject to topological sorting, each “node” is a point that cannot be crossed until all of itsdependencies are satisfied.Prior to Alembic’s support of merge points, the use case of databases sitting on different heads was basicallyimpossible to reconcile; having to manually splice the head files together invariably meant that one migrationwould occur before the other, thus being incompatible with databases that were present on the other migration.

7.1. Merging Branches 41


7.2 Working with Explicit Branches

The alembic upgrade command hinted at other options besides merging when dealing with multiple heads. Let’sback up and assume we’re back where we have as our heads just ae1027a6acf and 27c6a30d7c24:

$ alembic heads27c6a30d7c24ae1027a6acf

Earlier, when we did alembic upgrade head, it gave us an error which suggested please specify aspecific target revision, ’<branchname>@head’ to narrow to a specific head, or’heads’ for all heads in order to proceed without merging. Let’s cover those cases.

7.2.1 Referring to all heads at once

The heads identifier is a lot like head, except it explicitly refers to all heads at once. That is, it’s like telling Alembicto do the operation for both ae1027a6acf and 27c6a30d7c24 simultaneously. If we started from a fresh databaseand ran upgrade heads we’d see:

$ alembic upgrade headsINFO [alembic.migration] Context impl PostgresqlImpl.INFO [alembic.migration] Will assume transactional DDL.INFO [alembic.migration] Running upgrade -> 1975ea83b712, create account tableINFO [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a columnINFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table

Since we’ve upgraded to heads, and we do in fact have more than one head, that means these two distinct heads arenow in our alembic_version table. We can see this if we run alembic current:

$ alembic currentae1027a6acf (head)27c6a30d7c24 (head)

That means there’s two rows in alembic_version right now. If we downgrade one step at a time, Alembic willdelete from the alembic_version table each branch that’s closed out, until only one branch remains; then it willcontinue updating the single value down to the previous versions:

$ alembic downgrade -1INFO [alembic.migration] Running downgrade ae1027a6acf -> 1975ea83b712, add a column

$ alembic current27c6a30d7c24 (head)

$ alembic downgrade -1INFO [alembic.migration] Running downgrade 27c6a30d7c24 -> 1975ea83b712, add shopping cart table

$ alembic current1975ea83b712 (branchpoint)

$ alembic downgrade -1INFO [alembic.migration] Running downgrade 1975ea83b712 -> , create account table

$ alembic current



7.2.2 Referring to a Specific Version

We can pass a specific version number to upgrade. Alembic will ensure that all revisions upon which this version de-pends are invoked, and nothing more. So if we upgrade either to 27c6a30d7c24 or ae1027a6acf specifically,it guarantees that 1975ea83b712 will have been applied, but not that any “sibling” versions are applied:

$ alembic upgrade 27c6aINFO [alembic.migration] Running upgrade -> 1975ea83b712, create account tableINFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table

With 1975ea83b712 and 27c6a30d7c24 applied, ae1027a6acf is just a single additional step:

$ alembic upgrade ae102INFO [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column

7.2.3 Working with Branch Labels

To satisfy the use case where an environment has long-lived branches, especially independent branches as willbe discussed in the next section, Alembic supports the concept of branch labels. These are string valuesthat are present within the migration file, using the new identifier branch_labels. For example, if wewant to refer to the “shopping cart” branch using the name “shoppingcart”, we can add that name to our file27c6a30d7c24_add_shopping_cart_table.py:

"""add shopping cart table

"""

# revision identifiers, used by Alembic.revision = '27c6a30d7c24'down_revision = '1975ea83b712'branch_labels = ('shoppingcart',)

# ...

The branch_labels attribute refers to a string name, or a tuple of names, which will now apply to this revision,all descendants of this revision, as well as all ancestors of this revision up until the preceding branch point, in this case1975ea83b712. We can see the shoppingcart label applied to this revision:

$ alembic history1975ea83b712 -> 27c6a30d7c24 (shoppingcart) (head), add shopping cart table1975ea83b712 -> ae1027a6acf (head), add a column<base> -> 1975ea83b712 (branchpoint), create account table

With the label applied, the name shoppingcart now serves as an alias for the 27c6a30d7c24 revision specifi-cally. We can illustrate this by showing it with alembic show:

$ alembic show shoppingcartRev: 27c6a30d7c24 (head)Parent: 1975ea83b712Branch names: shoppingcartPath: foo/versions/27c6a30d7c24_add_shopping_cart_table.py



7.2. Working with Explicit Branches 43


However, when using branch labels, we usually want to use them using a syntax known as “branch at” syntax; thissyntax allows us to state that we want to use a specific revision, let’s say a “head” revision, in terms of a specificbranch. While normally, we can’t refer to alembic upgrade head when there’s multiple heads, we can refer tothis head specifcally using shoppingcart@head syntax:

$ alembic upgrade shoppingcart@headINFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table

The shoppingcart@head syntax becomes important to us if we wish to add new migration files to our versionsdirectory while maintaining multiple branches. Just like the upgrade command, if we attempted to add a newrevision file to our multiple-heads layout without a specific parent revision, we’d get a familiar error:

$ alembic revision -m "add a shopping cart column"FAILED: Multiple heads are present; please specify the head revision onwhich the new revision should be based, or perform a merge.

The alembic revision command is pretty clear in what we need to do; to add our new revision specifically to theshoppingcart branch, we use the --head argument, either with the specific revision identifier 27c6a30d7c24,or more generically using our branchname shoppingcart@head:

$ alembic revision -m "add a shopping cart column" --head shoppingcart@headGenerating /path/to/foo/versions/d747a8a8879_add_a_shopping_cart_column.py ... done

alembic history shows both files now part of the shoppingcart branch:

$ alembic history1975ea83b712 -> ae1027a6acf (head), add a column27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table<base> -> 1975ea83b712 (branchpoint), create account table

We can limit our history operation just to this branch as well:

$ alembic history -r shoppingcart:27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table

If we want to illustrate the path of shoppingcart all the way from the base, we can do that as follows:

$ alembic history -r :shoppingcart@head27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table<base> -> 1975ea83b712 (branchpoint), create account table

We can run this operation from the “base” side as well, but we get a different result:

$ alembic history -r shoppingcart@base:1975ea83b712 -> ae1027a6acf (head), add a column27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table<base> -> 1975ea83b712 (branchpoint), create account table

When we list from shoppingcart@base without an endpoint, it’s really shorthand for -rshoppingcart@base:heads, e.g. all heads, and since shoppingcart@base is the same “base” shared bythe ae1027a6acf revision, we get that revision in our listing as well. The <branchname>@base syntax can beuseful when we are dealing with individual bases, as we’ll see in the next section.

The <branchname>@head format can also be used with revision numbers instead of branch names, though this isless convenient. If we wanted to add a new revision to our branch that includes the un-labeled ae1027a6acf, if thisweren’t a head already, we could ask for the “head of the branch that includes ae1027a6acf” as follows:



$ alembic revision -m "add another account column" --head ae10@headGenerating /path/to/foo/versions/55af2cb1c267_add_another_account_column.py ... done

7.2.4 More Label Syntaxes

The heads symbol can be combined with a branch label, in the case that your labeled branch itself breaks off intomultiple branches:

$ alembic upgrade shoppingcart@heads

Relative identifiers, as introduced in Relative Migration Identifiers, work with labels too. For example, upgrading toshoppingcart@+2 means to upgrade from current heads on “shoppingcart” upwards two revisions:

$ alembic upgrade shoppingcart@+2

This kind of thing works from history as well:

$ alembic history -r current:shoppingcart@+2

The newer relnum+delta format can be combined as well, for example if we wanted to list along shoppingcartup until two revisions before the head:

$ alembic history -r :shoppingcart@head-2

7.3 Working with Multiple Bases

We’ve seen in the previous section that alembic upgrade is fine if we have multiple heads, alembicrevision allows us to tell it which “head” we’d like to associate our new revision file with, and branch labelsallow us to assign names to branches that we can use in subsequent commands. Let’s put all these together and referto a new “base”, that is, a whole new tree of revision files that will be semi-independent of the account/shopping cartrevisions we’ve been working with. This new tree will deal with database tables involving “networking”.

7.3.1 Setting up Multiple Version Directories

While optional, it is often the case that when working with multiple bases, we’d like different sets of version files toexist within their own directories; typically, if an application is organized into several sub-modules, each one wouldhave a version directory containing migrations pertinent to that module. So to start out, we can edit alembic.inito refer to multiple directories; we’ll also state the current versions directory as one of them:

# version location specification; this defaults# to foo/versions. When using multiple version# directories, initial revisions must be specified with --version-pathversion_locations = %(here)s/model/networking %(here)s/alembic/versions

The new directory %(here)s/model/networking is in terms of where the alembic.ini file is, as we areusing the symbol %(here)s which resolves to this location. When we create our first new revision targeted at thisdirectory, model/networking will be created automatically if it does not exist yet. Once we’ve created a revisionhere, the path is used automatically when generating subsequent revision files that refer to this revision tree.

7.3. Working with Multiple Bases 45


7.3.2 Creating a Labeled Base Revision

We also want our new branch to have its own name, and for that we want to apply a branch label to the base. In order toachieve this using the alembic revision command without editing, we need to ensure our script.py.makofile, used for generating new revision files, has the appropriate substitutions present. If Alembic version 0.7.0 or greaterwas used to generate the original migration environment, this is already done. However when working with an olderenvironment, script.py.mako needs to have this directive added, typically underneath the down_revisiondirective:

# revision identifiers, used by Alembic.revision = ${repr(up_revision)}down_revision = ${repr(down_revision)}

# add this here in order to use revision with branch_labelbranch_labels = ${repr(branch_labels)}

With this in place, we can create a new revision file, starting up a branch that will deal with database tables involvingnetworking; we specify the --head version of base, a --branch-label of networking, and the directory wewant this first revision file to be placed in with --version-path:

$ alembic revision -m "create networking branch" --head=base --branch-label=networking --version-path=model/networkingCreating directory /path/to/foo/model/networking ... doneGenerating /path/to/foo/model/networking/3cac04ae8714_create_networking_branch.py ... done

If we ran the above command and we didn’t have the newer script.py.mako directive, we’d get this error:

FAILED: Version 3cac04ae8714 specified branch_labels networking, howeverthe migration file foo/model/networking/3cac04ae8714_create_networking_branch.pydoes not have them; have you upgraded your script.py.mako to include the 'branch_labels'section?

When we receive the above error, and we would like to try again, we need to either delete the incorrectly generatedfile in order to run revision again, or we can edit the 3cac04ae8714_create_networking_branch.pydirectly to add the branch_labels in of our choosing.

7.3.3 Running with Multiple Bases

Once we have a new, permanent (for as long as we desire it to be) base in our system, we’ll always have multiple headspresent:

$ alembic heads3cac04ae8714 (networking) (head)27c6a30d7c24 (shoppingcart) (head)ae1027a6acf (head)

When we want to add a new revision file to networking, we specify networking@head as the --head. Theappropriate version directory is now selected automatically based on the head we choose:

$ alembic revision -m "add ip number table" --head=networking@headGenerating /path/to/foo/model/networking/109ec7d132bf_add_ip_number_table.py ... done

It’s important that we refer to the head using networking@head; if we only refer to networking, that refers toonly 3cac04ae8714 specifically; if we specify this and it’s not a head, alembic revision will make sure wedidn’t mean to specify the head:

$ alembic revision -m "add DNS table" --head=networkingFAILED: Revision 3cac04ae8714 is not a head revision; pleasespecify --splice to create a new branch from this revision



As mentioned earlier, as this base is independent, we can view its history from the base using history -rnetworking@base::

$ alembic history -r networking@base:109ec7d132bf -> 29f859a13ea (networking) (head), add DNS table3cac04ae8714 -> 109ec7d132bf (networking), add ip number table<base> -> 3cac04ae8714 (networking), create networking branch

At the moment, this is the same output we’d get at this point if we used -r :networking@head. However, thatwill change later on as we use additional directives.

We may now run upgrades or downgrades freely, among individual branches (let’s assume a clean database again):

$ alembic upgrade networking@headINFO [alembic.migration] Running upgrade -> 3cac04ae8714, create networking branchINFO [alembic.migration] Running upgrade 3cac04ae8714 -> 109ec7d132bf, add ip number tableINFO [alembic.migration] Running upgrade 109ec7d132bf -> 29f859a13ea, add DNS table

or against the whole thing using heads:

$ alembic upgrade headsINFO [alembic.migration] Running upgrade -> 1975ea83b712, create account tableINFO [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart tableINFO [alembic.migration] Running upgrade 27c6a30d7c24 -> d747a8a8879, add a shopping cart columnINFO [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a columnINFO [alembic.migration] Running upgrade ae1027a6acf -> 55af2cb1c267, add another account column

7.4 Branch Dependencies

When working with multiple roots, it is expected that these different revision streams will need to refer to one another.For example, a new revision in networking which needs to refer to the account table will want to establish55af2cb1c267, add another account column, the last revision that works with the account table, as adependency. From a graph perspective, this means nothing more that the new file will feature both 55af2cb1c267,add another account column and 29f859a13ea, add DNS table as “down” revisions, and looksjust as though we had merged these two branches together. However, we don’t want to consider these as “merged”;we want the two revision streams to remain independent, even though a version in networking is going to reachover into the other stream. To support this use case, Alembic provides a directive known as depends_on, whichallows a revision file to refer to another as a “dependency”, very similar to an entry in down_revision from a graphperspective, but different from a semantic perspective.

First we will build out our new revision on the networking branch in the usual way:

$ alembic revision -m "add ip account table" --head=networking@headGenerating /path/to/foo/model/networking/2a95102259be_add_ip_account_table.py ... done

Next, we’ll add an explicit dependency inside the file, by placing the directive depends_on=’55af2cb1c267’underneath the other directives:

# revision identifiers, used by Alembic.revision = '2a95102259be'down_revision = '29f859a13ea'branch_labels = Nonedepends_on='55af2cb1c267'

Currently, depends_on needs to be a real revision number, not a partial number or branch name. It can of courserefer to a tuple of any number of dependent revisions:

7.4. Branch Dependencies 47


# revision identifiers, used by Alembic.revision = '2a95102259be'down_revision = '29f859a13ea'branch_labels = Nonedepends_on = ('55af2cb1c267', 'd747a8a8879', 'fa4456a9201')

We can see the effect this directive has when we view the history of the networking branch in terms of “heads”,e.g., all the revisions that are descendants:

$ alembic history -r :networking@head29f859a13ea (55af2cb1c267) -> 2a95102259be (networking) (head), add ip account table109ec7d132bf -> 29f859a13ea (networking), add DNS table3cac04ae8714 -> 109ec7d132bf (networking), add ip number table<base> -> 3cac04ae8714 (networking), create networking branchae1027a6acf -> 55af2cb1c267 (effective head), add another account column1975ea83b712 -> ae1027a6acf, Add a column<base> -> 1975ea83b712 (branchpoint), create account table

What we see is that the full history of the networking branch, in terms of an “upgrade” to the “head”, will includethat the tree building up 55af2cb1c267, add another account column will be pulled in first. Interst-ingly, we don’t see this displayed when we display history in the other direction, e.g. from networking@base:

$ alembic history -r networking@base:29f859a13ea (55af2cb1c267) -> 2a95102259be (networking) (head), add ip account table109ec7d132bf -> 29f859a13ea (networking), add DNS table3cac04ae8714 -> 109ec7d132bf (networking), add ip number table<base> -> 3cac04ae8714 (networking), create networking branch

The reason for the discrepancy is that displaying history from the base shows us what would occur if we ran a down-grade operation, instead of an upgrade. If we downgraded all the files in networking using networking@base,the dependencies aren’t affected, they’re left in place.

We also see something odd if we view heads at the moment:

$ alembic heads2a95102259be (networking) (head)27c6a30d7c24 (shoppingcart) (head)55af2cb1c267 (effective head)

The head file that we used as a “dependency”, 55af2cb1c267, is displayed as an “effective” head, which we cansee also in the history display earlier. What this means is that at the moment, if we were to upgrade all versions to thetop, the 55af2cb1c267 revision number would not actually be present in the alembic_version table; this isbecause it does not have a branch of its own subsequent to the 2a95102259be revision which depends on it:

$ alembic upgrade headsINFO [alembic.migration] Running upgrade 29f859a13ea, 55af2cb1c267 -> 2a95102259be, add ip account table

$ alembic current2a95102259be (head)27c6a30d7c24 (head)

The entry is still displayed in alembic heads because Alembic knows that even though this revision isn’t a “real”head, it’s still something that we developers consider semantically to be a head, so it’s displayed, noting its specialstatus so that we don’t get quite as confused when we don’t see it within alembic current.

If we add a new revision onto 55af2cb1c267, the branch again becomes a “real” branch which can have its ownentry in the database:



$ alembic revision -m "more account changes" --head=55af2cb@headGenerating /path/to/foo/versions/34e094ad6ef1_more_account_changes.py ... done

$ alembic upgrade headsINFO [alembic.migration] Running upgrade 55af2cb1c267 -> 34e094ad6ef1, more account changes

$ alembic current2a95102259be (head)27c6a30d7c24 (head)34e094ad6ef1 (head)

For posterity, the revision tree now looks like:

$ alembic history29f859a13ea (55af2cb1c267) -> 2a95102259be (networking) (head), add ip account table109ec7d132bf -> 29f859a13ea (networking), add DNS table3cac04ae8714 -> 109ec7d132bf (networking), add ip number table<base> -> 3cac04ae8714 (networking), create networking branch1975ea83b712 -> 27c6a30d7c24 (shoppingcart) (head), add shopping cart table55af2cb1c267 -> 34e094ad6ef1 (head), more account changesae1027a6acf -> 55af2cb1c267, add another account column1975ea83b712 -> ae1027a6acf, Add a column<base> -> 1975ea83b712 (branchpoint), create account table

--- 27c6 --> d747 --> <head>/ (shoppingcart)

<base> --> 1975 -->\--- ae10 --> 55af --> <head>

^+--------+ (dependency)

||

<base> --> 3782 -----> 109e ----> 29f8 ---> 2a95 --> <head>(networking)

If there’s any point to be made here, it’s if you are too freely branching, merging and labeling, things can get prettycrazy! Hence the branching system should be used carefully and thoughtfully for best results.

7.4. Branch Dependencies 49

CHAPTER 8

Operation Reference

This file provides documentation on Alembic migration directives.

The directives here are used within user-defined migration files, within the upgrade() and downgrade() func-tions, as well as any functions further invoked by those.

All directives exist as methods on a class called Operations. When migration scripts are run, this objectis made available to the script via the alembic.op datamember, which is a proxy to an actual instance ofOperations. Currently, alembic.op is a real Python module, populated with individual proxies for each methodon Operations, so symbols can be imported safely from the alembic.op namespace.

A key design philosophy to the alembic.operations methods is that to the greatest degree possible, they inter-nally generate the appropriate SQLAlchemy metadata, typically involving Table and Constraint objects. This sothat migration instructions can be given in terms of just the string names and/or flags involved. The exceptions to thisrule include the add_column() and create_table() directives, which require full Column objects, thoughthe table metadata is still generated here.

The functions here all require that a MigrationContext has been configured within the env.py script first,which is typically via EnvironmentContext.configure(). Under normal circumstances they are called froman actual migration script, which itself would be invoked by the EnvironmentContext.run_migrations()method.

class alembic.operations.Operations(migration_context, impl=None)Define high level migration operations.

Each operation corresponds to some schema migration operation, executed against a particularMigrationContext which in turn represents connectivity to a database, or a file output stream.

While Operations is normally configured as part of the EnvironmentContext.run_migrations()method called from an env.py script, a standalone Operations instance can be made for use cases externalto regular Alembic migrations by passing in a MigrationContext:

from alembic.migration import MigrationContextfrom alembic.operations import Operations

conn = myengine.connect()ctx = MigrationContext.configure(conn)op = Operations(ctx)

op.alter_column("t", "c", nullable=True)

Construct a new Operations

Parameters migration_context – a MigrationContext instance.

51





add_column(table_name, column, schema=None)Issue an “add column” instruction using the current migration context.

e.g.:

from alembic import opfrom sqlalchemy import Column, String

op.add_column('organization',Column('name', String())

)

The provided Column object can also specify a ForeignKey, referencing a remote table name. Alembicwill automatically generate a stub “referenced” table and emit a second ALTER statement in order to addthe constraint separately:

from alembic import opfrom sqlalchemy import Column, INTEGER, ForeignKey

op.add_column('organization',Column('account_id', INTEGER, ForeignKey('accounts.id'))

)

Note that this statement uses the Column construct as is from the SQLAlchemy library. In particular,default values to be created on the database side are specified using the server_default parameter,and not default which only specifies Python-side defaults:

from alembic import opfrom sqlalchemy import Column, TIMESTAMP, func

# specify "DEFAULT NOW" along with the column addop.add_column('account',

Column('timestamp', TIMESTAMP, server_default=func.now()))

Parameters

• table_name – String name of the parent table.

• column – a sqlalchemy.schema.Column object representing the new column.

• schema – Optional schema name to operate within. To control quoting of the schemaoutside of the default behavior, use the SQLAlchemy construct quoted_name.

New in version 0.7.0: ‘schema’ can now accept a quoted_name construct.

alter_column(table_name, column_name, nullable=None, server_default=False,new_column_name=None, type_=None, autoincrement=None, exist-ing_type=None, existing_server_default=False, existing_nullable=None, exist-ing_autoincrement=None, schema=None, **kw)

Issue an “alter column” instruction using the current migration context.

Generally, only that aspect of the column which is being changed, i.e. name, type, nullability, default,needs to be specified. Multiple changes can also be specified at once and the backend should “do the rightthing”, emitting each change either separately or together as the backend allows.

MySQL has special requirements here, since MySQL cannot ALTER a column without a full specifica-tion. When producing MySQL-compatible migration files, it is recommended that the existing_type,existing_server_default, and existing_nullable parameters be present, if not being al-tered.

52 Chapter 8. Operation Reference





http://www.sqlalchemy.org/docs/core/sqlelement.html#sqlalchemy.sql.elements.quoted_name



Type changes which are against the SQLAlchemy “schema” types Boolean and Enum may also addor drop constraints which accompany those types on backends that don’t support them natively. Theexisting_server_default argument is used in this case as well to remove a previous constraint.

Parameters

• table_name – string name of the target table.

• column_name – string name of the target column, as it exists before the operation begins.

• nullable – Optional; specify True or False to alter the column’s nullability.

• server_default – Optional; specify a string SQL expression, text(), orDefaultClause to indicate an alteration to the column’s default value. Set to None tohave the default removed.

• new_column_name – Optional; specify a string name here to indicate the new namewithin a column rename operation.

• type_ – Optional; a TypeEngine type object to specify a change to the column’s type.For SQLAlchemy types that also indicate a constraint (i.e. Boolean, Enum), the con-straint is also generated.

• autoincrement – set the AUTO_INCREMENT flag of the column; currently under-stood by the MySQL dialect.

• existing_type – Optional; a TypeEngine type object to specify the previous type.This is required for all MySQL column alter operations that don’t otherwise specify a newtype, as well as for when nullability is being changed on a SQL Server column. It is alsoused if the type is a so-called SQLlchemy “schema” type which may define a constraint(i.e. Boolean, Enum), so that the constraint can be dropped.

• existing_server_default – Optional; The existing default value of the column.Required on MySQL if an existing default is not being changed; else MySQL removes thedefault.

• existing_nullable – Optional; the existing nullability of the column. Required onMySQL if the existing nullability is not being changed; else MySQL sets this to NULL.

• existing_autoincrement – Optional; the existing autoincrement of the column.Used for MySQL’s system of altering a column that specifies AUTO_INCREMENT.



batch_alter_table(*args, **kwds)Invoke a series of per-table migrations in batch.

Batch mode allows a series of operations specific to a table to be syntactically grouped together, and allowsfor alternate modes of table migration, in particular the “recreate” style of migration required by SQLite.

“recreate” style is as follows:

1.A new table is created with the new specification, based on the migration directives within the batch,using a temporary name.

2.the data copied from the existing table to the new table.

3.the existing table is dropped.

4.the new table is renamed to the existing table name.

53



http://www.sqlalchemy.org/docs/core/sqlelement.html#sqlalchemy.sql.expression.text

http://www.sqlalchemy.org/docs/core/defaults.html#sqlalchemy.schema.DefaultClause

http://www.sqlalchemy.org/docs/core/type_api.html#sqlalchemy.types.TypeEngine









The directive by default will only use “recreate” style on the SQLite backend, and only if directives arepresent which require this form, e.g. anything other than add_column(). The batch operation on otherbackends will proceed using standard ALTER TABLE operations.

The method is used as a context manager, which returns an instance of BatchOperations; this objectis the same as Operations except that table names and schema names are omitted. E.g.:

with op.batch_alter_table("some_table") as batch_op:batch_op.add_column(Column('foo', Integer))batch_op.drop_column('bar')

The operations within the context manager are invoked at once when the context is ended. When runagainst SQLite, if the migrations include operations not supported by SQLite’s ALTER TABLE, the entiretable will be copied to a new one with the new specification, moving all data across as well.

The copy operation by default uses reflection to retrieve the current structure of the table, and there-fore batch_alter_table() in this mode requires that the migration is run in “online” mode. Thecopy_from parameter may be passed which refers to an existing Table object, which will bypass thisreflection step.

Note: The table copy operation will currently not copy CHECK constraints, and may not copy UNIQUEconstraints that are unnamed, as is possible on SQLite. See the section Dealing with Constraints forworkarounds.

Parameters

• table_name – name of table

• schema – optional schema name.

• recreate – under what circumstances the table should be recreated. At its defaultof "auto", the SQLite dialect will recreate the table if any operations other thanadd_column(), create_index(), or drop_index() are present. Other optionsinclude "always" and "never".

• copy_from – optional Table object that will act as the structure of the table beingcopied. If omitted, table reflection is used to retrieve the structure of the table.

New in version 0.7.6: Fully implemented the copy_from parameter.

See also:

Working in Offline Mode

reflect_args

reflect_kwargs

• reflect_args – a sequence of additional positional arguments that will be applied tothe table structure being reflected / copied; this may be used to pass column and constraintoverrides to the table that will be reflected, in lieu of passing the whole Table usingcopy_from.


• reflect_kwargs – a dictionary of additional keyword arguments that will be appliedto the table structure being copied; this may be used to pass additional table and reflec-tion options to the table that will be reflected, in lieu of passing the whole Table usingcopy_from.







• table_args – a sequence of additional positional arguments that will be applied tothe new Table when created, in addition to those copied from the source table. Thismay be used to provide additional constraints such as CHECK constraints that may not bereflected.

• table_kwargs – a dictionary of additional keyword arguments that will be applied tothe new Table when created, in addition to those copied from the source table. This maybe used to provide for additional table options that may not be reflected.


Parameters naming_convention – a naming convention dictionary of the form describedat Integration of Naming Conventions into Operations, Autogenerate which will be applied tothe MetaData during the reflection process. This is typically required if one wants to dropSQLite constraints, as these constraints will not have names when reflected on this backend.Requires SQLAlchemy 0.9.4 or greater.

See also:

Dropping Unnamed or Named Foreign Key Constraints


Note: batch mode requires SQLAlchemy 0.8 or above.

See also:


bulk_insert(table, rows, multiinsert=True)Issue a “bulk insert” operation using the current migration context.

This provides a means of representing an INSERT of multiple rows which works equally well in the contextof executing on a live connection as well as that of generating a SQL script. In the case of a SQL script,the values are rendered inline into the statement.

e.g.:

from alembic import opfrom datetime import datefrom sqlalchemy.sql import table, columnfrom sqlalchemy import String, Integer, Date

# Create an ad-hoc table to use for the insert statement.accounts_table = table('account',

column('id', Integer),column('name', String),column('create_date', Date)

)

op.bulk_insert(accounts_table,[

{'id':1, 'name':'John Smith','create_date':date(2010, 10, 5)},

{'id':2, 'name':'Ed Williams','create_date':date(2007, 5, 27)},

{'id':3, 'name':'Wendy Jones','create_date':date(2008, 8, 15)},

])

55





When using –sql mode, some datatypes may not render inline automatically, such as dates and other specialtypes. When this issue is present, Operations.inline_literal() may be used:

op.bulk_insert(accounts_table,[

{'id':1, 'name':'John Smith','create_date':op.inline_literal("2010-10-05")},

{'id':2, 'name':'Ed Williams','create_date':op.inline_literal("2007-05-27")},

{'id':3, 'name':'Wendy Jones','create_date':op.inline_literal("2008-08-15")},

],multiinsert=False

)

When using Operations.inline_literal() in conjunction withOperations.bulk_insert(), in order for the statement to work in “online” (e.g. non –sql)mode, the multiinsert flag should be set to False, which will have the effect of individual INSERTstatements being emitted to the database, each with a distinct VALUES clause, so that the “inline” valuescan still be rendered, rather than attempting to pass the values as bound parameters.

New in version 0.6.4: Operations.inline_literal() can now be used withOperations.bulk_insert(), and the multiinsert flag has been added to assist in thisusage when running in “online” mode.

Parameters

• table – a table object which represents the target of the INSERT.

• rows – a list of dictionaries indicating rows.

• multiinsert – when at its default of True and –sql mode is not enabled, the INSERTstatement will be executed using “executemany()” style, where all elements in the list ofdictionaries are passed as bound parameters in a single list. Setting this to False resultsin individual INSERT statements being emitted per parameter set, and is needed in thosecases where non-literal values are present in the parameter sets.


create_check_constraint(name, source, condition, schema=None, **kw)Issue a “create check constraint” instruction using the current migration context.

e.g.:

from alembic import opfrom sqlalchemy.sql import column, func

op.create_check_constraint("ck_user_name_len","user",func.len(column('name')) > 5

)

CHECK constraints are usually against a SQL expression, so ad-hoc table metadata is usually needed. Thefunction will convert the given arguments into a sqlalchemy.schema.CheckConstraint boundto an anonymous table in order to emit the CREATE statement.

Parameters

• name – Name of the check constraint. The name is necessary so that an ALTER statementcan be emitted. For setups that use an automated naming scheme such as that described


http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.CheckConstraint


at Configuring Constraint Naming Conventions, name here can be None, as the eventlistener will apply the name to the constraint object when it is associated with the table.

• source – String name of the source table.

• condition – SQL expression that’s the condition of the constraint. Can be a string orSQLAlchemy expression language structure.

• deferrable – optional bool. If set, emit DEFERRABLE or NOT DEFERRABLE whenissuing DDL for this constraint.

• initially – optional string. If set, emit INITIALLY <value> when issuing DDL forthis constraint.



create_foreign_key(name, source, referent, local_cols, remote_cols, onupdate=None,ondelete=None, deferrable=None, initially=None, match=None,source_schema=None, referent_schema=None, **dialect_kw)

Issue a “create foreign key” instruction using the current migration context.

e.g.:

from alembic import opop.create_foreign_key(

"fk_user_address", "address","user", ["user_id"], ["id"])

This internally generates a Table object containing the necessary columns, then generates a newForeignKeyConstraint object which it then associates with the Table. Any event listeners as-sociated with this action will be fired off normally. The AddConstraint construct is ultimately used togenerate the ALTER statement.

Parameters

• name – Name of the foreign key constraint. The name is necessary so that an ALTERstatement can be emitted. For setups that use an automated naming scheme such as thatdescribed at Configuring Constraint Naming Conventions, name here can be None, as theevent listener will apply the name to the constraint object when it is associated with thetable.

• source – String name of the source table.

• referent – String name of the destination table.

• local_cols – a list of string column names in the source table.

• remote_cols – a list of string column names in the remote table.

• onupdate – Optional string. If set, emit ON UPDATE <value> when issuing DDL forthis constraint. Typical values include CASCADE, DELETE and RESTRICT.

• ondelete – Optional string. If set, emit ON DELETE <value> when issuing DDL forthis constraint. Typical values include CASCADE, DELETE and RESTRICT.


• source_schema – Optional schema name of the source table.

• referent_schema – Optional schema name of the destination table.

57







http://www.sqlalchemy.org/docs/core/ddl.html#sqlalchemy.schema.AddConstraint



create_index(name, table_name, columns, schema=None, unique=False, quote=None, **kw)Issue a “create index” instruction using the current migration context.

e.g.:

from alembic import opop.create_index('ik_test', 't1', ['foo', 'bar'])

Functional indexes can be produced by using the sqlalchemy.sql.expression.text() con-struct:

from alembic import opfrom sqlalchemy import textop.create_index('ik_test', 't1', [text('lower(foo)')])

New in version 0.6.7: support for making use of the text() construct in conjunction withOperations.create_index() in order to produce functional expressions within CREATE INDEX.

Parameters

• name – name of the index.

• table_name – name of the owning table.

• columns – a list consisting of string column names and/or text() constructs.



• unique – If True, create a unique index.

• quote – Force quoting of this column’s name on or off, corresponding to True orFalse. When left at its default of None, the column identifier will be quoted accord-ing to whether the name is case sensitive (identifiers with at least one upper case characterare treated as case sensitive), or if it’s a reserved word. This flag is only needed to forcequoting of a reserved word which is not known by the SQLAlchemy dialect.

• **kw – Additional keyword arguments not mentioned above are dialect specific, andpassed in the form <dialectname>_<argname>. See the documentation regardingan individual dialect at Dialects for detail on documented arguments.

create_primary_key(name, table_name, cols, schema=None)Issue a “create primary key” instruction using the current migration context.

e.g.:

from alembic import opop.create_primary_key(

"pk_my_table", "my_table",["id", "version"]

)

This internally generates a Table object containing the necessary columns, then generates a newPrimaryKeyConstraint object which it then associates with the Table. Any event listeners as-sociated with this action will be fired off normally. The AddConstraint construct is ultimately used togenerate the ALTER statement.

Parameters

• name – Name of the primary key constraint. The name is necessary so that an ALTERstatement can be emitted. For setups that use an automated naming scheme such as that







http://www.sqlalchemy.org/docs/dialects/index.html#dialect-toplevel


http://www.sqlalchemy.org/docs/core/constraints.html#sqlalchemy.schema.PrimaryKeyConstraint




described at Configuring Constraint Naming Conventions name here can be None, as theevent listener will apply the name to the constraint object when it is associated with thetable.

• table_name – String name of the target table.

• cols – a list of string column names to be applied to the primary key constraint.



create_table(name, *columns, **kw)Issue a “create table” instruction using the current migration context.

This directive receives an argument list similar to that of the traditional sqlalchemy.schema.Tableconstruct, but without the metadata:

from sqlalchemy import INTEGER, VARCHAR, NVARCHAR, Columnfrom alembic import op

op.create_table('account',Column('id', INTEGER, primary_key=True),Column('name', VARCHAR(50), nullable=False),Column('description', NVARCHAR(200)),Column('timestamp', TIMESTAMP, server_default=func.now())

)

Note that create_table() accepts Column constructs directly from the SQLAlchemy library. Inparticular, default values to be created on the database side are specified using the server_defaultparameter, and not default which only specifies Python-side defaults:

from alembic import opfrom sqlalchemy import Column, TIMESTAMP, func

# specify "DEFAULT NOW" along with the "timestamp" columnop.create_table('account',

Column('id', INTEGER, primary_key=True),Column('timestamp', TIMESTAMP, server_default=func.now())

)

The function also returns a newly created Table object, corresponding to the table specification given,which is suitable for immediate SQL operations, in particular Operations.bulk_insert():

from sqlalchemy import INTEGER, VARCHAR, NVARCHAR, Columnfrom alembic import op

account_table = op.create_table('account',Column('id', INTEGER, primary_key=True),Column('name', VARCHAR(50), nullable=False),Column('description', NVARCHAR(200)),Column('timestamp', TIMESTAMP, server_default=func.now())

)

op.bulk_insert(account_table,[

{"name": "A1", "description": "account 1"},

59








{"name": "A2", "description": "account 2"},]

)


Parameters

• name – Name of the table

• *columns – collection of Column objects within the table, as well as optionalConstraint objects and Index objects.



• **kw – Other keyword arguments are passed to the underlyingsqlalchemy.schema.Table object created for the command.

Returns

the Table object corresponding to the parameters given.

New in version 0.7.0: - the Table object is returned.

create_unique_constraint(name, source, local_cols, schema=None, **kw)Issue a “create unique constraint” instruction using the current migration context.

e.g.:

from alembic import opop.create_unique_constraint("uq_user_name", "user", ["name"])

This internally generates a Table object containing the necessary columns, then generates a newUniqueConstraint object which it then associates with the Table. Any event listeners associatedwith this action will be fired off normally. The AddConstraint construct is ultimately used to generatethe ALTER statement.

Parameters

• name – Name of the unique constraint. The name is necessary so that an ALTER statementcan be emitted. For setups that use an automated naming scheme such as that describedat Configuring Constraint Naming Conventions, name here can be None, as the eventlistener will apply the name to the constraint object when it is associated with the table.

• source – String name of the source table. Dotted schema names are supported.

• local_cols – a list of string column names in the source table.


• initially – optional string. If set, emit INITIALLY <value> when issuing DDL forthis constraint.



drop_column(table_name, column_name, **kw)Issue a “drop column” instruction using the current migration context.


















e.g.:

drop_column('organization', 'account_id')

Parameters

• table_name – name of table

• column_name – name of column



• mssql_drop_check – Optional boolean. When True, on Microsoft SQL Server only,first drop the CHECK constraint on the column using a SQL-script-compatible block thatselects into a @variable from sys.check_constraints, then exec’s a separate DROP CON-STRAINT for that constraint.

• mssql_drop_default – Optional boolean. When True, on Microsoft SQL Serveronly, first drop the DEFAULT constraint on the column using a SQL-script-compatibleblock that selects into a @variable from sys.default_constraints, then exec’s a separateDROP CONSTRAINT for that default.

• mssql_drop_foreign_key – Optional boolean. When True, on MicrosoftSQL Server only, first drop a single FOREIGN KEY constraint on the col-umn using a SQL-script-compatible block that selects into a @variable fromsys.foreign_keys/sys.foreign_key_columns, then exec’s a separate DROP CONSTRAINTfor that default. Only works if the column has exactly one FK constraint which refers toit, at the moment.


drop_constraint(name, table_name, type_=None, schema=None, **kw)Drop a constraint of the given name, typically via DROP CONSTRAINT.

Parameters

• name – name of the constraint.

• table_name – table name.

• type_ – optional, required on MySQL. can be ‘foreignkey’, ‘primary’, ‘unique’, or‘check’.



drop_index(name, table_name=None, schema=None, **kw)Issue a “drop index” instruction using the current migration context.

e.g.:

drop_index("accounts")

Parameters

• name – name of the index.

61






• table_name – name of the owning table. Some backends such as Microsoft SQL Serverrequire this.



drop_table(name, **kw)Issue a “drop table” instruction using the current migration context.

e.g.:

drop_table("accounts")

Parameters

• name – Name of the table



• **kw – Other keyword arguments are passed to the underlyingsqlalchemy.schema.Table object created for the command.

execute(sql, execution_options=None)Execute the given SQL using the current migration context.

In a SQL script context, the statement is emitted directly to the output stream. There is no return result,however, as this function is oriented towards generating a change script that can run in “offline” mode. Forfull interaction with a connected database, use the “bind” available from the context:

from alembic import opconnection = op.get_bind()

Also note that any parameterized statement here will not work in offline mode - INSERT, UPDATE andDELETE statements which refer to literal values would need to render inline expressions. For simple usecases, the inline_literal() function can be used for rudimentary quoting of string values. For“bulk” inserts, consider using bulk_insert().

For example, to emit an UPDATE statement which is equally compatible with both online and offlinemode:

from sqlalchemy.sql import table, columnfrom sqlalchemy import Stringfrom alembic import op

account = table('account',column('name', String)

)op.execute(

account.update().\where(account.c.name==op.inline_literal('account 1')).\values({'name':op.inline_literal('account 2')}))

Note above we also used the SQLAlchemy sqlalchemy.sql.expression.table() andsqlalchemy.sql.expression.column() constructs to make a brief, ad-hoc table construct justfor our UPDATE statement. A full Table construct of course works perfectly fine as well, though note it’s







http://www.sqlalchemy.org/docs/core/selectable.html#sqlalchemy.sql.expression.table

http://www.sqlalchemy.org/docs/core/sqlelement.html#sqlalchemy.sql.expression.column



a recommended practice to at least ensure the definition of a table is self-contained within the migrationscript, rather than imported from a module that may break compatibility with older migrations.

Parameters sql – Any legal SQLAlchemy expression, including:

•a string

•a sqlalchemy.sql.expression.text() construct.

•a sqlalchemy.sql.expression.insert() construct.

•a sqlalchemy.sql.expression.update(), sqlalchemy.sql.expression.insert(),or sqlalchemy.sql.expression.delete() construct.

•Pretty much anything that’s “executable” as described in SQL Expression Language Tutorial.

Parameters execution_options – Optional dictionary of execution options, will be passedto sqlalchemy.engine.Connection.execution_options().

f(name)Indicate a string name that has already had a naming convention applied to it.

This feature combines with the SQLAlchemy naming_convention feature to disambiguate constraintnames that have already had naming conventions applied to them, versus those that have not. This isnecessary in the case that the "%(constraint_name)s" token is used within a naming convention,so that it can be identified that this particular name should remain fixed.

If the Operations.f() is used on a constraint, the naming convention will not take effect:

op.add_column('t', 'x', Boolean(name=op.f('ck_bool_t_x')))

Above, the CHECK constraint generated will have the name ck_bool_t_x regardless of whether or nota naming convention is in use.

Alternatively, if a naming convention is in use, and ‘f’ is not used, names will be con-verted along conventions. If the target_metadata contains the naming convention {"ck":"ck_bool_%(table_name)s_%(constraint_name)s"}, then the output of the following:

op.add_column(‘t’, ‘x’, Boolean(name=’x’))

will be:

CONSTRAINT ck_bool_t_x CHECK (x in (1, 0)))

The function is rendered in the output of autogenerate when a particular constraint name is already con-verted, for SQLAlchemy version 0.9.4 and greater only. Even though naming_convention wasintroduced in 0.9.2, the string disambiguation service is new as of 0.9.4.


get_bind()Return the current ‘bind’.

Under normal circumstances, this is the Connection currently being used to emit SQL to the database.

In a SQL script context, this value is None. [TODO: verify this]

get_context()Return the MigrationContext object that’s currently in use.

inline_literal(value, type_=None)Produce an ‘inline literal’ expression, suitable for using in an INSERT, UPDATE, or DELETE statement.

63


http://www.sqlalchemy.org/docs/core/dml.html#sqlalchemy.sql.expression.insert

http://www.sqlalchemy.org/docs/core/dml.html#sqlalchemy.sql.expression.update

http://www.sqlalchemy.org/docs/core/dml.html#sqlalchemy.sql.expression.insert

http://www.sqlalchemy.org/docs/core/dml.html#sqlalchemy.sql.expression.delete

http://www.sqlalchemy.org/docs/core/tutorial.html#sqlexpression-toplevel

http://www.sqlalchemy.org/docs/core/connections.html#sqlalchemy.engine.Connection.execution_options

http://www.sqlalchemy.org/docs/core/connections.html#sqlalchemy.engine.Connection


When using Alembic in “offline” mode, CRUD operations aren’t compatible with SQLAlchemy’s defaultbehavior surrounding literal values, which is that they are converted into bound values and passed sep-arately into the execute() method of the DBAPI cursor. An offline SQL script needs to have theserendered inline. While it should always be noted that inline literal values are an enormous security holein an application that handles untrusted input, a schema migration is not run in this context, so literalsare safe to render inline, with the caveat that advanced types like dates may not be supported directly bySQLAlchemy.

See execute() for an example usage of inline_literal().

The environment can also be configured to attempt to render “literal” values in-line automatically, for those simple types that are supported by the dialect; seeEnvironmentContext.configure.literal_binds for this more recently added feature.

Parameters

• value – The value to render. Strings, integers, and simple numerics should be supported.Other types like boolean, dates, etc. may or may not be supported yet by various backends.

• type_ – optional - a sqlalchemy.types.TypeEngine subclass stating the typeof this value. In SQLAlchemy expressions, this is usually derived automatically from thePython type of the value itself, as well as based on the context in which the value is used.

See also:

EnvironmentContext.configure.literal_binds

rename_table(old_table_name, new_table_name, schema=None)Emit an ALTER TABLE to rename a table.

Parameters

• old_table_name – old name.

• new_table_name – new name.



class alembic.operations.BatchOperations(migration_context, impl=None)Modifies the interface Operations for batch mode.

This basically omits the table_name and schema parameters from associated methods, as these are a givenwhen running under batch mode.

See also:

Operations.batch_alter_table()

Construct a new Operations

Parameters migration_context – a MigrationContext instance.

add_column(column)Issue an “add column” instruction using the current batch migration context.

See also:

Operations.add_column()

alter_column(column_name, **kw)Issue an “alter column” instruction using the current batch migration context.

See also:






Operations.add_column()

create_check_constraint(name, condition, **kw)Issue a “create check constraint” instruction using the current batch migration context.

The batch form of this call omits the source and schema arguments from the call.

See also:

Operations.create_check_constraint()

create_foreign_key(name, referent, local_cols, remote_cols, **kw)Issue a “create foreign key” instruction using the current batch migration context.

The batch form of this call omits the source and source_schema arguments from the call.

e.g.:

with batch_alter_table("address") as batch_op:batch_op.create_foreign_key(

"fk_user_address","user", ["user_id"], ["id"])

See also:

Operations.create_foreign_key()

create_index(name, columns, **kw)Issue a “create index” instruction using the current batch migration context.

create_primary_key(name, cols, **kw)Issue a “create primary key” instruction using the current batch migration context.

The batch form of this call omits the table_name and schema arguments from the call.

See also:

Operations.create_primary_key()

create_unique_constraint(name, local_cols, **kw)Issue a “create unique constraint” instruction using the current batch migration context.

The batch form of this call omits the source and schema arguments from the call.

See also:

Operations.create_unique_constraint()

drop_column(column_name)Issue a “drop column” instruction using the current batch migration context.

See also:

Operations.drop_column()

drop_constraint(name, type_=None)Issue a “drop constraint” instruction using the current batch migration context.

The batch form of this call omits the table_name and schema arguments from the call.

See also:

Operations.drop_constraint()

drop_index(name, **kw)Issue a “drop index” instruction using the current batch migration context.

65

CHAPTER 9

Cookbook

A collection of “How-Tos”, highlighting various ways to extend Alembic.

Note: This is a new section where we hope to start cataloguing various “how-tos” we come up with based on userrequests. It is often the case that users will request a feature only to learn that simple customization can provide thesame thing. There’s only one recipe at the moment but we hope to get more soon!

9.1 Building an Up to Date Database from Scratch

There’s a theory of database migrations that says that the revisions in existence for a database should be able to gofrom an entirely blank schema to the finished product, and back again. Alembic can roll this way. Though we think it’skind of overkill, considering that SQLAlchemy itself can emit the full CREATE statements for any given model usingcreate_all(). If you check out a copy of an application, running this will give you the entire database in oneshot, without the need to run through all those migration files, which are instead tailored towards applying incrementalchanges to an existing database.

Alembic can integrate with a create_all() script quite easily. After running the create operation, tell Alembic tocreate a new version table, and to stamp it with the most recent revision (i.e. head):

# inside of a "create the database" script, first create# tables:my_metadata.create_all(engine)

# then, load the Alembic configuration and generate the# version table, "stamping" it with the most recent rev:from alembic.config import Configfrom alembic import commandalembic_cfg = Config("/path/to/yourapp/alembic.ini")command.stamp(alembic_cfg, "head")

When this approach is used, the application can generate the database using normal SQLAlchemy techniques insteadof iterating through hundreds of migration scripts. Now, the purpose of the migration scripts is relegated just tomovement between versions on out-of-date databases, not new databases. You can now remove old migration files thatare no longer represented on any existing environments.

To prune old migration files, simply delete the files. Then, in the earliest, still-remaining migration file, setdown_revision to None:

# replace this:#down_revision = '290696571ad2'

67

http://www.sqlalchemy.org/docs/core/metadata.html#sqlalchemy.schema.MetaData.create_all



# with this:down_revision = None

That file now becomes the “base” of the migration series.

9.2 Conditional Migration Elements

This example features the basic idea of a common need, that of affecting how a migration runs based on commandline switches.

The technique to use here is simple; within a migration script, inspect theEnvironmentContext.get_x_argument() collection for any additional, user-defined parameters. Then takeaction based on the presence of those arguments.

To make it such that the logic to inspect these flags is easy to use and modify, we modify our script.py.makotemplate to make this feature available in all new revision files:

"""${message}

Revision ID: ${up_revision}Revises: ${down_revision}Create Date: ${create_date}

"""

# revision identifiers, used by Alembic.revision = ${repr(up_revision)}down_revision = ${repr(down_revision)}

from alembic import opimport sqlalchemy as sa${imports if imports else ""}

from alembic import context

def upgrade():schema_upgrades()if context.get_x_argument(as_dictionary=True).get('data', None):

data_upgrades()

def downgrade():if context.get_x_argument(as_dictionary=True).get('data', None):

data_downgrades()schema_downgrades()

def schema_upgrades():"""schema upgrade migrations go here."""${upgrades if upgrades else "pass"}

def schema_downgrades():"""schema downgrade migrations go here."""${downgrades if downgrades else "pass"}

def data_upgrades():"""Add any optional data upgrade migrations here!"""pass

68 Chapter 9. Cookbook


def data_downgrades():"""Add any optional data downgrade migrations here!"""pass

Now, when we create a new migration file, the data_upgrades() and data_downgrades() placeholders willbe available, where we can add optional data migrations:

"""rev one

Revision ID: 3ba2b522d10dRevises: NoneCreate Date: 2014-03-04 18:05:36.992867

"""

# revision identifiers, used by Alembic.revision = '3ba2b522d10d'down_revision = None

from alembic import opimport sqlalchemy as safrom sqlalchemy import String, Columnfrom sqlalchemy.sql import table, column


def upgrade():schema_upgrades()if context.get_x_argument(as_dictionary=True).get('data', None):

data_upgrades()

def downgrade():if context.get_x_argument(as_dictionary=True).get('data', None):

data_downgrades()schema_downgrades()

def schema_upgrades():"""schema upgrade migrations go here."""op.create_table("my_table", Column('data', String))

def schema_downgrades():"""schema downgrade migrations go here."""op.drop_table("my_table")

def data_upgrades():"""Add any optional data upgrade migrations here!"""

my_table = table('my_table',column('data', String),

)

op.bulk_insert(my_table,[

{'data': 'data 1'},{'data': 'data 2'},{'data': 'data 3'},

])

9.2. Conditional Migration Elements 69


def data_downgrades():"""Add any optional data downgrade migrations here!"""

op.execute("delete from my_table")

To invoke our migrations with data included, we use the -x flag:

alembic -x data=true upgrade head

The EnvironmentContext.get_x_argument() is an easy way to support new commandline options withinenvironment and migration scripts.

9.3 Sharing a Connection with a Series of Migration Commands andEnvironments

It is often the case that an application will need to call upon a series of commands within alembic.command, whereit would be advantageous for all operations to proceed along a single transaction. The connectivity for a migration istypically solely determined within the env.py script of a migration environment, which is called within the scope ofa command.

The steps to take here are:

1. Produce the Connection object to use.

2. Place it somewhere that env.py will be able to access it. This can be either a. a module-level global some-where, or b. an attribute which we place into the Config.attributes dictionary (if we are on an olderAlembic version, we may also attach an attribute directly to the Config object).

3. The env.py script is modified such that it looks for this Connection and makes use of it, in lieu of buildingup its own Engine instance.

We illustrate using Config.attributes:

from alembic import command, config

cfg = config.Config("/path/to/yourapp/alembic.ini")with engine.begin() as connection:

cfg.attributes['connection'] = connectioncommand.upgrade(cfg, "head")

Then in env.py:

def run_migrations_online():connectable = config.attributes.get('connection', None)

if connectable is None:# only create Engine if we don't have a Connection# from the outsideconnectable = engine_from_config(

config.get_section(config.config_ini_section),prefix='sqlalchemy.',poolclass=pool.NullPool)

# when connectable is already a Connection object, calling# connect() gives us a *branched connection*.

with connectable.connect() as connection:




http://www.sqlalchemy.org/docs/core/connections.html#sqlalchemy.engine.Engine


context.configure(connection=connection,target_metadata=target_metadata

)


Branched Connections

Note that we are calling the connect() method, even if we are using a Connection object to start with.The effect this has when calling connect() is that SQLAlchemy passes us a branch of the original connection;it is in every way the same as the Connection we started with, except it provides nested scope; the contextwe have here as well as the close() method of this branched connection doesn’t actually close the outerconnection, which stays active for continued use.

New in version 0.7.5: Added Config.attributes.

9.3. Sharing a Connection with a Series of Migration Commands and Environments 71


http://www.sqlalchemy.org/docs/core/connections.html#sqlalchemy.engine.Connection.connect


http://www.sqlalchemy.org/docs/core/connections.html#sqlalchemy.engine.Connection.close

CHAPTER 10

API Details

This section describes some key functions used within the migration process, particularly those referenced within amigration environment’s env.py file.

10.1 Overview

The three main objects in use are the EnvironmentContext, MigrationContext, and Operations classes,pictured below.

An Alembic command begins by instantiating an EnvironmentContext object, then making it avail-able via the alembic.context proxy module. The env.py script, representing a user-configurablemigration environment, is then invoked. The env.py script is then responsible for calling upon theEnvironmentContext.configure(), whose job it is to create a MigrationContext object.

Before this method is called, there’s not yet any database connection or dialect-specific state set up. While manymethods on EnvironmentContext are usable at this stage, those which require database access, or at least ac-cess to the kind of database dialect in use, are not. Once the EnvironmentContext.configure() methodis called, the EnvironmentContext is said to be configured with database connectivity, available via a new

73


MigrationContext object. The MigrationContext is associated with the EnvironmentContext objectvia the EnvironmentContext.get_context() method.

Finally, env.py calls upon the EnvironmentContext.run_migrations() method. Within this method, anew Operations object, which provides an API for individual database migration operations, is established withinthe alembic.op proxy module. The Operations object uses the MigrationContext object ultimately as asource of database connectivity, though in such a way that it does not care if the MigrationContext is talking toa real database or just writing out SQL to a file.

10.2 The Environment Context

The EnvironmentContext class provides most of the API used within an env.py script. Within env.py,the instantated EnvironmentContext is made available via a special proxy module called alembic.context.That is, you can import alembic.context like a regular Python module, and each name you call upon it isultimately routed towards the current EnvironmentContext in use.

In particular, the key method used within env.py is EnvironmentContext.configure(), which establishesall the details about how the database will be accessed.

class alembic.environment.EnvironmentContext(config, script, **kw)Represent the state made available to an env.py script.

EnvironmentContext is normally instantiated by the commands present in the alembic.commandmodule. From within an env.py script, the current EnvironmentContext is available via thealembic.context datamember.

EnvironmentContext is also a Python context manager, that is, is intended to be used using the with:statement. A typical use of EnvironmentContext:

from alembic.config import Configfrom alembic.script import ScriptDirectory

config = Config()config.set_main_option("script_location", "myapp:migrations")script = ScriptDirectory.from_config(config)

def my_function(rev, context):'''do something with revision "rev", whichwill be the current database revision,and "context", which is the MigrationContextthat the env.py will create'''

with EnvironmentContext(config,script,fn = my_function,as_sql = False,starting_rev = 'base',destination_rev = 'head',tag = "sometag"

):script.run_env()

The above script will invoke the env.py script within the migration environment. If and when env.py callsMigrationContext.run_migrations(), the my_function() function above will be called by theMigrationContext, given the context itself as well as the current revision in the database.

74 Chapter 10. API Details


Note: For most API usages other than full blown invocation of migration scripts, the MigrationContextand ScriptDirectory objects can be created and used directly. The EnvironmentContext object isonly needed when you need to actually invoke the env.py module present in the migration environment.

Construct a new EnvironmentContext.

Parameters

• config – a Config instance.

• script – a ScriptDirectory instance.

• **kw – keyword options that will be ultimately passed along to the MigrationContextwhen EnvironmentContext.configure() is called.

begin_transaction()Return a context manager that will enclose an operation within a “transaction”, as defined by the environ-ment’s offline and transactional DDL settings.

e.g.:


begin_transaction() is intended to “do the right thing” regardless of calling context:

•If is_transactional_ddl() is False, returns a “do nothing” context manager which other-wise produces no transactional state or directives.

•If is_offline_mode() is True, returns a context manager that will invoke theDefaultImpl.emit_begin() and DefaultImpl.emit_commit() methods, which willproduce the string directives BEGIN and COMMIT on the output stream, as rendered by the targetbackend (e.g. SQL Server would emit BEGIN TRANSACTION).

•Otherwise, calls sqlalchemy.engine.Connection.begin() on the current online connec-tion, which returns a sqlalchemy.engine.Transaction object. This object demarcates a realtransaction and is itself a context manager, which will roll back if an exception is raised.

Note that a custom env.py script which has more specific transactional needs can of course manipulatethe Connection directly to produce transactional state in “online” mode.

config = NoneAn instance of Config representing the configuration file contents as well as other variables set program-matically within it.

configure(connection=None, url=None, dialect_name=None, transactional_ddl=None,transaction_per_migration=False, output_buffer=None, starting_rev=None,tag=None, template_args=None, render_as_batch=False, target_metadata=None,include_symbol=None, include_object=None, include_schemas=False, com-pare_type=False, compare_server_default=False, render_item=None, lit-eral_binds=False, upgrade_token=’upgrades’, downgrade_token=’downgrades’, alem-bic_module_prefix=’op.’, sqlalchemy_module_prefix=’sa.’, user_module_prefix=None,**kw)

Configure a MigrationContext within this EnvironmentContext which will provide databaseconnectivity and other configuration to a series of migration scripts.

Many methods on EnvironmentContext require that this method has been called in order to function,as they ultimately need to have database access or at least access to the dialect in use. Those which do aredocumented as such.

10.2. The Environment Context 75

http://www.sqlalchemy.org/docs/core/connections.html#sqlalchemy.engine.Connection.begin

http://www.sqlalchemy.org/docs/core/connections.html#sqlalchemy.engine.Transaction



The important thing needed by configure() is a means to determine what kind of database dialect isin use. An actual connection to that database is needed only if the MigrationContext is to be used in“online” mode.

If the is_offline_mode() function returns True, then no connection is needed here. Otherwise, theconnection parameter should be present as an instance of sqlalchemy.engine.Connection.

This function is typically called from the env.py script within a migration environment. It can be calledmultiple times for an invocation. The most recent Connection for which it was called is the one thatwill be operated upon by the next call to run_migrations().

General parameters:

Parameters

• connection – a Connection to use for SQL execution in “online” mode. Whenpresent, is also used to determine the type of dialect in use.

• url – a string database url, or a sqlalchemy.engine.url.URL object. The type ofdialect to be used will be derived from this if connection is not passed.

• dialect_name – string name of a dialect, such as “postgresql”, “mssql”, etc. The typeof dialect to be used will be derived from this if connection and url are not passed.

• transactional_ddl – Force the usage of “transactional” DDL on or off; this other-wise defaults to whether or not the dialect in use supports it.

• transaction_per_migration – if True, nest each migration script in a transactionrather than the full series of migrations to run.


• output_buffer – a file-like object that will be used for textual output when the --sqloption is used to generate SQL scripts. Defaults to sys.stdout if not passed here andalso not present on the Config object. The value here overrides that of the Configobject.

• output_encoding – when using --sql to generate SQL scripts, apply this encodingto the string output.

• literal_binds – when using --sql to generate SQL scripts, pass through theliteral_binds flag to the compiler so that any literal values that would ordinarilybe bound parameters are converted to plain strings.

Warning: Dialects can typically only handle simple datatypes like strings and num-bers for auto-literal generation. Datatypes like dates, intervals, and others may stillrequire manual formatting, typically using Operations.inline_literal().

Note: the literal_binds flag is ignored on SQLAlchemy versions prior to 0.8 wherethis feature is not supported.


See also:

Operations.inline_literal()

• starting_rev – Override the “starting revision” argument when using --sql mode.

• tag – a string tag for usage by custom env.py scripts. Set via the --tag option, canbe overridden here.





http://www.sqlalchemy.org/docs/core/engines.html#sqlalchemy.engine.url.URL


• template_args – dictionary of template arguments which will be added to the tem-plate argument environment when running the “revision” command. Note that the scriptenvironment is only run within the “revision” command if the –autogenerate option isused, or if the option “revision_environment=true” is present in the alembic.ini file.

• version_table – The name of the Alembic version table. The default is’alembic_version’.

• version_table_schema – Optional schema to place version table within.

Parameters specific to the autogenerate feature, when alembic revision is run with the--autogenerate feature:

Parameters

• target_metadata – a sqlalchemy.schema.MetaData object that will be con-sulted during autogeneration. The tables present will be compared against what is locallyavailable on the target Connection to produce candidate upgrade/downgrade opera-tions.

• compare_type – Indicates type comparison behavior during an autogenerate operation.Defaults to False which disables type comparison. Set to True to turn on default typecomparison, which has varied accuracy depending on backend. See Comparing Types foran example as well as information on other type comparison options.

See also:

Comparing Types

EnvironmentContext.configure.compare_server_default

• compare_server_default – Indicates server default comparison behavior during anautogenerate operation. Defaults to False which disables server default comparison. Setto True to turn on server default comparison, which has varied accuracy depending onbackend.

To customize server default comparison behavior, a callable may be specified which canfilter server default comparisons during an autogenerate operation. defaults during anautogenerate operation. The format of this callable is:

def my_compare_server_default(context, inspected_column,metadata_column, inspected_default, metadata_default,rendered_metadata_default):

# return True if the defaults are different,# False if not, or None to allow the default implementation# to compare these defaultsreturn None

context.configure(# ...compare_server_default = my_compare_server_default

)

inspected_column is a dictionary structure as returned bysqlalchemy.engine.reflection.Inspector.get_columns(), whereasmetadata_column is a sqlalchemy.schema.Column from the local modelenvironment.

A return value of None indicates to allow default server default comparison to proceed.Note that some backends such as Postgresql actually execute the two defaults on thedatabase side to compare for equivalence.




http://www.sqlalchemy.org/docs/core/reflection.html#sqlalchemy.engine.reflection.Inspector.get_columns



See also:

EnvironmentContext.configure.compare_type

• include_object – A callable function which is given the chance to return True orFalse for any object, indicating if the given object should be considered in the autogen-erate sweep.

The function accepts the following positional arguments:

– object: a SchemaItem object such as a Table, Column, IndexUniqueConstraint, or ForeignKeyConstraint object

– name: the name of the object. This is typically available via object.name.

– type: a string describing the type of object; currently "table", "column","index", "unique_constraint", or "foreign_key_constraint"

New in version 0.7.0: Support for indexes and unique constraints within theinclude_object hook.

New in version 0.7.1: Support for foreign keys within the include_object hook.

– reflected: True if the given object was produced based on table reflection, Falseif it’s from a local MetaData object.

– compare_to: the object being compared against, if available, else None.

E.g.:

def include_object(object, name, type_, reflected, compare_to):if (type_ == "column" and

not reflected andobject.info.get("skip_autogenerate", False)):return False

else:return True

context.configure(# ...include_object = include_object

)

EnvironmentContext.configure.include_object can alsobe used to filter on specific schemas to include or omit, when theEnvironmentContext.configure.include_schemas flag is set to True.The Table.schema attribute on each Table object reflected will indicate the name ofthe schema from which the Table originates.


See also:

EnvironmentContext.configure.include_schemas

• include_symbol – A callable function which, given a table name and schema name(may be None), returns True or False, indicating if the given table should be consideredin the autogenerate sweep.

Deprecated since version 0.6.0: EnvironmentContext.configure.include_symbolis superceded by the more generic EnvironmentContext.configure.include_objectparameter.


http://www.sqlalchemy.org/docs/core/metadata.html#sqlalchemy.schema.SchemaItem







E.g.:

def include_symbol(tablename, schema):return tablename not in ("skip_table_one", "skip_table_two")

context.configure(# ...include_symbol = include_symbol

)

See also:

EnvironmentContext.configure.include_schemas

EnvironmentContext.configure.include_object

• render_as_batch – if True, commands which alter elements within a table will beplaced under a with batch_alter_table(): directive, so that batch migrationswill take place.


See also:


• include_schemas – If True, autogenerate will scan across all schemas located bythe SQLAlchemy get_schema_names() method, and include all differences in ta-bles found across all those schemas. When using this option, you may want to alsouse the EnvironmentContext.configure.include_object option to specifya callable which can filter the tables/schemas that get included.

See also:

EnvironmentContext.configure.include_object

• render_item – Callable that can be used to override how any schema item, i.e. column,constraint, type, etc., is rendered for autogenerate. The callable receives a string describingthe type of object, the object, and the autogen context. If it returns False, the defaultrendering method will be used. If it returns None, the item will not be rendered in thecontext of a Table construct, that is, can be used to skip columns or constraints withinop.create_table():

def my_render_column(type_, col, autogen_context):if type_ == "column" and isinstance(col, MySpecialCol):

return repr(col)else:

return False

context.configure(# ...render_item = my_render_column

)

Available values for the type string include: "column", "primary_key","foreign_key", "unique", "check", "type", "server_default".

See also:

Affecting the Rendering of Types Themselves


http://www.sqlalchemy.org/docs/core/reflection.html#sqlalchemy.engine.reflection.Inspector.get_schema_names


• upgrade_token – When autogenerate completes, the text of the candidate upgradeoperations will be present in this template variable when script.py.mako is rendered.Defaults to upgrades.

• downgrade_token – When autogenerate completes, the text of the candidate down-grade operations will be present in this template variable when script.py.mako isrendered. Defaults to downgrades.

• alembic_module_prefix – When autogenerate refers to Alembicalembic.operations constructs, this prefix will be used (i.e. op.create_table)Defaults to “op.”. Can be None to indicate no prefix.

• sqlalchemy_module_prefix – When autogenerate refers to SQLAlchemyColumn or type classes, this prefix will be used (i.e. sa.Column("somename",sa.Integer)) Defaults to “sa.”. Can be None to indicate no prefix. Note that whendialect-specific types are rendered, autogenerate will render them using the dialect modulename, i.e. mssql.BIT(), postgresql.UUID().

• user_module_prefix – When autogenerate refers to a SQLAlchemy type (e.g.TypeEngine) where the module name is not under the sqlalchemy namespace, thisprefix will be used within autogenerate. If left at its default of None, the __module__attribute of the type is used to render the import module. It’s a good practice to set this andto have all custom types be available from a fixed module space, in order to future-proofmigration files against reorganizations in modules.

Changed in version 0.7.0: EnvironmentContext.configure.user_module_prefixno longer defaults to the value of EnvironmentContext.configure.sqlalchemy_module_prefixwhen left at None; the __module__ attribute is now used.

New in version 0.6.3: added EnvironmentContext.configure.user_module_prefix

See also:

Controlling the Module Prefix

Parameters specific to individual backends:

Parameters

• mssql_batch_separator – The “batch separator” which will be placed betweeneach statement when generating offline SQL Server migrations. Defaults to GO. Note thisis in addition to the customary semicolon ; at the end of each statement; SQL Serverconsiders the “batch separator” to denote the end of an individual statement execution,and cannot group certain dependent operations in one step.

• oracle_batch_separator – The “batch separator” which will be placed betweeneach statement when generating offline Oracle migrations. Defaults to /. Oracle doesn’tadd a semicolon between statements like most other backends.

execute(sql, execution_options=None)Execute the given SQL using the current change context.

The behavior of execute() is the same as that of Operations.execute(). Please see that func-tion’s documentation for full detail including caveats and limitations.

This function requires that a MigrationContext has first been made available via configure().

get_bind()Return the current ‘bind’.

In “online” mode, this is the sqlalchemy.engine.Connection currently being used to emit SQLto the database.






get_context()Return the current MigrationContext object.

If EnvironmentContext.configure() has not been called yet, raises an exception.

get_head_revision()Return the hex identifier of the ‘head’ script revision.

If the script directory has multiple heads, this method raises a CommandError;EnvironmentContext.get_head_revisions() should be preferred.

This function does not require that the MigrationContext has been configured.

See also:

EnvironmentContext.get_head_revisions()

get_head_revisions()Return the hex identifier of the ‘heads’ script revision(s).

This returns a tuple containing the version number of all heads in the script directory.



get_revision_argument()Get the ‘destination’ revision argument.

This is typically the argument passed to the upgrade or downgrade command.

If it was specified as head, the actual version number is returned; if specified as base, None is returned.


get_starting_revision_argument()Return the ‘starting revision’ argument, if the revision was passed using start:end.

This is only meaningful in “offline” mode. Returns None if no value is available or was configured.


get_tag_argument()Return the value passed for the --tag argument, if any.

The --tag argument is not used directly by Alembic, but is available for custom env.py configurationsthat wish to use it; particularly for offline generation scripts that wish to generate tagged filenames.


See also:

EnvironmentContext.get_x_argument() - a newer and more open ended system of extendingenv.py scripts via the command line.

get_x_argument(as_dictionary=False)Return the value(s) passed for the -x argument, if any.

The -x argument is an open ended flag that allows any user-defined value or values to be passed on thecommand line, then available here for consumption by a custom env.py script.

The return value is a list, returned directly from the argparse structure. If as_dictionary=True ispassed, the x arguments are parsed using key=value format into a dictionary that is then returned.



For example, to support passing a database URL on the command line, the standard env.py script can bemodified like this:

cmd_line_url = context.get_x_argument(as_dictionary=True).get('dbname')

if cmd_line_url:engine = create_engine(cmd_line_url)

else:engine = engine_from_config(

config.get_section(config.config_ini_section),prefix='sqlalchemy.',poolclass=pool.NullPool)

This then takes effect by running the alembic script as:

alembic -x dbname=postgresql://user:pass@host/dbname upgrade head



See also:

EnvironmentContext.get_tag_argument()

Config.cmd_opts

is_offline_mode()Return True if the current migrations environment is running in “offline mode”.

This is True or False depending on the the --sql flag passed.


is_transactional_ddl()Return True if the context is configured to expect a transactional DDL capable backend.

This defaults to the type of database in use, and can be overridden by the transactional_ddl argu-ment to configure()


run_migrations(**kw)Run migrations as determined by the current command line configuration as well as versioning informationpresent (or not) in the current database connection (if one is present).

The function accepts optional **kw arguments. If these are passed, they are sent directly to theupgrade() and downgrade() functions within each target revision file. By modifying thescript.py.mako file so that the upgrade() and downgrade() functions accept arguments, pa-rameters can be passed here so that contextual information, usually information to identify a particulardatabase in use, can be passed from a custom env.py script to the migration functions.


script = NoneAn instance of ScriptDirectory which provides programmatic access to version files within theversions/ directory.

static_output(text)Emit text directly to the “offline” SQL stream.

Typically this is for emitting comments that start with –. The statement is not treated as a SQL execution,no ; or batch separator is added, etc.



10.3 The Migration Context

class alembic.migration.MigrationContext(dialect, connection, opts, environ-ment_context=None)

Represent the database state made available to a migration script.

MigrationContext is the front end to an actual database connection, or alternatively a string output streamgiven a particular database dialect, from an Alembic perspective.

When inside the env.py script, the MigrationContext is available via theEnvironmentContext.get_context() method, which is available at alembic.context:

# from within env.py scriptfrom alembic import contextmigration_context = context.get_context()

For usage outside of an env.py script, such as for utility routines that want to check thecurrent version in the database, the MigrationContext.configure() method to create newMigrationContext objects. For example, to get at the current revision in the database usingMigrationContext.get_current_revision():

# in any application, outside of an env.py scriptfrom alembic.migration import MigrationContextfrom sqlalchemy import create_engine

engine = create_engine("postgresql://mydatabase")conn = engine.connect()

context = MigrationContext.configure(conn)current_rev = context.get_current_revision()

The above context can also be used to produce Alembic migration operations with an Operations instance:

# in any application, outside of the normal Alembic environmentfrom alembic.operations import Operationsop = Operations(context)op.alter_column("mytable", "somecolumn", nullable=True)

bindReturn the current “bind”.

In online mode, this is an instance of sqlalchemy.engine.Connection, and is suit-able for ad-hoc execution of any kind of usage described in SQL Expression LanguageTutorial as well as for usage with the sqlalchemy.schema.Table.create() andsqlalchemy.schema.MetaData.create_all() methods of Table, MetaData.

Note that when “standard output” mode is enabled, this bind will be a “mock” connection handler thatcannot return results and is only appropriate for a very limited subset of commands.

configReturn the Config used by the current environment, if any.


classmethod configure(connection=None, url=None, dialect_name=None, environ-ment_context=None, opts=None)

Create a new MigrationContext.

This is a factory method usually called by EnvironmentContext.configure().

Parameters

10.3. The Migration Context 83




http://www.sqlalchemy.org/docs/core/metadata.html#sqlalchemy.schema.Table.create





• connection – a Connection to use for SQL execution in “online” mode. Whenpresent, is also used to determine the type of dialect in use.

• url – a string database url, or a sqlalchemy.engine.url.URL object. The type ofdialect to be used will be derived from this if connection is not passed.

• dialect_name – string name of a dialect, such as “postgresql”, “mssql”, etc. The typeof dialect to be used will be derived from this if connection and url are not passed.

• opts – dictionary of options. Most other options accepted byEnvironmentContext.configure() are passed via this dictionary.

execute(sql, execution_options=None)Execute a SQL construct or string statement.

The underlying execution mechanics are used, that is if this is “offline mode” the SQL is written to theoutput buffer, otherwise the SQL is emitted on the current SQLAlchemy connection.

get_current_heads()Return a tuple of the current ‘head versions’ that are represented in the target database.

For a migration stream without branches, this will be a single value, synonymous with that ofMigrationContext.get_current_revision(). However when multiple unmerged branchesexist within the target database, the returned tuple will contain a value for each head.

If this MigrationContext was configured in “offline” mode, that is with as_sql=True, thestarting_rev parameter is returned in a one-length tuple.

If no version table is present, or if there are no revisions present, an empty tuple is returned.


get_current_revision()Return the current revision, usually that which is present in the alembic_version table in the database.

This method intends to be used only for a migration stream that does not contain unmergedbranches in the target database; if there are multiple branches present, an exception is raised. TheMigrationContext.get_current_heads() should be preferred over this method going forwardin order to be compatible with branch migration support.

If this MigrationContext was configured in “offline” mode, that is with as_sql=True, thestarting_rev parameter is returned instead, if any.

run_migrations(**kw)Run the migration scripts established for this MigrationContext, if any.

The commands in alembic.command will set up a function that is ultimately passed to theMigrationContext as the fn argument. This function represents the “work” that will be done whenMigrationContext.run_migrations() is called, typically from within the env.py script ofthe migration environment. The “work function” then provides an iterable of version callables and otherversion information which in the case of the upgrade or downgrade commands are the list of versionscripts to invoke. Other commands yield nothing, in the case that a command wants to run some otheroperation against the database such as the current or stamp commands.

Parameters **kw – keyword arguments here will be passed to each migration callable, that isthe upgrade() or downgrade() method within revision scripts.

stamp(script_directory, revision)Stamp the version table with a specific revision.

This method calculates those branches to which the given revision can apply, and updates those branchesas though they were migrated towards that revision (either up or down). If no current branches include therevision, it is added as a new branch head.



http://www.sqlalchemy.org/docs/core/engines.html#sqlalchemy.engine.url.URL



10.4 The Operations Object

Within migration scripts, actual database migration operations are handled via an instance of Operations. SeeOperation Reference for an overview of this object.

10.5 Commands

Alembic commands are all represented by functions in the alembic.command package. They all accept the samestyle of usage, being sent the Config object as the first argument.

Commands can be run programmatically, by first constructing a Config object, as in:

from alembic.config import Configfrom alembic import commandalembic_cfg = Config("/path/to/yourapp/alembic.ini")command.upgrade(alembic_cfg, "head")

In many cases, and perhaps more often than not, an application will wish to call upon a series of Alembic commandsand/or other features. It is usually a good idea to link multiple commands along a single connection and transaction,if feasible. This can be achieved using the Config.attributes dictionary in order to share a connection:

with engine.begin() as connection:alembic_cfg.attributes['connection'] = connectioncommand.upgrade(alembic_cfg, "head")

This recipe requires that env.py consumes this connection argument; see the example in Sharing a Connection witha Series of Migration Commands and Environments for details.

To write small API functions that make direct use of database and script directory information, rather than just runningone of the built-in commands, use the ScriptDirectory and MigrationContext classes directly.

alembic.command.branches(config, verbose=False)Show current branch points

alembic.command.current(config, verbose=False, head_only=False)Display the current revision for a database.

alembic.command.downgrade(config, revision, sql=False, tag=None)Revert to a previous version.

alembic.command.heads(config, verbose=False, resolve_dependencies=False)Show current available heads in the script directory

alembic.command.history(config, rev_range=None, verbose=False)List changeset scripts in chronological order.

alembic.command.init(config, directory, template=’generic’)Initialize a new scripts directory.

alembic.command.list_templates(config)List available templates

alembic.command.merge(config, revisions, message=None, branch_label=None, rev_id=None)Merge two revisions together. Creates a new migration file.


10.4. The Operations Object 85


See also:


alembic.command.revision(config, message=None, autogenerate=False, sql=False, head=’head’,splice=False, branch_label=None, version_path=None, rev_id=None)

Create a new revision file.

alembic.command.show(config, rev)Show the revision(s) denoted by the given symbol.

alembic.command.stamp(config, revision, sql=False, tag=None)‘stamp’ the revision table with the given revision; don’t run any migrations.

alembic.command.upgrade(config, revision, sql=False, tag=None)Upgrade to a later version.

10.6 Configuration

The Config object represents the configuration passed to the Alembic environment. From an API usage perspective,it is needed for the following use cases:

• to create a ScriptDirectory , which allows you to work with the actual script files in a migration environ-ment

• to create an EnvironmentContext, which allows you to actually run the env.py module within the mi-gration environment

• to programatically run any of the commands in the alembic.command module.

The Config is not needed for these cases:

• to instantiate a MigrationContext directly - this object only needs a SQLAlchemy connection or dialectname.

• to instantiate a Operations object - this object only needs a MigrationContext.

class alembic.config.Config(file_=None, ini_section=’alembic’, output_buffer=None, std-out=<open file ‘<stdout>’, mode ‘w’>, cmd_opts=None, con-fig_args=immutabledict({}), attributes=None)

Represent an Alembic configuration.

Within an env.py script, this is available via the EnvironmentContext.config attribute, which in turnis available at alembic.context:


some_param = context.config.get_main_option("my option")

When invoking Alembic programatically, a new Config can be created by passing the name of an .ini file tothe constructor:

from alembic.config import Configalembic_cfg = Config("/path/to/yourapp/alembic.ini")

With a Config object, you can then run Alembic commands programmatically using the directives inalembic.command.

The Config object can also be constructed without a filename. Values can be set programmatically, and newsections will be created as needed:



from alembic.config import Configalembic_cfg = Config()alembic_cfg.set_main_option("script_location", "myapp:migrations")alembic_cfg.set_main_option("url", "postgresql://foo/bar")alembic_cfg.set_section_option("mysection", "foo", "bar")

For passing non-string values to environments, such as connections and engines, use theConfig.attributes dictionary:

with engine.begin() as connection:alembic_cfg.attributes['connection'] = connectioncommand.upgrade(alembic_cfg, "head")

Parameters

• file_ – name of the .ini file to open.

• ini_section – name of the main Alembic section within the .ini file

• output_buffer – optional file-like input buffer which will be passed to theMigrationContext - used to redirect the output of “offline generation” when usingAlembic programmatically.

• stdout – buffer where the “print” output of commands will be sent. Defaults tosys.stdout.

New in version 0.4.

• config_args – A dictionary of keys and values that will be used for substitution in thealembic config file. The dictionary as given is copied to a new one, stored locally as theattribute .config_args. When the Config.file_config attribute is first invoked,the replacement variable here will be added to this dictionary before the dictionary ispassed to SafeConfigParser() to parse the .ini file.


• attributes – optional dictionary of arbitrary Python keys/values, which will be popu-lated into the Config.attributes dictionary.


See also:

Sharing a Connection with a Series of Migration Commands and Environments

Construct a new Config

attributesA Python dictionary for storage of additional state.

This is a utility dictionary which can include not just strings but engines, connections, schema objects, oranything else. Use this to pass objects into an env.py script, such as passing a Connection when callingcommands from alembic.command programmatically.


See also:


Config.attributes

10.6. Configuration 87


cmd_opts = NoneThe command-line options passed to the alembic script.

Within an env.py script this can be accessed via the EnvironmentContext.config attribute.


See also:

EnvironmentContext.get_x_argument()

config_file_name = NoneFilesystem path to the .ini file in use.

config_ini_section = NoneName of the config file section to read basic configuration from. Defaults to alembic, that is the[alembic] section of the .ini file. This value is modified using the -n/--name option to the Alembicrunnier.

file_configReturn the underlying ConfigParser object.

Direct access to the .ini file is available here, though the Config.get_section() andConfig.get_main_option() methods provide a possibly simpler interface.

get_main_option(name, default=None)Return an option from the ‘main’ section of the .ini file.

This defaults to being a key from the [alembic] section, unless the -n/--name flag were used toindicate a different section.

get_section(name)Return all the configuration options from a given .ini file section as a dictionary.

get_section_option(section, name, default=None)Return an option from the given section of the .ini file.

get_template_directory()Return the directory where Alembic setup templates are found.

This method is used by the alembic init and list_templates commands.

print_stdout(text, *arg)Render a message to standard out.

set_main_option(name, value)Set an option programmatically within the ‘main’ section.

This overrides whatever was in the .ini file.

set_section_option(section, name, value)Set an option programmatically within the given section.

The section is created if it doesn’t exist already. The value here will override whatever was in the .ini file.

alembic.config.main(argv=None, prog=None, **kwargs)The console runner function for Alembic.

10.7 Script Directory

The ScriptDirectory object provides programmatic access to the Alembic version files present in the filesystem.



class alembic.script.Script(module, rev_id, path)Represent a single revision file in a versions/ directory.

The Script instance is returned by methods such as ScriptDirectory.iterate_revisions().

docReturn the docstring given in the script.

longdocReturn the docstring given in the script.

module = NoneThe Python module representing the actual script itself.

path = NoneFilesystem path of the script.

class alembic.script.ScriptDirectory(dir, file_template=’%(rev)s_%(slug)s’, trun-cate_slug_length=40, version_locations=None, source-less=False, output_encoding=’utf-8’)

Provides operations upon an Alembic script directory.

This object is useful to get information as to current revisions, most notably being able to get at the “head”revision, for schemes that want to test if the current revision in the database is the most recent:

from alembic.script import ScriptDirectoryfrom alembic.config import Configconfig = Config()config.set_main_option("script_location", "myapp:migrations")script = ScriptDirectory.from_config(config)

head_revision = script.get_current_head()

as_revision_number(id_)Convert a symbolic revision, i.e. ‘head’ or ‘base’, into an actual revision number.

classmethod from_config(config)Produce a new ScriptDirectory given a Config instance.

The Config need only have the script_location key present.

generate_revision(revid, message, head=None, refresh=False, splice=False,branch_labels=None, version_path=None, depends_on=None, **kw)

Generate a new revision file.

This runs the script.py.mako template, given template arguments, and creates a new file.

Parameters

• revid – String revision id. Typically this comes from alembic.util.rev_id().

• message – the revision message, the one passed by the -m argument to the revisioncommand.

• head – the head revision to generate against. Defaults to the current “head” if no branchesare present, else raises an exception.


• splice – if True, allow the “head” version to not be an actual head; otherwise, theselected head must be a head (e.g. endpoint) revision.

• refresh – deprecated.

10.7. Script Directory 89


get_base()Return the “base” revision as a string.

This is the revision number of the script that has a down_revision of None.

If the script directory has multiple bases, an error is raised; ScriptDirectory.get_bases() shouldbe preferred.

get_bases()return all “base” revisions as strings.

This is the revision number of all scripts that have a down_revision of None.


get_current_head()Return the current head revision.

If the script directory has multiple heads due to branching, an error is raised;ScriptDirectory.get_heads() should be preferred.

Returns a string revision number.

See also:

ScriptDirectory.get_heads()

get_heads()Return all “versioned head” revisions as strings.

This is normally a list of length one, unless branches are present. TheScriptDirectory.get_current_head() method can be used normally when a script di-rectory has only one head.

Returns a tuple of string revision numbers.

get_revision(id_)Return the Script instance with the given rev id.

See also:

ScriptDirectory.get_revisions()

get_revisions(id_)Return the Script instance with the given rev identifier, symbolic name, or sequence of identifiers.


iterate_revisions(upper, lower)Iterate through script revisions, starting at the given upper revision identifier and ending at the lower.

The traversal uses strictly the down_revision marker inside each migration script, so it is a requirementthat upper >= lower, else you’ll get nothing back.

The iterator yields Script objects.

See also:

RevisionMap.iterate_revisions()

run_env()Run the script environment.

This basically runs the env.py script present in the migration environment. It is called exclusively by thecommand functions in alembic.command.



walk_revisions(base=’base’, head=’heads’)Iterate through all revisions.

Parameters

• base – the base revision, or “base” to start from the empty revision.

• head – the head revision; defaults to “heads” to indicate all head revisions. May also be“head” to indicate a single head revision.

Changed in version 0.7.0: the “head” identifier now refers to the head of a non-branchedrepository only; use “heads” to refer to the set of all head branches simultaneously.

10.8 Revision

The RevisionMap object serves as the basis for revision management, used exclusively by ScriptDirectory .

class alembic.revision.Revision(revision, down_revision, dependencies=None,branch_labels=None)

Base class for revisioned objects.

The Revision class is the base of the more public-facing Script object, which represents a migration script.The mechanics of revision management and traversal are encapsulated within Revision, while Script ap-plies this logic to Python files in a version directory.

branch_labels = NoneOptional string/tuple of symbolic names to apply to this revision’s branch

dependencies = NoneAdditional revisions which this revision is dependent on.

From a migration standpoint, these dependencies are added to the down_revision to form the full iteration.However, the separation of down_revision from “dependencies” is to assist in navigating a history thatcontains many branches, typically a multi-root scenario.

down_revision = NoneThe down_revision identifier(s) within the migration script.

Note that the total set of “down” revisions is down_revision + dependencies.

is_baseReturn True if this Revision is a ‘base’ revision.

is_branch_pointReturn True if this Script is a branch point.

A branchpoint is defined as a Script which is referred to by more than one succeeding Script, that ismore than one Script has a down_revision identifier pointing here.

is_headReturn True if this Revision is a ‘head’ revision.

This is determined based on whether any other Script within the ScriptDirectory refers to thisScript. Multiple heads can be present.

is_merge_pointReturn True if this Script is a merge point.

nextrev = frozenset([])following revisions, based on down_revision only.

10.8. Revision 91


revision = NoneThe string revision number.

class alembic.revision.RevisionMap(generator)Maintains a map of Revision objects.

RevisionMap is used by ScriptDirectory to maintain and traverse the collection of Script objects,which are themselves instances of Revision.

Construct a new RevisionMap.

Parameters generator – a zero-arg callable that will generate an iterable of Revision in-stances to be used. These are typically Script subclasses within regular Alembic use.

add_revision(revision, _replace=False)add a single revision to an existing map.

This method is for single-revision use cases, it’s not appropriate for fully populating an entire revisionmap.

basesAll “base” revisions as strings.

These are revisions that have a down_revision of None, or empty tuple.


get_current_head(branch_label=None)Return the current head revision.

If the script directory has multiple heads due to branching, an error is raised;ScriptDirectory.get_heads() should be preferred.

Parameters branch_label – optional branch name which will limit the heads considered tothose which include that branch_label.

Returns a string revision number.

See also:

ScriptDirectory.get_heads()

get_revision(id_)Return the Revision instance with the given rev id.

If a symbolic name such as “head” or “base” is given, resolves the identifier into the current head or baserevision. If the symbolic name refers to multiples, MultipleHeads is raised.

Supports partial identifiers, where the given identifier is matched against all identifiers that start with thegiven characters; if there is exactly one match, that determines the full revision.

get_revisions(id_)Return the Revision instances with the given rev id or identifiers.

May be given a single identifier, a sequence of identifiers, or the special symbols “head” or “base”. Theresult is a tuple of one or more identifiers, or an empty tuple in the case of “base”.

In the cases where ‘head’, ‘heads’ is requested and the revision map is empty, returns an empty tuple.

Supports partial identifiers, where the given identifier is matched against all identifiers that start with thegiven characters; if there is exactly one match, that determines the full revision.

headsAll “head” revisions as strings.

This is normally a tuple of length one, unless unmerged branches are present.




iterate_revisions(upper, lower, implicit_base=False, inclusive=False, as-sert_relative_length=True)

Iterate through script revisions, starting at the given upper revision identifier and ending at the lower.

The traversal uses strictly the down_revision marker inside each migration script, so it is a requirementthat upper >= lower, else you’ll get nothing back.

The iterator yields Revision objects.

10.9 Autogeneration

Alembic 0.3 introduces a small portion of the autogeneration system as a public API.

alembic.autogenerate.compare_metadata(context, metadata)Compare a database schema to that given in a MetaData instance.

The database connection is presented in the context of a MigrationContext object, which provides databaseconnectivity as well as optional comparison functions to use for datatypes and server defaults - see the “auto-generate” arguments at EnvironmentContext.configure() for details on these.

The return format is a list of “diff” directives, each representing individual differences:

from alembic.migration import MigrationContextfrom alembic.autogenerate import compare_metadatafrom sqlalchemy.schema import SchemaItemfrom sqlalchemy.types import TypeEnginefrom sqlalchemy import (create_engine, MetaData, Column,

Integer, String, Table)import pprint

engine = create_engine("sqlite://")

engine.execute('''create table foo (

id integer not null primary key,old_data varchar,x integer

)''')

engine.execute('''create table bar (

data varchar)''')

metadata = MetaData()Table('foo', metadata,

Column('id', Integer, primary_key=True),Column('data', Integer),Column('x', Integer, nullable=False)

)Table('bat', metadata,

Column('info', String))

mc = MigrationContext.configure(engine.connect())

10.9. Autogeneration 93



diff = compare_metadata(mc, metadata)pprint.pprint(diff, indent=2, width=20)

Output:

[ ( 'add_table',Table('bat', MetaData(bind=None),

Column('info', String(), table=<bat>), schema=None)),( 'remove_table',

Table(u'bar', MetaData(bind=None),Column(u'data', VARCHAR(), table=<bar>), schema=None)),

( 'add_column',None,'foo',Column('data', Integer(), table=<foo>)),

( 'remove_column',None,'foo',Column(u'old_data', VARCHAR(), table=None)),

[ ( 'modify_nullable',None,'foo',u'x',{ 'existing_server_default': None,'existing_type': INTEGER()},

True,False)]]

Parameters

• context – a MigrationContext instance.

• metadata – a MetaData instance.

10.10 DDL Internals

These are some of the constructs used to generate migration instructions. The APIs here build off of thesqlalchemy.schema.DDLElement and sqlalchemy.ext.compiler systems.

For programmatic usage of Alembic’s migration directives, the easiest route is to use the higher level functions givenby alembic.operations.

class alembic.ddl.base.AddColumn(name, column, schema=None)

class alembic.ddl.base.AlterColumn(name, column_name, schema=None, existing_type=None, ex-isting_nullable=None, existing_server_default=None)

class alembic.ddl.base.AlterTable(table_name, schema=None)Represent an ALTER TABLE statement.

Only the string name and optional schema name of the table is required, not a full Table object.

class alembic.ddl.base.ColumnDefault(name, column_name, default, **kw)

class alembic.ddl.base.ColumnName(name, column_name, newname, **kw)

class alembic.ddl.base.ColumnNullable(name, column_name, nullable, **kw)

class alembic.ddl.base.ColumnType(name, column_name, type_, **kw)



http://www.sqlalchemy.org/docs/core/ddl.html#sqlalchemy.schema.DDLElement

http://www.sqlalchemy.org/docs/core/compiler.html#module-sqlalchemy.ext.compiler


class alembic.ddl.base.DropColumn(name, column, schema=None)

class alembic.ddl.base.RenameTable(old_table_name, new_table_name, schema=None)

alembic.ddl.base.add_column(compiler, column, **kw)

alembic.ddl.base.alter_column(compiler, name)

alembic.ddl.base.alter_table(compiler, name, schema)

alembic.ddl.base.drop_column(compiler, name)

alembic.ddl.base.format_column_name(compiler, name)

alembic.ddl.base.format_server_default(compiler, default)

alembic.ddl.base.format_table_name(compiler, name, schema)

alembic.ddl.base.format_type(compiler, type_)

alembic.ddl.base.quote_dotted(name, quote)quote the elements of a dotted name

alembic.ddl.base.visit_add_column(element, compiler, **kw)

alembic.ddl.base.visit_column_default(element, compiler, **kw)

alembic.ddl.base.visit_column_name(element, compiler, **kw)

alembic.ddl.base.visit_column_nullable(element, compiler, **kw)

alembic.ddl.base.visit_column_type(element, compiler, **kw)

alembic.ddl.base.visit_drop_column(element, compiler, **kw)

alembic.ddl.base.visit_rename_table(element, compiler, **kw)

class alembic.ddl.impl.DefaultImpl(dialect, connection, as_sql, transactional_ddl, output_buffer,context_opts)

Provide the entrypoint for major migration operations, including database-specific behavioral variances.

While individual SQL/DDL constructs already provide for database-specific implementations, variances hereallow for entirely different sequences of operations to take place for a particular migration, such as SQL Server’sspecial ‘IDENTITY INSERT’ step for bulk inserts.

add_column(table_name, column, schema=None)

add_constraint(const)

alter_column(table_name, column_name, nullable=None, server_default=False, name=None,type_=None, schema=None, autoincrement=None, existing_type=None, exist-ing_server_default=None, existing_nullable=None, existing_autoincrement=None)

autogen_column_reflect(inspector, table, column_info)A hook that is attached to the ‘column_reflect’ event for when a Table is reflected from the database duringthe autogenerate process.

Dialects can elect to modify the information gathered here.

bind

bulk_insert(table, rows, multiinsert=True)

command_terminator = ‘;’

compare_server_default(inspector_column, metadata_column, rendered_metadata_default, ren-dered_inspector_default)

compare_type(inspector_column, metadata_column)

10.10. DDL Internals 95


correct_for_autogen_constraints(conn_uniques, conn_indexes, meta-data_unique_constraints, metadata_indexes)

create_index(index)

create_table(table)

drop_column(table_name, column, schema=None, **kw)

drop_constraint(const)

drop_index(index)

drop_table(table)

emit_begin()Emit the string BEGIN, or the backend-specific equivalent, on the current connection context.

This is used in offline mode and typically via EnvironmentContext.begin_transaction().

emit_commit()Emit the string COMMIT, or the backend-specific equivalent, on the current connection context.

This is used in offline mode and typically via EnvironmentContext.begin_transaction().

execute(sql, execution_options=None)

classmethod get_by_dialect(dialect)

prep_table_for_batch(table)perform any operations needed on a table before a new one is created to replace it in batch mode.

the PG dialect uses this to drop constraints on the table before the new one uses those same names.

rename_table(old_table_name, new_table_name, schema=None)

requires_recreate_in_batch(batch_op)Return True if the given BatchOperationsImpl would need the table to be recreated and copied inorder to proceed.

Normally, only returns True on SQLite when operations other than add_column are present.

start_migrations()A hook called when EnvironmentContext.run_migrations() is called.

Implementations can set up per-migration-run state here.

static_output(text)

transactional_ddl = False

class alembic.ddl.impl.ImplMeta(classname, bases, dict_)

10.10.1 MySQL

class alembic.ddl.mysql.MySQLAlterDefault(name, column_name, default, schema=None)Bases: alembic.ddl.base.AlterColumn

class alembic.ddl.mysql.MySQLChangeColumn(name, column_name, schema=None, new-name=None, type_=None, nullable=None, de-fault=False, autoincrement=None)

Bases: alembic.ddl.base.AlterColumn

class alembic.ddl.mysql.MySQLImpl(dialect, connection, as_sql, transactional_ddl, output_buffer,context_opts)

Bases: alembic.ddl.impl.DefaultImpl





correct_for_autogen_constraints(conn_unique_constraints, conn_indexes, meta-data_unique_constraints, metadata_indexes)

transactional_ddl = False

class alembic.ddl.mysql.MySQLModifyColumn(name, column_name, schema=None, new-name=None, type_=None, nullable=None, de-fault=False, autoincrement=None)

Bases: alembic.ddl.mysql.MySQLChangeColumn

10.10.2 MS-SQL

class alembic.ddl.mssql.MSSQLImpl(*arg, **kw)Bases: alembic.ddl.impl.DefaultImpl


batch_separator = ‘GO’

bulk_insert(table, rows, **kw)

drop_column(table_name, column, **kw)

emit_begin()

emit_commit()

transactional_ddl = True

alembic.ddl.mssql.mssql_add_column(compiler, column, **kw)

alembic.ddl.mssql.visit_add_column(element, compiler, **kw)

alembic.ddl.mssql.visit_column_default(element, compiler, **kw)

alembic.ddl.mssql.visit_column_nullable(element, compiler, **kw)

alembic.ddl.mssql.visit_column_type(element, compiler, **kw)

alembic.ddl.mssql.visit_rename_column(element, compiler, **kw)

alembic.ddl.mssql.visit_rename_table(element, compiler, **kw)

10.10.3 Postgresql

class alembic.ddl.postgresql.PostgresqlImpl(dialect, connection, as_sql, transactional_ddl,output_buffer, context_opts)


autogen_column_reflect(inspector, table, column_info)


10.10. DDL Internals 97



prep_table_for_batch(table)

transactional_ddl = True

alembic.ddl.postgresql.visit_rename_table(element, compiler, **kw)

10.10.4 SQLite

class alembic.ddl.sqlite.SQLiteImpl(dialect, connection, as_sql, transactional_ddl, out-put_buffer, context_opts)


add_constraint(const)



drop_constraint(const)

requires_recreate_in_batch(batch_op)Return True if the given BatchOperationsImpl would need the table to be recreated and copied inorder to proceed.

Normally, only returns True on SQLite when operations other than add_column are present.

transactional_ddl = FalseSQLite supports transactional DDL, but pysqlite does not: see: http://bugs.python.org/issue10740


http://bugs.python.org/issue10740

CHAPTER 11

Changelog

11.1 0.7.7

Released: July 22, 2015

11.1.1 feature

• [feature] [batch] Implemented support for BatchOperations.create_primary_key() andBatchOperations.create_check_constraint(). Additionally, table keyword arguments arecopied from the original reflected table, such as the “mysql_engine” keyword argument. ¶ References: #305

11.1.2 bug

• [bug] [versioning] Fixed critical issue where a complex series of branches/merges would bog down the iterationalgorithm working over redundant nodes for millions of cycles. An internal adjustment has been made so thatduplicate nodes are skipped within this iteration. ¶ References: #310

• [bug] [environment] The MigrationContext.stamp() method, added as part of the versioning refactorin 0.7 as a more granular version of command.stamp(), now includes the “create the alembic_version tableif not present” step in the same way as the command version, which was previously omitted. ¶ References:#300

• [bug] [autogenerate] Fixed bug where foreign key options including “onupdate”, “ondelete” wouldnot render within the op.create_foreign_key() directive, even though they render within a fullForeignKeyConstraint directive. ¶ References: #298

• [bug] [tests] Repaired warnings that occur when running unit tests against SQLAlchemy 1.0.5 or greater in-volving the “legacy_schema_aliasing” flag. ¶

11.2 0.7.6

Released: May 5, 2015

11.2.1 feature

• [feature] [versioning] Fixed bug where the case of multiple mergepoints that all have the identical set ofancestor revisions would fail to be upgradable, producing an assertion failure. Merge points were previously

99

https://bitbucket.org/zzzeek/alembic/issue/305/





assumed to always require at least an UPDATE in alembic_revision from one of the previous revs to the newone, however in this case, if one of the mergepoints has already been reached, the remaining mergepoints haveno row to UPDATE therefore they must do an INSERT of their target version. ¶ References: #297

• [feature] [autogenerate] Added support for type comparison functions to be not just per environment, but alsopresent on the custom types themselves, by supplying a method compare_against_backend. Added anew documentation section Comparing Types describing type comparison fully. ¶ References: #296

• [feature] [operations] Added a new option EnvironmentContext.configure.literal_binds,which will pass the literal_binds flag into the compilation of SQL constructs when using “offline”mode. This has the effect that SQL objects like inserts, updates, deletes as well as textual statementssent using text() will be compiled such that the dialect will attempt to render literal values “inline”automatically. Only a subset of types is typically supported; the Operations.inline_literal()construct remains as the construct used to force a specific literal representation of a value. TheEnvironmentContext.configure.literal_binds flag is added to the “offline” section of theenv.py files generated in new environments. ¶ References: #255

11.2.2 bug

• [bug] [batch] Fully implemented the copy_from parameter for batch mode, which previously was not func-tioning. This allows “batch mode” to be usable in conjunction with --sql. ¶ References: #289

• [bug] [batch] Repaired support for the BatchOperations.create_index() directive, which was mis-named internally such that the operation within a batch context could not proceed. The create index operationwill proceed as part of a larger “batch table recreate” operation only if recreate is set to “always”, or if thebatch operation includes other instructions that require a table recreate. ¶ References: #287

11.3 0.7.5

Released: March 19, 2015

11.3.1 feature

• [feature] [commands] Added a new feature Config.attributes, to help with the use case of sharing statesuch as engines and connections on the outside with a series of Alembic API calls; also added a new cookbooksection to describe this simple but pretty important use case.

See also:


¶

• [feature] [environment] The format of the default env.py script has been refined a bit; it now uses contextmanagers not only for the scope of the transaction, but also for connectivity from the starting engine. The engineis also now called a “connectable” in support of the use case of an external connection being passed in. ¶

• [feature] [versioning] Added support for “alembic stamp” to work when given “heads” as an argument, whenmultiple heads are present. ¶ References: #267

100 Chapter 11. Changelog








11.3.2 bug

• [bug] [autogenerate] The --autogenerate option is not valid when used in conjunction with “offline”mode, e.g. --sql. This now raises a CommandError, rather than failing more deeply later on. Pull requestcourtesy Johannes Erdfelt. ¶ References: #266, pull request bitbucket:39

• [bug] [operations] [mssql] Fixed bug where the mssql DROP COLUMN directive failed to include modifierssuch as “schema” when emitting the DDL. ¶ References: #284

• [bug] [postgresql] [autogenerate] Postgresql “functional” indexes are necessarily skipped from the autogen-erate process, as the SQLAlchemy backend currently does not support reflection of these structures. A warningis emitted both from the SQLAlchemy backend as well as from the Alembic backend for Postgresql when suchan index is detected. ¶ References: #282

• [bug] [autogenerate] [mysql] Fixed bug where MySQL backend would report dropped unique indexes and/orconstraints as both at the same time. This is because MySQL doesn’t actually have a “unique constraint”construct that reports differently than a “unique index”, so it is present in both lists. The net effect though isthat the MySQL backend will report a dropped unique index/constraint as an index in cases where the objectwas first created as a unique constraint, if no other information is available to make the decision. This differsfrom other backends like Postgresql which can report on unique constraints and unique indexes separately. ¶References: #276

• [bug] [commands] Fixed bug where using a partial revision identifier as the “starting revision” in --sql modein a downgrade operation would fail to resolve properly.

As a side effect of this change, the EnvironmentContext.get_starting_revision_argument()method will return the “starting” revision in its originally- given “partial” form in all cases, whereas previouslywhen running within the command.stamp() command, it would have been resolved to a full number beforepassing it to the EnvironmentContext. The resolution of this value to a real revision number has basicallybeen moved to a more fundamental level within the offline migration process. ¶ References: #269

11.4 0.7.4

Released: January 12, 2015

11.4.1 bug

• [bug] [postgresql] [autogenerate] Repaired issue where a server default specified without text() that rep-resented a numeric or floating point (e.g. with decimal places) value would fail in the Postgresql-specific checkfor “compare server default”; as PG accepts the value with quotes in the table specification, it’s still valid. Pullrequest courtesy Dimitris Theodorou. ¶ References: #241, pull request bitbucket:37

• [bug] [autogenerate] The rendering of a ForeignKeyConstraint will now ensure that the names of thesource and target columns are the database-side name of each column, and not the value of the .key attributeas may be set only on the Python side. This is because Alembic generates the DDL for constraints as standaloneobjects without the need to actually refer to an in-Python Table object, so there’s no step that would resolvethese Python-only key names to database column names. ¶ References: #259

• [bug] [autogenerate] Fixed bug in foreign key autogenerate where if the in-Python table used custom columnkeys (e.g. using the key=’foo’ kwarg to Column), the comparison of existing foreign keys to those specifiedin the metadata would fail, as the reflected table would not have these keys available which to match up. Foreignkey comparison for autogenerate now ensures it’s looking at the database-side names of the columns in all cases;this matches the same functionality within unique constraints and indexes. ¶ References: #260

11.4. 0.7.4 101


https://bitbucket.org/zzzeek/alembic/pull-request/39












• [bug] [autogenerate] Fixed issue in autogenerate type rendering where types that belong to modules that havethe name “sqlalchemy” in them would be mistaken as being part of the sqlalchemy. namespace. Pull reqcourtesy Bartosz Burclaf. ¶ References: #261, pull request github:17

11.5 0.7.3

Released: December 30, 2014

11.5.1 bug

• [bug] [versioning] Fixed regression in new versioning system where upgrade / history operation would fail onAttributeError if no version files were present at all. ¶ References: #258

11.6 0.7.2


11.6.1 bug

• [bug] [sqlite] [autogenerate] Adjusted the SQLite backend regarding autogen of unique constraints to workfully with the current SQLAlchemy 1.0, which now will report on UNIQUE constraints that have no name. ¶

• [bug] [batch] Fixed bug in batch where if the target table contained multiple foreign keys to the same targettable, the batch mechanics would fail with a “table already exists” error. Thanks for the help on this from LucasKahlert. ¶ References: #254

• [bug] [mysql] Fixed an issue where the MySQL routine to skip foreign-key-implicit indexes would also catchunnamed unique indexes, as they would be named after the column and look like the FK indexes. Pull requestcourtesy Johannes Erdfelt. ¶ References: #251, pull request bitbucket:35

• [bug] [oracle] [mssql] Repaired a regression in both the MSSQL and Oracle dialects whereby the overridden_exec() method failed to return a value, as is needed now in the 0.7 series. ¶ References: #253

11.7 0.7.1


11.7.1 feature

• [feature] [autogenerate] Support for autogenerate of FOREIGN KEY constraints has been added. Theseare delivered within the autogenerate process in the same manner as UNIQUE constraints, includinginclude_object support. Big thanks to Ann Kamyshnikova for doing the heavy lifting here. ¶ References:#178, pull request bitbucket:32

• [feature] [batch] Added naming_convention argument to Operations.batch_alter_table(),as this is necessary in order to drop foreign key constraints; these are often unnamed on the target database, andin the case that they are named, SQLAlchemy is as of the 0.9 series not including these names yet.

See also:



https://github.com/zzzeek/alembic/pull/17









Dropping Unnamed or Named Foreign Key Constraints

¶

• [feature] [batch] Added two new arguments Operations.batch_alter_table.reflect_args andOperations.batch_alter_table.reflect_kwargs, so that arguments may be passed directly tosuit the Table object that will be reflected.

See also:

Controlling Table Reflection

¶

11.7.2 bug

• [bug] [batch] The render_as_batch flag was inadvertently hardcoded to True, so all autogenerates werespitting out batch mode...this has been fixed so that batch mode again is only when selected in env.py. ¶

• [bug] [batch] Fixed bug where the “source_schema” argument was not correctly passed when callingBatchOperations.create_foreign_key(). Pull request courtesy Malte Marquarding. ¶ References:pull request bitbucket:34

• [bug] [batch] Repaired the inspection, copying and rendering of CHECK constraints and so-called “schema”types such as Boolean, Enum within the batch copy system; the CHECK constraint will not be “doubled” whenthe table is copied, and additionally the inspection of the CHECK constraint for its member columns will nolonger fail with an attribute error. ¶ References: #249

11.8 0.7.0

Released: November 24, 2014

11.8.1 changed

• [changed] [commands] The --head_only option to the alembic current command is deprecated; thecurrent command now lists just the version numbers alone by default; use --verbose to get at additionaloutput. ¶

• [changed] [autogenerate] The default value of the EnvironmentContext.configure.user_module_prefixparameter is no longer the same as the SQLAlchemy prefix. When omitted, user-defined types will now usethe __module__ attribute of the type class itself when rendering in an autogenerated module. ¶ References:#229

• [changed] [compatibility] Minimum SQLAlchemy version is now 0.7.6, however at least 0.8.4 is stronglyrecommended. The overhaul of the test suite allows for fully passing tests on all SQLAlchemy versions from0.7.6 on forward. ¶

11.8.2 feature

• [feature] [versioning] The “multiple heads / branches” feature has now landed. This is by far the most signif-icant change Alembic has seen since its inception; while the workflow of most commands hasn’t changed, andthe format of version files and the alembic_version table are unchanged as well, a new suite of featuresopens up in the case where multiple version files refer to the same parent, or to the “base”. Merging of branches,operating across distinct named heads, and multiple independent bases are now all supported. The feature incurs

11.8. 0.7.0 103






radical changes to the internals of versioning and traversal, and should be treated as “beta mode” for the nextseveral subsequent releases within 0.7.

See also:


¶ References: #167

• [feature] [versioning] In conjunction with support for multiple independent bases, the specific version direc-tories are now also configurable to include multiple, user-defined directories. When multiple directories exist,the creation of a revision file with no down revision requires that the starting directory is indicated; the creationof subsequent revisions along that lineage will then automatically use that directory for new files.

See also:

Setting up Multiple Version Directories

¶ References: #124

• [feature] [operations] [sqlite] Added “move and copy” workflow, where a table to be altered is copied to anew one with the new structure and the old one dropped, is now implemented for SQLite as well as all databasebackends in general using the new Operations.batch_alter_table() system. This directive providesa table-specific operations context which gathers column- and constraint-level mutations specific to that table,and at the end of the context creates a new table combining the structure of the old one with the given changes,copies data from old table to new, and finally drops the old table, renaming the new one to the existing name.This is required for fully featured SQLite migrations, as SQLite has very little support for the traditional ALTERdirective. The batch directive is intended to produce code that is still compatible with other databases, in thatthe “move and copy” process only occurs for SQLite by default, while still providing some level of sanity toSQLite’s requirement by allowing multiple table mutation operations to proceed within one “move and copy”as well as providing explicit control over when this operation actually occurs. The “move and copy” featuremay be optionally applied to other backends as well, however dealing with referential integrity constraints fromother tables must still be handled explicitly.

See also:


¶ References: #21

• [feature] [commands] Relative revision identifiers as used with alembic upgrade, alembicdowngrade and alembic history can be combined with specific revisions as well, e.g. alembicupgrade ae10+3, to produce a migration target relative to the given exact version. ¶

• [feature] [commands] New commands added: alembic show, alembic heads and alembic merge.Also, a new option --verbose has been added to several informational commands, such as alembichistory, alembic current, alembic branches, and alembic heads. alembic revisionalso contains several new options used within the new branch management system. The output of commandshas been altered in many cases to support new fields and attributes; the history command in particular nowreturns it’s “verbose” output only if --verbose is sent; without this flag it reverts to it’s older behavior ofshort line items (which was never changed in the docs). ¶

• [feature] [config] Added new argument Config.config_args, allows a dictionary of replacement vari-ables to be passed which will serve as substitution values when an API-produced Config consumes the .inifile. Pull request courtesy Noufal Ibrahim. ¶ References: pull request bitbucket:33

• [feature] [operations] The Table object is now returned when the Operations.create_table()method is used. This Table is suitable for use in subsequent SQL operations, in particular theOperations.bulk_insert() operation. ¶ References: #205









• [feature] [autogenerate] Indexes and unique constraints are now included in theEnvironmentContext.configure.include_object hook. Indexes are sent with type "index"and unique constraints with type "unique_constraint". ¶ References: #203

• [feature] SQLAlchemy’s testing infrastructure is now used to run tests. This system supports both nose andpytest and opens the way for Alembic testing to support any number of backends, parallel testing, and 3rd partydialect testing. ¶

11.8.3 bug

• [bug] [commands] The alembic revision command accepts the --sql option to suit some very ob-scure use case where the revision_environment flag is set up, so that env.py is run when alembicrevision is run even though autogenerate isn’t specified. As this flag is otherwise confusing, error messagesare now raised if alembic revision is invoked with both --sql and --autogenerate or with --sqlwithout revision_environment being set. ¶ References: #248

• [bug] [postgresql] [autogenerate] Added a rule for Postgresql to not render a “drop unique” and “drop index”given the same name; for now it is assumed that the “index” is the implicit one Postgreql generates. Futureintegration with new SQLAlchemy 1.0 features will improve this to be more resilient. ¶ References: #247

• [bug] [autogenerate] A change in the ordering when columns and constraints are dropped; autogenerate willnow place the “drop constraint” calls before the “drop column” calls, so that columns involved in those con-straints still exist when the constraint is dropped. ¶ References: #247

• [bug] [oracle] The Oracle dialect sets “transactional DDL” to False by default, as Oracle does not supporttransactional DDL. ¶ References: #245

• [bug] [autogenerate] Fixed a variety of issues surrounding rendering of Python code that contains unicodeliterals. The first is that the “quoted_name” construct that SQLAlchemy uses to represent table and columnnames as well as schema names does not repr() correctly on Py2K when the value contains unicode charac-ters; therefore an explicit stringification is added to these. Additionally, SQL expressions such as server defaultswere not being generated in a unicode-safe fashion leading to decode errors if server defaults contained non-asciicharacters. ¶ References: #243

• [bug] [operations] The Operations.add_column() directive will now additionally emit the appropri-ate CREATE INDEX statement if the Column object specifies index=True. Pull request courtesy DavidSzotten. ¶ References: #174, pull request bitbucket:29

• [bug] [autogenerate] Bound parameters are now resolved as “literal” values within the SQL expression insideof a CheckConstraint(), when rendering the SQL as a text string; supported for SQLAlchemy 0.8.0 and forward.¶ References: #219

• [bug] [autogenerate] Added a workaround for SQLAlchemy issue #3023 (fixed in 0.9.5) where a column that’spart of an explicit PrimaryKeyConstraint would not have its “nullable” flag set to False, thus producing a falseautogenerate. Also added a related correction to MySQL which will correct for MySQL’s implicit server defaultof ‘0’ when a NULL integer column is turned into a primary key column. ¶ References: #199

• [bug] [autogenerate] [mysql] Repaired issue related to the fix for #208 and others; a composite foreign keyreported by MySQL would cause a KeyError as Alembic attempted to remove MySQL’s implicitly generatedindexes from the autogenerate list. ¶ References: #240

• [bug] [autogenerate] If the “alembic_version” table is present in the target metadata, autogenerate will skipthis also. Pull request courtesy Dj Gilcrease. ¶ References: #28

• [bug] [autogenerate] The EnvironmentContext.configure.version_table andEnvironmentContext.configure.version_table_schema arguments are now honored duringthe autogenerate process, such that these names will be used as the “skip” names on both the database reflectionand target metadata sides. ¶ References: #77

11.8. 0.7.0 105
















• [bug] [templates] Revision files are now written out using the ’wb’ modifier to open(), since Mako readsthe templates with ’rb’, thus preventing CRs from being doubled up as has been observed on windows. Theencoding of the output now defaults to ‘utf-8’, which can be configured using a newly added config file parameteroutput_encoding. ¶ References: #234

• [bug] [operations] Added support for use of the quoted_name construct when using the schema argumentwithin operations. This allows a name containing a dot to be fully quoted, as well as to provide configurablequoting on a per-name basis. ¶ References: #230

• [bug] [postgresql] [autogenerate] Added a routine by which the Postgresql Alembic dialect inspectsthe server default of INTEGER/BIGINT columns as they are reflected during autogenerate for the patternnextval(<name>...) containing a potential sequence name, then queries pg_catalog to see if thissequence is “owned” by the column being reflected; if so, it assumes this is a SERIAL or BIGSERIAL columnand the server default is omitted from the column reflection as well as any kind of server_default compari-son or rendering, along with an INFO message in the logs indicating this has taken place. This allows SE-RIAL/BIGSERIAL columns to keep the SEQUENCE from being unnecessarily present within the autogenerateoperation. ¶ References: #73

• [bug] [autogenerate] The system by which autogenerate renders expressions within aIndex, the server_default of Column, and the existing_server_default ofOperations.alter_column() has been overhauled to anticipate arbitrary SQLAlchemy SQL con-structs, such as func.somefunction(), cast(), desc(), and others. The system does not, as might bepreferred, render the full-blown Python expression as originally created within the application’s source code,as this would be exceedingly complex and difficult. Instead, it renders the SQL expression against the targetbackend that’s subject to the autogenerate, and then renders that SQL inside of a text() construct as a literalSQL string. This approach still has the downside that the rendered SQL construct may not be backend-agnosticin all cases, so there is still a need for manual intervention in that small number of cases, but overall themajority of cases should work correctly now. Big thanks to Carlos Rivera for pull requests and support on this.¶ References: #197, #196, #64

• [bug] [operations] The “match” keyword is not sent to ForeignKeyConstraint byOperations.create_foreign_key() when SQLAlchemy 0.7 is in use; this keyword was addedto SQLAlchemy as of 0.8.0. ¶

11.9 0.6.7

Released: September 9, 2014

11.9.1 feature

• [feature] Added support for functional indexes when using the Operations.create_index() directive.Within the list of columns, the SQLAlchemy text() construct can be sent, embedding a literal SQL expres-sion; the Operations.create_index() will perform some hackery behind the scenes to get the Indexconstruct to cooperate. This works around some current limitations in Index which should be resolved on theSQLAlchemy side at some point. ¶ References: #222

11.9.2 bug

• [bug] [mssql] Fixed bug in MSSQL dialect where “rename table” wasn’t using sp_rename() as is requiredon SQL Server. Pull request courtesy Łukasz Bołdys. ¶ References: pull request bitbucket:26















11.10 0.6.6

Released: August 7, 2014

11.10.1 feature

• [feature] Added a new accessor MigrationContext.config, when used in conjunction with aEnvironmentContext and Config, this config will be returned. Patch courtesy Marc Abramowitz. ¶References: pull request github:10

11.10.2 bug

• [bug] A file named __init__.py in the versions/ directory is now ignored by Alembic when the col-lection of version files is retrieved. Pull request courtesy Michael Floering. ¶ References: #95, pull requestbitbucket:24

• [bug] Fixed Py3K bug where an attempt would be made to sort None against string values when autogeneratewould detect tables across multiple schemas, including the default schema. Pull request courtesy paradoxxxzero.¶ References: pull request bitbucket:23

• [bug] Autogenerate render will render the arguments within a Table construct using *[...] when the numberof columns/elements is greater than 255. Pull request courtesy Ryan P. Kelly. ¶ References: pull requestgithub:15

• [bug] Fixed bug where foreign key constraints would fail to render in autogenerate when a schema name waspresent. Pull request courtesy Andreas Zeidler. ¶ References: pull request github:14

• [bug] Some deep-in-the-weeds fixes to try to get “server default” comparison working better across platformsand expressions, in particular on the Postgresql backend, mostly dealing with quoting/not quoting of variousexpressions at the appropriate time and on a per-backend basis. Repaired and tested support for such defaults asPostgresql interval and array defaults. ¶ References: #212

• [bug] Liberalized even more the check for MySQL indexes that shouldn’t be counted in autogenerate as “drops”;this time it’s been reported that an implicitly created index might be named the same as a composite foreign keyconstraint, and not the actual columns, so we now skip those when detected as well. ¶ References: #208

11.10.3 misc

• [enhancement] When a run of Alembic command line fails due to CommandError, the output now prefixesthe string with "FAILED:", and the error is also written to the log output using log.error(). ¶ References:#209

11.11 0.6.5

Released: May 3, 2014

11.11.1 feature

• [feature] [environment] Added new feature EnvironmentContext.configure.transaction_per_migration,which when True causes the BEGIN/COMMIT pair to incur for each migration individually, rather than for

11.10. 0.6.6 107













the whole series of migrations. This is to assist with some database directives that need to be within individualtransactions, without the need to disable transactional DDL entirely. ¶ References: #201

11.11.2 bug

• [bug] [autogenerate] [mysql] This releases’ “autogenerate index detection” bug, when a MySQL table includesan Index with the same name as a column, autogenerate reported it as an “add” even though its not; this isbecause we ignore reflected indexes of this nature due to MySQL creating them implicitly. Indexes that arenamed the same as a column are now ignored on MySQL if we see that the backend is reporting that it alreadyexists; this indicates that we can still detect additions of these indexes but not drops, as we cannot distinguish abackend index same-named as the column as one that is user generated or mysql-generated. ¶ References: #202

• [bug] [autogenerate] Fixed bug where the include_object() filter would not receive the originalColumn object when evaluating a database-only column to be dropped; the object would not include the par-ent Table nor other aspects of the column that are important for generating the “downgrade” case where thecolumn is recreated. ¶ References: #200

• [bug] [environment] Fixed bug where EnvironmentContext.get_x_argument() would fail if theConfig in use didn’t actually originate from a command line call. ¶ References: #195

• [bug] [autogenerate] Fixed another bug regarding naming conventions, continuing from #183, whereadd_index() drop_index() directives would not correctly render the f() construct when the index containeda convention-driven name. ¶ References: #194

11.12 0.6.4

Released: March 28, 2014

11.12.1 feature

• [feature] The command.revision() command now returns the Script object corresponding to the newlygenerated revision. From this structure, one can get the revision id, the module documentation, and everythingelse, for use in scripts that call upon this command. Pull request courtesy Robbie Coomber. ¶ References: pullrequest bitbucket:20

11.12.2 bug

• [bug] [mssql] Added quoting to the table name when the special EXEC is run to dropany existing server defaults or constraints when the drop_column.mssql_drop_check ordrop_column.mssql_drop_default arguments are used. ¶ References: #186

• [bug] [mysql] Added/fixed support for MySQL “SET DEFAULT” / “DROP DEFAULT” phrases, which willnow be rendered if only the server default is changing or being dropped (e.g. specify None to alter_column()to indicate “DROP DEFAULT”). Also added support for rendering MODIFY rather than CHANGE when thecolumn name isn’t changing. ¶ References: #103

• [bug] Added support for the initially, match keyword arguments as well as dialect-specific keywordarguments to Operations.create_foreign_key().

tags feature

tickets 163













Altered the support for “sourceless” migration files (e.g. only .pyc or .pyo present) so that the flag “source-less=true” needs to be in alembic.ini for this behavior to take effect. ¶ References: #190

• [bug] [mssql] The feature that keeps on giving, index/unique constraint autogenerate detection, has even morefixes, this time to accommodate database dialects that both don’t yet report on unique constraints, but the back-end does report unique constraints as indexes. The logic Alembic uses to distinguish between “this is an index!”vs. “this is a unique constraint that is also reported as an index!” has now been further enhanced to not produceunwanted migrations when the dialect is observed to not yet implement get_unique_constraints() (e.g. mssql).Note that such a backend will no longer report index drops for unique indexes, as these cannot be distinguishedfrom an unreported unique index. ¶ References: #185

• [bug] Extensive changes have been made to more fully support SQLAlchemy’s new naming conventions fea-ture. Note that while SQLAlchemy has added this feature as of 0.9.2, some additional fixes in 0.9.4 are neededto resolve some of the issues:

1. The Operations object now takes into account the naming conventions that are present on theMetaData object that’s associated using target_metadata. When Operations renders a con-straint directive like ADD CONSTRAINT, it now will make use of this naming convention when it pro-duces its own temporary MetaData object.

2. Note however that the autogenerate feature in most cases generates constraints like foreign keys and uniqueconstraints with the final names intact; the only exception are the constraints implicit with a schema-typelike Boolean or Enum. In most of these cases, the naming convention feature will not take effect for theseconstraints and will instead use the given name as is, with one exception....

3. Naming conventions which use the "%(constraint_name)s" token, that is, produce a new namethat uses the original name as a component, will still be pulled into the naming convention converterand be converted. The problem arises when autogenerate renders a constraint with it’s already-generatedname present in the migration file’s source code, the name will be doubled up at render time due to thecombination of #1 and #2. So to work around this, autogenerate now renders these already-tokenizednames using the new Operations.f() component. This component is only generated if SQLAlchemy0.9.4 or greater is in use.

Therefore it is highly recommended that an upgrade to Alembic 0.6.4 be accompanied by an upgrade ofSQLAlchemy 0.9.4, if the new naming conventions feature is used.

See also:

Integration of Naming Conventions into Operations, Autogenerate

¶ References: #183

• [bug] Suppressed IOErrors which can raise when program output pipe is closed under a program like head;however this only works on Python 2. On Python 3, there is not yet a known way to suppress the BrokenPipeEr-ror warnings without prematurely terminating the program via signals. ¶ References: #160

• [bug] Fixed bug where Operations.bulk_insert() would not function properly whenOperations.inline_literal() values were used, either in –sql or non-sql mode. The values willnow render directly in –sql mode. For compatibility with “online” mode, a new flag multiinsert can be setto False which will cause each parameter set to be compiled and executed with individual INSERT statements.¶ References: #179

• [bug] [py3k] Fixed a failure of the system that allows “legacy keyword arguments” to be understood, whicharose as of a change in Python 3.4 regarding decorators. A workaround is applied that allows the code to workacross Python 3 versions. ¶ References: #175

11.12. 0.6.4 109








11.13 0.6.3

Released: February 2, 2014

11.13.1 feature

• [feature] Added new argument EnvironmentContext.configure.user_module_prefix. Thisprefix is applied when autogenerate renders a user-defined type, which here is defined as any type that isfrom a module outside of the sqlalchemy. hierarchy. This prefix defaults to None, in which case theEnvironmentContext.configure.sqlalchemy_module_prefix is used, thus preserving the cur-rent behavior. ¶ References: #171

• [feature] The ScriptDirectory system that loads migration files from a versions/ directory nowsupports so-called “sourceless” operation, where the .py files are not present and instead .pyc or .pyo filesare directly present where the .py files should be. Note that while Python 3.3 has a new system of locating.pyc/.pyo files within a directory called __pycache__ (e.g. PEP-3147), PEP-3147 maintains support forthe “source-less imports” use case, where the .pyc/.pyo are in present in the “old” location, e.g. next to the.py file; this is the usage that’s supported even when running Python3.3. ¶ References: #163

11.13.2 bug

• [bug] Added a workaround for when we call fcntl.ioctl() to get at TERMWIDTH; if the function returnszero, as is reported to occur in some pseudo-ttys, the message wrapping system is disabled in the same way asif ioctl() failed. ¶ References: #172

• [bug] Added support for autogenerate covering the use case where Table objects specified in the metadatahave an explicit schema attribute whose name matches that of the connection’s default schema (e.g. “public”for Postgresql). Previously, it was assumed that “schema” was None when it matched the “default” schema,now the comparison adjusts for this. ¶ References: #170

• [bug] The compare_metadata() public API function now takes into ac-count the settings for EnvironmentContext.configure.include_object,EnvironmentContext.configure.include_symbol, and EnvironmentContext.configure.include_schemas,in the same way that the --autogenerate command does. Pull request courtesy Roman Podoliaka. ¶References: pull request github:9

• [bug] Calling bulk_insert() with an empty list will not emit any commands on the current connection.This was already the case with --sql mode, so is now the case with “online” mode. ¶ References: #168

• [bug] Enabled schema support for index and unique constraint autodetection; previously these were non-functional and could in some cases lead to attribute errors. Pull request courtesy Dimitris Theodorou. ¶ Refer-ences: pull request bitbucket:17

• [bug] More fixes to index autodetection; indexes created with expressions like DESC or functional indexes willno longer cause AttributeError exceptions when attempting to compare the columns. ¶ References: #164

11.14 0.6.2

Released: Fri Dec 27 2013











11.14.1 feature

• [feature] [mssql] Added new argument mssql_drop_foreign_key toOperations.drop_column(). Like mssql_drop_default and mssql_drop_check, willdo an inline lookup for a single foreign key which applies to this column, and drop it. For a column with morethan one FK, you’d still need to explicitly use Operations.drop_constraint() given the name, eventhough only MSSQL has this limitation in the first place. ¶

11.14.2 bug

• [bug] Autogenerate for op.create_table() will not include a PrimaryKeyConstraint() that hasno columns. ¶

• [bug] Fixed bug in the not-internally-used ScriptDirectory.get_base() method which would fail ifcalled on an empty versions directory. ¶

• [bug] An almost-rewrite of the new unique constraint/index autogenerate detection, to accommodate a varietyof issues. The emphasis is on not generating false positives for those cases where no net change is present, asthese errors are the ones that impact all autogenerate runs:

– Fixed an issue with unique constraint autogenerate detection where a named UniqueConstraint onboth sides with column changes would render with the “add” operation before the “drop”, requiring theuser to reverse the order manually.

– Corrected for MySQL’s apparent addition of an implicit index for a foreign key column, so that it doesn’tshow up as “removed”. This required that the index/constraint autogen system query the dialect-specificimplementation for special exceptions.

– reworked the “dedupe” logic to accommodate MySQL’s bi-directional duplication of unique indexes asunique constraints, and unique constraints as unique indexes. Postgresql’s slightly different logic of dupli-cating unique constraints into unique indexes continues to be accommodated as well. Note that a uniqueindex or unique constraint removal on a backend that duplicates these may show up as a distinct “re-move_constraint()” / “remove_index()” pair, which may need to be corrected in the post-autogenerate ifmultiple backends are being supported.

– added another dialect-specific exception to the SQLite backend when dealing with unnamed unique con-straints, as the backend can’t currently report on constraints that were made with this technique, hencethey’d come out as “added” on every run.

– the op.create_table() directive will be auto-generated with the UniqueConstraint objectsinline, but will not double them up with a separate create_unique_constraint() call, which mayhave been occurring. Indexes still get rendered as distinct op.create_index() calls even when thecorresponding table was created in the same script.

– the inline UniqueConstraint within op.create_table() includes all the options likedeferrable, initially, etc. Previously these weren’t rendering.

¶ References: #157

• [bug] [mssql] The MSSQL backend will add the batch separator (e.g. "GO") in --sql mode after the finalCOMMIT statement, to ensure that statement is also processed in batch mode. Courtesy Derek Harland. ¶References: pull request bitbucket:13

11.15 0.6.1

Released: Wed Nov 27 2013

11.15. 0.6.1 111




11.15.1 feature

• [feature] Expanded the size of the “slug” generated by “revision” to 40 characters, which is also configurable bynew field truncate_slug_length; and also split on the word rather than the character; courtesy Frozen-ball. ¶ References: pull request bitbucket:12

• [feature] Support for autogeneration detection and rendering of indexes and unique constraints has been added.The logic goes through some effort in order to differentiate between true unique constraints and unique indexes,where there are some quirks on backends like Postgresql. The effort here in producing the feature and tests iscourtesy of IJL. ¶ References: #107

11.15.2 bug

• [bug] [mysql] Fixed bug where op.alter_column() in the MySQL dialect would fail to apply quotes tocolumn names that had mixed casing or spaces. ¶ References: #152

• [bug] Fixed the output wrapping for Alembic message output, so that we either get the terminal width for“pretty printing” with indentation, or if not we just output the text as is; in any case the text won’t be wrappedtoo short. ¶ References: #135

• [bug] Fixes to Py3k in-place compatibity regarding output encoding and related; the use of the new io.* packageintroduced some incompatibilities on Py2k. These should be resolved, due to the introduction of new adaptertypes for translating from io.* to Py2k file types, StringIO types. Thanks to Javier Santacruz for help with this.¶ References: pull request bitbucket:9

• [bug] Fixed py3k bug where the wrong form of next() was being called when using the list_templatescommand. Courtesy Chris Wilkes. ¶ References: #145

• [bug] Fixed bug introduced by new include_object argument where the inspected column would bemisinterpreted when using a user-defined type comparison function, causing a KeyError or similar expression-related error. Fix courtesy Maarten van Schaik. ¶

• [bug] Added the “deferrable” keyword argument to op.create_foreign_key() so that DEFERRABLEconstraint generation is supported; courtesy Pedro Romano. ¶

• [bug] Ensured that strings going to stdout go through an encode/decode phase, so that any non-ASCII charactersget to the output stream correctly in both Py2k and Py3k. Also added source encoding detection using Mako’sparse_encoding() routine in Py2k so that the __doc__ of a non-ascii revision file can be treated as unicode inPy2k. ¶ References: #137

11.16 0.6.0

Released: Fri July 19 2013

11.16.1 feature

• [feature] Added new kw argument to EnvironmentContext.configure() include_object. Thisis a more flexible version of the include_symbol argument which allows filtering of columns as well astables from the autogenerate process, and in the future will also work for types, constraints and other constructs.The fully constructed schema object is passed, including its name and type as well as a flag indicating if theobject is from the local application metadata or is reflected. ¶ References: #101

• [feature] The output of the alembic history command is now expanded to show information about eachchange on multiple lines, including the full top message, resembling the formatting of git log. ¶











• [feature] Added alembic.config.Config.cmd_opts attribute, allows access to the argparse op-tions passed to the alembic runner. ¶

• [feature] Added new command line argument -x, allows extra arguments to be appended to the command linewhich can be consumed within an env.py script by looking at context.config.cmd_opts.x, or moresimply a new method EnvironmentContext.get_x_argument(). ¶ References: #120

• [feature] Added -r argument to alembic history command, allows specification of [start]:[end]to view a slice of history. Accepts revision numbers, symbols “base”, “head”, a new symbol “current” represent-ing the current migration, as well as relative ranges for one side at a time (i.e. -r-5:head, -rcurrent:+3).Courtesy Atsushi Odagiri for this feature. ¶

• [feature] Source base is now in-place for Python 2.6 through 3.3, without the need for 2to3. Support for Python2.5 and below has been dropped. Huge thanks to Hong Minhee for all the effort on this! ¶ References: #55

11.16.2 bug

• [bug] Added support for options like “name” etc. to be rendered within CHECK constraints in autogenerate.Courtesy Sok Ann Yap. ¶ References: #125

• [bug] Repaired autogenerate rendering of ForeignKeyConstraint to include use_alter argument, if present. ¶

11.16.3 misc

• [misc] Source repository has been moved from Mercurial to Git. ¶

11.17 0.5.0

Released: Thu Apr 4 2013

Note: Alembic 0.5.0 now requires at least version 0.7.3 of SQLAlchemy to run properly. Support for 0.6 has beendropped.

11.17.1 feature

• [feature] Added version_table_schema argument to EnvironmentContext.configure(),complements the version_table argument to set an optional remote schema for the version table. CourtesyChristian Blume. ¶ References: #76

• [feature] Added output_encoding option to EnvironmentContext.configure(), used with--sql mode to apply an encoding to the output stream. ¶ References: #90

• [feature] Added Operations.create_primary_key() operation, will genenerate an ADD CON-STRAINT for a primary key. ¶ References: #93

• [feature] upgrade and downgrade commands will list the first line of the docstring out next to the versionnumber. Courtesy Hong Minhee. ¶ References: #115

• [feature] Added –head-only option to “alembic current”, will print current version plus the symbol “(head)” ifthis version is the head or not. Courtesy Charles-Axel Dein. ¶

• [feature] The rendering of any construct during autogenerate can be customized, in particular to allowspecial rendering for user-defined column, constraint subclasses, using new render_item argument toEnvironmentContext.configure(). ¶ References: #108

11.17. 0.5.0 113










11.17.2 bug

• [bug] [postgresql] Fixed format of RENAME for table that includes schema with Postgresql; the schema nameshouldn’t be in the “TO” field. ¶ References: #32

• [bug] [mssql] Fixed bug whereby double quoting would be applied to target column name during ansp_rename operation. ¶ References: #109

• [bug] [sqlite] [mysql] transactional_ddl flag for SQLite, MySQL dialects set to False. MySQL doesn’t supportit, SQLite does but current pysqlite driver does not. ¶ References: #112

• [bug] Autogenerate will render additional table keyword arguments like “mysql_engine” and others withinop.create_table(). ¶ References: #110

• [bug] Fixed bug whereby create_index() would include in the constraint columns that are added to all Tableobjects using events, externally to the generation of the constraint. This is the same issue that was fixed forunique constraints in version 0.3.2. ¶

• [bug] Worked around a backwards-incompatible regression in Python3.3 regarding argparse; running “alembic”with no arguments now yields an informative error in py3.3 as with all previous versions. Courtesy AndreyAntukh. ¶

• [bug] A host of argument name changes within migration operations for consistency. Keyword arguments willcontinue to work on the old name for backwards compatibility, however required positional arguments will not:

Operations.alter_column() - name -> new_column_name - old name will work forbackwards compatibility.

Operations.create_index() - tablename -> table_name - argument is positional.

Operations.drop_index() - tablename -> table_name - old name will work for back-wards compatibility.

Operations.drop_constraint() - tablename -> table_name - argument is positional.

Operations.drop_constraint() - type -> type_ - old name will work for backwardscompatibility

¶ References: #104

11.17.3 misc

• [change] SQLAlchemy 0.6 is no longer supported by Alembic - minimum version is 0.7.3, full support is as of0.7.9. ¶

11.18 0.4.2

Released: Fri Jan 11 2013

11.18.1 feature

• [feature] Added a README.unittests with instructions for running the test suite fully. ¶ References: #96









11.18.2 bug

• [bug] [autogenerate] Fixed bug where autogenerate would fail if a Column to be added to a table made use ofthe ”.key” paramter. ¶ References: #99

• [bug] [sqlite] The “implicit” constraint generated by a type such as Boolean or Enum will not generate anALTER statement when run on SQlite, which does not support ALTER for the purpose of adding/removingconstraints separate from the column def itself. While SQLite supports adding a CHECK constraint at thecolumn level, SQLAlchemy would need modification to support this. A warning is emitted indicating thisconstraint cannot be added in this scenario. ¶ References: #98

• [bug] Added a workaround to setup.py to prevent “NoneType” error from occuring when “setup.py test” is run.¶ References: #96

• [bug] Added an append_constraint() step to each condition withintest_autogenerate:AutogenRenderTest.test_render_fk_constraint_kwarg if the SQLAlchemy version isless than 0.8, as ForeignKeyConstraint does not auto-append prior to 0.8. ¶ References: #96

11.19 0.4.1

Released: Sun Dec 9 2012

11.19.1 feature

• [feature] Explicit error message describing the case when downgrade –sql is used without specifying specificstart/end versions. ¶ References: #66

11.19.2 bug

• [bug] Added support for autogenerate render of ForeignKeyConstraint options onupdate, ondelete, initially,and deferred. ¶ References: #92

• [bug] Autogenerate will include “autoincrement=False” in the rendered table metadata if this flag was set tofalse on the source Column object. ¶ References: #94

• [bug] Removed erroneous “emit_events” attribute from operations.create_table() documentation. ¶ References:#81

• [bug] Fixed the minute component in file_template which returned the month part of the create date. ¶

11.20 0.4.0

Released: Mon Oct 01 2012

11.20.1 feature

• [feature] Support for tables in alternate schemas has been added fully to all operations, as well as to the autogen-erate feature. When using autogenerate, specifying the flag include_schemas=True to Environment.configure()will also cause autogenerate to scan all schemas located by Inspector.get_schema_names(), which is supportedby some (but not all) SQLAlchemy dialects including Postgresql. Enormous thanks to Bruno Binet for a hugeeffort in implementing as well as writing tests. . ¶ References: #33

11.19. 0.4.1 115











• [feature] The command line runner has been organized into a reusable CommandLine object, so that otherfront-ends can re-use the argument parsing built in. ¶ References: #70

• [feature] Added “stdout” option to Config, provides control over where the “print” output of commands like“history”, “init”, “current” etc. are sent. ¶ References: #43

• [feature] Added support for alteration of MySQL columns that have AUTO_INCREMENT, as well as enablingthis flag. Courtesy Moriyoshi Koizumi. ¶

11.20.2 bug

• [bug] Fixed the “multidb” template which was badly out of date. It now generates revision files using the con-figuration to determine the different upgrade_<xyz>() methods needed as well, instead of needing to hardcodethese. Huge thanks to BryceLohr for doing the heavy lifting here. ¶ References: #71

• [bug] Fixed the regexp that was checking for .py files in the version directory to allow any .py file through.Previously it was doing some kind of defensive checking, probably from some early notions of how this directoryworks, that was prohibiting various filename patterns such as those which begin with numbers. ¶ References:#72

• [bug] Fixed MySQL rendering for server_default which didn’t work if the server_default was a generated SQLexpression. Courtesy Moriyoshi Koizumi. ¶

11.21 0.3.6

Released: Wed Aug 15 2012

11.21.1 feature

• [feature] Added include_symbol option to EnvironmentContext.configure(), specifies a callable which willinclude/exclude tables in their entirety from the autogeneration process based on name. ¶ References: #27

• [feature] Added year, month, day, hour, minute, second variables to file_template. ¶ References: #59

• [feature] Added ‘primary’ to the list of constraint types recognized for MySQL drop_constraint(). ¶

• [feature] Added –sql argument to the “revision” command, for the use case where the “revision_environment”config option is being used but SQL access isn’t desired. ¶

11.21.2 bug

• [bug] Repaired create_foreign_key() for self-referential foreign keys, which weren’t working at all. ¶

• [bug] ‘alembic’ command reports an informative error message when the configuration is missing the‘script_directory’ key. ¶ References: #63

• [bug] Fixes made to the constraints created/dropped alongside so-called “schema” types such as Boolean andEnum. The create/drop constraint logic does not kick in when using a dialect that doesn’t use constraintsfor these types, such as postgresql, even when existing_type is specified to alter_column(). Additionally, theconstraints are not affected if existing_type is passed but type_ is not, i.e. there’s no net change in type. ¶References: #62

• [bug] Improved error message when specifiying non-ordered revision identifiers to cover the case when the“higher” rev is None, improved message overall. ¶ References: #66












11.22 0.3.5

Released: Sun Jul 08 2012

11.22.1 feature

• [feature] Implemented SQL rendering for CheckConstraint() within autogenerate upgrade, including for literalSQL as well as SQL Expression Language expressions. ¶

11.22.2 bug

• [bug] Fixed issue whereby reflected server defaults wouldn’t be quoted correctly; uses repr() now. ¶ References:#31

• [bug] Fixed issue whereby when autogenerate would render create_table() on the upgrade side for a table thathas a Boolean type, an unnecessary CheckConstraint() would be generated. ¶ References: #58

11.23 0.3.4

Released: Sat Jun 02 2012

11.23.1 bug

• [bug] Fixed command-line bug introduced by the “revision_environment” feature. ¶

11.24 0.3.3

Released: Sat Jun 02 2012

11.24.1 feature

• [feature] New config argument “revision_environment=true”, causes env.py to be run unconditionally when the“revision” command is run, to support script.py.mako templates with dependencies on custom “template_args”.¶

• [feature] Added “template_args” option to configure() so that an env.py can add additional arguments to thetemplate context when running the “revision” command. This requires either –autogenerate or the configurationdirective “revision_environment=true”. ¶

• [feature] Added version_table argument to EnvironmentContext.configure(), allowing for the configuration ofthe version table name. ¶ References: #34

• [feature] Added support for “relative” migration identifiers, i.e. “alembic upgrade +2”, “alembic downgrade-1”. Courtesy Atsushi Odagiri for this feature. ¶

11.22. 0.3.5 117





11.24.2 bug

• [bug] Added “type” argument to op.drop_constraint(), and implemented full constraint drop support forMySQL. CHECK and undefined raise an error. MySQL needs the constraint type in order to emit a DROPCONSTRAINT. ¶ References: #44

• [bug] Fixed bug whereby directories inside of the template directories, such as __pycache__ on Pypy, wouldmistakenly be interpreted as files which are part of the template. ¶ References: #49

11.25 0.3.2

Released: Mon Apr 30 2012

11.25.1 feature

• [feature] Basic support for Oracle added, courtesy shgoh. ¶ References: #40

• [feature] Added support for UniqueConstraint in autogenerate, courtesy Atsushi Odagiri ¶

11.25.2 bug

• [bug] Fixed support of schema-qualified ForeignKey target in column alter operations, courtesy AlexanderKolov. ¶

• [bug] Fixed bug whereby create_unique_constraint() would include in the constraint columns that are added toall Table objects using events, externally to the generation of the constraint. ¶

11.26 0.3.1

Released: Sat Apr 07 2012

11.26.1 bug

• [bug] bulk_insert() fixes:

1. bulk_insert() operation was not working most likely since the 0.2 series when used with an engine.

2. Repaired bulk_insert() to complete when used against a lower-case-t table and executing with only one setof parameters, working around SQLAlchemy bug #2461 in this regard.

3. bulk_insert() uses “inline=True” so that phrases like RETURNING and such don’t get invoked for single-row bulk inserts.

4. bulk_insert() will check that you’re passing a list of dictionaries in, raises TypeError if not detected.

¶ References: #41

11.27 0.3.0

Released: Thu Apr 05 2012







11.27.1 feature

• [feature] Added a bit of autogenerate to the public API in the form of the function alem-bic.autogenerate.compare_metadata. ¶

11.27.2 misc

• [general] The focus of 0.3 is to clean up and more fully document the public API of Alembic, includingbetter accessors on the MigrationContext and ScriptDirectory objects. Methods that are not considered to bepublic on these objects have been underscored, and methods which should be public have been cleaned up anddocumented, including:

MigrationContext.get_current_revision() ScriptDirectory.iterate_revisions() ScriptDirec-tory.get_current_head() ScriptDirectory.get_heads() ScriptDirectory.get_base() ScriptDirec-tory.generate_revision()

¶

11.28 0.2.2

Released: Mon Mar 12 2012

11.28.1 feature

• [feature] Informative error message when op.XYZ directives are invoked at module import time. ¶

• [feature] Added execution_options parameter to op.execute(), will call execution_options() on the Connectionbefore executing.

The immediate use case here is to allow access to the new no_parameters option in SQLAlchemy 0.7.6, whichallows some DBAPIs (psycopg2, MySQLdb) to allow percent signs straight through without escaping, thusproviding cross-compatible operation with DBAPI execution and static script generation. ¶

• [feature] script_location can be interpreted by pkg_resources.resource_filename(), if it is a non-absolute URIthat contains colons. This scheme is the same one used by Pyramid. ¶ References: #29

• [feature] added missing support for onupdate/ondelete flags for ForeignKeyConstraint, courtesy Giacomo Bag-noli ¶

11.28.2 bug

• [bug] Fixed inappropriate direct call to util.err() and therefore sys.exit() when Config failed to locate the configfile within library usage. ¶ References: #35

• [bug] Autogenerate will emit CREATE TABLE and DROP TABLE directives according to foreign key depen-dency order. ¶

• [bug] implement ‘tablename’ parameter on drop_index() as this is needed by some backends. ¶

• [bug] setup.py won’t install argparse if on Python 2.7/3.2 ¶

• [bug] fixed a regression regarding an autogenerate error message, as well as various glitches in the Pylonssample template. The Pylons sample template requires that you tell it where to get the Engine from now.courtesy Marcin Kuzminski ¶ References: #30

11.28. 0.2.2 119





• [bug] drop_index() ensures a dummy column is added when it calls “Index”, as SQLAlchemy 0.7.6 will warnon index with no column names. ¶

11.29 0.2.1

Released: Tue Jan 31 2012

11.29.1 bug

• [bug] Fixed the generation of CHECK constraint, regression from 0.2.0 ¶ References: #26

11.30 0.2.0

Released: Mon Jan 30 2012

11.30.1 feature

• [feature] API rearrangement allows everything Alembic does to be represented by contextual objects, includingEnvironmentContext, MigrationContext, and Operations. Other libraries and applications can now use thingslike “alembic.op” without relying upon global configuration variables. The rearrangement was done such thatexisting migrations should be OK, as long as they use the pattern of “from alembic import context” and “fromalembic import op”, as these are now contextual objects, not modules. ¶ References: #19

• [feature] The naming of revision files can now be customized to be some combination of “rev id” and “slug”,the latter of which is based on the revision message. By default, the pattern “<rev>_<slug>” is used for new files.New script files should include the “revision” variable for this to work, which is part of the newer script.py.makoscripts. ¶ References: #24

• [feature] Can create alembic.config.Config with no filename, use set_main_option() to add values. Also addedset_section_option() which will add sections. ¶ References: #23

11.30.2 bug

• [bug] env.py templates call connection.close() to better support programmatic usage of commands; useNullPool in conjunction with create_engine() as well so that no connection resources remain afterwards. ¶References: #25

• [bug] fix the config.main() function to honor the arguments passed, remove no longer used “scripts/alembic”as setuptools creates this for us. ¶ References: #22

• [bug] Fixed alteration of column type on MSSQL to not include the keyword “TYPE”. ¶

11.31 0.1.1

Released: Wed Jan 04 2012









11.31.1 feature

• [feature] PyPy is supported. ¶

• [feature] Python 2.5 is supported, needs __future__.with_statement ¶

• [feature] Add alembic_module_prefix argument to configure() to complement sqlalchemy_module_prefix. ¶References: #18

11.31.2 bug

• [bug] Clean up file write operations so that file handles are closed. ¶

• [bug] Fix autogenerate so that “pass” is generated between the two comments if no net migrations were present.¶

• [bug] Fix autogenerate bug that prevented correct reflection of a foreign-key referenced table in the list of “toremove”. ¶ References: #16

• [bug] Fix bug where create_table() didn’t handle self-referential foreign key correctly ¶ References: #17

• [bug] Default prefix for autogenerate directives is “op.”, matching the mako templates. ¶ References: #18

• [bug] fix quotes not being rendered in ForeignKeConstraint during autogenerate ¶ References: #14

11.32 0.1.0

Released: Wed Nov 30 2011

• Initial release. Status of features: ¶

• Alembic is used in at least one production environment, but should still be considered ALPHA LEVEL SOFT-WARE as of this release, particularly in that many features are expected to be missing / unimplemented. MajorAPI changes are not anticipated but for the moment nothing should be assumed.

The author asks that you please report all issues, missing features, workarounds etc. to the bugtracker, athttps://bitbucket.org/zzzeek/alembic/issues/new . ¶

• Python 3 is supported and has been tested. ¶

• The “Pylons” and “MultiDB” environment templates have not been directly tested - these should be consideredto be samples to be modified as needed. Multiple database support itself is well tested, however. ¶

• Postgresql and MS SQL Server environments have been tested for several weeks in a production environment.In particular, some involved workarounds were implemented to allow fully-automated dropping of default- orconstraint-holding columns with SQL Server. ¶

• MySQL support has also been implemented to a basic degree, including MySQL’s awkward style of modifyingcolumns being accommodated. ¶

• Other database environments not included among those three have not been tested, at all. This includes Fire-bird, Oracle, Sybase. Adding support for these backends should be straightforward. Please report all missing/incorrect behaviors to the bugtracker! Patches are welcome here but are optional - please just indicate the exactformat expected by the target database. ¶

• SQLite, as a backend, has almost no support for schema alterations to existing databases. The author wouldstrongly recommend that SQLite not be used in a migration context - just dump your SQLite database into anintermediary format, then dump it back into a new schema. For dev environments, the dev installer should

11.32. 0.1.0 121






https://bitbucket.org/zzzeek/alembic/issues/new


be building the whole DB from scratch. Or just use Postgresql, which is a much better database for non-trivial schemas. Requests for full ALTER support on SQLite should be reported to SQLite’s bug tracker athttp://www.sqlite.org/src/wiki?name=Bug+Reports, as Alembic will not be implementing the “rename the tableto a temptable then copy the data into a new table” workaround. Note that Alembic will at some point offer anextensible API so that you can implement commands like this yourself. ¶

• Well-tested directives include add/drop table, add/drop column, including support for SQLAlchemy “schema”types which generate additional CHECK constraints, i.e. Boolean, Enum. Other directives not included herehave not been strongly tested in production, i.e. rename table, etc. ¶

• Both “online” and “offline” migrations, the latter being generated SQL scripts to hand off to a DBA, have beenstrongly production tested against Postgresql and SQL Server. ¶

• Modify column type, default status, nullable, is functional and tested across PG, MSSQL, MySQL, but not yetwidely tested in production usage. ¶

• Many migrations are still outright missing, i.e. create/add sequences, etc. As a workaround, execute() canbe used for those which are missing, though posting of tickets for new features/missing behaviors is stronglyencouraged. ¶

• Autogenerate feature is implemented and has been tested, though only a little bit in a production setting. Inparticular, detection of type and server default changes are optional and are off by default; they can alsobe customized by a callable. Both features work but can have surprises particularly the disparity betweenBIT/TINYINT and boolean, which hasn’t yet been worked around, as well as format changes performed by thedatabase on defaults when it reports back. When enabled, the PG dialect will execute the two defaults to becompared to see if they are equivalent. Other backends may need to do the same thing.

The autogenerate feature only generates “candidate” commands which must be hand-tailored in any case, so isstill a useful feature and is safe to use. Please report missing/broken features of autogenerate! This will be agreat feature and will also improve SQLAlchemy’s reflection services. ¶

• Support for non-ASCII table, column and constraint names is mostly nonexistent. This is also a straightforwardfeature add as SQLAlchemy itself supports unicode identifiers; Alembic itself will likely need fixes to logging,column identification by key, etc. for full support here. ¶


http://www.sqlite.org/src/wiki?name=Bug+Reports

CHAPTER 12

Indices and tables

• genindex

• modindex

• search

123


124 Chapter 12. Indices and tables

Python Module Index

aalembic.command, 85alembic.config, 86alembic.ddl, 94alembic.ddl.base, 94alembic.ddl.impl, 95alembic.ddl.mssql, 97alembic.ddl.mysql, 96alembic.ddl.postgresql, 97alembic.ddl.sqlite, 98alembic.environment, 74alembic.migration, 83alembic.operations, 51alembic.revision, 91alembic.script, 88

125


126 Python Module Index

Index

Aadd_column() (alembic.ddl.impl.DefaultImpl method),

95add_column() (alembic.operations.BatchOperations

method), 64add_column() (alembic.operations.Operations method),

51add_column() (in module alembic.ddl.base), 95add_constraint() (alembic.ddl.impl.DefaultImpl method),

95add_constraint() (alembic.ddl.sqlite.SQLiteImpl

method), 98add_revision() (alembic.revision.RevisionMap method),

92AddColumn (class in alembic.ddl.base), 94alembic.command (module), 85alembic.config (module), 86alembic.ddl (module), 94alembic.ddl.base (module), 94alembic.ddl.impl (module), 95alembic.ddl.mssql (module), 97alembic.ddl.mysql (module), 96alembic.ddl.postgresql (module), 97alembic.ddl.sqlite (module), 98alembic.environment (module), 74alembic.migration (module), 83alembic.operations (module), 51alembic.revision (module), 91alembic.script (module), 88alter_column() (alembic.ddl.impl.DefaultImpl method),

95alter_column() (alembic.ddl.mssql.MSSQLImpl

method), 97alter_column() (alembic.ddl.mysql.MySQLImpl

method), 96alter_column() (alembic.operations.BatchOperations

method), 64alter_column() (alembic.operations.Operations method),

52alter_column() (in module alembic.ddl.base), 95

alter_table() (in module alembic.ddl.base), 95AlterColumn (class in alembic.ddl.base), 94AlterTable (class in alembic.ddl.base), 94as_revision_number() (alembic.script.ScriptDirectory

method), 89attributes (alembic.config.Config attribute), 87autogen_column_reflect() (alembic.ddl.impl.DefaultImpl

method), 95autogen_column_reflect() (alem-

bic.ddl.postgresql.PostgresqlImpl method),97

Bbases (alembic.revision.RevisionMap attribute), 92batch_alter_table() (alembic.operations.Operations

method), 53batch_separator (alembic.ddl.mssql.MSSQLImpl at-

tribute), 97BatchOperations (class in alembic.operations), 64begin_transaction() (alem-

bic.environment.EnvironmentContext method),75

bind (alembic.ddl.impl.DefaultImpl attribute), 95bind (alembic.migration.MigrationContext attribute), 83branch_labels (alembic.revision.Revision attribute), 91branches() (in module alembic.command), 85bulk_insert() (alembic.ddl.impl.DefaultImpl method), 95bulk_insert() (alembic.ddl.mssql.MSSQLImpl method),

97bulk_insert() (alembic.operations.Operations method), 55

Ccmd_opts (alembic.config.Config attribute), 87ColumnDefault (class in alembic.ddl.base), 94ColumnName (class in alembic.ddl.base), 94ColumnNullable (class in alembic.ddl.base), 94ColumnType (class in alembic.ddl.base), 94command_terminator (alembic.ddl.impl.DefaultImpl at-

tribute), 95compare_metadata() (in module alembic.autogenerate),

93

127


compare_server_default() (alembic.ddl.impl.DefaultImplmethod), 95

compare_server_default() (alem-bic.ddl.mysql.MySQLImpl method), 97

compare_server_default() (alem-bic.ddl.postgresql.PostgresqlImpl method),97

compare_server_default() (alembic.ddl.sqlite.SQLiteImplmethod), 98

compare_type() (alembic.ddl.impl.DefaultImpl method),95

config (alembic.environment.EnvironmentContextattribute), 75

config (alembic.migration.MigrationContext attribute),83

Config (class in alembic.config), 86config_file_name (alembic.config.Config attribute), 88config_ini_section (alembic.config.Config attribute), 88configure() (alembic.environment.EnvironmentContext

method), 75configure() (alembic.migration.MigrationContext class

method), 83correct_for_autogen_constraints() (alem-

bic.ddl.impl.DefaultImpl method), 95correct_for_autogen_constraints() (alem-

bic.ddl.mysql.MySQLImpl method), 97correct_for_autogen_constraints() (alem-


correct_for_autogen_constraints() (alem-bic.ddl.sqlite.SQLiteImpl method), 98

create_check_constraint() (alem-bic.operations.BatchOperations method),65

create_check_constraint() (alem-bic.operations.Operations method), 56

create_foreign_key() (alem-bic.operations.BatchOperations method),65

create_foreign_key() (alembic.operations.Operationsmethod), 57

create_index() (alembic.ddl.impl.DefaultImpl method),96

create_index() (alembic.operations.BatchOperationsmethod), 65

create_index() (alembic.operations.Operations method),57

create_primary_key() (alem-bic.operations.BatchOperations method),65

create_primary_key() (alembic.operations.Operationsmethod), 58

create_table() (alembic.ddl.impl.DefaultImpl method), 96

create_table() (alembic.operations.Operations method),59

create_unique_constraint() (alem-bic.operations.BatchOperations method),65

create_unique_constraint() (alem-bic.operations.Operations method), 60

current() (in module alembic.command), 85

DDefaultImpl (class in alembic.ddl.impl), 95dependencies (alembic.revision.Revision attribute), 91doc (alembic.script.Script attribute), 89down_revision (alembic.revision.Revision attribute), 91downgrade() (in module alembic.command), 85drop_column() (alembic.ddl.impl.DefaultImpl method),

96drop_column() (alembic.ddl.mssql.MSSQLImpl

method), 97drop_column() (alembic.operations.BatchOperations

method), 65drop_column() (alembic.operations.Operations method),

60drop_column() (in module alembic.ddl.base), 95drop_constraint() (alembic.ddl.impl.DefaultImpl

method), 96drop_constraint() (alembic.ddl.sqlite.SQLiteImpl

method), 98drop_constraint() (alembic.operations.BatchOperations

method), 65drop_constraint() (alembic.operations.Operations

method), 61drop_index() (alembic.ddl.impl.DefaultImpl method), 96drop_index() (alembic.operations.BatchOperations

method), 65drop_index() (alembic.operations.Operations method), 61drop_table() (alembic.ddl.impl.DefaultImpl method), 96drop_table() (alembic.operations.Operations method), 62DropColumn (class in alembic.ddl.base), 94

Eemit_begin() (alembic.ddl.impl.DefaultImpl method), 96emit_begin() (alembic.ddl.mssql.MSSQLImpl method),

97emit_commit() (alembic.ddl.impl.DefaultImpl method),

96emit_commit() (alembic.ddl.mssql.MSSQLImpl

method), 97EnvironmentContext (class in alembic.environment), 74execute() (alembic.ddl.impl.DefaultImpl method), 96execute() (alembic.environment.EnvironmentContext

method), 80execute() (alembic.migration.MigrationContext method),

84

128 Index


execute() (alembic.operations.Operations method), 62

Ff() (alembic.operations.Operations method), 63file_config (alembic.config.Config attribute), 88format_column_name() (in module alembic.ddl.base), 95format_server_default() (in module alembic.ddl.base), 95format_table_name() (in module alembic.ddl.base), 95format_type() (in module alembic.ddl.base), 95from_config() (alembic.script.ScriptDirectory class

method), 89

Ggenerate_revision() (alembic.script.ScriptDirectory

method), 89get_base() (alembic.script.ScriptDirectory method), 89get_bases() (alembic.script.ScriptDirectory method), 90get_bind() (alembic.environment.EnvironmentContext

method), 80get_bind() (alembic.operations.Operations method), 63get_by_dialect() (alembic.ddl.impl.DefaultImpl class

method), 96get_context() (alembic.environment.EnvironmentContext

method), 81get_context() (alembic.operations.Operations method),

63get_current_head() (alembic.revision.RevisionMap

method), 92get_current_head() (alembic.script.ScriptDirectory

method), 90get_current_heads() (alem-

bic.migration.MigrationContext method),84

get_current_revision() (alem-bic.migration.MigrationContext method),84

get_head_revision() (alem-bic.environment.EnvironmentContext method),81

get_head_revisions() (alem-bic.environment.EnvironmentContext method),81

get_heads() (alembic.script.ScriptDirectory method), 90get_main_option() (alembic.config.Config method), 88get_revision() (alembic.revision.RevisionMap method),

92get_revision() (alembic.script.ScriptDirectory method),

90get_revision_argument() (alem-


get_revisions() (alembic.revision.RevisionMap method),92

get_revisions() (alembic.script.ScriptDirectory method),90

get_section() (alembic.config.Config method), 88get_section_option() (alembic.config.Config method), 88get_starting_revision_argument() (alem-


get_tag_argument() (alem-bic.environment.EnvironmentContext method),81

get_template_directory() (alembic.config.Configmethod), 88

get_x_argument() (alem-bic.environment.EnvironmentContext method),81

Hheads (alembic.revision.RevisionMap attribute), 92heads() (in module alembic.command), 85history() (in module alembic.command), 85

IImplMeta (class in alembic.ddl.impl), 96init() (in module alembic.command), 85inline_literal() (alembic.operations.Operations method),

63is_base (alembic.revision.Revision attribute), 91is_branch_point (alembic.revision.Revision attribute), 91is_head (alembic.revision.Revision attribute), 91is_merge_point (alembic.revision.Revision attribute), 91is_offline_mode() (alem-


is_transactional_ddl() (alem-bic.environment.EnvironmentContext method),82

iterate_revisions() (alembic.revision.RevisionMapmethod), 93

iterate_revisions() (alembic.script.ScriptDirectorymethod), 90

Llist_templates() (in module alembic.command), 85longdoc (alembic.script.Script attribute), 89

Mmain() (in module alembic.config), 88merge() (in module alembic.command), 85MigrationContext (class in alembic.migration), 83module (alembic.script.Script attribute), 89mssql_add_column() (in module alembic.ddl.mssql), 97MSSQLImpl (class in alembic.ddl.mssql), 97MySQLAlterDefault (class in alembic.ddl.mysql), 96MySQLChangeColumn (class in alembic.ddl.mysql), 96

Index 129


MySQLImpl (class in alembic.ddl.mysql), 96MySQLModifyColumn (class in alembic.ddl.mysql), 97

Nnextrev (alembic.revision.Revision attribute), 91

OOperations (class in alembic.operations), 51

Ppath (alembic.script.Script attribute), 89PostgresqlImpl (class in alembic.ddl.postgresql), 97prep_table_for_batch() (alembic.ddl.impl.DefaultImpl

method), 96prep_table_for_batch() (alem-


print_stdout() (alembic.config.Config method), 88

Qquote_dotted() (in module alembic.ddl.base), 95

Rrename_table() (alembic.ddl.impl.DefaultImpl method),

96rename_table() (alembic.operations.Operations method),

64RenameTable (class in alembic.ddl.base), 95requires_recreate_in_batch() (alem-

bic.ddl.impl.DefaultImpl method), 96requires_recreate_in_batch() (alem-

bic.ddl.sqlite.SQLiteImpl method), 98revision (alembic.revision.Revision attribute), 91Revision (class in alembic.revision), 91revision() (in module alembic.command), 86RevisionMap (class in alembic.revision), 92run_env() (alembic.script.ScriptDirectory method), 90run_migrations() (alem-


run_migrations() (alembic.migration.MigrationContextmethod), 84

Sscript (alembic.environment.EnvironmentContext at-

tribute), 82Script (class in alembic.script), 88ScriptDirectory (class in alembic.script), 89set_main_option() (alembic.config.Config method), 88set_section_option() (alembic.config.Config method), 88show() (in module alembic.command), 86SQLiteImpl (class in alembic.ddl.sqlite), 98

stamp() (alembic.migration.MigrationContext method),84

stamp() (in module alembic.command), 86start_migrations() (alembic.ddl.impl.DefaultImpl

method), 96static_output() (alembic.ddl.impl.DefaultImpl method),

96static_output() (alembic.environment.EnvironmentContext

method), 82

Ttransactional_ddl (alembic.ddl.impl.DefaultImpl at-

tribute), 96transactional_ddl (alembic.ddl.mssql.MSSQLImpl

attribute), 97transactional_ddl (alembic.ddl.mysql.MySQLImpl

attribute), 97transactional_ddl (alembic.ddl.postgresql.PostgresqlImpl

attribute), 98transactional_ddl (alembic.ddl.sqlite.SQLiteImpl at-

tribute), 98

Uupgrade() (in module alembic.command), 86

Vvisit_add_column() (in module alembic.ddl.base), 95visit_add_column() (in module alembic.ddl.mssql), 97visit_column_default() (in module alembic.ddl.base), 95visit_column_default() (in module alembic.ddl.mssql), 97visit_column_name() (in module alembic.ddl.base), 95visit_column_nullable() (in module alembic.ddl.base), 95visit_column_nullable() (in module alembic.ddl.mssql),

97visit_column_type() (in module alembic.ddl.base), 95visit_column_type() (in module alembic.ddl.mssql), 97visit_drop_column() (in module alembic.ddl.base), 95visit_rename_column() (in module alembic.ddl.mssql),

97visit_rename_table() (in module alembic.ddl.base), 95visit_rename_table() (in module alembic.ddl.mssql), 97visit_rename_table() (in module alembic.ddl.postgresql),

98

Wwalk_revisions() (alembic.script.ScriptDirectory

method), 90

130 Index

alembic documentation · 2021. 1. 30. · alembic documentation, release 0.7.7 1.3.1dependencies...

Documents