migrate 140123161042-phpapp02
DESCRIPTION
TRANSCRIPT
migrating to drupal 8
a brief historydrupal.org/upgrade
update vs. upgradewhat types drupal “renewing” we have?
drupal update
drupal update minor version update
7.24 > 7.25
drupal upgrade
drupal update minor version update
7.24 > 7.25
drupal upgrade
drupal update minor version update
7.24 > 7.25
major version upgrade7.x > 8.x
drupal upgrade
drupal update minor version update
7.24 > 7.25
major version upgrade7.x > 8.x
the migrate module“in service” since 2009
drupal.org/project/migrate
the migrate module
• How it worked?
• Migrations = classes extending Migration.
• Main elements: source, destination, map, mappings, “hooks” (prepareRow, prepare, complete, createStub, etc).
• Each migration has to extend the Migration class or one of its successors.
migrate in D8 core
disclaimer
disclaimer
• the migrate system is under heavy development right now.
disclaimer
• the migrate system is under heavy development right now.
• some of the features or APIs may change in the future
disclaimer
• the migrate system is under heavy development right now.
• some of the features or APIs may change in the future
• not all the current work is pushed to 8.x.
disclaimer
• the migrate system is under heavy development right now.
• some of the features or APIs may change in the future
• not all the current work is pushed to 8.x.
• The work is in the sandbox athttps://drupal.org/sandbox/chx/2105305
creditsKároly Négyesi (chx)Mike Ryan (mikeryan)
Moshe Weitzman (moshe weitzman)Ben Dougherty (benjy)
drupal 8 migration
note
note
• While a significant portion of the code and the interaction between the elements is brand new, the actual migrate-y code is coming straight from D7: highwater marks, track changes, id map, this is here to
note
• While a significant portion of the code and the interaction between the elements is brand new, the actual migrate-y code is coming straight from D7: highwater marks, track changes, id map, this is here to
• The new interaction allows for really nice and powerful migrations but at the same time we are most definitely not reinventing wheel.
structure/modules
Migratecore/modules/migrate/
Migratecore/modules/migrate/
• provides general API for all migrations
Migratecore/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).
Migratecore/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).
• provides a plugin manager for manipulation on migration plugins.
Migratecore/modules/migrate/
• provides general API for all migrations
• provides interfaces and base classes for all migration plugin components (source, destination, process, id_map, row).
• provides a plugin manager for manipulation on migration plugins.
• provides the migrate configurable (configuration entity type).
Migrate Drupalcore/modules/migrate_drupal/
Migrate Drupalcore/modules/migrate_drupal/
• the first module using the new Migrate API.
Migrate Drupalcore/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
Migrate Drupalcore/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.
Migrate Drupalcore/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.
•Defines migrations for all system components:
Migrate Drupalcore/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.
•Defines migrations for all system components:
•Drupal 6 settings (site name, slogan, roles, etc)
Migrate Drupalcore/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.
•Defines migrations for all system components:
•Drupal 6 settings (site name, slogan, roles, etc)
•Content definitions (vocabularies, node types, etc)
Migrate Drupalcore/modules/migrate_drupal/
• the first module using the new Migrate API.
• kind of migrate_d2d successor.
•migrates out-of-the-box from Drupal 6 and 7 into Drupal 8.
•Defines migrations for all system components:
•Drupal 6 settings (site name, slogan, roles, etc)
•Content definitions (vocabularies, node types, etc)
•Content (noded, terms, users, etc).
understanding migrations
migrations are configurables
small peekinto configurables
what is a configurable?
• “Configurables” are configuration entities.
what is a configurable?
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.
what is a configurable?
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.
what is a configurable?
\Drupal\Core\Entity\Entity
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.
what is a configurable?
\Drupal\Core\Entity\Entity
\Drupal\Core\Entity\ContentEntityBase
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.
what is a configurable?
\Drupal\Core\Entity\Entity
\Drupal\Core\Entity\ContentEntityBase
\Drupal\Core\Config\ConfigEntityBase
• “Configurables” are configuration entities.
• In Drupal 8 the content is separated from configuration. Both are classes and are sharing the same ancestor: the Entity class.
what is a configurable?
\Drupal\Core\Entity\Entity
\Drupal\Core\Entity\ContentEntityBase
\Drupal\Core\Config\ConfigEntityBase
• A configurable is the way Drupal 8 stores the configuration of a specific functionality. E.g. the the definition of a node type is stored in a configuration entity of type “node_type”.
• Configuration entity types are annotated classes, meaning that the object meta information is stored in annotation rather than in info hooks - as it was in Drupal <= 7.
• Imagine configurables as entities storing their data in config YAML files rather than DB.
• The “fields” of a configurable are the public properties exposed by the configurable object.
what is a configurable?
what is a configurable?
how it’s stored?example
migration pluginsparts implemented by specific migrations
source plugins
• plugins returning information and data from the source of migration.
• usually: the list of fields, the source iterator (used retrieve data from source).
• each migration should configure a source.
destination plugins
• are handling data at the destination: import, rollback.
• different plugins for different destination components: entity, config, etc.
• are defined in the base module (migrate) as destination is always drupal 8 but if necessary it can be extended.
• each migration should specify a destination.
id map plugin
• plugins of this type are handling and storing the relation between primary IDs of source and destination.
• without this, rollback and continuous migrations are impossible.
• in 99% of the cases you’ll use the sql id map plugin (Sql) that keeps the map of each migration in a table.
• table name migrate_map_MIGRATION_ID
processors
• plugins that are performing small but very specialized operations against values to be migrated.
• Some simple examples: DefaultValue, Concat, etc.
• The most important interface method: transform().
the anatomy of a migrationmigrating user roles from a dupal 6 site
creating the config fileconfig/migrate.migration.d6_user_role.yml
relative to core/modules/migrate_drupal
config .yml file content
config .yml file content• id: same as the last part of filename (d6_user_role)
config .yml file content• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
config .yml file content• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the source plugin to be used
config .yml file content• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the source plugin to be used
• process: describe the list of processors to be applied per destination field.
config .yml file content• id: same as the last part of filename (d6_user_role)
• sourceIds: Source fields, providing a primary ID.
• source: configure the source of data, usually the source plugin to be used
• process: describe the list of processors to be applied per destination field.
• destination: destination configuration, usually the destination plugin.
id
id• this is the configurable unique id.
id• this is the configurable unique id.
• it must be exactly as the same as the last part of filename: d6_user_role.
id• this is the configurable unique id.
• it must be exactly as the same as the last part of filename: d6_user_role.
id: d6_user_role
sourceIds
sourceIds• look in D6 schema to find the role primary ID.
sourceIds• look in D6 schema to find the role primary ID.
• lines 107 - 115 of drupal/modules/user/user.install.
sourceIds• look in D6 schema to find the role primary ID.
• lines 107 - 115 of drupal/modules/user/user.install.
$schema['role'] = array( 'description' => 'Stores user roles.', 'fields' => array( 'rid' => array( 'type' => 'serial', 'unsigned' => TRUE, 'not null' => TRUE, 'description' => 'Primary Key: Unique role id.',
sourceIds
sourceIds• use TypedData identifiers for data type.
sourceIds• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.
sourceIds• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.
sourceIds: rid: type: integer
sourceIds• use TypedData identifiers for data type.
• Here are the .yml lines that we need to add.
sourceIds: rid: type: integer
Note: sourceIds will be removed in the near future and the source plugin will set also the primary id.
source
source
• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.
source
• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.
• let’s see how it should look (code).
source
• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.
• let’s see how it should look (code).
• add the source plugin id in the configuration .yml file.
source
• we need to implement a source plugin first, that provides the list of fields and the iterator by querying the D6 backend.
• let’s see how it should look (code).
• add the source plugin id in the configuration .yml file.
source: plugin: drupal6_user_role
process
process• process keys are destination “fields”.
process• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
process• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from baseFieldDefinitions
process• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from baseFieldDefinitions
• let’s see how it looks! (code).
process• process keys are destination “fields”.
• for configurables: the public properties (except uuid)
• for content: the keys from baseFieldDefinitions
• let’s see how it looks! (code).
process: id: label: weight: permissions:
destination
destination
• should point to the destination plugin.
destination
• should point to the destination plugin.
• in this case we’re importing into user_role entity, so we’re passing also the entity_type argument.
destination
• should point to the destination plugin.
• in this case we’re importing into user_role entity, so we’re passing also the entity_type argument.
destination: plugin: entity entity_type: user_role
running a migration
• via drush
• There will be a brief UI implemented in core (to come!)
final notes
final notes• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
final notes• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.
final notes• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.
• The underlying Migrate API is source-agnostic. You can easily migrate into D8 from MS SQL, Oracle, piles of HTML files, XML feeds, CSV files, etc.
final notes• Minor version updates are unchanged. Developers
continue to use hook_update_N() for those.
• Contrib and custom modules are encouraged to ship with migrations of their data from D6/D7 to D8. Use core modules as model.
• The underlying Migrate API is source-agnostic. You can easily migrate into D8 from MS SQL, Oracle, piles of HTML files, XML feeds, CSV files, etc.
• Similarly, Drupal 4.x and Drupal 5.x sites are able to migrate using this same approach.
Questions?Thank you.