data synchronization with linkmove in the context of ddd

DB Synchronization with LinkMove:a missing piece in your Domain-Driven Design Architectureby Andrus Adamchik @andrus_a

About Me• I used to be a WO developer in the past life.• I write open source code.• I run ObjectStyle.

Domain-Driven Design

Domain: Sports Organization

Model: Overlapping PartsScheduling

Website

Model: Distinct PartsScheduling

Website

Domain-Driven Design

• Business domain is split into "bounded contexts".• Ubiquitous language within each context.• ... many other good ideas...

How to Integrate Bounded Contexts?

Integration Strategies

• REST services.• Messaging (domain events).

Integrating Bounded Contexts: Challenges

• Source context needs to proactively build an integration layer.

• Consuming context need an "anticorruption layer" to avoid coupling with the producer model.

• What to do when the source is down?

ETL (Extract-Transform-Load) as Integration Strategy

• Can be applied to legacy sources.• Built-in "anti-corruption" layer.• Consuming context persists data, and can operate when the

source is unavailable.

ETL Challenges• Most ETL literature and solutions are data warehouse

specific.• Most solutions are not open-source.• Most solutions are standalone services with extra integration

challenges.

Enter LinkMove...One of the least promoted and most used open source tools in our team.

https://github.com/nhl/link-move

LinkMove - Model-Driven Embedded ETL

Demo - JDBC Extractor

LinkMove Solution• Embedded in your app.• Version controlled extractor descriptors and task code.• Driven by consumer's ORM model (Cayenne-based).• Extracts data from any source (can work with legacy

sources).

LinkMove - Connectors• JDBC• LDAP (including ActiveDirectory)• Files• URL• you can add your own

LinkMove - Extractors• JDBC• LDAP (including ActiveDirectory)• XML• JSON• CSV• you can add your own

Demo - JSON Extractor

Connecting Sources to Targets

Match by Target ID (default, implicit)

Match by Target Property (or Properties)

Match with Mapping Strategy

Transforming Data.. Goals:• Adapt source to conform to target format.• Clean up source data.• Filter source data.• Simplify matching.• Apply business rules.

Transforming in Extractors

Transforming with Listeners

Performance Optimization

Batching• Matching and committing targets in batches.• Default batch size is 500 items.• Size can be customized.

Non-Blocking Parallel Execution

Summary• LinkMove is an embedded model-driven ETL.• Reuses your Cayenne model.• Built-in anti-corruption layer.• Ideal for synchronizing "bounded contexts" in your system.• .. or as a programmable general purpose ETL framework.

Questions?

• You can ★ us here: https://github.com/nhl/link-move