© Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0

How to Overcome Data Challenges When Refactoring Monoliths to Microservices

Kenny Bastani, Developer Advocate, PivotalTwitter: @kennybastani

Jason Mimick, Technical Director, MongoDBTwitter: @jmimick

Today’s Speakers

Kenny BastaniSpring Developer Advocate

Jason MimickTechnical Director, Partners

Pivotal Partner Ecosystem

Agenda

Overcoming Data Challenges When Refactoring Monoliths to Microservices

● From Monolith to Microservices

● Migration Strategies

○ Splitting the Monolith

○ Strangling the Monolith

● From RDBMS to NoSQL

● Pivotal Cloud Foundry + MongoDB

● Reference Architecture & Open Source Example

● Q/A

Safe Harbor Statement

This presentation contains “forward-looking statements” within the meaning of Section 27A of the Securities Act of 1933, as amended, and Section 21E of the Securities Exchange Act of 1934, as amended. Such forward-looking statements are subject to a number of risks, uncertainties, assumptions and other factors that could cause actual results and the timing of certain events to differ materially from future results expressed or implied by the forward-looking statements. Factors that could cause or contribute to such differences include, but are not limited to, those identified our filings with the Securities and Exchange Commission. You should not rely upon forward-looking statements as predictions of future events. Furthermore, such forward-looking statements speak only as of the date of this presentation.

In particular, the development, release, and timing of any features or functionality described for MongoDB products remains at MongoDB’s sole discretion. This information is merely intended to outline our general product direction and it should not be relied on in making a purchasing decision nor is this a commitment, promise or legal obligation to deliver any material, code, or functionality. Except as required by law, we undertake no obligation to update any forward-looking statements to reflect events or circumstances after the date of such statements.

Abandoning the Status Quo

From Monolith to Microservices

The Monolith

Make JAR not WAR. – Josh Long

Monolith – One Release Cycle to Rule Them All

● Slows down velocity of deploying new features

● Takes too long to ramp up new developers

● The code base becomes too large to fully comprehend

● Centralized change management slows progress (DBA, Ops)

● Deployments batch together changes from different teams

● Ops team manages infrastructure

● Ops team drives execution environment and upgrades

● Deploy everything at once or nothing at all

The SOA

Independent Release Cycles

Single Responsibility Principle

A class should have only one reason to change. – Uncle Bob

Distributed Monolith

If your [services] must be deployed as a complete set in a specific order, please put them back in a monolith and save yourself some pain. – Matt Stine

SOA – The Curious Case of Shared Ownership

● Services typically share one database

● Services typically share ownership of tables

● Services share libraries containing domain objects

● Boundaries of ownership become blurred over time

Single Responsibility Principle

A class should have only one reason to change. – Uncle Bob

Distributed Monolith

If your [services] must be deployed as a complete set in a specific order, please put them back in a monolith and save yourself some pain. – Matt Stine

SOA – Coordinated Releases

● Account service makes a change to customer record

○ Can be deployed independently

● Account service makes a change to address record

○ Must deploy all three services together

Microservices

● Independent Release Cycles

● No table sharing

● No library sharing

● Teams do not share ownership

Conway’s Law

Organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.

– Melvin Conway

Microservices – Share-nothing Architecture

● Small teams focusing on business capabilities

● Share-nothing architecture

● Independent release cycles

● Increases number of pathways to production

Microservices - Encapsulation of Data

Data Encapsulation

For us service orientation means encapsulating the data with the business logic that operates on the data, with the only access through a published service interface. No direct database access is allowed from outside the service, and there’s no data sharing among the services.

– Werner Vogels, CTO, Amazon

Microservices - Encapsulation of Data

Logical Separation

Horizontal Scalability

Replacing the Airplane’s Engine While In-flight

Migration Strategies

Splitting the Monolith

Monolith

Shared Tables

Splitting the MonolithMonolith

New Microservice

The monolith grows over time. It acquires new functionality and lines of code at an alarming rate. Before long it becomes a big, scary giant presence in our organization that people are scared to touch or change. But all is not lost! With the right tools at our disposal, we can slay this beast.

– Sam Newman

Splitting the Monolith – Fracture & Refactor

● Fracture and refactor modules into encapsulated services

● Impractical for most large enterprises

● Foreign-key relationships are not easy to break

● High risk of breaking data warehouse integrations and ETLs

Strangling the Monolith

Gradually create a new system around the edges of the old, letting it grow slowly over several years until the old system is strangled.

– Martin Fowler

The strangler vine seeds itself in the upper branches of a fig tree and works its way down to the roots, slowly starving the tree of resources.

Strangling the Monolith

Monolith

Create an edge around the legacy system and slowly migrate data away

Strangling the Monolith

Monolith

API Gateway

New Microservices

Move new feature development to microservices

“Gradually create a new system around the edges of the old, letting it grow slowly over several years until the old system is strangled.”

– Martin Fowler

Strangling the Monolith – Abstract Away & Deprecate

● Minimizes wasted development cycles

● Focuses on new feature development with microservices

● Integrates with legacy system using an API gateway

● Slowly moves data away from large shared database

Strangling the Monolith

API Gateway

● API gateway automatically adapts between old and new formats

● Uses cache pattern to migrate system of record to microservices

● Needs to replicate writes back to large shared database

● Legacy system also routes requests through API gateway

● Router can identify system of record for domain data by primary key

The Struggle of Systems of Record

From RDBMS to NoSQL

Scalability& Performance

Always On,Global Deployments

FlexibilityExpressive Query Language& Secondary Indexes

Strong Consistency

Enterprise Management& Integrations

Non relational (NoSQL) and relational/tabular

Scalability& Performance

Always On,Global Deployments

FlexibilityExpressive Query Language& Secondary Indexes

Strong Consistency

Enterprise Management& Integrations

Modern database requirements

28

Freedom to run anywhere

Database that runs the same

everywhere

Coverage in any geography

Leverage the benefits of a

multi-cloud strategy

Avoid lock-in

Mainframe

Database as a service

ServerLaptop Self-managed in the cloud

Modern database requirements

Database as a service

Self-service, elastic, and automated

Secure by defaultGlobal and highly available

Continuous backups

Real-time monitoring and optimization

Cloud agnostic

A Cloud-native Platform for Microservices

Pivotal Cloud Foundry+ MongoDB

MongoDB as a service on Pivotal Cloud Foundry, build faster and deliver faster with reliability

BUILD FASTER DELIVER FASTER, RELIABILY

DEV

• On demand production ready MongoDB

• Integrate and bind easily with app and other services

OPS

• Provisioning and scaling • Management • Continuous backup and recovery• Monitoring

• Automate and simplify:

• Security and control

How does the integration work

Value

Deliver value from day 1. MongoDB Enterprise on Pivotal Cloud Foundry aligns with existing DevOps (CI/CD) workflows and provides on demand production ready MongoDB and application cluster in minutes, enabling you to rapidly build apps and micro-services

De‐risk your deployments. Our solution offers enterprise grade with high availability, blue-green deployments, continuous backups and point-in-time recovery to keep your mission critical deployments always-on

Drive operational excellence. MongoDB Ops Manager with BOSH automate and simplify managing, patching, configuring, scaling, tuning, and monitoring MongoDB deployments to reduce IT capital and operational expenses

Supporting modern applications

New and coming features:

• Change Streams• Change Data Capture/Event Sourcing

• Schema Validation• Data governance

• Transactions - COMING MONGODB 4.0 • Coordinating multi-system & complex data relationships

Change Streams

Enabling developers to build reactive, real-time servicesC

hang

e St

ream

s A

PI

Business Apps

User Data

Sensors

Clickstream

Real-Time Event Notifications

Message Queue

Schema Validation

Enforces strict schema structure over a complete collection for data governance & quality

• Builds on document validation introduced by restricting new content that can be added to a document

• Enforces presence, type, and values for document content, including nested array

• Simplifies application logic

Tunable: enforce document structure, log warnings, or allow complete schema flexibility

Queryable: identify all existing documents that do not comply

MongoDB Multi-Document Transactions

Just like relational transactions● Multi-statement, familiar relational syntax● Easy to add to any application● Multiple documents in 1 or many collections

ACID guarantees● Snapshot isolation, all or nothing execution● No performance impact for non-transactional

operations

Schedule● MongoDB 4.0, Summer ‘18: replica set● MongoDB 4.2: extended to sharded clusters

Cloud Native Java + PCF + MongoDB

Reference Architecture

$ cf dev start

Open Source Example

https://github.com/kbastani/cloud-native-microservice-strangler-example

http://www.kennybastani.com/2016/08/strangling-legacy-microservices-spring-cloud.html

Open Source Example on GitHub

© Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0

How to Overcome Data Challenges When Refactoring Monoliths to Microservices

Kenny Bastani, Developer Advocate, PivotalTwitter: @kennybastani

Jason Mimick, Technical Director, MongoDBTwitter: @jmimick

The First Law of Microservices – “It Depends”

Q/A