duplicating data or replicating data in micro services
TRANSCRIPT
![Page 1: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/1.jpg)
in a Microservices Architecture
Dennis van der SteltSoftware Architect
![Page 2: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/2.jpg)
v a n d e r s t e l t
2Your presenter
Dennis van der Stelt
http://dennis.bloggingabout.net/
Software Architect
Geek
Developer
Architect
Gamer
Enthousiast
Professional Skills
Something about me that you should know
About Dennis
Innovative software architect
Almost 20 years of experience in development ofdistributed systems. Has a continuous drive tolearn and improve knowledge in differentarchitectural styles, including quality in softwaredevelopment. Highly motivated to share hisknowledge via articles, presentations and his blog.
Awards
98%
80%
60%
40%
100%
NServiceBus Champ
Typemock MVP
Certified Trainer
![Page 3: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/3.jpg)
v a n d e r s t e l t
3
AgendaIn a polyglot architectural landscape with a lot of business logic, data and information, how can we build solid loosely coupled systems?
Let’s have a look at the differences between those opposites
Monolithic vs Microservices
Dividing your system’s business logic over logical boundaries
Service Boundaries
Why duplicating data is a bad practice, but replicating data is perfectly fine
Replicating vs Duplicating
![Page 4: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/4.jpg)
v a n d e r s t e l t
Monolithic application
Wikipedia : large, powerful, and intractably indivisible and uniform.
A monolithic application is build and deployed as a single unit.
WordPress is a successful & scalable monolithic application used by millions!
Monolith is therefor not by default a badly designed system
4
![Page 5: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/5.jpg)
v a n d e r s t e l t
5Microservices
a software architectural style for complex applications of small, independent processes.
services are small, highly decoupled and focus on a single task
communicate via language agnostic APIs
boundaries are explicitservices are autonomous
share schema & contract, not classescompatibility is based on policy
![Page 6: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/6.jpg)
v a n d e r s t e l t
Monolithic application 6
single deployment unit
![Page 7: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/7.jpg)
v a n d e r s t e l t
Microservices 7
multiple deployable services
![Page 8: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/8.jpg)
v a n d e r s t e l t
Authentication
8Coupling
OSS
ViewModel
Change
Product
Database
Razor
![Page 9: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/9.jpg)
v a n d e r s t e l t
9
First law
“My First Law of Distributed Object Design: Don't distribute your objects”
Martin Fowler in Patterns of Enterprise Application Architecture
![Page 10: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/10.jpg)
v a n d e r s t e l t
10Monolithic application problems
Problem Analysis
of all monolithic systems are buildwith high time constraints, resultingin big balls of mud in which nodeveloper can find their way.
42%
MaintenanceThe coupling makes it hard to change anything
Documentation
Cost to add new featuresBuilding the 100th feature as fast as the 1st
Project Ownership
![Page 11: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/11.jpg)
v a n d e r s t e l t
Monolithic application 11
![Page 12: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/12.jpg)
v a n d e r s t e l t
12
![Page 13: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/13.jpg)
v a n d e r s t e l t
13
Microservices“don't even consider microservices unless you have a system that's too complex to manage as a monolith”
Martin Fowler
![Page 14: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/14.jpg)
SERVICE BOUNDARIESDividing your system’s business logic over logical boundaries
![Page 15: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/15.jpg)
v a n d e r s t e l t
15Microservices architecture
DeploymentDon’t necessarily need to separately deploy every single service
Services boundariesDefine boundaries around logical business processes
MonitoringDocument message flows and monitor them with great tools
MessagingAsynchronous communication
via messaging & publish/subscribe
![Page 16: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/16.jpg)
v a n d e r s t e l t
Any organization that designs a system will produce a design whose structure is a
copy of the organization's communication structure.“
”Conway’s law – 1968
![Page 17: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/17.jpg)
v a n d e r s t e l t
Service boundaries 17
Ship
pin
gFin
ance
Sale
sM
arketing
![Page 18: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/18.jpg)
v a n d e r s t e l t
Deployment 18
Ship
pin
gFin
ance
Sale
sM
arketing
SQL
SQL
{ }
![Page 19: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/19.jpg)
v a n d e r s t e l t
Architectural style 19
Ship
pin
gFin
ance
Sale
sM
arketing
CQRS
Event SourcingCRUD
there is no
top level
architecture
![Page 20: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/20.jpg)
v a n d e r s t e l t
Coupling aspects 20
SpatialTemporalPlatform
![Page 21: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/21.jpg)
v a n d e r s t e l t
Messaging
21
Ship
pin
gFin
ance
Sale
sM
arketing
SQL
![Page 22: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/22.jpg)
v a n d e r s t e l t
Messaging
22
Ship
pin
gFin
ance
Sale
sM
arketing
SQL
![Page 23: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/23.jpg)
v a n d e r s t e l t
Messaging
23
Ship
pin
gFin
ance
Sale
sM
arketing
SQLPublish / Subscribe
![Page 24: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/24.jpg)
v a n d e r s t e l t
24
Messaging
Handle business events through
Inversion of communication
by supplementing SOA with EDA
“”
![Page 25: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/25.jpg)
v a n d e r s t e l t
25Resume
Service BoundariesTry to see if your organizational communication structure can be the model for your logical service boundaries.
MessagingAvoid RPC style communication between services. Use messaging with an event driven architecture.
DeploymentDon’t necessarily see every microservice as a deployable unit. Do keep an eye out for coupling. Avoid joins in your database between entities.
![Page 26: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/26.jpg)
DUPLICATION vs REPLICATIONWhy you shouldn’t duplicate, but replicate data
![Page 27: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/27.jpg)
v a n d e r s t e l t
27
Do not duplicate
across service boundaries
You can share…
some information
Publish / Subscribe
to notify in advance
![Page 28: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/28.jpg)
v a n d e r s t e l t
28Duplicating data
What event just took placebusiness process related
events that other services require to properly function
and take action on
Status changes Identifiers Date Time Info
Who was the subjectthe triggered event has a
specific subject in our domain
When does the status expirewhen status updates expire or
are no longer relevant at a point in time, we provide
expiration information
![Page 29: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/29.jpg)
v a n d e r s t e l t
Sales
Messaging 29
Taxi
CustomersTaxiRequested
TaxiRequested
RequestTaxi
![Page 30: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/30.jpg)
v a n d e r s t e l t
Sales
Messaging 30
TaxiTaxiRequested
RequestTaxi
FinanceTaxiAssigned
Customers
![Page 31: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/31.jpg)
v a n d e r s t e l t
Sales
Messaging 31
TaxiRequestCancelled
Finance
RequestCancelled
CancelRequest
![Page 32: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/32.jpg)
v a n d e r s t e l t
32Replicating vs Duplicating data
So duplication of data is not good But what about this
replicating data?
![Page 33: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/33.jpg)
v a n d e r s t e l t
33Replicating vs Duplicating data
So duplication of data is not good But what about this
replicating data?
Replicating within a service boundary is okay
for performance or connectivity issues
![Page 34: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/34.jpg)
v a n d e r s t e l t
34Replicating data options
Performance optimization
Suitable for all categories business and personal presentation
In memory cache
Prepare for optimal querying
Prepare data for querying, for example into a specific readmodel / viewmodel
Resolve latency issues
Replicating data across physical machines does not break service boundaries
Resolve network issues
Different sites across the globe, or a moving site without connection,
like a large container ship
Viewmodels
Different machines
Different sites
![Page 35: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/35.jpg)
v a n d e r s t e l t
35Composite UI
SalesLists requests made by
customer
CustomerDisplays customer information such as name and current location
TaxiShows where taxi drivers are and at what time they arrive at your location
FinanceDisplays status of all
financial transactions and outstanding bills
![Page 36: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/36.jpg)
MONITORINGyour services
![Page 37: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/37.jpg)
v a n d e r s t e l t
37Monitoring
![Page 38: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/38.jpg)
v a n d e r s t e l t
38Monitoring
![Page 39: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/39.jpg)
v a n d e r s t e l t
39Monitoring
![Page 40: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/40.jpg)
v a n d e r s t e l t
40Resume; how we build microservices
Define BoundariesDefine boundaries of your services. Your organizationalstructure may help. Do not duplicate data across services.Share status changes, identifiers and date-times.
ImplementImplement your microservices that handle a single task.Use publish/subscribe to notify other services with statuschanges.
Composite UIEverything from data up until the user interface shouldbelong in a service, resulting in a composite ui thatretrieves data based on the mentioned identifiers.
DeployDeploy your microservices, but not necessarily as singleunits. A possibility is a deployment unit per serviceboundary.
![Page 41: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/41.jpg)
v a n d e r s t e l t
41
Microservices can be hard to build, deploy & monitor. Get help from people with experience
Think about temporal coupling between services and how to solve this, possibly with messaging
Do not duplicate data between services, as at some point business logic will follow
Conclusion
![Page 42: Duplicating data or replicating data in Micro Services](https://reader033.vdocuments.site/reader033/viewer/2022042716/55cc9f81bb61eb0b698b46be/html5/thumbnails/42.jpg)
v a n d e r s t e l t
42Contact Me
AddressRotterdam, The Netherlands
Twitter@dvdstelt
Websitehttp://dennis.bloggingabout.net
Thank you for your time…