Download - Oak , the Architecture of the new Repository
![Page 1: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/1.jpg)
APACHE SLING & FRIENDS TECH MEETUPBERLIN, 22-24 SEPTEMBER 2014
Oak, the Architecture of the new RepositoryMichael Dürig, Adobe Research
![Page 2: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/2.jpg)
Design goals
adaptTo() 2014 2
Scalable Big repositories Clustering Customisable, flexible OSGi friendly
![Page 3: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/3.jpg)
Outline
adaptTo() 2014 3
CRUD Changes Search
![Page 4: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/4.jpg)
Tree model
adaptTo() 2014 4
a d
b c
![Page 5: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/5.jpg)
Updating
adaptTo() 2014 5
?
x
a d
b c
![Page 6: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/6.jpg)
MVCC
adaptTo() 2014 6
a d
b c
r2: /dr1: /d
r1: /a/br2: /a/b
r1: / r2: /
r2: /d/x
HEAD
![Page 7: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/7.jpg)
adaptTo() 2014 7
Refresh and Garbage Collection
![Page 8: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/8.jpg)
garbage
Refresh
adaptTo() 2014 8
![Page 9: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/9.jpg)
garbage
Garbage collection
adaptTo() 2014 9
![Page 10: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/10.jpg)
adaptTo() 2014 10
Concurrency and Conflicts
![Page 11: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/11.jpg)
r2a
Concurrent updates
adaptTo() 2014 11
r2br1
![Page 12: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/12.jpg)
Merging
adaptTo() 2014 12
r1 r3merge
r2b
r2a
upates
![Page 13: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/13.jpg)
Conflict handling: serialisation
adaptTo() 2014 13
Fully serialised Fail, no concurrent update
Partially serialised Concurrent conflict free updates
![Page 14: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/14.jpg)
Conflict handling strategies: merging
adaptTo() 2014 14
Partial merge Conflict markers, deferred resolution
Full merge Need to choose victim
![Page 15: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/15.jpg)
adaptTo() 2014 15
Replicas and Sharding
![Page 16: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/16.jpg)
Replica and caches
adaptTo() 2014 16
master copy full replica cache
![Page 17: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/17.jpg)
Sharding strategies
adaptTo() 2014 17
by path by level by hash with caching
![Page 18: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/18.jpg)
adaptTo() 2014 18
Implementations
![Page 19: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/19.jpg)
MicroKernel / NodeStore
adaptTo() 2014 19
Tree / Revision model implementation
Responsible for
Clustering
Sharding
Caching
Conflict handling
Not responsible for
Validation
Access control
Search
Versioning
![Page 20: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/20.jpg)
Current implementations
adaptTo() 2014 20
DocumentMK TarMK (SegmentMK)
Persistence MongoDB, JDBC Local FS
Conflict handling Partial serialisation Full serialisation
Clustering MongoDB clustering Simple failover
Sharding MongoDB sharding N/A
Node Performance Moderate High
Key use cases Large deployments (>1TB), concurrent writes
Small/medium deployments, mostly read
![Page 21: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/21.jpg)
adaptTo() 2014 21
Access Control
![Page 22: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/22.jpg)
Accessible paths
adaptTo() 2014 22
a d
cb
![Page 23: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/23.jpg)
xistentialism
adaptTo() 2014 23
All paths traversable Node may not exist Decorator on NodeStore
E
root.getChildNode("a").exists();
root.getChildNode("a") .getChildNode("b").exists();
⟹ false
⟹ true
![Page 24: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/24.jpg)
adaptTo() 2014 24
Comparing Revisions
![Page 25: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/25.jpg)
Content diff
adaptTo() 2014 25
What changed between trees Cornerstone for
Validation Indexing Observation …
![Page 26: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/26.jpg)
What changed?
adaptTo() 2014 26
∆
![Page 27: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/27.jpg)
Example: merging
adaptTo() 2014 27
r1
r2a
r2b
r3
r1 ➞ r2a“a” modified
“b” removed
∆
r1 ➞ r2b “d” modified
“x” added
∆
![Page 28: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/28.jpg)
adaptTo() 2014 28
Commit Hooks
![Page 29: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/29.jpg)
Commit hooks
adaptTo() 2014 29
Key plugin mechanism Higher level functionality
Validation (node type, access control, …) Trigger (auto create, defaults, …) Updates (index, …)
![Page 30: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/30.jpg)
Editing a commit
adaptTo() 2014 30
∆ ∆ + x
![Page 31: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/31.jpg)
Commit hooks
adaptTo() 2014 31
Based on content diff pass a commit fail a commit edit a commit
Applied in sequence
![Page 32: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/32.jpg)
Type of hooks
adaptTo() 2014 32
CommitHook Editor Validator
Content diff Optional Always Always
Can modify Yes Yes No
Programming model
Simple Callbacks Callbacks
Performance impact
High Medium Low
![Page 33: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/33.jpg)
adaptTo() 2014 33
Observers
![Page 34: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/34.jpg)
Observers
adaptTo() 2014 34
Observe changes After commit Often does a content diff Asynchronous Optionally synchronous
Local cluster node only
![Page 35: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/35.jpg)
Examples
adaptTo() 2014 35
JCR observation External index update Cache invalidation Logging
![Page 36: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/36.jpg)
adaptTo() 2014 36
Search
![Page 37: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/37.jpg)
Query Engine
adaptTo() 2014 37
SELECTWHERE x=y
/a//*
Parser
Parser
Parser
Index
Index
Traverse
Parser
Index
parse execute post process
![Page 38: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/38.jpg)
Index Implementations
adaptTo() 2014 38
Property (ordered) Reference Lucene
In-content or file system Solr
Embedded or external
![Page 39: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/39.jpg)
adaptTo() 2014 39
Big Picture
![Page 40: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/40.jpg)
Big picture
adaptTo() 2014 40
MicroKernel
Oak Core
Oak JCR
Oak API
NodeStore API
JCR API
Plugins
![Page 41: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/41.jpg)
Resources
adaptTo() 2014 41
http://jackrabbit.apache.org/oak/
![Page 42: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/42.jpg)
adaptTo() 2014 42
Appendix
![Page 43: Oak , the Architecture of the new Repository](https://reader036.vdocuments.site/reader036/viewer/2022062422/56813f29550346895da9c7ac/html5/thumbnails/43.jpg)
Resources
adaptTo() 2014 43
http://jackrabbit.apache.org/oak/
http://jackrabbit.apache.org/oak/docs/
https://svn.apache.org/repos/asf/jackrabbit/oak/trunk
/