xml store christian theil have, [email protected]@itu.dk rené kofoed, [email protected]@itu.dk...
Post on 21-Dec-2015
220 views
TRANSCRIPT
![Page 1: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/1.jpg)
XML Store
Christian Theil Have, [email protected]é Kofoed, [email protected]
References: Kasper Pedersen & Jesper Pedersen, Value-oriented XML StoreMads Pultz, Garbage Collection in Distributed Value-Oriented Storage System
![Page 2: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/2.jpg)
XML Store overview XML store
Stores values Name service
Like a DNS service
![Page 3: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/3.jpg)
XML Store overview What’s in an XML store
Values Value references
XML store helps you to store values, and locating them again with references.
![Page 4: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/4.jpg)
What’s a value ? A piece of data of some sort, maybe the
number 5 :) Are 5 and 5 and 5 the same values? Nope – there is only one value: 5 The other 5 and 5 are occurences of the
first 5.
![Page 5: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/5.jpg)
What’s a value reference? A shortname for a value :) A shortname is shorter than the data being
stored.
![Page 6: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/6.jpg)
Value references
to xml or not to xml
<speech><speaker>BBCD01</speaker>
<line>BBCD02</line></speech>
Hamlet
Values
Value references
![Page 7: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/7.jpg)
Shared valuesDocument A<speech>
<speaker>BBCD01</speaker>
<line>BBCD02</line></speech>
Hamlet
Document B<speech>
<speaker>BBCD01</speaker>
<line>BBCD03</line></speech>
to be
or not to be
Shared value
![Page 8: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/8.jpg)
And updates? By changing speaker: ”Hamlet” in
document A, will also change document B This will be problematic!
![Page 9: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/9.jpg)
Therefore…. Perform non-destructional updates by:
Creating a new value in a new tree Replacing existing references to the old value
to point to the new value
![Page 10: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/10.jpg)
Changing Hamlet to OpheliaDocument A<speech>
<speaker>BBCD01</speaker>
<line>BBCD02</line></speech>
Document A’<speech>
<speaker>BBCD04</speaker>
<line>BBCD03</line></speech>
Hamlet
Document B<speech>
<speaker>BBCD01</speaker>
<line>BBCD03</line></speech>
Ophelia
![Page 12: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/12.jpg)
XML - format for representing tree-structured data XML models:
Document Object Model (DOM) - loads the entire document into memory in order to manipulate it.
XML Enabled Databases (XED) – stores XML data in an existing relational database.
Native XML Databases – internal model designed for persisting XML
Document Value Model DVM – a way to program XML centric applications
![Page 14: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/14.jpg)
Implementations 1. Based on Chord 2. Based on IP-multicast
Uses locators and a reference server to map a value reference to a location independant value.
![Page 15: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/15.jpg)
XMLStore architechture Overview
Layered architechture Application layer DVM layer Disk layer
A concrete XMLStore is build with plugable modules.
![Page 16: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/16.jpg)
Layers in XMLStore
”SPEECH”Name server:
6FE02A
42.63.145.108:6949/XAA/42,8
<speaker>5A7012,87D311</speaker>
Value reference(hashed value)
Lookup (p2p routing)
load
Locator
Value
Lookup
Document
XMLStore
![Page 17: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/17.jpg)
Modules (1)XMLStore is organized in a modular way.
Funtionality can be added using decorators:
LoadSave
9A01F0451234
Save Load
Spawnsthread
![Page 18: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/18.jpg)
Modules (2)Example modules:
Buffering Caching Asynchronous operation Replication (peer-to-peer) distribution Garbage collection
![Page 19: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/19.jpg)
Name service Binds a human readable name to a value reference. Must provide provide destructive updates
Lost update problem Pessimistic locking Optimistic concurrency control
Should be Atomic
![Page 21: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/21.jpg)
Garbage collection in XMLStore Problem: Since values cannot be deleted
from XMLStore, it could eventually be flooded.
The solution: Garbage collection!
Garbage collection in XMLStore is concerned with reclaiming values that cannot be reached using references.
![Page 22: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/22.jpg)
Garbage collection Live values - can be reached using an existing value
reference. The existing value references comprise the ”Root set”
Mutator – A program that accesses memory Collector – The garbage collector Local vs. Global garbage collection Different approaches
Reference counting Reference listing Tracing
![Page 23: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/23.jpg)
Garbage collection1234F0459A01
<a>F3B2</a>”foo””bar”
FF01 ”foobar”
Name server
9A01F0451234
……
F3B2FF01F3B2
……
111
11
F3B2 … 0
Stub Skeleton
Reference count
![Page 24: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/24.jpg)
Garbage collection XMLStore needs extended API to to keep track
of value references.
![Page 25: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/25.jpg)
Garbage collection Reference counting
Basic idea: count the number of references to a given object, and reclaim it when the count reaches zero.
A reference count is associated with each value occurence
Scales well Doesn’t prevent cycles
This is not a problem since XMLStore contains only acyclic data structures
No resilience to lost messages Low fault tolerance
![Page 26: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/26.jpg)
Garbage collection Reference listing
Each value occurence has a separate skeleton for each client…
Mapping of peers to value references
F3B2
Client A
Client B
Peer
hamlet
Skeleton
![Page 27: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/27.jpg)
Garbage collection Tracing
Live objects are recursively traced from the roots and unreachable objects are determined to be garbage
Stop-and-collect vs. Incremental collection Group based tracing Pros and cons
Can detect cycles Scalability issues
![Page 29: XML Store Christian Theil Have, cth@itu.dkcth@itu.dk René Kofoed, rkofoed@itu.dkrkofoed@itu.dk References: Kasper Pedersen & Jesper Pedersen, Value-oriented](https://reader030.vdocuments.site/reader030/viewer/2022032522/56649d6d5503460f94a4d8b0/html5/thumbnails/29.jpg)
Approach adapted for xmlstore
Local garbage collection Reference listing Tracing (copying collector)
.. So cycles can be collected (only relevant with cells/mutable data)
Global garbage collection Tracing