how to survive in a base world
DESCRIPTION
This talk I gave at the NoSQL matters 2013. It quickly introduces BASE transactions and then illustrates the challenges a developer encounters when switching from ACID to BASE using a (very) simple example application. First the ACID variant is shown. Then a naive BASE transformation is done. After that the typical problems with that approach are shown one by one a how to handle them on a conceptual and code level. Of course the voice track is missing again but lots of stuff should become clear from the slides, though.TRANSCRIPT
![Page 1: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/1.jpg)
How to survive in a BASE world A practical guide for the dauntless developer
Uwe Friedrichsen, codecentric AG, 2013
![Page 2: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/2.jpg)
Uwe Friedrichsen
@ufried
![Page 3: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/3.jpg)
ACID vs. BASE
![Page 4: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/4.jpg)
A C I D Atomicity Consistency Isolation Durability
![Page 5: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/5.jpg)
B A S E Basically Available Soft State Eventually
Consistent
![Page 6: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/6.jpg)
ACID
• Strong consistency
• Isolation
• Focus on “commit”
• Nested transactions
• Availability?
• Conservative (pessimistic)
• Difficult evolution (e.g.
schema)
BASE
• Weak consistency (stale data OK)
• Availability first
• Best effort
• Approximate answers OK
• Aggressive (optimistic)
• Simpler!
• Faster
• Easier evolution
But I think it’s a spectrum
ACID vs. BASE
Source: Eric Brewer, Towards Robust Distributed Systems, PODC Keynote, July-19-2000
![Page 7: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/7.jpg)
Consequences
• No ACID properties across entities
• No ACID properties across nodes
• ACID properties for single entity on single node
![Page 8: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/8.jpg)
A (very) simple example
![Page 9: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/9.jpg)
customer
id : long
name : string
book
id : long
title : string
author : string
loaned by loaned
0..1 0..n
loan(customer, book) : void
return(customer, book) : void
booksLoaned(customer) : list of book
isBookLoaned(book) : boolean
Data model
Actions
![Page 10: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/10.jpg)
Iteration #0
RDBMS – ACID – JPA
![Page 11: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/11.jpg)
JPA Entity Class Customer
@Entity
public class Customer {
@Id
@GeneratedValue
private long id;
private String name;
@OneToMany(mappedBy = "customer")
private final List<Book> loanedBooks;
<Methods>
}
![Page 12: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/12.jpg)
JPA Entity Class Book
@Entity
public class Book {
@Id
@GeneratedValue
private long id;
private String title;
private String author;
private Customer customer;
<Methods>
}
![Page 13: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/13.jpg)
Database model
Foreign Key to
CUSTOMER.ID
TABLE CUSTOMER
ID
NAME
TABLE BOOK
ID
TITLE
AUTHOR
CUSTOMER (FK)
![Page 14: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/14.jpg)
Action „loan“ using JPA
public void loan(long customerId, long bookId) {
<Get EntityManager>
em.getTransaction().begin();
<Read customer and book from database>
customer.getLoanedBooks().add(book);
book.setCustomer(customer);
em.persist(customer);
em.persist(book);
em.getTransaction().commit();
<Cleanup>
}
Same for action „return“
![Page 15: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/15.jpg)
Action „isBookLoaned“ using JPA
public boolean isBookLoaned(long bookId) {
<Get EntityManager>
Query q = em.createQuery("SELECT b FROM Book b WHERE b.id = :id");
q.setParameter("id", bookId);
Book book = (Book) q.getSingleResult();
return book.getCustomer() != null;
}
![Page 16: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/16.jpg)
Action „booksLoaned“ using JPA
public List<Book> booksLoaned(long customerId) {
<Get EntityManager>
Query q =
em.createQuery("SELECT c FROM Customer c WHERE c.id = :id");
q.setParameter("id", customerId);
Customer customer = (Customer) q.getSingleResult();
return customer.getLoanedBooks();
}
Boils down to an implicitly executed SQL query that looks like
SELECT … FROM BOOK WHERE CUSTOMERID = <customerId>
![Page 17: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/17.jpg)
Mutating action
• begin transaction
• do required changes
• commit
Non-mutating action
• create query
• execute query
• navigate in result set (implicitly executing additional queries if required)
Wrapup ACID model
![Page 18: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/18.jpg)
Iteration #1
NoSQL – BASE – naïve
![Page 19: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/19.jpg)
Mutating action
• begin transaction
• do required changes
• commit
Non-mutating action
• create query
• execute query
• navigate in result set (implicitly executing additional queries if required)
Naïve BASE model
???
![Page 20: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/20.jpg)
Data model
Customer
TABLE CUSTOMER
ID
NAME
Book
TABLE BOOK
ID
TITLE
AUTHOR
CUSTOMER (FK)
{
"id" : <id>,
"name" : "<name>"
}
{
"id" : <id>,
"title" : "<title>",
"author" : "<author>",
"customerId" : <id>
}
![Page 21: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/21.jpg)
JPA Entity Class Customer
@Entity
public class Customer {
@Id
@GeneratedValue
private long id;
private String name;
@OneToMany(mappedBy = "customer")
private final List<Book> loanedBooks;
<Methods>
}
Need to take care of ID generation ourselves
Gone for the moment (naïve approach)
![Page 22: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/22.jpg)
Class Customer
public class Customer {
private long id;
private String name;
<Methods>
}
![Page 23: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/23.jpg)
JPA Entity Class Book
@Entity
public class Book {
@Id
@GeneratedValue
private long id;
private String title;
private String author;
private Customer customer;
<Methods>
}
Need to take care of ID generation ourselves
Gone for the moment (naïve approach)
![Page 24: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/24.jpg)
Class Book
public class Book {
private long id;
private String title;
private String author;
private long customerId;
<Methods>
}
Only the ID
![Page 25: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/25.jpg)
Action „loan“, „return“ & „isBookLoaned“
public void loan(long customerId, long bookId) {
Book book = readBook(bookId);
book.setCustomerId(customerId);
writeBook(book);
}
public void return(long customerId, long bookId) {
Book book = readBook(bookId);
book.setCustomerId(0L);
writeBook(book);
}
public boolean isBookLoaned(long bookId) {
Book book = readBook(bookId);
return book.getCustomerId() > 0L;
}
![Page 26: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/26.jpg)
readBook
private Book readBook(long id) {
String json = readBookFromDb(id);
return toBook(json);
}
private Book toBook(String json) {
JSONObject jo = new JSONObject(json);
Book b = new Book(jo.getLong("id"), jo.getString("title"),
jo.getString("author"), jo.getLong("customerId"));
return b;
}
Same for „readCustomer “ and „toCustomer “
![Page 27: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/27.jpg)
writeBook
private void writeBook(Book b) {
long key = b.getId();
String json = fromBook(b);
writeBookToDb(key, json);
}
private String fromBook(Book b) {
JSONObject jo = new JSONObject();
jo.put("id", b.getId());
jo.put("title", c.getTitle());
jo.put("author", c.getAuthor());
jo.put("customerId", b.getCustomerId());
return jo.toString();
}
Same for „writeCustomer “ and „fromCustomer “
![Page 28: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/28.jpg)
Action „booksLoaned“
public List<Book> booksLoaned(long customerId) {
List<Book> loanedBooks = new ArrayList<Book>();
Iterator<Book> it = readAllBooks();
for (Book b : it) {
if (book.getCustomerId() == customerId) {
loanedBooks.add(b);
}
}
return loanedBooks;
}
private Iterator<Book> readAllBooks() {
DBIterator dbi = readAllBooksFromDb(); // Iterator<String> (json)
return new BookIterator(dbi);
}
![Page 29: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/29.jpg)
BookIterator
public class BookIterator implements Iterator<Book> {
private final DBIterator dbi;
public BookIterator(DBIterator dbi) {
this.dbi = dbi;
}
@Override public boolean hasNext() {
return dbi.hasNext();
}
@Override public Book next() {
return toBook(dbi.next());
}
@Override public void remove() {
throw new UnsupportedOperationException();
}
}
![Page 30: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/30.jpg)
Iteration #2
Handling inconsistencies
across replica sets
![Page 31: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/31.jpg)
Assume the following code sequence
loan(2312L, 4711L);
boolean isLoaned = isBookLoaned(4711L);
Question: Which value has isLoaned?
Quick exercise
![Page 32: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/32.jpg)
Alternative #1
Replica
Node 1
Replica
Node 3
Client
replicate
2
loan 1 isLoaned 3
Replica
Node 2
![Page 33: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/33.jpg)
Alternative #2
Replica
Node 1
Replica
Node 3
Client
replicate
3
loan 1 isLoaned 2
Replica
Node 2
![Page 34: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/34.jpg)
Read your own writes
Solution #1
![Page 35: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/35.jpg)
Read your
own writes
Replica
Node 1
Replica
Node 3
Client
replicate
?
loan 1 isLoaned 2
Replica
Node 2
![Page 36: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/36.jpg)
Read from master
Solution #2
![Page 37: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/37.jpg)
Read from
master
Master Slave 2
Client
replicate
?
loan 1 isLoaned 2
Slave 1
![Page 38: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/38.jpg)
Quorum
• W > N/2
• R + W > N
Solution #3
![Page 39: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/39.jpg)
readBook & writeBook
private Book readBook(long id) {
String json = readBookFromDb(id, READ_QUORUM);
return toBook(json);
}
private void writeBook(Book b) {
long key = b.getId();
String json = fromBook(b);
writeBookToDb(key, json, WRITE_QUORUM);
}
Same for „readCustomer “ and „writeCustomer “
![Page 40: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/40.jpg)
Iteration #3
Handling siblings
due to partitioning
![Page 41: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/41.jpg)
Assume the following code sequence
boolean isLoaned = isBookLoaned(4711L);
Question: What do you get?
Quick exercise
![Page 42: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/42.jpg)
isBookLoaned(4711L)
public boolean isBookLoaned(4711L) {
Book book = readBook(4711L);
return book.getCustomerId() > 0L;
}
private Book readBook(4711L) {
String json = readBookFromDb(4711L);
return toBook(json);
}
private Book toBook(String json) {
JSONObject jo = new JSONObject(json);
Book b = new Book(jo.getLong("id"), jo.getString("title"),
jo.getString("author"), jo.getLong("customerId"));
return b;
}
Expects a “normal” book JSON
![Page 43: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/43.jpg)
… but your JSON looks like this
{ "siblings" : [
{
"id" : 4711L,
"title" : "The Catcher in the Rye",
"author" : "J. D. Salinger",
"customerId" : 2312L
},
{
"id" : 4711L,
"title" : "The Catcher in the Rye",
"author" : "J. D. Salinger",
"customerId" : 0L
}
]
}
![Page 44: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/44.jpg)
Leave the decision to the caller
Solution #1
![Page 45: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/45.jpg)
Build a resolver
Solution #2
![Page 46: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/46.jpg)
Extended JSON data model
Customer:
{
"id" : <id>,
"name" : "<name>",
"timestamp" : <timestamp>
}
Book:
{
"id" : <id>,
"title" : "<title>",
"author" : "<author>",
"customerId" : <id>,
"timestamp" : <timestamp>
}
![Page 47: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/47.jpg)
Domain Classes
public class Customer {
private Long id;
private String name;
<Methods>
}
public class Book {
private long id;
private String title;
private String author;
private long customerId;
<Methods>
}
Timestamp not required
in the domain classes
![Page 48: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/48.jpg)
fromBook
private String fromBook(Book b) {
JSONObject jo = new JSONObject();
jo.put("id", b.getId());
jo.put("title", c.getTitle());
jo.put("author", c.getAuthor());
jo.put("customerId", b.getCustomerId());
jo.put("timestamp", System.currentTimeMillis());
return jo.toString();
}
Same for „fromCustomer “
Timestamp added
in mapping
![Page 49: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/49.jpg)
toBook
private Book toBook(String json) {
JSONObject jo = new JSONObject(resolveBook(json));
Book b = new Book(jo.getLong("id"), jo.getString("title"),
jo.getString("author"), jo.getLong("customerId"));
return b;
}
Same for „toCustomer “
![Page 50: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/50.jpg)
resolveBook
private String resolveBook(String json) {
JSONObject jo = new JSONObject(json);
JSONArray siblings = jo.optJSONArray("siblings");
if (siblings == null) {
return json;
}
int index = 0;
long timestamp = 0L
for (int i = 0; i < siblings.length(); i++) {
long t = siblings.getJSONObject(i).getLong("timestamp");
if (t > timestamp) {
index = i;
timestamp = t;
}
}
return siblings.getJSONObject(index).toString();
}
![Page 51: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/51.jpg)
Conflict-free replicated data types
Solution #3
![Page 52: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/52.jpg)
Iteration #4
Read optimizations
![Page 53: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/53.jpg)
Assume the following method to retrieve the
number of books a customer has loaned
public int numberOfBooksLoaned(long cId) {
return booksLoaned(cId).size();
}
Question: How is the performance?
Quick exercise
![Page 54: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/54.jpg)
Remember the implementation
public List<Book> booksLoaned(long customerId) {
List<Book> loanedBooks = new ArrayList<Book>();
Iterator<Book> it = readAllBooks();
for (Book b : it) {
if (book.getCustomerId() == customerId) {
loanedBooks.add(b);
}
}
return loanedBooks;
}
private Iterator<Book> readAllBooks() {
DBIterator dbi = readAllBooksFromDb(); // Iterator<String> (json)
return new BookIterator(dbi);
}
Reads all book entities distributed across all nodes
![Page 55: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/55.jpg)
Use secondary indices
Solution #1
![Page 56: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/56.jpg)
Re-implementation using indices
public List<Book> booksLoaned(long customerId) {
List<Book> loanedBooks = new ArrayList<Book>();
Iterator<Book> it = readAllBooksLoanedByACustomer(customerId);
for (Book b : it) {
loanedBooks.add(b);
}
return loanedBooks;
}
private Iterator<Book> readAllBooksLoanedByACustomer(long customerId) {
DBIterator dbi =
readAllBooksUsingSecondaryIndexFromDb(long customerId);
return new BookIterator(dbi);
}
![Page 57: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/57.jpg)
• Only works on ACID nodes
• Duplicates while rebalancing
• Partial answers if node is down
Tradeoffs of indices
![Page 58: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/58.jpg)
… but actually you are interested in a property
that logically belongs to a single customer
![Page 59: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/59.jpg)
Denormalize
Solution #2
![Page 60: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/60.jpg)
Denormalized Customer entity
{
"id" : <id>,
"name" : "<name>",
"loanedBooks" : [
{
"id" : <id>,
"title" : "<title>",
"author" : "<author>"
},
...
],
"timestamp" : <timestamp>
}
Can be empty
![Page 61: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/61.jpg)
Denormalized Book entity
{
"id" : <id>,
"title" : "<title>",
"author" : "<author>",
"loanedBy" :
{
"id" : <id>,
"name" : "<name>"
},
"timestamp" : <timestamp>
}
Optional value
![Page 62: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/62.jpg)
Denormalized Customer domain class
public class Customer {
private Long id;
private String name;
private List<BookRef> loanedBooks;
<Methods>
}
public class BookRef { object
private Long id;
private String title;
private String author;
<Methods>
}
Here we go again
![Page 63: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/63.jpg)
Denormalized Book domain class
public class Book {
private Long id;
private String title;
private String author;
private CustomerRef customer;
<Methods>
}
public class CustomerRef {
private Long id;
private String name;
<Methods>
}
Here we go again
![Page 64: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/64.jpg)
Actions „loan“ & „return“
public void loan(long customerId, long bookId) {
Customer customer = readCustomer(customerId);
Book book = readBook(bookId);
customer.getLoanedBooks().add(new BookRef(book));
book.setCustomer(new CustomerRef(customer));
writeCustomer(customer);
writeBook(book);
}
public void return(long customerId, long bookId) {
Customer customer = readCustomer(customerId);
Book book = readBook(bookId);
customer.getLoanedBooks().remove(new BookRef(book));
book.setCustomer(null);
writeCustomer(customer);
writeBook(book);
}
Also mutates
the customer
![Page 65: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/65.jpg)
Actions „booksLoaned“ & „isBookLoaned“
public List<BookRef> booksLoaned(long customerId) {
Customer customer = readCustomer(customerId);
return customer.getLoanedBooks();
}
public boolean isBookLoaned(long bookId) {
Book book = readBook(bookId);
return book.getCustomer() != null;
}
Only one read
required
Check becomes
more expressive
Method “readAllBooks” not required anymore in this scenario
![Page 66: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/66.jpg)
fromCustomer
private String fromCustomer(Customer c) {
JSONObject jo = new JSONObject();
jo.put("id", c.getId());
jo.put("name", c.getName());
JSONArray ja = new JSONArray();
for (BookRef b : c.getLoanedBooks()) {
JSONObject jb = new JSONObject();
jb.put("id", b.getId());
jb.put("title", c.getTitle());
jb.put("author", c.getAuthor());
ja.put(jb);
}
jo.put("loanedBooks", ja);
jo.put("timestamp", System.currentTimeMillis());
return jo.toString();
}
Also map
book references
![Page 67: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/67.jpg)
toCustomer
private Customer toCustomer(String json) {
JSONObject jo = new JSONObject(json);
JSONArray ja = jo.getJSONArray("loanedBooks");
List<BookRef> loanedBooks = new ArrayList<BookRef>();
for (int i = 0; i < ja.length(); i++) {
JSONObject jb = ja.getJSONObject(i);
BookRef b = new BookRef(jb.getLong("id"),
jb.getString("title"),
jb.getString("author"));
loanedBooks.add(b);
}
Customer c = new Customer(jo.getLong("id"), jo.getString("name"),
loanedBooks);
return c;
}
Also map
book references
![Page 68: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/68.jpg)
fromBook
private String fromBook(Book b) {
JSONObject jo = new JSONObject();
jo.put("id", b.getId());
jo.put("title", c.getTitle());
jo.put("author", c.getAuthor());
if (c.getCustomer() != null) {
JSONObject jc = new JSONObject();
jc.put("id", c.getCustomer().getId());
jc.put("name", c.getCustomer().getName());
jo.put("loanedBy", jc);
}
jo.put("timestamp", System.currentTimeMillis());
return jo.toString();
}
Also map
customer
reference
![Page 69: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/69.jpg)
toBook
private Book toBook(String json) {
JSONObject jo = new JSONObject(json);
CustomerRef c = null;
JSONObject jc = jo.optJSONObject("loanedBy");
if (jc != null) {
c = new CustomerRef(jc.getLong("id"), jc.getString("name"));
}
Book b = new Book(jo.getLong("id"), jo.getString("title"),
jo.getString("author"), c);
return b;
}
Also map
customer
reference
![Page 70: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/70.jpg)
• All data within one entity
• Coarse grained entities
• Often huge read speedups
• No OR-Mapper equivalent
• Can result in inconsistencies across entities
Wrapup denormalization
![Page 71: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/71.jpg)
Iteration #5
Handling inconsistencies
across entities
![Page 72: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/72.jpg)
Assume the following code sequence
loan(2312L, 4711L);
long bookId = readCustomer(2312L)
.getLoanedBooks().get(0).getId();
long customerId = readBook(4711L)
.getCustomer().getId();
Question: What do you get?
Quick exercise
![Page 73: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/73.jpg)
Remember the implementation
public void loan(2312L, 4711L) {
Customer customer = readCustomer(2312L);
Book book = readBook(4711L);
customer.getLoanedBooks().add(new BookRef(book));
book.setCustomer(new CustomerRef(customer));
writeCustomer(customer);
writeBook(book);
}
This call fails
![Page 74: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/74.jpg)
Leads to the following entity state
{
"id" : 2312L,
"name" : "Uwe Friedrichsen",
"loanedBooks" : [
{
"id" : 4711L,
"title" : "The Catcher in the Rye",
"author" : "J. D. Salinger"
}
],
"timestamp" : <timestamp>
}
{
"id" : 4711L,
"title" : "The Catcher in the Rye",
"author" : "J. D. Salinger",
"timestamp" : <timestamp>
}
![Page 75: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/75.jpg)
On-the-fly resolver
Solution #1
![Page 76: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/76.jpg)
Actions „loan“ & „return“
public void loan(long customerId, long bookId) {
Customer customer = readCustomer(customerId);
Book book = readBook(bookId);
customer.getLoanedBooks().add(new BookRef(book));
book.setCustomer(new CustomerRef(customer));
writeBook(book);
writeCustomer(customer);
}
public void return(long customerId, long bookId) {
Customer customer = readCustomer(customerId);
Book book = readBook(bookId);
customer.getLoanedBooks().remove(new BookRef(book));
book.setCustomer(null);
writeBook(book);
writeCustomer(customer);
}
Switched
write order
![Page 77: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/77.jpg)
resolveIfRequired
private void resolveIfRequired(Customer c, Book b) {
BookRef br = new BookRef(b.getId(), b.getTitle(), b.getAuthor());
if (!c.getLoanedBooks().contains(br)) {
c.getLoanedBooks.add(br);
}
}
// some code that (for some reason) reads both related entities
...
Book book = readBook(bookId);
Customer customer = null;
If (book.getCustomer() != null) {
readCustomer(book.getCustomer().getId());
resolveIfRequired(customer, book);
}
// Do stuff with customer and book
...
![Page 78: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/78.jpg)
On-the-fly resolver with write-back (“read repair”)
Solution #2
![Page 79: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/79.jpg)
resolveIfRequired
private void resolveIfRequired(Customer c, Book b) {
BookRef br = new BookRef(b.getId(), b.getTitle(), b.getAuthor());
if (!c.getLoanedBooks().contains(br)) {
c.getLoanedBooks.add(br);
writeCustomer(c);
}
}
Best effort principle
![Page 80: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/80.jpg)
Consistency crawler
Compensating actions
Change journal
…
Advanced solutions
???
![Page 81: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/81.jpg)
Wrap-up
• BASE is different
• New challenges for a developer
• Some databases promise to hide BASE …
… but you can‘t fool the laws of distribution
• Inconsistencies will happen – be prepared
![Page 82: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/82.jpg)
The real world is also BASE
Welcome to reality … ;-)
![Page 83: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/83.jpg)
Uwe Friedrichsen
@ufried
http://www.slideshare.net/ufried/
http://blog.codecentric.de/author/ufr
![Page 84: How to survive in a BASE world](https://reader034.vdocuments.site/reader034/viewer/2022052303/54b7a45f4a79592f188b456f/html5/thumbnails/84.jpg)