Meetup Gothenburg Dec 28 2015 Per-Åke Minborgminborg@speedment.com

Palo Alto Office Speedment Java One Presentation

Background

Before Speedment

Connector

JDBC

ORM

JPA

ApplicationPROSWidely spread standard

• Many providers• Many developers

You can work with a relational database as if it were object oriented Increased productivity Provides a certain degree of abstraction (you can replace your DBMS)

CONS Slows down the application and gives unpredictable performance You can not access all Java 8 featuresTime consuming to write applications for existing databases, You must still write SQL, HQL, et. AlHigher the risk for errors

Java

ORM

JPA

Speedment- a Java Development Tool

Connector

JDBC

Application Object Oriented functional programming

Fast to develop Fast Execution Easy to test Secure

Make use of all Java 8 features Domain Model reflects your existing

datasource Automatic generation of Code (”error free”)

Java

Speedment

Java 8

Application

Speedment- a Java Development Tool

Code Examples• Embrace Java 8 paradigms with stream(), filter(), sort(), limit() etc.

• Forget about SQL, JQL et. al

Database Table “hare”

mysql> explain hare;+-------+-------------+------+-----+---------+----------------+| Field | Type | Null | Key | Default | Extra |+-------+-------------+------+-----+---------+----------------+| id | int(11) | NO | PRI | NULL | auto_increment || name | varchar(45) | NO | | NULL | || color | varchar(45) | NO | | NULL | || age | int(11) | NO | | NU | |+-------+-------------+------+-----+---------+----------------+4 rows in set (0.01 sec)

Code Generationpublic interface Hare extends Entity<Hare> {

public final static ReferenceComparableField<Hare, Integer> ID = … public final static ReferenceComparableStringField<Hare> NAME = … public final static ReferenceComparableStringField<Hare> COLOR = … public final static ReferenceComparableField<Hare, Integer> AGE = …

Integer getId(); String getName(); String getColor(); Integer getAge();

Hare setId(Integer id); Hare setName(String name); Hare setColor(String color); Hare setAge(Integer age);

/** Graph-like traversal methods eliminating JOINs */ Stream<Carrot> findCarrotsByOwner(); Stream<Carrot> findCarrotsByRival(); Stream<Carrot> findCarrots();}

Querying using Java 8 StreamsList<Hare> oldHares = hares.stream() .filter(Hare.AGE.greaterThan(4)) .collect(toList());

SELECT * from `hare`WHERE`age` > 4;

Stream Fundamentals Detailed description Slide 35-45

Depends on Terminating OperationOptional<Hare> oldHare = hares.stream() .filter(Hare.AGE.greaterThan(4)) .findAny();

SELECT * from `hare`WHERE`age` > 4 LIMIT 1;

Depends on Terminating Operationlong noOldHares = hares.stream() .filter(Hare.AGE.greaterThan(4)) .count();

SELECT count(*) from `hare`WHERE`age` > 4;

Entities are linked

Optional<Carrot> carrot = hares.stream() .filter(Hare.NAME.equal(”Spire”)) // Carrot is a foreign key. .flatmap(Hare::findCarrots) .findAny();

Multi-threading

hares.stream() .parallel() .forEach(doExpensiveOperation());

PersistenceHare harry = hares.newInstance() .setName("Harry") .setColor("Gray") .setAge(3) .persist();

// OR JPA-style

entityManager.persist(harry);

Delete Using Streams users.stream()   .filter(User.ID.greaterThan(4))   .forEach(User::delete);

DELETE FROM `user`WHERE `id` > 4;

Initialization

Speedment speedment = new HareApplication().build();Manager<Hare> hares = speedment.managerOf(Hare.class);

Speedment Insane Mode•Extreme Performance, real time big Data

•Convert large databases into In-JVM-memory Java Objects

•Scales Linearly -More CPUs, More Insane

In Memory Initialization

Speedment speedment = new HareApplication()

.with(OnHeapReadOnlyCacheComponent.class) .build();

How does it Work?

23

1 In-JVM-memory cache2 Java Objects in Graph View3 Separates Reads and WritesReflection in

real time

ApplicationJVM

Read

Write

Why will in-JVM-Memory give Insane Speed?

Database In JVM Memory

Compare latencies using the Speed of Light

Why will in-JVM Memory give Insane Speed?• Keep everything in-JVM

• No need to check if an object is in cache

• We can organize objects knowing that we have them all

• We do not even have to look at individual objects -> O(1)

• Everything is Java

Size

Time

O(1)

O(n)

Speedment – OSS and Enterprise

Cloud Deployment

RDBMS

Speedment

Speedment

Speedment Speedment

Speedment

• Spin up nodes as you grow

• Partition data over nodes

How can I fit my full Database in JVM Memory?

~24 GB

>1 TB

>4 TB

∞

On-HeapOff-Heap

SSD

SQL

Database Size(After Compression)

Scale out with our partner Hazelcast• Set up a cluster in the Cloud• Scale out your existing database• Set up Terabytes of RAM

Properties:• The API will remain the same regardless of selected Storage Engine

• Massively concurrent and lock free

• Scale with CPU performance and number of cores

• Eventually consistent

• Transaction safe, if you want

Configuration• Groovy script like Gradle

• Config file is placed at the same location as the POM

dbms { schema { table { name = ”hare"; column { name = "name"; } } }}

Ease of use:• Single Maven dependency in POM

• Maven targets

• Works with Gradle

• GUI to automatically derive the groovy config file from existing databases

Properties:• The database ”owns” the data

• Data is reflected into Materialized Object Views (MOVs)

• MOVs and application resides in the same JVM

• Most operations are O(1)

• persons.byName(“Bob") -> Map<PK,Person> with 1000 persons in 100 ns (30 m)

• TCP/IP RTT 100 us -> +1000 "finds" per TCP RT.

• In-memory DBs with zero query latency are snails by definition

http://localhost:4567/trades/1001/200? from=0&to=1441147840000&callback=Ext.data.JsonP.callback1

Ext.data.JsonP.callback1([{'start':1420131600000,'stop':1420236681000,'average':100876,'open':100000,'close':102209,'high':104060,'low':98714},{'start':1420236682000,'stop':1420243199000,'average':102339,'open':102224,'close':103348,'high':103366,'low':101603},{'start':1420477200000,'stop':1420502399000,'average':102272,'open':103353,'close':103316,'high':103608,'low':100819},{'start':1420563600000,'stop':1420657005000,'average':101682,'open':103315,'close':102292,'high':103755,'low':99241},

… (200 rows in total) ...{'start':1441147839000,'stop':1441147839000,'average':142165,'open':142165,'close':142165,'high':142165,'low':142165}]);

Objectives:

Stream Fundamentals Per-Åke Minborgminborg@speedment.com

Setup

36

DB App

Pwd ?

In-JVM Cache

Stream<Trade> stream = trades.stream();

37

DB

Data Source

Trade5 Trade4 Trade3 Trade2 Trade1

trades.stream().parallel();

38

TradeN

Trade4

Trade3

Trade2

Trade1

TradeX

Thread 1

Thread 2

Thread N

DB

Data Source

trades.stream().filter(Predicate<Trade> predicate);

39

Trade2 Trade1predicate.test(trade)Trade5 Trade4

Trade3

false

true

trades.stream().map(Function<Trade,String> mapper);

40

Json2 Json1Trade5 Trade4mapper.

apply(trade)

e.g. Trade::toJson

trades.stream().collect(Collector collector);

41

TradeN

Trade4

Trade3

Trade2

Trade1

TradeX

apply(trade)

apply(trade)

apply(trade)

Complete Stream

42

DB

Data Source

Finding first and last date/time in an interval

43

from to

selected

trades.stream().parallel().filter(DATE.between(from, to)).filter(STOCK.equal(stock)).mapToLong(Trade::getDate).summaryStatistics();

44

2 3 4

2 3 4

2 3 4

5

12345

DB

Data Source

45

DB

Data Source 1

4

4

4

52

3

trades.stream().parallel().filter(DATE.between(from, to)).filter(STOCK.equal(stock)).mapToLong(Trade::getDate).summaryStatistics();

12345

Generating a REST APIIntroduction to Speedment and Sencha

Emil Forslundemil@speedment.com

Outline●The Speedment + Sencha Software Stack●Using the Pivotgrid Plugin for Speedment

Overview of the Software Stack

DB

Javascript Framework

ServerFramework

DB Abstraction Layer

HTML5RESTJava 8JDBC

Database Layer●Holds all the data collected from various sources●High Level of Normalization

Sale

ProductSalesperson

Customer

Country

Region

Office

Cost Price

Count

Database Abstraction Layer●Presents an Object-Oriented view of the database●Write expressive and powerful queries●Optimize reading by using in-memory caching●Better testability and higher productivity

speedment.managerOf(Sale.class) .stream() .filter(Sale.PRODUCT.equal(105)) .collect(toList());

Server Framework●A lightweight webserver●Separates client/server-responsabilitys●Execute selected business logic as response to REST commands●Low Level of Abstraction

http://localhost:8123/salesinfo/?c=callback&filter=[{...}]

Javascript Framework●Presents data in user-friendly components●User can navigate the data in the browser●Automatically requests additional data from the server

Introduction to the Pivot Grid for Speedment●Generate a full-fledged server application from your database●Select exactly which data to expose in the API●Define aliases for columns and tables●Map between different data types●Create virtual columns calculated in real-time

Step 1 – Create a New Project

Step 2 – Find Pivotgrid Archetype

Step 3 – Name Project

Step 4 – Run Speedment GUI

Step 5 – Press Generate

Step 6 – Run Application

Example Application: Salesinfo

Thanks!