thinking beyond orm in jpa

Thinking Beyond ORM in JPA

Patrycja WegrzynowiczJavaOne 2015

About Me• 15+ professional experience

– Software engineer, architect, head of software R&D • Author and speaker

– JavaOne, Devoxx, JavaZone, TheServerSide Java Symposium, Jazoon, OOPSLA, ASE, others

• Finalizing PhD in Computer Science • Founder and CTO of Yonita

– Bridge the gap between the industry and the academia – Automated detection and refactoring of software defects– Security, performance, concurrency, databases

• @yonlabs

Outline• Motivation• Why?– App-centric vs. data-centric

• What?– Use cases and performance

• How?– JPA (2.1)

• Conclusion

Database

DatabaseThe Mordor of Java Devs

App-Centric vs. Data-Centric• App-centric– Java code drives database design– One app accesses data– CRUD more often than complex queries

• Data-centric– Database design drives Java code– Several apps access data– CRUD as often as complex queries

Use CasesLegacy systemsReportingComplex queriesBulk operations

Performance

SpeedLatencyThroughput

Use Cases and Performance

Spee

dLa

tenc

yTh

roug

hput

Legacy systemsReportingComplex queriesBulk operations

Legacy Systems

Several Apps -> One Source

Database-Level Abstraction• Views• Stored procedures• Triggers

Stored Procedures in JPA• 2.0 and before– Native queries to call stored procedures– No OUT/INOUT parameters– Database dependent CALL syntax

• 2.1– EntityManager.createStoredProcedureQu

ery–@NamedStoredProcedureQuery

Example: Stored ProcedureResult Set

-- MY SQLCREATE PROCEDURE GET_EMPLOYEES()BEGIN

SELECT *FROM EMPLOYEES;

END

Example: Stored ProcedureEntityManager API

// create and setup a stored procedure queryStoredProcedureQuery q = em.createStoredProcedureQuery(

"GET_EMPLOYEES", Employee.class);

// gather the results (an implicit call to execute)List<Employee> list = (List<Employee>) q.getResultList();

Example: Stored ProcedureOUT Parameter

-- MY SQLCREATE PROCEDURE SUM_SALARIES(

OUT TOTAL INT)BEGIN

SELECT SUM(SALARY) INTO TOTALFROM EMPLOYEES;

END



"SUM_SALARIES");q.registerStoredProcedureParameter(

"TOTAL", Integer.class, ParameterMode.OUT);

// execute the query...q.execute();

// ...to obtain the output valueInteger total = (Integer) q.getOutputParameterValue("TOTAL");

Example: Stored ProcedureAll in One

-- MY SQLCREATE PROCEDURE GET_EMPLOYEES(

IN GIVEN_COUNTRY VARCHAR(255), OUT TOTAL INT

)BEGIN

SELECT SUM(SALARY) INTO TOTALFROM EMPLOYEESWHERE COUNTRY = GIVEN_COUNTRY; SELECT *FROM EMPLOYEESWHERE COUNTRY = GIVEN_COUNTRY;

END



"GET_EMPLOYEES", Employee.class);q.registerStoredProcedureParameter(

"COUNTRY", String.class, ParameterMode.IN);q.registerStoredProcedureParameter(


// setup the parametersq.setParameter("COUNTRY", "Poland");

// obtain the employees...List<Employee> list = (List<Employee>) q.getResultList();

// ...and the output valueInteger total = (Integer) q.getOutputParameterValue("TOTAL");







// obtain the employeesList<Employee> list = (List<Employee>) q.getResultList();

// do we need execute here?Integer total = (Integer) q.getOutputParameterValue("TOTAL");







// first, an implicit call to executeList<Employee> list = (List<Employee>) q.getResultList();

// ...then, we can safely obtain the output value Integer total = (Integer) q.getOutputParameterValue("TOTAL");







// what if we reorder the lines?Integer total = (Integer) q.getOutputParameterValue("TOTAL");

// an implicit call to executeList<Employee> list = (List<Employee>) q.getResultList();







// the other way around it doesn’t work!Integer total = (Integer) q.getOutputParameterValue("TOTAL");

// an implicit call to executeList<Employee> list = (List<Employee>) q.getResultList();







// execute must be called before getOutputParameterValue// explicitely or implicitelyq.execute();Integer total = (Integer) q.getOutputParameterValue("TOTAL");

// an implicit call to execute List<Employee> list = (List<Employee>) q.getResultList();








// deos it call execute once more time?List<Employee> list = (List<Employee>) q.getResultList();








// an implicit call to execute only if not executed yet!List<Employee> list = (List<Employee>) q.getResultList();



"GET_EMPLOYEES", Employee.class);// what about the order here?q.registerStoredProcedureParameter(









"TOTAL", Integer.class, ParameterMode.OUT);q.registerStoredProcedureParameter(

"COUNTRY", String.class, ParameterMode.IN);







"TOTAL", Integer.class, ParameterMode.OUT);q.registerStoredProcedureParameter(

"COUNTRY", Integer.class, ParameterMode.IN);






"GET_EMPLOYEES", Employee.class);// what about the order of the positional parameters?q.registerStoredProcedureParameter(

2, Integer.class, ParameterMode.OUT);q.registerStoredProcedureParameter(

1, String.class, ParameterMode.IN);

// setup the parametersq.setParameter(1, "Poland");

// execute must be called before getOutputParameterValue// explicitely or implicitelyq.execute();Integer total = (Integer) q.getOutputParameterValue(2);




"GET_EMPLOYEES", Employee.class);// what about the order of the positional parameters?q.registerStoredProcedureParameter(

2, Integer.class, ParameterMode.OUT);q.registerStoredProcedureParameter(

1, String.class, ParameterMode.IN);




Example: Stored ProcedureAnnotation

@NamedStoredProcedureQuery{ name = "getEmployees", procedureName = "GET_EMPLOYEES", resultClasses = Employee.class, parameters = { @StoredProcedureParameter(name = "COUNTRY",

mode = ParameterMode.IN, type = String.class), @StoredProcedureParameter(name = "TOTAL",

mode = ParameterMode.OUT, type = Integer.class), }}@Entitypublic class Employee {...}


// create and setup a stored procedure queryStoredProcedureQuery q = em.createNamedStoredProcedureQuery(

”getEmployees");

// setup the parametersq.setParameter(”COUNTRY", "Poland");

// first, an implicit call to executeList<Employee> list = (List<Employee>) q.getResultList();

// ...then, we can safely obtain the output value Integer total = (Integer) q.getOutputParameterValue("TOTAL");

Example: Stored Procedure-- PostgreSQLCREATE OR REPLACE FUNCTION GET_EMPLOYEES(

IN GIVEN_COUNTRY VARCHAR(255),OUT TOTAL INT

) RETURNS REFCURSOR AS

$BODY$DECLARE

EMPS REFCURSOR; BEGIN

OPEN EMPS FOR SELECT *FROM EMPLOYEEWHERE COUNTRY = GIVEN_COUNTRY;RETURN EMPS;

END;$BODY$LANGUAGE plpgsql




1, void.class, ParameterMode.REF_CURSOR);q.registerStoredProcedureParameter(

"COUNTRY", String.class, ParameterMode.IN);


// first, an implicit call to execute...List<Employee> list = (List<Employee>) q.getResultList();

Example: Stored ProcedureAnnotation

@NamedStoredProcedureQuery{ name = "getEmployees", procedureName = "GET_EMPLOYEES", resultClasses = Employee.class, parameters = { @StoredProcedureParameter(mode = ParameterMode.REFCUR, type = void.class), @StoredProcedureParameter(mode = ParameterMode.IN,

type = String.class)}@Entitypublic class Employee {...}


// create and setup a stored procedure queryStoredProcedureQuery q = em.createNamedStoredProcedureQuery(

”getEmployees”);


// obtain the resultList<Employee> list = (List<Employee>) q.getResultList();

Stored Procedures in JPA 2.1Wrap-up

• Annotation– @NamedStoredProcedureQuery

• EntityManager API– createStoredProcedureQuery– registerStoredProcedureParameter

• Use cases– Existing database– Abstraction on database level (e.g., for several

applications)• Still differences between databases!– Much smaller though

Reporting

Reporting Anti-Patterns• Direct usage of an object-oriented

domain model• Too much data loaded• Heavy processing on the Java side

Reporting Anti-PatternsExample

Reporting Anti-PatternsEmployee Entity

@Entitypublic class Employee { @Id @GeneratedValue private Long id; private String firstName; private String lastName; private BigDecimal salary; private BigDecimal bonus; @Temporal(TemporalType.DATE) private Date startDate; @Temporal(TemporalType.DATE) private Date endDate; @ManyToOne @JoinColumn(name = "manager_id") private Employee manager; @OneToOne @JoinColumn(name = "address_id") private Address address; private String country; @OneToMany(mappedBy = "owner") private Collection<Phone> phones; @ManyToMany(mappedBy = "employees”) private Collection<Project> projects; …}

Sum of Salaries By CountrySelect All (1)

TypedQuery<Employee> query = em.createQuery("SELECT e FROM Employee e", Employee.class);

List<Employee> list = query.getResultList();

// calculate sum of salaries by country// map: country->sumMap<String, BigDecimal> results = new HashMap<>();for (Employee e : list) { String country = e.getAddress().getCountry(); BigDecimal total = results.get(country); if (total == null) total = BigDecimal.ZERO; total = total.add(e.getSalary()); results.put(country, total);}

Sum of Salaries by CountrySelect Join Fetch (2)

TypedQuery<Employee> query = em.createQuery("SELECT e FROM Employee e

JOIN FETCH e.address", Employee.class);List<Employee> list = query.getResultList();

// calculate sum of salaries by country// map: country->sumMap<String, BigDecimal> results = new HashMap<>();for (Employee e : list) { String country = e.getAddress().getCountry(); BigDecimal total = results.get(country); if (total == null) total = BigDecimal.ZERO; total = total.add(e.getSalary()); results.put(country, total);}

Reporting Anti-PatternsProjection (3)

Query query = em.createQuery("SELECT e.salary, e.address.country

FROM Employee e”);List<Object[]> list = query.getResultList();

// calculate sum of salaries by country// map: country->sumMap<String, BigDecimal> results = new HashMap<>();for (Object[] e : list) { String country = (String) e[1]; BigDecimal total = results.get(country); if (total == null) total = BigDecimal.ZERO; total = total.add((BigDecimal) e[0]); results.put(country, total);}

Reporting Anti-PatternsAggregation JPQL (4)

Query query = em.createQuery("SELECT SUM(e.salary), e.address.country

FROM Employee e GROUP BY e.address.country”);List<Object[]> list = query.getResultList();// already calculated!

Reporting Anti-PatternsAggregation SQL (5)

Query query = em.createNativeQuery("SELECT SUM(e.salary), a.country

FROM employee e JOIN address a ON e.address_id = a.id GROUP BY a.country");List list = query.getResultList();// already calculated!

Comparison 1-4100 000 employees, EclipseLink

MySQL PostgreSQLSelect all (1+N) (1) 25704ms 18120msSelect join fetch (2) 6211ms 3954msProjection (3) 533ms 569msAggregation JPQL (4) 410ms 380msAggregation SQL (5) 380ms 409ms

ProjectionJPQL -> Value Object

Query query = em.createQuery("SELECT new com.yonita.jpa.vo.EmployeeVO(

e.salary, e.address.country) FROM Employee e”);// List<EmployeeVO>List list = query.getResultList();

ProjectionJPQL -> Value Object

Query query = em.createQuery( "SELECT new com.yonita.jpa.CountryStatVO(

sum(e.salary), e.address.country) FROM Employee e GROUP BY e.address.country"”);// List<CountryStatVO>List list = query.getResultList();

ProjectionSQL -> Value Object

@SqlResultSetMapping( name = "countryStatVO", classes = { @ConstructorResult( targetClass = CountryStatVO.class, columns = { @ColumnResult( name = "ssum", type = BigDecimal.class), @ColumnResult( name = "country", type = String.class) }) })

ProjectionSQL -> Value Object

Query query = em.createNativeQuery("SELECT SUM(e.salary), a.country

FROM employee e JOIN address a ON e.address_id = a.id GROUP BY a.country", "countryStatVO");// List<CountryStatVO>List list = query.getResultList();

Projection Wrap-up

• JPA 2.0 – Only JPQL query to directly produce a value object!

• JPA 2.1– JPQL and native queries to directly produce a value object!

• Managed object– Sync with database– L1/L2 cache

• Use cases for Direct Value Object– Reporting, statistics, history– Read-only data, GUI data– Performance:

• No need for managed objects• Rich (or fat) managed objects• Subset of attributes required• Gain speed• Offload an app server

AggregationWrap-up

• JPA 2.0 – Selected aggregation functions: COUNT, SUM, AVG, MIN, MAX

• JPA 2.1– All function as supported by a database– Call any database function with new FUNCTION keyword

• Database-specific aggregate functions– MS SQL: STDEV, STDEVP, VAR, VARP,…– MySQL: BIT_AND, BIT_OR, BIT_XOR,…– Oracle: MEDIAN, PERCENTILE,…– More…

• Use cases– Reporting, statistics– Performance

• Gain speed• Offload an app server to a database!

Complex Queries

JPQL vs. SQL• Column/table visibility

– JPA 2.0/2.1: Only mapped columns and tables• Operations on sets

– UNION, INTERSECT, EXCEPT– JPA 2.0/2.1: No support

• Nested fetch joins– JPA 2.0: No support– JPA 2.1: entity graphs for different fetching strategies

• ON– JPA 2.0: No support– JPA 2.1: Only on “connected entities”

JPQL vs. SQL• Database functions– Aggregate functions– Conversion functions– Extraction functions–Manipulation functions– Functions, functions, functions…– JPA 2.0: Selected functions– JPA 2.1: FUNCTION keyword

Helper Functions JPA 2.1• String functions:

– CONCAT, SUBSTRING, TRIM, LOWER, UPPER, LENGTH, LOCATE

• Arithmetic functions: – ABS, SQRT, MOD, SIZE, INDEX

• Datatime functions: – CURRENT_DATE, CURRENT_TIME, CURRENT_TIMESTAMP

• Aggregate functions:– COUNT, SUM, MIN, MAX, AVG

• Invocation of predefined or user-defined functions: – FUNCTION(function_name {, function_args}*)

Helper Functions JPA 2.1• Invocation of predefined or user-

defined functions: – FUNCTION(function_name {,

function_args}*)– SELECT c

FROM Customer cWHERE FUNCTION(‘hasGoodCredit’, c.balance

c.creditLimit)

JPA 2.1

Wrap-up• JPA 2.1– Stored procedures support– Projections

• Direct value object for native queries– Richer JPQL

• Performance– Don’t load if you don’t need– Don’t execute many small queries if you can

execute one big query– Don’t calculate if a database can

Continuous Learning Paradigm• A fool with a tool is still a fool• Let’s educate ourselves!

Q&A

[email protected]@yonlabs