my jdbc performance sucks and what i did about it!
Post on 23-Feb-2016
67 Views
Preview:
DESCRIPTION
TRANSCRIPT
©2010 PROS Holdings, Inc. All rights reserved.
My JDBC Performance Sucks and What I Did About It!
©2010 PROS Holdings, Inc. All rights reserved. 2
Who am I? Peter Tran Senior Performance Technical Lead Over 12 years with PROS 8 years working on server-side performance 6 years Oracle tuning 3 years SQL Server tuning ptran@prospricing.com
©2010 PROS Holdings, Inc. All rights reserved. 3
Topics Assumptions Transaction Isolation Levels Connection Batching Queries Parsing Misc
©2010 PROS Holdings, Inc. All rights reserved. 4
Assumptions Will not compare different driver type
- Recommendations should be driver type agnostic Examples in presentation uses Type IV Thin Driver Examples uses SQL Server 2008 and Oracle 10g
databases Will not cover SQL optimization Will not cover JDBC resource leak
©2010 PROS Holdings, Inc. All rights reserved. 5
Transaction Isolation Levels Big impact on performance Critical for correct business logic java.sql.Connection
Transaction Isolation Level Dirty-Reads Non-Repeatable Phantom Performance
TRANSACTION_NONEN/A N/A N/A Fastest
TRANSACTION_READ_UNCOMMITEDYes Yes Yes Fast
TRANSACTION_READ_COMMITTEDNo Yes Yes Medium
TRANSACTON_REPEATABLE_READNo No Yes Slow
TRANSACTION_SERIALIZABLENo No No Slowest
©2010 PROS Holdings, Inc. All rights reserved. 6
Connection java.sql.Connection Use connection pooling
- Big performance penalty to create new physical connection to database
1 10000
1
2
3
4
5
6
7
Time to Create Physical Connection
# of Physical Database Connections
Seco
nds
©2010 PROS Holdings, Inc. All rights reserved. 7
Batching Batch UPDATE and INSERT SQLs Use
- java.sql.(Prepared/Callable)Statement- Added benefit of preventing SQL Injection (security)
Do not use- java.sql.Statement
Remember to setAutoCommit to FALSE SQL statement reuse Reduces parse
©2010 PROS Holdings, Inc. All rights reserved. 8
Batch Insert Example – Wrong! Wrong way to do batch update.//turn off autocommit
con.setAutoCommit(false);
Statement stmt = con.createStatement();
stmt.addBatch("INSERT INTO employees VALUES (1000, 'Joe Jones')");
stmt.addBatch("INSERT INTO departments VALUES (260, 'Shoe')");
stmt.addBatch("INSERT INTO emp_dept VALUES (1000, 260)");
// submit a batch of update commands for execution
int[] updateCounts = stmt.executeBatch();
©2010 PROS Holdings, Inc. All rights reserved. 9
Batch Insert Example Right way to do batch update using PreparedStatement// turn off autocommit
con.setAutoCommit(false);
PreparedStatement stmt = con.prepareStatement( "INSERT INTO employees VALUES (?, ?)");
stmt.setInt(1, 2000);
stmt.setString(2, "Kelly Kaufmann");
stmt.addBatch();
stmt.setInt(1, 3000);
stmt.setString(2, "Bill Barnes");
stmt.addBatch();
// submit the batch for execution
int[] updateCounts = stmt.executeBatch();
©2010 PROS Holdings, Inc. All rights reserved. 10
Batch Query Example Right way to do batch query using CallableStatement
// turn off autocommit
con.setAutoCommit(false);
CallableStatement stmt = con.prepareCall("sp_GetEmployees (?, ?)");
stmt.setInt(1, 2000);
stmt.setString(2, "Kelly Kaufmann");
ResultSet rs = stmt.executeQuery()
©2010 PROS Holdings, Inc. All rights reserved. 11
Batch Insert Test Results
Test 1: Statement – No Batch Test 2: PreparedStatement – No batch/no reuse Test 3: PreparedStatement – No batch/reuse Test 4: PreparedStatement – With batch/reuse
1 2 3 40
1
2
3
4
5
6
7
Batch Tests
Test #
Seco
nds
©2010 PROS Holdings, Inc. All rights reserved. 12
Batch Pitfalls Forgetting to set autocommit to FALSE Excessively large batch size
- Writer blocks readers- Use up more memory on database server- ORA-01555 snapshot too old
©2010 PROS Holdings, Inc. All rights reserved. 13
Query Performance Set fetch size
- Settable java.sql.ResultSet java.sql.Statement java.sql.(Prepared/Callable)Statement
- Oracle 11.2g driver defaults to 10- SQL Server v3.0 driver defaults to 128- Reduces Physical/Logical I/O- Uses more memory in client to hold data
©2010 PROS Holdings, Inc. All rights reserved. 14
Query Performance Fetch only data that is needed
- SQL Optimization – Physical/Logical IOs- Reduce Network
Use appropriate getXXX() method for retrieving data- Minimize data conversion guessing- Eliminates nasty conversion bug
©2010 PROS Holdings, Inc. All rights reserved. 15
Microsoft Query Set Fetch Size Test Results
Test 1: Query 1M rows – fetch size = 10 Test 2: Query 1M rows – fetch size = 128 (default) Test 3: Query 1M rows – fetch size = 1M
1 2 30
0.05
0.1
0.15
0.2
0.25
0.3Microsoft Query Fetch Size Tests
Test #
Seco
nds
©2010 PROS Holdings, Inc. All rights reserved. 16
Oracle Query Set Fetch Size Test Results
Test 1: Query 1M rows – fetch size = 10 (default) Test 2: Query 1M rows – fetch size = 500 Test 3: Query 1M rows – fetch size = 1M
1 2 30
0.05
0.1
0.15
0.2
0.25
Oracle Query Fetch Size Tests
Test #
Seco
nds
©2010 PROS Holdings, Inc. All rights reserved. 17
Parsing Parsing is expensive!
- Scalability killer for high volume OLTP applications Not as big of concern for OLAP applications Consumes CPU intensive
- Database engine not doing real-work Increase latch contention Increase memory usage SQL statement cache is a singleton
- Oracle: Shared Pool- SQL Server: Plan Cache
©2010 PROS Holdings, Inc. All rights reserved. 18
Parsing Best place to fix is in the application code
- Use PreparedStatement or CallableStatement No Parse – Predicates uses bind variables and
Statement reused Soft Parse – Predicate uses bind variables but
Statement not reused Hard Parse – Predicate with literal values
Parse Type PerformanceParse Once BestSoft Parse GoodHard Parse Bad
©2010 PROS Holdings, Inc. All rights reserved. 19
Parse Fix - Hack Do not have access to application code May select a suboptimal query plan Buggy
©2010 PROS Holdings, Inc. All rights reserved. 20
Oracle Parsing Example Oracle 11.2g JDBC Driver
Test 1: Hard Parse (using Statement with literals) Test 2: Soft Parse Test 3: Parse Once
1 2 302468
1012
Parse Test
Test #
Seco
nds
©2010 PROS Holdings, Inc. All rights reserved. 21
Microsoft Parsing Example Microsoft JDBC Driver version 3.0
Test 1: Hard Parse (using Statement with literals) Test 2: Soft Parse Test 3: Parse Once
1 2 301234567
Parse Test
Test #
Seco
nds
©2010 PROS Holdings, Inc. All rights reserved. 22
jTDS Parsing Example for SQL Server jTDS JDBC Driver version 1.2.5
Test 1: Hard Parse (using Statement with literals) Test 2: Soft Parse Test 3: Parse Once
1 2 30
0.51
1.52
2.53
3.54
4.5
Parse Test
Test #
Seco
nds
©2010 PROS Holdings, Inc. All rights reserved. 23
Parse Fix - Hack SQL Server
- Use PARAMETERIZATION = FORCED- Set at database level (Database Properties -> Options)- Restrictive: Check Books-On-Line (BOL)- YMMV – 5% improvements
©2010 PROS Holdings, Inc. All rights reserved. 24
Parse Fix - Hack Oracle
- CURSOR_SHARING = SIMILAR | FORCED Instance level or session level Use trigger to hack session level for each login
CREATE OR REPLACE TRIGGER after_usr_logon
AFTER LOGON ON <USER>.SCHEMA
BEGIN EXECUTE IMMEDIATE 'ALTER SESSION SET CURSOR_SHARING=FORCE';
END;
Super buggy Use SIMILAR – mixed work-load OLTP/OLAP Use FORCE – pure OLTP Use EXACT (default) – DSS/DW OLAP
©2010 PROS Holdings, Inc. All rights reserved. 25
Driver Specific Improvements Requires casting to drive specific implementation Your Performance May Vary (YPMV) Microsoft JDBC Driver version 3.0
- Adaptive Buffer – reduces memory footprint in client to minimize Out of Memory but requires more round-trip to database (default=adaptive as of version 2.0)
Oracle 11.2g JDBC Driver- Statement and ResultSet Caching- Connection Caching
jTDS JDBC Driver version 1.2.5- Implicitly caches PreparedStatement – not a new
feature
©2010 PROS Holdings, Inc. All rights reserved. 26
Database JDBC Driver Tuning Parameters
SQL Server- Implicit Unicode conversion
Use setStringParametersAsUnicode Bypass index and use table scan
©2010 PROS Holdings, Inc. All rights reserved. 27
My JDBC Performance Workout Plan Maximize the usage of batching Set appropriate fetch size for queries Set the appropriate isolation level Reduce the hard/soft parses
Before After
©2010 PROS Holdings, Inc. All rights reserved. 28
Q&A
top related