advanced and complex sql coding - gateway/400...1 iseries application development advanced and...
TRANSCRIPT
1
iSeries
Application Development
Advanced and
Complex SQL Coding
Sheryl M. Larsen
Sheryl M. Larsen, Inc.
May 12, 2011
Platform:
Gateway 400 User Group, St. Louis, MO
2 © Sheryl M. Larsen, Inc. 2000-2009
Contact: [email protected] (630) 399-3330 WWW.SMLSQL.COM
Sheryl M. Larsen, Inc. DB2 SQL Consulting & Education
Sheryl has 25 years experience
in DB2, has published articles,
white papers, webtv, animated
access paths:
Google Sheryl Larsen @
Bmc.com Ca.com Ibm.com
Sheryl Larsen is an internationally
recognized researcher, consultant
and lecturer, specializing in DB2
and is known for her extensive
expertise in SQL. She co-authored
a book, DB2 Answers, Osborne-
McGraw-Hill, 1999.
She was voted an inaugural IBM
Information Champion 2009 ,
IDUG Speaker Hall of Fame and, a
member of IBM’s DB2 Gold
Consultants program since 1994.
Currently, she is President of the
Midwest Database Users Group
(mwdug.org), and owns Sheryl M.
Larsen, Inc., a firm specializing in
Advanced DB2 Consulting and
Education.
3 © Sheryl M. Larsen, Inc. 2000-2011
Joins
Subqueries
Outer Joins
Common Table Expressions
Materialized Query Tables
EXCEPT
INTERCEPT
Browsing Result Sets
Table of Contents
SQL Skill Self Assessment Level Assessment = YOU CAN FULLY UNDERSTAND THE FEATURE AND PROPER USE OF:
0 You think SQL is a new energy drink
1 Simple SELECT statements, WITH clause, ORDER BY
2
WHERE clauses, BETWEEN, LIKE, IN(list), =, >=, >, <, <=, <>, NOT IN(list),
NOT LIKE, NOT BETWEEN, DISTINCT
3
Table joins (inner, outer, full), UNION, UNION ALL, CONCAT, static CURSORs,
FOR UPDATE OF, ROW_NUMBER, EXCEPT, INTERCEPT
4
noncorrelated and correlated subqueries, EXISTS, NOT EXISTS, FETCH
FIRST x ROWS ONLY, OPTIMIZE FOR x ROWS, MERGE, TRUNCATE
5
Indexable, Stage1 and Stage 2 predicate evaluation, multirow
FETCH/INSERT, GET DIAGNOSTICS, Scalar full SELECT
6
Table expressions/ common table expressions, GROUP BY, HAVING, IS NOT
DISTINCT FROM, embedded dynamic SQL, ORDER BY/FETCH FIRST in
subselect, FETCH CONTINUE, EXCEPT/INTERCEPT
7
CASE expressions, Global Temporary Table (GTT), Declared Temporary
Table (DTT), Dynamic Scrollable cursors, SEQUENCES/IDENTITY
8
Queries involving > 10 tables, INSERT within (SELECT , UPDATE, DELETE,
MERGE), Star Schema, GROUP BY expression, Browsing result sets
9
MQT (Materialized Query Tables), Recursive SQL, UNION in Views,
> 30 useful Built-in Functions, DENSE_RANK, RANK
10
Codes effective and efficient SQL applying performance rules and knows
when to use each SQL feature appropriately
Name: __________________
Before:_______
After:________
5 © Sheryl M. Larsen, Inc. 2000-2008
225 Tables in SQL Statement Limit
225 directly or indirectly identifying base table references:
SELECT …….. FROM T1, T2, ….T200 WHERE C1 IN
(SELECT ….. FROM T300 more tables
WHERE )
UNION ALL
(SELECT …… FROM T400 more tables
WHERE)
ORDER BY
FETCH FIRST 22 ROWS ONLY
WITH UR;
Full
SE
LE
CT
Subselect
Subquery
6 © Sheryl M. Larsen, Inc. 2000-2008
Inner Join
SELECT T42.PLCY_ACCT_NUM
,T52.SERV_ASGN_START_DT
FROM CLIENT T42
,AGENT T52
WHERE T42.PLCY_ID = T52.PLCY_ID
AND T42.LOB = ‘S’
AND T42.CLNT_PLCY_END_DT IS NULL
AND T52.SERV_ASGN_END_DT IS NULL
•List Most Restrictive Condition first within predicate type
•= , IN
•BETWEEN, >=, >,<,<=
•LIKE,
•Noncorrelated Subqueries
•Correlated Subqueries
•Produce a list of the accounts numbers, current servicing agent start date
for all active Insurance Payment Plan accounts.
7 © Sheryl M. Larsen, Inc. 2000-2008
Self Join Needed for Intra-Table Column Comparisons
» WHERE A.COL2 = A.COL5
» On different rows (accounts, policies, items, orders,
other unique ids, etc.)
» What Students Have the Same Classes?
SELECT SSS.SID, SSS.CLASS_ID, SSS.TIME_ID FROM SMLU_STUDENT_SCHED SS ,SMLU_STUDENT_SCHED SSS WHERE SSS.CLASS_ID = SS.CLASS_ID AND SSS.TIME_ID = SS.TIME_ID AND SSS.SID <> SS.SID AND SSS.SID NOT IN(list challenged student IDs) AND SS.SID = :challenged-sid
Self Join to get same
class/time rows Point to
Different
Rows
Homework
Challenged
8 © Sheryl M. Larsen, Inc. 2000-2008
Subqueries over Joins
Needed for Intra-Table Column Comparisons
» What Students Have at least one of the same Classes
as a given student?
SELECT SS.SID FROM SMLU_STUDENT_SCHED SS WHERE SS.SID NOT IN(list challenged student IDs)
AND EXISTS
(SELECT ‘TRUE OR FALSE’ FROM SMLU_STUDENT_SCHED SSS WHERE SSS.CLASS_ID = SS.CLASS_ID AND SSS.TIME_ID = SS.TIME_ID AND SSS.SID <> SS.SID AND SSS.SID = :challenged-sid)
Homework
Challenged
9 © Sheryl M. Larsen, Inc. 2000-2008
Left/Right Outer Joins
Get all of Client ids for single males
» Regardless if there are any email addresses
SELECT T40.CLNT_ID,
COALESCE(T71.EML_ADDR, 'NONE AVAIL')
FROM CLIENT T40
LEFT OUTER JOIN
EMAIL T71
ON T40.CLNT_ID = T71.CLNT_ID
WHERE T40.MRTL_STTS = 'S'
AND T40.GNDR = 'M'
Client
T40
T71
SELECT Columns
FROM TABX X
LEFT JOIN TABY Y
ON X.COL1 = Y.COL1
10 © Sheryl M. Larsen, Inc. 2000-2008
Inner Join versus Left Join
Inner Join
TABX
Exceptions
TABY
Exceptions
SELECT Columns
FROM TABX X, TABY Y
WHERE X.COL1 = Y.COL1
Nested Common Table Expressions WITH COMMON1 (column list) AS
(SELECT PART, SUBPART, 1
FROM LEGO
WHERE PART = ‘L1’)
, COMMON2 (column list) AS
SELECT L.PART, L.SUBPART, N+1
FROM COMMON1 T1, LEGO L
WHERE T1.SUBPART = L.PART
AND N < 10 )
, COMMON3 (column list) AS
SELECT L.PART, L.SUBPART, N+1
FROM COMMON2 T1, BUILDING_BLOCKS L
WHERE T1.SUBPART = L.PART
AND N < 10 )
SELECT * FROM COMMON1
, TODDLERS2
WHERE …………..;
11 © Sheryl M. Larsen, Inc. 2000-2008
12 © Sheryl M. Larsen, Inc. 2000-2008
Materialized Query Tables Automatic Summary Tables (ASTs) are a subset
of MQTs
Can be ANY valid SELECT statement
CREATE TABLE MQT1 AS
SELECT dozens of aggregates
seven table join
nine column GROUP BY…
Or use your imagination .. ;
13 © Sheryl M. Larsen, Inc. 2000-2008
MQTs
Use them as “multi-table” indexes with or without aggregates
Optimizer is aware of all MQTs (for dynamically bound only for z/OS!) or direct reference for static
Auto query rewrite of base table queries to use MQTs if:
» CURRENT REFRESH AGE = „ANY‟
» CURRENT MAINTINED TABLE TYPES = „SYSTEM‟, „USER‟, „ALL‟
» ENABLE QUERY OPTIMIZATION is used
» Referential constraints between base tables exists
Matters more than you think…
14 © Sheryl M. Larsen, Inc. 2000-2008
…. To Build or not to Build
Generic MQT strategies
»HUGE MQT with many indexes
»Many small MQTs with few indexes
Optimal MQT strategy
»Most popular queries as MQTs • Common tables
• Common predicate sets
• Common data translations
»Indexes on the popular ones to allow access to subset of MQT
15 © Sheryl M. Larsen, Inc. 2000-2008
Popular Table Pivoting Syntax SELECT
SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END) AS JAN_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END) AS FEB_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END) AS MARCH_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END) AS APRIL_AMT , SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END) AS MAY_AMT FROM SALES WHERE CUST_ID = :hv
JAN_AMT FEB_AMT MARCH_AMT APRIL_AMT MAY_AMT
238,50. 392.39 8,029.39 2,380.83 3,838.22
Generated
at
Execution
16 © Sheryl M. Larsen, Inc. 2000-2008
MQT
MQT INDEX:
CUST_ID
CREATE TABLE CUST_PIVOT AS (
SELECT CUST_ID
, SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END)
AS JAN_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END)
AS FEB_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END)
AS MARCH_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END)
AS APRIL_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END)
AS MAY_AMT
FROM SALES
GROUP BY CUST_ID)
DATA INITIALLY DEFERRED
REFRESH DEFERRED
MAINTAINED BY SYSTEM
ENABLE QUERY OPTMIZATION;
17 © Sheryl M. Larsen, Inc. 2000-2008
Query Speed Up
CUST_ID JAN_AMT FEB_AMT MARCH_AMT APRIL_AMT MAY_AMT
11 238,50. 342.39 1,029.39 5,380.83 3,438.22
12 298,50. 352.30 729.39 2,580.22 9,838.96
13 738,50. 362.32 8,769.39 7,360.33 538.48
14 1,038,50. 372.31 2,029.39 2,387.55 3,825.00
SELECT CUST_ID
, SUM(CASE WHEN MONTH(SALES_DT) = 1 THEN AMOUNT END) AS JAN_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 2 THEN AMOUNT END)AS FEB_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 3 THEN AMOUNT END)AS MARCH_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 4 THEN AMOUNT END)AS APRIL_AMT
, SUM(CASE WHEN MONTH(SALES_DT) = 5 THEN AMOUNT END)AS MAY_AMT
FROM SALES
WHERE CUST_ID = 12
Accessed
at
Execution SELECT * FROM CUST_PIVOT
WHERE CUST_ID = 12
CUST_PIVOT
18 © Sheryl M. Larsen, Inc. 2000-2008
EXCEPT, INTERSECT NAME
Able
Able
Baker
Baker
R1
NAME
Able
Able
Charlie
Charlie
R2
SELECT
……
……
……
AS R1
UNION
SELECT
…….
…….
AS R2
NAME
Able
Baker
Charlie
SELECT
……
……
……
AS R1
UNION
ALL
SELECT
…….
…….
AS R2
NAME
Able
Able
Baker
Baker
Able
Able
Charlie
Charlie
SELECT
……
……
……
AS R1
EXCEPT
SELECT
…….
…….
AS R2
NAME
Baker
SELECT
……
……
……
AS R1
EXCEPT
ALL
SELECT
…….
…….
AS R2
NAME
Baker
Baker
SELECT
……
……
……
AS R1
INTERSECT
SELECT
…….
…….
AS R2
NAME
Able
SELECT
……
……
……
AS R1
INTERSECT
ALL
SELECT
…….
…….
AS R2
NAME
Able
Able
Able
Able
Baker Baker
Able Able
Able Able
Charlie Charlie
Find Things that Have ALL
List the clients that have at least one active policy in ALL lines of
business offered by Insurance Company.
SELECT DISTINCT PR1.CLNT_ID
FROM POLICY_ROLE PR1
WHERE NOT EXISTS (SELECT DISTINCT P1. LOB
FROM POLICY P1 )
EXCEPT
(SELECT DISTINCT PR2.LOB
FROM POLICY_ROLE PR2
WHERE
PR2.CLNT_ID = PR1.CLNT_ID
AND PR2.CLNT_PLCY_END_DT IS NULL )
19 © Sheryl M. Larsen, Inc. 2000-2009
A
F
H
L
- =
A
F
H
L
H
A
F
L
All
701 801
A
F
H
L
All
- =
False
True
EXCEPT Example If the employee works on every project located in Denver, then
list the employee‟s social security number and name.
SELECT NAME, SSN
FROM EMPLOYEE E
WHERE NOT EXISTS
((SELECT PROJECT.PNUMBER
FROM PROJECT
WHERE PLOCATION = „DENVER')
EXCEPT
(SELECT W.PNUMBER
FROM WORKSON
WHERE E.SSN = W.SSN));
20 © Sheryl M. Larsen, Inc. 2000-2009
P11
P26
P84
P99
All
Test one SSN at a time
If their list matches
They win!
21
z/OS
Application Development
Repositioning
Cursors
Sheryl M. Larsen
Sheryl M. Larsen, Inc.
Platform:
22 © Sheryl M. Larsen, Inc. 2000-2009
EMP Table and Cursor EMPNO EMP_NAME EMP_ADDR EMP_PHONE
1 JOE DOWNERS GROVE 5552833820
2 SAM ELMHURST 5553824857
4 JIM AURORA 5559382737
6 KELLY LIBERTYVILLE 5551293874
8 MIKE BARRINGTON 5551267480
DELCLARE EMPCSR CURSOR FOR SELECT EMP_NAME, EMP_ADDR, EMP_PHONE FROM EMP WHERE EMPNO > :lastempno ORDER BY EMPNO FETCH FIRST 2 ROWS ONLY;
© Sheryl M. Larsen, Inc. 2000-2009
23
Get First 2 Rows EMPNO EMP_NAME EMP_ADDR EMP_PHONE
1 JOE DOWNERS GROVE 5552833820
2 SAM ELMHURST 5553824857
4 JIM AURORA 5559382737
6 KELLY LIBERTYVILLE 5551293874
8 MIKE BARRINGTON 5551267480
SET :lastempno = 0 , :page0 in COM AREA
OPEN EMPCSR;
FETCH EMPCSR FOR 2 ROWS;
Store :lastempno = :last-fetched-empno in COM AREA and :page1
Display screen
24 © Sheryl M. Larsen, Inc. 2000-2009
Get Next 2 Rows EMPNO EMP_NAME EMP_ADDR EMP_PHONE
1 JOE DOWNERS GROVE 5552833820
2 SAM ELMHURST 5553824857
4 JIM AURORA 5559382737
6 KELLY LIBERTYVILLE 5551293874
8 MIKE BARRINGTON 5551267480
Get :lastempno from COM AREA (=2 for page1)
OPEN EMPCSR;
FETCH EMPCSR FOR 2 ROWS;
Store :lastempno = :last-fetched-empno in COM AREA and :page2
25 © Sheryl M. Larsen, Inc. 2000-2009
Go Back 1 Page EMPNO EMP_NAME EMP_ADDR EMP_PHONE
1 JOE DOWNERS GROVE 5552833820
2 SAM ELMHURST 5553824857
4 JIM AURORA 5559382737
6 KELLY LIBERTYVILLE 5551293874
8 MIKE BARRINGTON 5551267480
Get :lastempno from COM AREA for Page 0 (=0)
OPEN EMPCSR;
FETCH EMPCSR;
Store :firstempno and :page2 in COM AREA
FETCH EMPCSR;
Store :lastempno = :last-fetched-empno in COM AREA
Always go an extra page back so the > works
26 © Sheryl M. Larsen, Inc. 2000-2009
BRWSUM Table
COL1 COL2 COL3 COL4 COL5
A 1 TT 99 Data ….
A 1 UU 77 Data ….
A 4 SS 66 Data ….
B 2 RR 66 Data ….
B 3 RR 77 Data ….
B 3 RR 88 Data ….
B 3 SS 66 Data ….
B 4 SS 99 Data ….
B 4 UU 88 Data ….
C 1 SS 66 Data ….
C 1 SS 77 Data ….
Makes row unique, should have index COL1.COL2.COL3.COL4
27 © Sheryl M. Larsen, Inc. 2000-2009
Declare Cursor
COL1 COL2 COL3 COL4 COL5
A 1 TT 99 Data ….
A 1 UU 77 Data ….
A 4 SS 66 Data ….
B 2 RR 66 Data ….
B 3 RR 77 Data ….
B 3 RR 88 Data ….
B 3 SS 66 Data ….
B 4 SS 99 Data ….
B 4 UU 88 Data ….
C 1 SS 66 Data ….
C 1 SS 77 Data ….
DELCLARE BRWSUM1 CURSOR FOR
SELECT COL5 FROM BRWSUM
WHERE ? AND ? AND ? AND ? AND ? AND ? AND ?
ORDER BY COL1, COL2, COL3, COL4
FETCH FIRST 5 ROWS ONLY;
What should these be?
28 © Sheryl M. Larsen, Inc. 2000-2009
Declare BRWSUM1Cursor
COL1 COL2 COL3 COL4 COL5
A 1 TT 99 Data ….
A 1 UU 77 Data ….
A 4 SS 66 Data ….
B 2 RR 66 Data ….
B 3 RR 77 Data ….
B 3 RR 88 Data ….
B 3 SS 66 Data ….
B 4 SS 99 Data ….
B 4 UU 88 Data ….
C 1 SS 66 Data ….
C 1 SS 77 Data ….
DELCLARE BRWSUM1 CURSOR FOR SELECT COL5
FROM BRWSUM WHERE ((COL1 = :col1-last AND COL2 = :col2-last AND COL3 = :col3-last AND COL4 > :col4-last) OR (COL1 = :col1-last AND COL2 = :col2-last AND COL3 > :col3-last) OR (COL1 = :col1-last AND COL2 > :col2-last) OR (COL1 > :col1-last))
ORDER BY COL1, COL2, COL3, COL4 FETCH FIRST 5 ROWS ONLY;
29 © Sheryl M. Larsen, Inc. 2000-2009
Get First 5 Rows
OPEN BRWSUM1;
FETCH BRWSUM1 FOR 5 ROWS;
Store :col1-4-last = :last-fetched-row in COM AREA and :page1
Display screen
COL1 COL2 COL3 COL4 COL5
A 1 TT 99 Data ….
A 1 UU 77 Data ….
A 4 SS 66 Data ….
B 2 RR 66 Data ….
B 3 RR 77 Data ….
B 3 RR 88 Data ….
B 3 SS 66 Data ….
B 4 SS 99 Data ….
B 4 UU 88 Data ….
C 1 SS 66 Data ….
C 1 SS 77 Data ….
SET
:col1-last = low values, :col2-last = 0, :col3-last = low values,:col4-last = 0,
:page0 in COM AREA
30 © Sheryl M. Larsen, Inc. 2000-2009
Get Next 5 Rows
OPEN BRWSUM1;
( (COL1 = ‘B’ AND COL2 = 3 AND COL3 = ‘RR’ AND COL4 > 77) OR (COL1 = ‘B’ AND COL2 = 3 AND COL3 > ‘RR’) OR
(COL1 = ‘B’ AND COL2 > 3) OR (COL1 > ‘B’))
FETCH BRWSUM1 FOR 5 ROWS;
Store :col1-4-last = :last-fetched-row in COM AREA and :page2
Display screen
COL1 COL2 COL3 COL4 COL5
A 1 TT 99 Data ….
A 1 UU 77 Data ….
A 4 SS 66 Data ….
B 2 RR 66 Data ….
B 3 RR 77 Data ….
B 3 RR 88 Data ….
B 3 SS 66 Data ….
B 4 SS 99 Data ….
B 4 UU 88 Data ….
C 1 SS 66 Data ….
C 1 SS 77 Data ….
Get from COM AREA
:col1-last = „B‟, :col2-last = 3, :col3-last = „RR‟,:col4-last =77
First Screen
31 © Sheryl M. Larsen, Inc. 2000-2009
Go Back 1 Page
OPEN BRWSUM1;
( (COL1 = low AND COL2 = 0 AND COL3 = low AND COL4 > 0) OR (COL1 = low AND COL2 = 0 AND COL3 > low) OR (COL1 = low AND COL2 > 0) OR (COL1 > 0))
FETCH BRWSUM1;
Loop 5 times
Store :col1-4-last = :last-fetched-row in COM AREA and :page1
Display screen
COL1 COL2 COL3 COL4 COL5
A 1 TT 99 Data ….
A 1 UU 77 Data ….
A 4 SS 66 Data ….
B 2 RR 66 Data ….
B 3 RR 77 Data ….
B 3 RR 88 Data ….
B 3 SS 66 Data ….
B 4 SS 99 Data ….
B 4 UU 88 Data ….
C 1 SS 66 Data ….
C 1 SS 77 Data ….
Get from COM AREA for page0
:col1-last = low values :col2-last = 0, :col3-last = low values,:col4-last =0
Last Screen
32
z/OS
Application Development
Thank You!
Sheryl M. Larsen
Sheryl M. Larsen, Inc.
www.smlsql.com
Platform:
iSeries Information Center, Version 5 Release 3
http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp