reducing redo
Post on 30-Jan-2016
52 Views
Preview:
DESCRIPTION
TRANSCRIPT
1 © 2005 Julian Dyke
ReducingRedo
Julian Dyke
Independent Consultant
Web Version
juliandyke.com
2
© 2005 Julian Dykejuliandyke.com
Agenda
Introduction Tests
Indexes Number of columns processed SELECT FOR UPDATE Number of rows processed COMMIT Batch size Global temporary tables External tables
Conclusion
3
© 2005 Julian Dykejuliandyke.com
Redo Records
Redo Block Header
16 bytes
Redo Block 512 or
1024 bytes
Redo Block Body
496 bytes
Redo Record 1
Redo Record 2
Wastage
Redo Record 3
Wastage
Header
Body
Body
Header
Spare
Header
Body
Spare
STOP
4
© 2005 Julian Dykejuliandyke.com
Change Vectors
Redo Record
Header
Body
Header
Body
Header
Body
Header
Body
ChangeVectors
ChangeVector 3
ChangeVector 1
ChangeVector 2
STOP
5
© 2005 Julian Dykejuliandyke.com
Header
Change Vector
Header
Body
STOP
6
© 2005 Julian Dykejuliandyke.com
Example Examples in this presentation taken from Formula 1 database
Contains full details of all races from 1961 to 2004
Updated annually in November (end of season)
Currently 20 cars per race 19 races per season `Approximately 360 new rows per season
juliandyke.net
7
© 2005 Julian Dykejuliandyke.com
Schema
CLASSIFICATION
SEASON
GRANDPRIX
RACE TEAMDRIVER
COUNTRYCIRCUIT
ENGINE
CAR
8
© 2005 Julian Dykejuliandyke.com
Cars Each season has up to 18 races (19 in 2005)
Each race has up to 39 entrants (13 races in 1989)
Each car has driver, team and engine laps completed (may be zero) optional notes
Results are classified as follows
C Classified
DNF Did not finish
DNS Did not start
DNQ Did not qualify
DIS Disqualified
9
© 2005 Julian Dykejuliandyke.com
Points Points basically awarded to driver and team as follows
1st 2nd 3rd 4th 5th 6th 7th 8th
Pre 1991 9 6 4 3 2 1
1991-2002 10 6 4 3 2 1
2003 onwards 10 8 6 5 4 3 2 1
But . . . not always straightforward Half points awarded for incomplete races Split races (two half point races aggregated) Driver and / or team disqualifications e.g. Tyrrell in 1984 Up to 1980 only best scores counted for each half of
season e.g. Best 5 results from first 7 races and best 5 results from
last 7 races 1982-1990 only best 11 results counted for drivers 1961-1978 only first car to finish counted for each team
10
© 2005 Julian Dykejuliandyke.com
Input file - car.csv Comma separated file 16181 rows Fields are:
season_key race_key position driver_key team_key engine_key laps_completed classification_key notes (optional)
2004 17 17 ZBAU MIN FOR 41 DNF Spin2004 17 18 DCOU MCL MER 38 DNF Accident2004 17 19 RBAR FER FER 38 DNF Accident2004 17 20 MWEB JAG FOR 20 DNF Overheated2004 18 1 JMON WIL BMW 71 C2004 18 2 KRAI MCL MER 71 C2004 18 3 RBAR FER FER 71 C2004 18 4 FALO REN REN 71 C2004 18 5 RSCH WIL BMW 71 C2004 18 6 TSAT BAR HON 71 C2004 18 7 MSCH FER FER 71 C2004 18 8 FMAS SAU FER 71 C2004 18 9 GFIS SAU FER 71 C2004 18 10 JVIL REN REN 70 C2004 18 11 DCOU MCL MER 70 C2004 18 12 JTRU TOY TOY 70 C2004 18 13 RZON TOY TOY 70 C2004 18 14 CKLI JAG FOR 69 C2004 18 15 TGLO JOR FOR 69 C2004 18 16 ZBAU MIN FOR 67 C2004 18 17 GBRU MIN FOR 67 C2004 18 18 MWEB JAG FOR 23 DNF Accident2004 18 19 NHEI JOR FOR 15 DNF Clutch2004 18 20 JBUT BAR HON 3 DNF Engine
11
© 2005 Julian Dykejuliandyke.com
Input file - points.csv Comma separated file 16181 rows Fields are:
season_key race_key position driver_points team_point
2004 17 17 0 02004 17 18 0 02004 17 19 0 02004 17 20 0 02004 18 1 10 102004 18 2 8 82004 18 3 6 62004 18 4 5 52004 18 5 4 42004 18 6 3 32004 18 7 2 22004 18 8 1 12004 18 9 0 02004 18 10 0 02004 18 11 0 02004 18 12 0 02004 18 13 0 02004 18 14 0 02004 18 15 0 02004 18 16 0 02004 18 17 0 02004 18 18 0 02004 18 19 0 02004 18 20 0 0
12
© 2005 Julian Dykejuliandyke.com
CAR table CAR table and index definitions
CREATE TABLE car(
season_key NUMBER NOT NULL,race_key NUMBER NOT NULL,position NUMBER NOT NULL,driver_key VARCHAR2(4) NOT NULL,team_key VARCHAR2(3) NOT NULL,engine_key VARCHAR2(3) NOT NULL,laps_completed NUMBER NOT NULL,classification_key VARCHAR2(4) NOT NULL,notes VARCHAR2(100),driver_points NUMBER NOT NULL DEFAULT 0,team_points NUMBER NOT NULL DEFAULT 0
);
ALTER TABLE car ADD CONSTRAINT car_pkPRIMARY KEY (season_key,race_key,position);
CREATE INDEX car_driver ON car (season_key,driver_key,driver_points);
13
© 2005 Julian Dykejuliandyke.com
CAR CAR table relational integrity definitions
ALTER TABLE car ADD CONSTRAINT car_raceFOREIGN KEY (season_key,race_key)REFERENCES race (season_key,race_key);
ALTER TABLE car ADD CONSTRAINT car_driverFOREIGN KEY (driver_key)REFERENCES driver (driver_key);
ALTER TABLE car ADD CONSTRAINT car_teamFOREIGN KEY (team_key)REFERENCES team (team_key);
ALTER TABLE car ADD CONSTRAINT car_engineFOREIGN KEY (engine_key)REFERENCES engine (engine_key);
ALTER TABLE car ADD CONSTRAINT car_classificationFOREIGN KEY (classification_key)REFERENCES classification (classification_key);
14
© 2005 Julian Dykejuliandyke.com
For each line in car.csv{
read :season_key, :race_key, :position, :driver_key, :team_key,:engine_key, :laps_completed, :classification_key, :notes;
INSERT INTO car (season_key, race_key, position,driver_key, team_key, engine_key,laps_completed, classification_key, notes)
VALUES(:season_key,:race_key,:position,:driver_key, :team_key, :engine_key,:laps_completed, :classification_key, :notes)
COMMIT;}
Baseline - Insert
15
© 2005 Julian Dykejuliandyke.com
Baseline - Insert
Redo Generation for each Insert Statement
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo insert row in CAR table INSERT
11.2Redo Insert row in CAR tableINSERT
Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT
Redo 10.2 Insert row into CAR_PK indexINSERT
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexINSERT
Redo 10.2 Insert row into CAR_DRIVER indexINSERT
Commit 5.4 End TransactionCOMMIT
Oracle 9.2 and below
16
© 2005 Julian Dykejuliandyke.com
Insert Statement
Redo Generation for eachInsert Statement
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo insert row in CAR table INSERT
11.2Redo Insert row in CAR tableINSERT
Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT
Redo 10.2 Insert row into CAR_PK indexINSERT
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexINSERT
Redo 10.2 Insert row into CAR_DRIVER indexINSERT
Commit 5.4 End TransactionCOMMIT
Oracle 10.1 and above
17
© 2005 Julian Dykejuliandyke.com
Baseline - UpdateFor each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
SELECT driver_key, team_key, engine_key, laps_completed, classification_key, notes
INTO :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes
FROM carWHERE season_key = :season_keyAND race_key = :race_keyAND position = :positionFOR UPDATE;
UPDATE car SETdriver_key = :driver_key, team_key = :team_key,engine_key = :engine_key,laps_completed = :laps_completed,classification_key = :classification_key,notes = :notes, driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_keyAND position = :position;
COMMIT;}
18
© 2005 Julian Dykejuliandyke.com
Baseline - Update
Redo Generation for eachUpdate Statement
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo update row in CAR table UPDATE
11.5Redo Update row in CAR tableUPDATE
Undo Undo delete row from CAR_DRIVER index5.1 (10.22)UPDATE
Redo 10.4 Delete row from CAR_DRIVER indexUPDATE
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexUPDATE
Redo 10.2 Insert row into CAR_DRIVER indexUPDATE
5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE
11.4Redo Lock row in CAR tableSELECT FOR UPDATE
Commit 5.4 End TransactionCOMMIT
Oracle 9.2 and below
19
© 2005 Julian Dykejuliandyke.com
Baseline - Update
Redo Generation
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo update row in CAR table UPDATE
11.5Redo Update row in CAR tableUPDATE
Undo Undo delete row from CAR_DRIVER index5.1 (10.22)UPDATE
Redo 10.4 Delete row from CAR_DRIVER indexUPDATE
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexUPDATE
Redo 10.2 Insert row into CAR_DRIVER indexUPDATE
5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE
11.4Redo Lock row in CAR tableSELECT FOR UPDATE
Commit 5.4 End TransactionCOMMIT
Oracle 10.1 and aboveRedo Generation for eachUpdate Statement
20
© 2005 Julian Dykejuliandyke.com
Baseline - Results Redo Generation in Bytes
Operation INSERT(car.csv)
UPDATE(points.csv)
Total
Baseline 20448852 14409676 34858528
Note Amount of redo generated by both INSERT and UPDATE
can be variable due to Undo segment management Recursive DDL statements e.g. extent allocation Block cleanouts
21
© 2005 Julian Dykejuliandyke.com
Test 1 Check for unused indexes
CAR_PK indexes columns SEASON_KEY RACE_KEY POSITION
supports primary key therefore mandatory
CAR_DRIVER indexes columns SEASON_KEY DRIVER_KEY DRIVER_POINTS
no longer required by current version of application
DROP INDEX car_driver;
22
© 2005 Julian Dykejuliandyke.com
Test 1 - Insert
Redo Generation for each Insert Statement
Header 5.2 Start Transaction
11.2Redo Insert row in CAR tableINSERT
Redo 10.2 Insert row into CAR_PK indexINSERT
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexINSERT
Redo 10.2 Insert row into CAR_DRIVER indexINSERT
5.1 (11.1)Undo Undo insert row in CAR table INSERT
Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT
Commit 5.4 End TransactionCOMMIT
STOP
23
© 2005 Julian Dykejuliandyke.com
Test 1 - Update
Redo Generation
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo update row in CAR table UPDATE
11.5Redo Update row in CAR tableUPDATE
Undo Undo delete row from CAR_DRIVER index5.1 (10.22)UPDATE
Redo 10.4 Delete row from CAR_DRIVER indexUPDATE
5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexUPDATE
Redo 10.2 Insert row into CAR_DRIVER indexUPDATE
5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE
11.4Redo Lock row in CAR tableSELECT FOR UPDATE
Commit 5.4 End TransactionCOMMIT
Redo Generation for eachUpdate Statement
STOP
24
© 2005 Julian Dykejuliandyke.com
Test 1 - Results Redo Generation in Bytes
Operation INSERT(car.csv)
UPDATE(points.csv)
Total
Baseline 20448852 14409676 34858528
Test 1 14687756 12467400 27155156
Conclusion Eliminating redundant index reduced
insert redo generation by 5761096 bytes update redo generation by 1942276 bytes
25
© 2005 Julian Dykejuliandyke.com
Test 2 In UPDATE statements
For tables undo and redo is generated for all columns in SET clause
For indexes undo and redo are only generated for index keys that have changed
Statements often update all columns to reduce parsing e.g.:
UPDATE car SETdriver_key = :driver_key, team_key = :team_key,engine_key = :engine_key,laps_completed = :laps_completed,classification_key = :classification_key,notes = :notes, driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_keyAND position = :position;
26
© 2005 Julian Dykejuliandyke.com
Test 2 - Update
Redo Generation
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo update row in CAR table UPDATE
11.5Redo Update row in CAR tableUPDATE
5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE
11.4Redo Lock row in CAR tableSELECT FOR UPDATE
Commit 5.4 End TransactionCOMMIT
Redo Generation for eachUpdate Statement
Slot = 23Col 3 = JMONCol 4 = WILCol 5 = BMWCol 6 = 71Col 7 = C Col 8 = <Null>Col 9 = 10Col 10= 10
Slot = 23Col 3 = JMONCol 4 = WILCol 5 = BMWCol 6 = 71Col 7 = C Col 8 = <Null>Col 9 = 0Col 10= 0
11.5Redo
5.1 (11.1)Undo
Slot = 23Col 3 = JMONCol 4 = WILCol 5 = BMWCol 6 = 71Col 7 = C Col 8 = <Null>Col 9 = 10Col 10= 10
Slot = 23Col 3 = JMONCol 4 = WILCol 5 = BMWCol 6 = 71Col 7 = C Col 8 = <Null>Col 9 = 0Col 10= 0
STOP
27
© 2005 Julian Dykejuliandyke.com
Test 2 Only update columns which can have new values
DRIVER_POINTS TEAM_POINTS
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
SELECT ... FOR UPDATE;
UPDATE car SETdriver_key = :driver_key, team_key = :team_key,engine_key = :engine_key,laps_completed = :laps_completed,classification_key = :classification_key,notes = :notes, driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_keyAND position = :position;
COMMIT;}
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
SELECT ... FOR UPDATE;
UPDATE car SETdriver_key = :driver_key, team_key = :team_key,engine_key = :engine_key,laps_completed = :laps_completed,classification_key = :classification_key,notes = :notes, driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_keyAND position = :position;
COMMIT;}
28
© 2005 Julian Dykejuliandyke.com
Test 2 - Results Redo Generation in Bytes
Operation INSERT(car.csv)
UPDATE(points.csv)
Total
Baseline 20448852 14409676 34858528
Test1 14687756 12467400 27155156
Test2 14560052 11584760 26144812
Conclusion Eliminating unnecessary columns from update statements
reduced update redo generation by 882640 bytes Would be significantly more if unchanged columns
included long fields e.g. CHAR, or VARCHAR2
29
© 2005 Julian Dykejuliandyke.com
Test 3 Eliminate unnecessary SELECT FOR UPDATE statements
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
SELECT driver_key, team_key, engine_key, laps_completed, classification_key, notes
INTO :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes
FROM carWHERE season_key = :season_key AND race_key = :race_key AND position = :positionFOR UPDATE;
UPDATE car SET driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_key AND position = :position;
COMMIT;}
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
SELECT driver_key, team_key, engine_key, laps_completed, classification_key, notes
INTO :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes
FROM carWHERE season_key = :season_key AND race_key = :race_key AND position = :positionFOR UPDATE;
UPDATE car SET driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_key AND position = :position;
COMMIT;}
30
© 2005 Julian Dykejuliandyke.com
Test 3 - Update
Redo Generation
Header 5.2 Start Transaction
5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE
11.4Redo Lock row in CAR tableSELECT FOR UPDATE
Redo Generation for eachUpdate Statement
11.5Redo Update row in CAR tableUPDATE
5.1 (11.1)Undo Undo update row in CAR table UPDATE
Commit 5.4 End TransactionCOMMIT
STOP
31
© 2005 Julian Dykejuliandyke.com
Test 3 - Results Redo Generation in Bytes
Operation INSERT(car.csv)
UPDATE(points.csv)
Total
Baseline 20448852 14409676 34858528
Test 1 14687756 12467400 27155156
Test 2 14560052 11584760 26144812
Test 3 14554428 8475484 23029912
Conclusion Eliminating SELECT FOR UPDATE statement reduced
update redo generation by 3109276 bytes
32
© 2005 Julian Dykejuliandyke.com
Test 4 Rows are inserted with default values of 0 for driver_points
and team_points Points only scored by
first eight cars - 2003 onwards first six cars - pre 2003
Only update rows with non-zero rows for driver_points and/or team_points
Team
Driver
Points No Points
Points 3514 30
No Points 324 12313
Driver Team
324 3514 30 12313
STOP
33
© 2005 Julian Dykejuliandyke.com
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
Test 4 - Update
Redo GenerationRedo Generation for eachUpdate Statement
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
UPDATE car SET driver_points = 1 team_points = 1
WHERE ...
col9 = 0 col10 = 0
col9 = 1 col10 = 1
UPDATE car SET driver_points = 0 team_points = 0
WHERE ...
col9 = 0 col10 = 0
col9 = 0 col10 = 0
UPDATE car SET driver_points = 0 team_points = 0
WHERE ...
col9 = 0 col10 = 0
col9 = 0 col10 = 0
UPDATE car SET driver_points = 9 team_points = 9
WHERE ...
col9 = 0 col10 = 0
col9 = 9 col10 = 9
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
Header 5.2
11.5Redo
5.1 (11.1)Undo
Commit 5.4
UPDATE car SET driver_points = 9 team_points = 9
WHERE ...
col9 = 0 col10 = 0
col9 = 9 col10 = 9
STOP
34
© 2005 Julian Dykejuliandyke.com
Test 4 Only update rows with non-zero rows for driver_points and/or
team_points
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
UPDATE car SETdriver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_keyAND position = :position;
COMMIT;
}
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
IF driver_points != 0 OR team_points != 0 THEN {
UPDATE car SETdriver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_keyAND position = :position;
COMMIT;}
}
35
© 2005 Julian Dykejuliandyke.com
Test 4 - Results Redo Generation in Bytes
Operation INSERT(car.csv)
UPDATE(points.csv)
Total
Baseline 20448852 14409676 34858528
Test 1 14687756 12467400 27155156
Test 2 14560052 11584760 26144812
Test 3 14554428 8475484 23029912
Test 4 14683408 2070316 16753724
Conclusions Eliminating unnecessary update statements reduced
update redo generation by 6405168 bytes
36
© 2005 Julian Dykejuliandyke.com
For each line in car.csv{
read :season_key, :race_key, :position, :driver_key, :team_key,:engine_key, :laps_completed, :classification_key, :notes;
INSERT INTO car (season_key, race_key, position,driver_key, team_key, engine_key,laps_completed, classification_key, notes)
VALUES(:season_key,:race_key,:position,:driver_key, :team_key, :engine_key,:laps_completed, :classification_key, :notes)
COMMIT;}
For each line in car.csv{
read :season_key, :race_key, :position, :driver_key, :team_key,:engine_key, :laps_completed, :classification_key, :notes;
INSERT INTO car (season_key, race_key, position,driver_key, team_key, engine_key,laps_completed, classification_key, notes)
VALUES(:season_key,:race_key,:position,:driver_key, :team_key, :engine_key,:laps_completed, :classification_key, :notes)
COMMIT;}
COMMIT;
Test 5 Eliminate unnecessary COMMIT statements
37
© 2005 Julian Dykejuliandyke.com
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
IF driver_points != 0 OR team_points != 0 THEN {
UPDATE car SET driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_key AND position = :position;
COMMIT;}
}
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
IF driver_points != 0 OR team_points != 0 THEN {
UPDATE car SET driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_key AND position = :position;
COMMIT;}
}
COMMIT;
Test 5 Eliminate unnecessary COMMIT statements (continued)
38
© 2005 Julian Dykejuliandyke.com
Test 5 - Insert
Redo GenerationRedo Generation for eachInsert Statement
Header 5.2 Start Transaction
11.2Redo Insert row in CAR tableINSERT
Redo 10.2 Insert row into CAR_PK indexINSERT
5.1 (11.1)Undo Undo insert row in CAR table INSERT
Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT
Commit 5.4 End TransactionCOMMIT
Header 5.2 Start Transaction
Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT
11.2Redo Insert row in CAR tableINSERT
Redo 10.2 Insert row into CAR_PK indexINSERT
5.1 (11.1)Undo Undo insert row in CAR table INSERT
Commit 5.4 End TransactionCOMMIT
STOP
39
© 2005 Julian Dykejuliandyke.com
Test 5 - Results Redo Generation in Bytes
Operation INSERT(car.csv)
UPDATE(points.csv)
Total
Baseline 20448852 14409676 34858528
Test 1 14687756 12467400 27155156
Test 2 14560052 11584760 26144812
Test 3 14554428 8475484 23029912
Test 4 14683408 2070316 16753724
Test 5 9516512 1028084 10544596
Conclusion Eliminating COMMIT statements reduced
insert redo generation by 5166896 bytes update redo generation by 1042232 bytes
40
© 2005 Julian Dykejuliandyke.com
Test 6 Default batch size is 1 Test INSERT and UPDATE with different batch sizes
Batch Size INSERT Redo UPDATE Redo Total Redo
1 9517096 1028084 10545180
2 5654136 1028084 6682220
4 3927092 1028440 4955532
8 3011944 1028084 4040028
16 2588540 1028636 3617176
32 2375884 1028172 3404056
64 2254936 1028040 3282976
128 2195876 1028084 3223960
256 2179404 1028440 3207844
512 2163816 1028084 3191900
1024 2163084 1028084 3191168
2048 2160012 1028084 3188096
41
© 2005 Julian Dykejuliandyke.com
Test 6 - Results
0
1000000
2000000
3000000
4000000
5000000
6000000
7000000
8000000
9000000
10000000
1 2 4 8 16 32 64 128 256 512 1024 2048
Batch Size
Red
o (b
ytes
)
42
© 2005 Julian Dykejuliandyke.com
Test 6 - Results Redo Generation in Bytes
Operation INSERT(car.csv)
UPDATE(points.csv)
Total
Baseline 20448852 14409676 34858528
Test 1 14687756 12467400 27155156
Test 2 14560052 11584760 26144812
Test 3 14554428 8475484 23029912
Test 4 14683408 2070316 16753724
Test 5 9516512 1028084 10544596
Test 6 2195876 1028084 3223960
Conclusion Batch Size of 128
reduced insert redo generation by 7320636 bytes update redo generation unaffected
43
© 2005 Julian Dykejuliandyke.com
Test 7 Create global temporary table
CREATE GLOBAL TEMPORARY TABLE temporary_car (
season_key VARCHAR2(4),race_key VARCHAR2(2),position NUMBER,driver_key VARCHAR2(4),team_key VARCHAR2(3), engine_key VARCHAR2(3),laps_completed NUMBER,classification_key VARCHAR2(4),notes VARCHAR2(100),driver_points NUMBER,team_points NUMBER
)ON COMMIT PRESERVE ROWS;
44
© 2005 Julian Dykejuliandyke.com
Test 7 Insert rows into global temporary table
For each line in car.csv{
read :season_key, :race_key, :position, :driver_key, :team_key,:engine_key, :laps_completed, :classification_key, :notes;
INSERT INTO temporary_car (season_key, race_key, position,driver_key, team_key, engine_key,laps_completed, classification_key, notes)
VALUES(:season_key,:race_key,:position,:driver_key, :team_key, :engine_key,:laps_completed, :classification_key, :notes)
COMMIT;}
Generated 64140 bytes of redo
45
© 2005 Julian Dykejuliandyke.com
Test 7 Update points in global temporary table
For each line in points.csv{
read :season_key, :race_key, :position, :driver_points, :team_points;
IF driver_points != 0 OR team_points != 0 THEN {
UPDATE temporary car SET driver_points = :driver_points, team_points = :team_points
WHERE season_key = :season_keyAND race_key = :race_key AND position = :position;
}}
COMMIT;
Generated 652884 bytes of redo
46
© 2005 Julian Dykejuliandyke.com
Test 7 Copy rows from global temporary table to permanent table
INSERT INTO car (
season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes, driver_points, team_points ) SELECT
season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes, driver_points, team_points FROM temporary_car;
Generated 2166724 bytes of redo
APPEND hint had no effect
47
© 2005 Julian Dykejuliandyke.com
Test 7 - Results Redo Generation in Bytes
Conclusion Global Temporary Table reduced total redo generation by
340212 bytes
Operation Description Total
Baseline Update all rows 34858528
Test 1 Update affected rows 27155156
Test 2 Update affected columns 26144812
Test 3 Drop index 23029912
Test 4 SELECT FOR UPDATE 16753724
Test 5 COMMIT 10544596
Test 6 Increase Batch Size 3223960
Test 7 Global Temporary Table 2883748
48
© 2005 Julian Dykejuliandyke.com
Test 8 Create external tables
CREATE OR REPLACE DIRECTORY external_dir AS '/u01/app/oracle/gp';
CREATE TABLE external_points (
season_key VARCHAR2(4),race_key VARCHAR2(2),position NUMBER,driver_points NUMBER,team_points NUMBER
)ORGANIZATION EXTERNAL(
TYPE ORACLE_LOADERDEFAULT DIRECTORY external_dirACCESS PARAMETERS(
RECORDS DELIMITED BY NEWLINEFIELDS TERMINATED BY ','
)LOCATION ('points.csv')
);
49
© 2005 Julian Dykejuliandyke.com
Test 8CREATE TABLE external_car (
season_key VARCHAR2(4),race_key VARCHAR2(2),position NUMBER,driver_key VARCHAR2(4),team_key VARCHAR2(3), engine_key VARCHAR2(3),laps_completed NUMBER,classification_key VARCHAR2(4),notes VARCHAR2(100)
)ORGANIZATION EXTERNAL(
TYPE ORACLE_LOADERDEFAULT DIRECTORY external_dirACCESS PARAMETERS(
RECORDS DELIMITED BY NEWLINEFIELDS TERMINATED BY ','MISSING FIELD VALUES ARE NULL
)LOCATION ('car.csv')
);
50
© 2005 Julian Dykejuliandyke.com
Test 8 Insert directly into permanent table joining contents of both
external tables
INSERT INTO car (
season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes,
driver_points, team_points ) SELECT
c.season_key, c.race_key, c.position, c.driver_key, c.team_key, c.engine_key, c.laps_completed, c.classification_key, c.notes, p.driver_points, p.team_points
FROM external_car c, external_points p WHERE c.season_key = p.season_key AND c.race_key = p.race_keyAND c.position = p.position";
Generated 2166724 bytes of redo
51
© 2005 Julian Dykejuliandyke.com
Test 8 - Results Redo Generation in Bytes
Conclusion External Tables reduced total redo generation by 717024
bytes
Operation Description Total
Baseline Update all rows 34858528
Test 1 Update affected rows 27155156
Test 2 Update affected columns 26144812
Test 3 Drop index 23029912
Test 4 SELECT FOR UPDATE 16753724
Test 5 COMMIT 10544596
Test 6 Increase Batch Size 3223960
Test 7 Global Temporary Table 2883748
Test 8 External Table 2166724
52
© 2005 Julian Dykejuliandyke.com
Conclusion We have seen that the following techniques can be used to
reduce the amount of redo generated:
Eliminating redundant indexes Reducing the number of columns updated Eliminating redundant SELECT FOR UPDATE statements Reducing the number of rows processed Eliminating COMMIT statements Increasing the batch size Using Global Temporary Tables Using External Tables
53
© 2005 Julian Dykejuliandyke.com
Thank you for your interest
For more information and to provide feedback
please contact me
My e-mail address is:info@juliandyke.com
My website address is:
www.juliandyke.com
top related