Using Oracle RdbPartitioned Lock Trees

Using Oracle RdbPartitioned Lock Trees

Norman LastovicaOracle Rdb Engineering

November 13, 06

2

Agenda

• Locking Review

• Partitioned Lock Trees in OpenVMS Clusters

• Performance tests

3

Disclaimers

• Tests represented are not strictly scientific

• Shared SAN storage performance is impactedby every IO from every system connected

• Different systems, versions, storage yielddifferent results

4

VMS Distributed Lock Manager

• Used by cooperating applications

• Manages “locks” on “resources”

• Resources & Sub Resources organized in trees

• Cluster transparent

5

Lock Trees

• Database lock is root resource• Contains root file DEVID+FILID

Page1 Page177

Area1

Page53 Page88

Record Record

Logical Area

Area2

Page88 Page94

Area3

DB

6

Remote Locking

• Tree is “mastered” on one node in cluster• Access to tree from master node is “local”• Access from other nodes is “remote”

• Master node is first established on node where rootlock is created (ie, where database is first opened)

• Dynamic lock remastering• VMS moves tree to system with most activity• Be aware of “bouncing”• Can be disabled

7

Remote vs. LocalRequest Latency

149

187

5

157

4

186

3

156141

0

50

100

150

200

mic

rose

cond

s

GS1280 rx8620 1.6 rx4640 1.1

GS1280

rx8620 1.6

rx4640 1.1

GS1280rx8620 1.6rx4640 1.1

8

Rdb Partitioned Lock Trees

• One tree per area• Root lock name contains area DEVID+FILID

• Database tree for non-area resources• Allows area trees to be mastered on different

nodes to reduce remote locking

Page1 Page177 Page422

Area1

Area1Root

Page61 Page177 Page859

Area2

Area2Root

OtherStuff

DB

9

Partitioned Lock TreePage Locks & ALG

PLN PLN PLN

Level -110 Pages

Level -2100 Pages

Level -31000 Pages

PLN PLN

Level -110 Pages

Level -2100 Pages

Level -31000 Pages

Logical Area

PLN

Level -1

Level -2

Level -3

Logical Area

Page Page

Physical Area

Root Area

10

Rdb Partitioned Lock Trees

• Most helpful when application access can bebiased by node/area

• Offline database modification

•ALTER DATABASE FILE …LOCK PARTITIONING IS ENABLED

11

Test Environment

• Cluster of two rx4640 (& an ES40 not used)• 100mb Ethernet interconnect• EVA3000 storage with V8.2-1 & Rdb 7.2.0.1

+--------------------------------------------------------+---------+| SYSTEMS | MEMBERS |+--------+--------------------------------+--------------+---------+| NODE | HW_TYPE | SOFTWARE | STATUS |+--------+--------------------------------+--------------+---------+| SAGAN | HP rx4640 (1.50GHz/6.0MB) | VMS V8.2-1 | MEMBER || WELLS | HP rx4640 (1.50GHz/6.0MB) | VMS V8.2-1 | MEMBER || MONGO | AlphaServer ES40 | VMS F8.3 | MEMBER |+--------+--------------------------------+--------------+---------+

12

Test Database

• 100 Storage areas

• 100 Tables & 100 indexes

13

Populating Database

• “Even” areas/tables/indexes on WELLS

• “Odd” areas/tables/indexes on SAGAN

14

Populating DatabaseWaiting for Page 1:x?

Node: SAGAN (1/2/2) Oracle Rdb V7.2-010 Perf. Monitor 12-APR-2006 12:39:34.87Rate: 3.00 Seconds Active User Stall Messages Elapsed: 00:05:46.69Page: 1 of 1 DISK$RDBTEST_SYS1:[BIGDB]BIGDB.RDB;1 Mode: Online--------------------------------------------------------------------------------Process.ID Since...... T Stall.reason.............................Lock.ID.204003B2:1 12:39:32.74 W waiting for page 1:1497 (PW) 09013F34204003B1:1 12:39:32.89 W waiting for page 1:852 (PW) 0900AAE4204003B3:1 12:39:23.87 W waiting for page 1:852 (PW) 08011E27204003B0:1 12:39:32.84 W waiting for page 1:852 (PW) 0B013151204003AB:1 12:39:32.76 W waiting for page 1:1495 (PW) 0A012271204003B4:1 12:39:23.84 W waiting for page 1:852 (PW) 080138CF204003AD:1 12:39:32.74 W waiting for page 72:598 (PW) 080150E3204003B5:1 12:39:24.07 W waiting for page 1:853 (PW) 08011FB4204003AE:1 12:39:32.90 W waiting for page 1:851 (PW) 1900DAB2204003AF:1 12:39:32.88 W waiting for page 1:851 (PW) 0A01333D

15

Reduce Contention

• Reduce RDB$SYSTEM page size to 2 blocks?

• Disable cardinality collection

16

Populating DatabasePLT disabled

DISTRIBUTED LOCK MANAGEMENT STATISTICS on node WELLS 12-APR-2006 12:37:04.67

CUR AVE MIN MAX

New ENQ Rate (Local) 2511.88 609.09 0.00 3341.84 (Incoming) 2396.55 547.69 0.00 3066.52 (Outgoing) 2.66 1.30 0.00 18.33 Converted ENQ Rate (Local) 2574.21 838.49 0.00 4834.77 (Incoming) 4388.13 878.33 0.00 6353.37 (Outgoing) 0.00 0.00 0.00 0.00 DEQ Rate (Local) 2146.90 508.51 0.00 3015.52 (Incoming) 2258.56 452.32 0.00 3248.85 (Outgoing) 2.66 1.30 0.00 18.33 Blocking AST Rate (Local) 269.65 132.94 0.00 740.96 (Incoming) 0.00 0.00 0.00 0.00 (Outgoing) 838.96 159.62 0.00 1465.59 Dir Functn Rate (Incoming) 164.99 73.30 8.33 282.98 (Outgoing) 0.00 4.21 0.00 95.66 Deadlock Message Rate 1.33 4.89 0.99 15.33

17

Records Stored, PLT Disabled

18

Populating DatabasePLT enabled

DISTRIBUTED LOCK MANAGEMENT STATISTICS on node WELLS 12-APR-2006 12:08:41.72

CUR AVE MIN MAX

New ENQ Rate (Local) 2056.90 1416.90 0.99 2166.90 (Incoming) 355.98 195.52 13.99 479.31 (Outgoing) 644.97 316.35 8.66 703.30 Converted ENQ Rate (Local) 1913.24 1548.52 4.33 2688.87 (Incoming) 344.31 281.95 19.99 484.97 (Outgoing) 275.32 124.69 2.33 275.32 DEQ Rate (Local) 2358.22 1432.23 0.00 2610.21 (Incoming) 106.66 123.32 3.33 440.64 (Outgoing) 425.31 236.32 0.00 514.30 Blocking AST Rate (Local) 51.33 75.59 2.99 201.65 (Incoming) 28.66 19.46 0.33 38.99 (Outgoing) 56.33 79.29 1.99 179.99 Dir Functn Rate (Incoming) 184.65 139.96 28.66 316.65 (Outgoing) 63.99 24.06 0.00 78.32 Deadlock Message Rate 1.66 3.59 1.66 9.66

19

Record Stores, PLT Enabled

20

Test OLTP Workload

• 1 User per table/area/index

• Update 100 random records per transaction

• 8 Users per system

21

WorkloadPLT Disabled – From WELLS

DISTRIBUTED LOCK MANAGEMENT STATISTICS on node WELLS 12-APR-2006 15:42:00.04

CUR AVE MIN MAX

New ENQ Rate (Local) 6609.94 6273.02 4389.90 8027.90 (Incoming) 4402.19 4286.60 2819.92 5520.37 (Outgoing) 0.66 1.40 0.00 4.64 Converted ENQ Rate (Local) 2583.85 2452.34 1722.03 3129.22 (Incoming) 1724.56 1673.39 1100.87 2165.90 (Outgoing) 0.00 0.00 0.00 0.00 DEQ Rate (Local) 6608.95 6273.02 4388.58 8027.90 (Incoming) 4401.86 4286.56 2818.59 5520.70 (Outgoing) 0.66 1.40 0.00 4.64 Blocking AST Rate (Local) 57.52 52.22 34.92 68.85 (Incoming) 0.00 0.00 0.00 0.00 (Outgoing) 40.23 41.80 27.61 59.21 Dir Functn Rate (Incoming) 124.37 52.50 7.97 193.87 (Outgoing) 0.00 0.00 0.00 0.00 Deadlock Message Rate 0.00 0.00 0.00 0.00

22

WorkloadPLT Disabled – From SAGAN

DISTRIBUTED LOCK MANAGEMENT STATISTICS on node SAGAN 12-APR-2006 15:43:55.87

CUR AVE MIN MAX

New ENQ Rate (Local) 10.97 28.59 10.97 46.22 (Incoming) 1.33 1.33 1.33 1.33 (Outgoing) 5683.14 4933.59 4184.04 5683.14 Converted ENQ Rate (Local) 15.96 53.70 15.96 91.44 (Incoming) 0.00 0.00 0.00 0.00 (Outgoing) 2234.68 1932.23 1629.78 2234.68 DEQ Rate (Local) 10.97 28.59 10.97 46.22 (Incoming) 1.33 1.33 1.33 1.33 (Outgoing) 5681.81 4932.76 4183.71 5681.81 Blocking AST Rate (Local) 0.00 0.00 0.00 0.00 (Incoming) 63.51 51.54 39.57 63.51 (Outgoing) 0.00 0.00 0.00 0.00 Dir Functn Rate (Incoming) 26.60 22.28 17.95 26.60 (Outgoing) 0.00 0.00 0.00 0.00 Deadlock Message Rate 0.00 0.00 0.00 0.00

23

TPS, PLT Disabled

24

Enable PLT

SQL> ALTER DATABASE FILE X$LOCK PARTION ENABLE;

25

WorkloadPLT Enabled

DISTRIBUTED LOCK MANAGEMENT STATISTICS on node WELLS 12-APR-2006 16:00:08.67

CUR AVE MIN MAX

New ENQ Rate (Local) 6065.90 5907.35 4483.66 6776.87 (Incoming) 121.71 35.38 0.99 121.71 (Outgoing) 0.33 1.72 0.33 3.65 Converted ENQ Rate (Local) 2426.89 2333.36 1750.50 2661.33 (Incoming) 205.17 180.35 152.30 205.17 (Outgoing) 0.00 0.00 0.00 0.00 DEQ Rate (Local) 6063.90 5907.35 4483.33 6776.87 (Incoming) 116.38 34.18 0.99 116.38 (Outgoing) 0.33 1.72 0.33 3.65 Blocking AST Rate (Local) 62.18 51.93 46.22 62.18 (Incoming) 0.00 0.00 0.00 0.00 (Outgoing) 68.17 60.85 46.22 68.17 Dir Functn Rate (Incoming) 67.50 72.68 17.29 165.27 (Outgoing) 0.00 0.00 0.00 0.00 Deadlock Message Rate 0.00 0.00 0.00 0.00

26

WorkloadPLT Enabled – From SAGAN

DISTRIBUTED LOCK MANAGEMENT STATISTICS on node SAGAN 12-APR-2006 16:00:53.32

CUR AVE MIN MAX

New ENQ Rate (Local) 7163.50 6016.79 4735.33 8649.06 (Incoming) 250.31 82.80 4.99 250.31 (Outgoing) 4.66 1.06 0.00 4.66 Converted ENQ Rate (Local) 2581.82 2181.64 1711.88 3126.11 (Incoming) 0.00 0.00 0.00 0.00 (Outgoing) 224.31 183.67 151.32 255.31 DEQ Rate (Local) 7162.50 6016.76 4733.66 8648.40 (Incoming) 250.98 85.57 7.33 250.98 (Outgoing) 4.66 1.06 0.00 4.66 Blocking AST Rate (Local) 0.00 0.00 0.00 0.00 (Incoming) 76.66 63.62 52.99 87.32 (Outgoing) 0.00 0.00 0.00 0.00 Dir Functn Rate (Incoming) 302.97 132.57 42.99 382.30 (Outgoing) 0.00 0.00 0.00 0.00 Deadlock Message Rate 0.00 0.00 0.00 0.00

27

TPS, PLT Enabled

28

Where PLT May Help Most

• Mostly segregated workload by area by node

• Mostly RW or long RO transactions

• More cluster nodes to distribute among

29

Where PLT MayNot Help As Much

• Lots of short RO transactions• Root/TSNBLK access dominates

• Workload difficult/impossible to partition

30

Considerations

• Evaluate use of• PE1• LOCKDIRWT• LOCKRMWT

• Faster cluster interconnects

• Faster storage (quicker IO = less time locksare held)

31

norman.lastovica@oracle.comhttp://www.oracle.com/rdb

Q&AA N S W E R SA N S W E R SQ U E S T I O N SQ U E S T I O N S

With ThanksKevin JenkinsGreg Jordan

Christian MoserPaul MeadJeff JalbertKeith Hare