choosing database technology

Jens ColdeweyColdewey Consulting

Uhdestraße 12D-81477 München

[email protected]://www.coldewey.com

Choosing a DatabaseTechnology

Tutorial at Object World 98Frankfurt, October, 1st1998

© Jens Coldewey, Coldewey Consulting, 1998 - All rights reserved

Roadmap through the next hours

4 Introduction

4 The Problem: StoringObjects

4 Requirements of theStakeholders

4 The important forces indepth

4 Turning forces into adecision


What are we talking about?

4 You...

– develop medium tolarge size systems

– use object technology

– need to store yourobjects

– have to decide for adatabase technology

4 It doesn't matter...

– what domain you arein

– whether you work for aproduct or a project

– what language youuse

! I'm not talking about specific products !


Deciding for a database you have totake care of the different stakeholders

EndUsers

Opera-tions

ProjectManage-

ment

ClientManage-

ment

TeamUsers Project


You have to find the best balance

4Every group hasdifferent objectives

4These "forces" oftencontradict each other

4A good decision is apragmatic balance ofthese forces

This tutorial analyzesthe forces. It's yourterm to decide.


Before we talk about solutions we haveto identify the problem: Storing objects

4 Introduction

è The Problem:Storing Objects



4 Turning forces intodecisions Database

?

X Y

Z


Storing objects you want OO features tobe preserved...

4Complex Objects

4Object Identity

4Encapsulation

4Classes

4Class Hierarchy

4Polymorphism

4Extensibility


...while new requirements emerge

4Persistence

4Query Facility

4Concurrency

4Recovery

4Security

4Secondary Storage Management

(Modified from [Atk+89])


What are the options?

Database

?

X Y

Z

ObjectDatabase

Object/RelationalCoupling

StreamPersistence

ObjectServer

PageServer

ORDBMS AccessLayer


Object databases directly store objectsin the database

X Y

Z

X Y

Z

4 Tight languageintegration

– Access

– Typing

– References

4 ODMG defines"standard" interface

4 Products: ObjectStore,Versant, Objectivity/DB,Poet, GemStone,...

4 Literature: [Cat94]


Relational databases need a complexmapping mechanism

4 Relational Paradigm

– Tables and table-operations

– References via foreignkeys

– No extended typing

– No inheritance

4 SQL - Standardized

4 Products: TOPLink,Persistence

4 Literature: [Kel98]

X Y

Z

ZY X

O/R Access Layer


Object / Relational databases try tocombine both worlds

4 Extends RelationalParadigm

– Complex Objects

– Typing

– Nested Tables

4 E-SQL (vendor specific)

4 No standards by now

4 Products: DB/2,Informix, Oracle 8,Unidata

4 Literature: [StM96]

X Y

Z

Y Z

X


To start with the analysis of the forceswe turn back to our stakeholders

4 Introduction


è Requirements ofthe Stakeholders


4 Turning forces intodecisions

EndUsers

Opera-tions

ProjectManage-

ment

ClientManage-

ment

TeamUsers Project


Client's management usually has fivethings in mind

Opera-ting Cost

Invest-ment

Protec-tion

Develop-mentCost

Time toMarket

Func-tionality

Client'sManage-

ment


The end user's requirements are usually"obvious"

Func-tionality

Relia-bility

Perfor-mance

EndUser


Operations' requirements are often lessobvious - and they will haunt you!

Main-tain-

ability

Re-covery

AccessControl

Relia-bility

AdminEffort

Re-sources

Oper-ations


Some forces derive from your ownproject management

VendorSupport

Invest-ment

Protec-tion

Devel-opmentEffort

TrainingLicenseCost

ProjectManage-

ment


And finally the development team needsto get optimal support for their work

VendorSupport

Stability

DBFeatures

ToolSupport

Archi-tecture

Team


Functionality

Performance

Reliability

AccessControl

Resources

Maintain-ability

ArchitecturalConfor-mance

DB Features

Stability

ToolSupport

Training

OperatingCost

Time toMarket

Admin Effort

License Cost

Develop-ment Effort

DevelopmentInvestmentProtection

Client'sInvestmentProtection

VendorSupport


In the following discussion I concentrateon discriminating issues

4Some of these issues depend more on theproduct than on the technology

4The following concentrates on whattechnology heavily influences:

– Architectural Conformance

– Functionality and DB Features

– Performance

– Maintainability

4The rest is just sketched


4 Introduction



è The importantforces in depth (I)


ArchitecturalConformance

Functionalityand Database

Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest


The Architectural Conformance splitsinto three areas of interest

Architec-tural Confor-

mance

Data Architecture

methodology D

istri

butio

n


Many large systems consist of severalapplications on the same database

Contracts Liability CommisionRepor-

tingStatistics

Enterprise Database


This is the architecture most (relational)database experts have in mind

+ Well-known architecture

+ Easy integration withlegacy applications

+ Easy Client/Serverdesign

+ Many useful tools

− Database designchanges propagatemerciless ⇒Application specificviews needed

− Elephantiasis of thedatabase


The object view is different

Data

Functionality

Func-tionalty

DataFunc-

tionalty

Data

Func-tionalty

Data


The object approach avoids someproblems but generates new ones

+ Good encapsulation ofdata

+ Defined responsibilitieskeep databasesmanageable

+ Missing centraldatabase model speedsup development

− Additionalcommunication is newand complex

− Analyzing information ofseveral applications ishard

− Tool market is still in itsinfancy

− Paradigm shift is hard


Both DB technologies support botharchitectures but have different focus

+ Feasible- Schema evolution

may become anightmare

+ ProposedArchitecture

+ Schema evolutionsupported well

+ ProposedArchitecture

+ Good tool support

+ Feasible- "Brushing against

the tools"

Object DB (O)RDB

Dat

abas

eO

rien

ted

Ob

ject

Ori

ente

d



mance

DataArchitecture

methodology D

istri

butio

n


Storing objects you want to preserveyour methodology

4Language integration– A single language (and type system) should apply, no

matter whether information is transient or persistent

4 Information Hiding– Only the object itself knows what data it is storing

4Separation of concerns– Every part of the system should have its well-defined,

exclusive responsibility

Many successful system have been built withoutsticking to these rules dogmatically


The paradigm gap is quite obvious here

OODB ORDB RDB

+ Very well- Vendor

specificLan

g.

Inte

gr. - Extended

SQL- Explizit

transfer

+ FullLanguageFeatures

- No intrinsicencapsu-lationIn

form

.H

idin

g o Depends onUsage

Sep

ar. o

fC

on

c.

+ Full ObjectPower

o Depends onUsage

- Data centricapproach

- SQL- Explizit

transfer



mance

DataArchitecture

methodology D

istri

butio

n


All solutions support single serverarchitectures with specific strengths

DatabaseEngine

QueryEngine

Model

QueryEngine

QueryEngine

ModelModelModel

DatabaseProxy

DatabaseClient DB Client

Access Layer

PageServer

ObjectServer

ORDB RDB

Requests P

ages

Requests O

bjec

ts

R-E

SQ

L

Rel

atio

ns

R-E

SQ

L

Rel

atio

ns


For multi-server architectures you needextra services

LockServer

Server1

DB 1Server 2

DB2

Server 3

DB3Trans-action

Service

Server1

DB 1Server 2

DB2

Server 3

DB3

Page Server TM / CORBA


The product is more important than thetechnology

OODB ORDB and RDB

+ Good supportS

ing

le S

erve

r

+ Page Server:homogenous ser-vers supported well

- Others: Coming up

+ Established trans-action processorsand servers

+ Standards for distri-buted transactionsM

ult

iple

Ser

vers

+ Good support


Architectural considerations don'tsuppose a single solution

Database Orientation

Language Intergration

Information Hiding

Separation ofConcerns

Object Orientation

Single Server Support

Multiple HomogenousServers

MultipleHeterogenous

Servers

Database Orientation

Language Intergration

Information Hiding

Separation ofConcerns

Object Orientation

Single Server Support

Multiple HomogenousServers

MultipleHeterogenous

Servers

ODBMS

ORDBMS

RDBMS


4 Introduction



è The importantforces in depth (II)


4 Summary



Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest


It is important not to miss any of theaspects that comprise functionality

Functio-nality & DBFeatures

Datastorage

Stru

ctur

e

Dynam

ics


There are a lot of details to beconsidered

Structure

4 Objectgranularity

4 Inheritance

4 Number ofClasses

4 Collections

4 Versioningand History

Data Storage

4 TransactionModel

4 Concurrencyand Locking

4 Scalability

Dynamics

4 AccessCharacteristics

4 QueryComplexity andFlexibility

4 Mass Updates


Object granularity is an important factor

4 Analysis and designtechniques influenceobject size

– Large objects resultfrom a data-focussedanalysis

– Small fine-grainedobjects result from ause-case drivenapproach

4 High reusability oftenresults in fine-grainedobjects

Ver

y S

mal

l

Sm

all

Med

ium

Larg

e

Ver

y La

rge

Per

form

ance

ODBMS (Page) ODBMS (Object)

O/RDBMS RDBMS


Inheritance support is one of the crucialdifferences

4 Inheritance supportimplies:

– Subtypes

– Polymorphic queries

– Complete tree support

– Polymorphicaggregations

– Fast addition of newsubclasses

– (Multiple Inheritance)

Subtypes

PolymorphicQueries

Full Tree

PolymorphicAggregations

Addition ofSubclasses

MultipleInheritance

ODBMS ORDBMSRDBMS


The number of classes prevent a purelyrelational solution

4 A fine-grained designusually results in manyclasses

4 The more classes youhave the more you relyon polymorphism

4 A direct O/R approachmaps classes totables...

Ver

y F

ew

Few

Med

ium

Man

y

Hug

e

Per

form

ance

ODBMS O/RDBMS RDBMS


Support for templates and collections isless natural than you may guess

4Compliance to class library

4Different database representations

– Indexed Collections → Direct Storage (Bi-directional cursors!)

– Dictionaries →Indices

– Sets →Special (ODB) or UNIQUE (RDB)

4ODBs offer additional bi-directionalAssociations

Important differences between products!


Support for versions and history aresometimes helpful

4 Implementingversioning with arelational database iswell-understood butneeds resources

4 Many object databasessupport one-dimensional multi-thread versions

4 More of a goodie than acrucial feature


Concerning structure ODBMS fit best

Fine-grainedObjects

Inheritance

Many ClassesCollections

Versioning

ODBMS ORDBMS RDBMS



Datastorage

Stru

ctur

e Dynam

ics


Database applications show typicalaccess characteristics

OpenTask

OpenProject

Work withObjects

AnotherContract

FindContract

Contractswhere...

Navigation Queries


ODBMS support navigation,(O)RDBMS querying

4 ODBMS

– store objectsregardless of class

– are optimized to followreferences

– provide clustering foradditional navigationsupport

– have weak queryengines

4 (O)RDBMS

– store objectsaccording to class

– use (slow) joins tonavigate

– have excellent queryengines

4 ORDBMS

– can navigateaggregations fast

This is one of the most important forces


Consequently (O)RDBMS offer muchbetter support for queries

4 ODBMS

– Medium support forcomplex objectselections

– Attribute queriesviolate encapsulation

– No ad-hoc queries(Caution: DataWarehouses...)

4 (O)RDBMS

– Excellent support forcomplex queries

– Paradigm supportsfree queries on alldata (everything isglobal!)

– Good support for ad-hoc queries


ORDBMS usually don't support massupdates or stored procedures

Processing

Processing

Batch Control

ObjectDatabase

(Object)RelationalDatabase

Requests O

bjec

ts

UP

DA

TE

WH

ER

E Ret

urn

Cod

e

Clie

ntS

erve

rN

etw

ork


Summary: Navigational applications callfor ODBMS, querying apps for RDBMS

Navigation

Mass-Updates

QueryingAd-hoc Queries

QueryComplexity

ORDBMS RDBMS ODBMS



Datastorage

Stru

ctur

e

Dynam

ics


Today's applications feature twodifferent transaction styles

4 Check-in/Check-outsemantics

4 Long Transactions

– May last for hours oreven weeks

– High probability ofconflicts

– Only few transactionsper second

– Call for nestedtransactions

4 Classic semantics

4 Short transactions

– Last for a fraction ofseconds

– Low probability ofconflicts

– More than 1000transactions persecond


The differences become relevant inextreme areas only

+ Up to 100 Tx/sec+ Up to 1000 users

+ Support for nestedtransactions

+ Timestamps andversioning toresolve conflicts

++ Up to 1000+ Tx/sec++ Several 1000 users

o Feasible but noexplizit support

Object DB (O)RDB

Sh

ort

Tra

nsa

ctio

ns

Lo

ng

Tra

nsa

ctio

ns


There are two different lockingtechniques in the ODBMS world

4 Object Locking

+ Low risk of conflictsand deadlocks

– High overhead forlocking

4 Page Locking

– Higher risk of conflictsand deadlocks

+ Better performance

4 Unless you haveextreme performancerequirements, the besttradeoff is hard to find

4 Some products offerboth strategies

4 You have to experimentwith your project'sprofile


Locking can cause disasters with naive O/R Mapping

Class A

Class B Class C

Class D Class E Class F

Table A

Table B Table C

Table D Table ETable E


All technologies provide good scalability

4 Both technologies workwith extremly largedatabases

4 Access time for simpleretrieval:

– ODBMS: const

– RDBMS: logm n


Some hard data from the MMIS project

00,5

11,5

22,5

ODBMS tuned

ODBMS

RDBMS0

20

40

60

80

100

120

140

160

180

200[R

etri

eval

Tim

e m

sec]

[Million Records]

ODBMS tuned

ODBMS RDBMS

Source: A. Chaudhri: Object Databases in Practice - Tutorial; OOPSLA'97 Workshop on Experiences using Object Data Management in the Real Worldhttp://www.soi.city.ac.uk/~akmal/oopsla97.dir/workshop.html


Beware of the locking characteristics ofan O/R access layer!

ODBMS

ORDBMS

RDBMS

Long Tx

Short Tx

ScalabilityLocking low load

Locking high load

Long Tx

Short Tx

ScalabilityLocking low load

Locking high load


4 Introduction



è The importantforces in depth (III)




Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest


In general ODBMS have significantlybetter performance

4 Ratio between ODBMSand RDBMS

– Navigational Lookup:10-100 times

– Queries: 0,1-10 times

– Insert: 1-10 times

4 Access layers make iteven worse

4 Advantage decreaseswith transaction load

4 Collection of bench-marks:www.soi.city.ac.uk/~akmal/

4 Don't take thesenumbers for granted:Make your own tests!


Performance depends on several factors

NetworkLoad

Tuning

Locking

AccessCharac-teristics

Tx Over-head

Caching

Perfor-mance


Key to performance: Direct access andclustering instead of foreign keys

X Y

Z

x:=Lookupx->y->z.do()

read

OID

-> P

hysicalP

osition

Cache

X Y

Z

4 The more the appli-cation uses navigationthe higher your gain

4 Significant differencesbetween products

– Page servers showfaster access butworse locking behavior

– Moving an object maychange OID

4 ORDBMS also use OIDs(called REF)


Tuning techniques and support differ

4 ODBMS

– Clustering

– Cache adjustment

– Indices and queryoptimization

4 Mediocre tools forprofiling

4 (O)RDBMS

– Denormalization (!)

– Query optimization

– Indices

– Technical parameters

4 Excellent profilingsupport


A real world comparison...(Object Store for NT 5.0 against JDBC 1.1.3/Oracle 7.3)

Komplex ClassGIF Image

JPEG Image

ODBMS

RDBMS / JDBC0,1

1

10

100

1000

10000

ODBMSRDBMS / JDBC

Source: Strategic Technology Resources: Java Data Management - Comparing ODBMS and RDBMS Implementations of Quantum Objects , White Paper, Chicago, Illinois; 1997; http://www.odi.com/content/white_papers/str-odb-rdb.pdf


...and a more elaborate benchmark

Traversalcold

Traversalwarm

RandomLookup

cold

RandomLookupwarm

Insertcold Insert

warm

OODBMS Small

ODMS large

RDBMS Small

RDBMS large

0

20

40

60

80

100

120

140OODBMS SmallODMS largeRDBMS SmallRDBMS large

Source: R. Catell: An Engineering Database Benchmark in: J. Gray (ed.): The Benchmark Handbook for Database and Transaction Systems, Morgan-

Kaufman, San Mateo, California. 1991


A good benchmark takes all this intoaccount

4 A benchmark shouldcontain

– Emulation of accesscharacteristics

– Database size

– Simulation of heavymulti-user operation

– Checks onfragmentation

4 Please keep in mind:

– Performance onlyneeds to be "goodenough"

– Benchmarks shouldbe time-boxed


4 Introduction



è The importantforces in depth (IV)




Features

Performance

Maintainabilityand

DevelopmentSupport

The Rest


Maintainability is more thanjust using OO

Skills

SchemaEvolution

MarketPosition

SourceCom-plexity

Maintain-ability


Over the time the object model changes("schema evolution")

4 ODBMS

– Provide support foradding attributes

– Provide APIs toprogram morecomplex changes(lazy update)

4 RDBMS

– Views can handlemost changes viaadministration


The vendor's market position isimportant to estimate long-term support

4ODBMS

– Large vendors exist for a decade now

– Total revenue less than Oracle's expenses onadvertising

– Long-term survival of some vendors unknown

4 (O)RDBMS

– Vendors have established market positions

– Long-term survival near to guaranteed

– O/R Databases have unknown future


Complex sources only annoy developersbut may cause real problems later

4 More sources mean

– More complex confi-guration management

– More points to change

– More opportunities forerrors

4 ODBMS

– Language integrationprovides single-source(with exceptions)

4 (O)RDBMS

– Database definitionand mapping definitionin extra files

– Behavior in OOlanguage and (E)SQL(and storedprocedures)


Which solution fits your maintenanceneeds depends on your requirements

ODBMS

ORDBMS

RDBMS

Schema Evolution

SourceManagement

Market Position

Skills

Schema Evolution

SourceManagement

Market Position

Skills


Some final remarks on developmentsupport

3rd PartyTools

Profiling

TestingTools

DebuggerSupport

VendorSupport

IDESupport

ToolSupport


4 Introduction




è Turning forcesinto decisions


Now that we have identified the forces,it's up to you to find a balance...

4 Identify relevant forces

4 Find your project'scorresponding needs

4 Identify the options thatstill remain

4 Evaluate products

– attend classes

– write prototypes

4 Come up withsuggestions


A minimal set of forces to decidetechnology may look like this

4Basic architecture (Database oriented versusobject oriented)

4Distribution architecture

4Object granularity

4Access characteristics

4Number of classes

4Transaction load

4Market Position


This list of forces clearly showsstrengths and weaknesses of both

ODBMS

ORDBMS

RDBMS

Distribution

DatabaseOrientation

Tx Load

Query support

Market PositionObject Size

Object Server

Number ofClasses

NavigationalAccess

Distribution

DatabaseOrientation

Tx Load

Query support

Market PositionObject Size

Object Server

Number ofClasses

NavigationalAccess


... and it's up to you tofight the political stands

4 Try to see theevaluation with the eyesof the stakeholders

4 Try to foresee theirobjections

4 Try to understandpersonal fears

4 Don't be dogmatic

"People hate changes"T. DeMarco


Thanks for their support

For additional input

4 Akmal Chaudhri,Logica UK Ltd.

4 Uwe Beßle,Versant

4 Rüdiger Eberlein,Oracle

For fruitful discussions

4 Wolfgang Keller,EA Generali


References

[Atk+89] M. Atkinson, F. Bancilhon, D. DeWitt, K. Dittrich, D. Maier, S Zdonik: The Object-Oriented Database System Manifesto in: Deductive and Object-Oriented Databases,Proceedings of the First International Conference on Deductive and Object-OrientedDatabases (DOOD'89), pp. 223-240

[Cat91] R. G.G. Catell: An Engineering Database Benchmark in: J. Gray (ed.): The BenchmarkHandbook for Database and Transaction Systems, Morgan-Kaufman, San Mateo, California.1991

[Cat94] R. G. G. Catell: Object Data Management - Object-Oriented and Extended RelationalDatabase Systems - Revised Edition; Addison-Wesley, Reading, Massachusetts, 1994

[Cha97] A. Chaudhri: Object Databases in Practice - Tutorial; OOPSLA'97 Workshop onExperiences using Object Data Management in the Real Worldhttp://www.soi.city.ac.uk/~akmal/oopsla97.dir/workshop.html

[Kel98] W. Keller: Object/Relational Access Layers - A Roadmap, Missing Links, and More Patternsin: J. Coldewey, P. Dyson (eds.): Proceedings of the 3rd European Conference on PatternLanguages of Programming and Computing, Universitäts Verlag Konstanz, to be publishedhttp://www.coldewey.com/europlop98/

[StM96] M. Stonebraker, D. Moore: Object-Relational DBMSs - The Next Great Wave; Morgan-Kaufman, San Mateo, California, 1996

[STS97] Strategic Technology Resources: Java Data Management - Comparing ODBMS andRDBMS Implementations of Quantum Objects, White Paper, Chicago, Illinois; 1997;http://www.odi.com/content/white_papers/str-odb-rdb.pdf

choosing database technology

Documents