postgresql enterprise class features and capabilities

Copyright 2016 FUJITSU LIMITED

I am..

Venkata B Nagothi

Call me..

VENKAT


Agenda

Migration Challenges (Oracle to PostgreSQL)

PostgreSQL capabilities


Migration Challenges

Design and Architecture challenges

Development effort required for successful database migration to PostgreSQL

Minimize development effort when migrating Applications


Migration Assessment

Infrastructure

Database Objects

Data Migration

PL/SQL Objects

Database Environment

Application


Infrastructure Migration


High Availability

Oracle Dataguard Broker

PostgreSQL Streaming Replication

Migration Impact

• More simpler to implement

• Supports all the protection modes in Oracle

• Supports cascading replication with some limitations


High Availability

Oracle Dataguard Observer

PostgreSQL pgPool-II

Master

Standby

Switches over when the

master crashes

pgPool-II Automatically promotes the

standby to be the new

master

pgPool-II Automatic failover disable

Application


High Availability

Role reversal – Limitations in PostgreSQL

Master

Standby (new master)

Streaming Replication

Standby can be promoted to Standalone new master

manually

Old master cannot be made standby. Needs to rebuilt

from new master’s backup


High Availability

PostgreSQL Streaming Replication – Limitations

Filesystem structure has to be identical

v

Master

Streaming Replication /data/tbs01

/data/tbs02

/data/tbs03

Standby

/data/tbs01

/data/tbs02

/data/tbs03


Tablespaces

Oracle Tablespaces

PostgreSQL Tablespaces

Migration Impact

• More OS dependent

• Re-Architect database disk architecture

• Performance testing and benchmarking

• Re-do Capacity Planning


Tablespaces

/disk1/datafile-1

/disk2/datafile-2

/disk3/datafile-3

Logical Structure

Logical Structure

Physical Structure

Physical Structure

/disk1/tbs01

Oracle Tablespace Structure PostgreSQL Tablespace Structure

Table 1 Table 2

Index 1

Table 1 Table 2

Index 1

Datafile 1 Datafile 2

Datafile 3


Tablespaces

PostgreSQL Disk Architecture

/disk1/tbs01

/disk2/tbs02

/disk3/tbs03

/disk4/tbs04

/disk5/tbs05

Each partition has to be an extendable partition for

better capacity management

A single partition over multiple disk spindles

(RAID)

/data/pgdata


Tablespaces

Limitations - PostgreSQL

Tablespaces alone cannot be backed up

Transportable tablespaces not possible

Tablespace restore/recovery is not quite possible


Databases

DB01

Instance 1

DB02

Instance 2

DB03

Instance 3

DB01

Instance 1

DB02

DB03

Oracle PostgreSQL

• Database Size

• Database performance

• Application

Architecture/behaviour

• Database operations

• Security

• Multitenancy

• Other business and

operational

constraints


Database Objects Migration


Schemas

Oracle Users

PostgreSQL Users & Schemas

Migration Impact

• Users and Schemas must be separately created in PostgreSQL

• A user can own multiple schemas

• Flexibility around changing ownerships


Database Objects

Oracle Objects PostgreSQL Objects

Materialized Views Materialized Views (limitations)

Index-Organized-Tables Cluster a Table (partial support)

Public Synonyms search_path

Global Temporary Tables Unlogged Tables


Data Types

Oracle Data type PostgreSQL Data Type

VARCHAR2 VARCHAR

NUMBER INTEGER

DATE TIMESTAMP

CLOB TEXT / JSON / JSONB

BLOB BYTEA / JSONB LO


Data Migration


Data Migration

BLOB

BYTEA (1 GB LIMIT)

LO

BLOB JSONB


Data Migration

Oracle BLOB

PostgreSQL BYTEA

(max limit 1 GB) Ora2pg (automatic)

Migration Impact

• Oracle BLOBs are automatically migrated to PostgreSQL bytea by

ora2pg

• Unstable Application performance can be expected which would hit the

benchmarking metrics

• Can accommodate more than 1 GB due to PostgreSQL compression

algorithm

• Triggers an Application code change


Application Impact (Example : JAVA)

getBytes() BYTEA

Migration Impact

• Only getBytes() function can be used against BYTEA data type in PostgreSQL

• getBytes() attempts to read the whole BYTEA into the memory at a time which will impose performance problems resulting in excessive usage of Memory and Network bandwidth

• Cannot stream BYTEA in small chunks

BYTEA Column

Size

Text File 10K

Text File 10K

Image 20M

Image 300M

Text file 100K

Data Migration

getBLOB() BLOB


Data Migration

Oracle BLOB

PostgreSQL LO Manual

Migration Impact

• Custom ETLs must be built

• Migration effort and time can be a challenge

• Application code change is expected

• Application functionality and Performance testing


Application Impact (Example : JAVA)

getBlob()

LO

Migration Impact

• getBlob() function is used against LO similar to Oracle

• getBlob() is the recommended approach for accessing LO (Large Objects)

datatypes from PostgreSQL

• Large Objects can be streamed

• BLOBs can be streamed in multiple chunks resulting in effective performance

Data Migration

BLOB


Data Migration

Oracle BLOB with JSON data

PostgreSQL JSONB Manual

Migration Impact

• Can result in an heavy Application design and code change


• Heavy development can be on the cards

SELECT UTL_RAW.CAST_TO_VARCHAR2 select col->>‘value' FROM table;


Data Migration

Oracle CLOB with JSON data

PostgreSQL TEXT AUTOMATIC (Ora2pg)

Migration Impact

• Can result in an heavy Application design and code change


PostgreSQL JSONB

Manual


PL/SQL Migration


PL/SQL Migration

PL/SQL Object PostgreSQL Alternative

Procedures Functions with RETURNS VOID

Functions Functions

Triggers Triggers

Packages Schemas + Temp tables + Functions


System Packages

SYSTEM PACKAGES PostgreSQL Alternative

DBMS* PACKAGES • ORAFCE external contrib module

• PostgreSQL alternatives • Custom functions


System Packages

Some of the system packages supported by PostgreSQL

PostgreSQL - ORAFCE

• DBMS_OUTPUT • DBMS_ALERT • DBMS_PIPE • DBMS_UTILITY • DBMS_ASSERT • DBMS_RANDOM


System Packages

Challenges in migrating DBMS_JOBS system package

Oracle PostgreSQL

• DBMS_JOBS • pg_agent


Packages

Packages

PACKAGES PostgreSQL Alternative

PACKAGE DEFINITION SCHEMA

GLOBAL VARIABLES TEMP TABLES

PACKAGE BODY FUNCTION(S)


Migrating Oracle Queries


Oracle Hints

Oracle Hints

PostgreSQL NO HINTS

Migration Impact

• HINTS are not supported in PostgreSQL - NOT REQUIRED

• Code changes required

• Functionality testing

• Performance testing and Benchmarking required


Oracle Outer joins

Oracle LEFT OUTER JOIN PostgreSQL LEFT OUTER JOIN

SELECT e.ename, d.dname 2 FROM emp e,

dept d 3 WHERE e.deptno = d.deptno(+);

SELECT e.ename, d.dname FROM emp e

LEFT OUTER JOIN dept d ON (e.deptno = d.deptno);

Migration Impact

• Application code change • Functionality testing


Oracle PostgreSQL SELECT sequencename.nextval from dual SELECT nextval(‘sequencename'); SELECT sequencename.currval from dual SELECT currval(‘sequencename’);

Sequences


Subqueries

Every subquery has to have an alias

Oracle PostgreSQL

select * from (select * from emp) select * from (select * from emp) as foo


Implicit Type Casting

Implicit type casting can be highly beneficial and can help reduced the need to change the application code to a greater extent

Increases the possibility usage of Indexes

Avoids the need to create function-based Indexes

Table PostgreSQL

select * from table where col1::int > 2;

create cast(varchar as integer) with inout as implicit;

select * from table where col1 > 2;

Oracle

select * from table where col1 > 2;

varchar varchar

2 Sydney

3 London

4 Singapore

3 London

4 Singapore

3 London

4 Singapore


Hierarchical Queries

Oracle Hierarchical Queries

PostgreSQL Hierarchical Queries

(There is no straight forward way)

Oracle PostgreSQL

• CONNECT BY, START WITH, SYS_CONNECT_BY_PATH and CONNECT_BY_ROOT

• WITH RECURSIVE • connectby() function part of tablefunc

contrib module • UNION and UNION ALL • pl-pgsql functions


Application Migration Challenges


JDBC Driver

Oracle JDBC Driver

PostgreSQL JDBC Driver

Migration Impact

AUTOCOMMIT OFF • Application behaviour is different in autocommit off mode • Multiple transactions will be automatically part of a transaction block, which means COMMIT ALL or

NONE • Heavy application code change may be required • Functionality testing • Performance testing • Changing legacy code might impose further more challenges


Monitoring

Automation

Integration

Database Environment


PostgreSQL Capabilities


Replication Slots

wal_keep_segments Earlier to 9.4

upgrade Replication Slots

9.4

WAL Optimization

PostgreSQL – 9.4, the amount of WAL data to be written has been reduced in-case of updates. This results in IO performance gain and reduction in the size of WAL files.

Example : When one column part of a 4 column row gets updated, only the updated part of the row is written to WAL files

Table WAL File (9.3)

Row 1

Row 2

Col 1 Col 2 Col 3 Col 4

Row 1

Row 2

WAL File (9.4 )

Row 1

Row 2



WAL Optimization

Table Size – 19GB

4 Columns

Updated 1 column

0

5

10

15

20

25

WALs (GB) in 9.3 WALs (GB) in 9.4

Series1


Questions ?

postgresql enterprise class features and capabilities

Technology