postgresql enterprise class features and capabilities
Post on 21-Jan-2018
548 Views
Preview:
TRANSCRIPT
Copyright 2016 FUJITSU LIMITED
I am..
Venkata B Nagothi
Call me..
VENKAT
Copyright 2016 FUJITSU LIMITED
Agenda
Migration Challenges (Oracle to PostgreSQL)
PostgreSQL capabilities
Copyright 2016 FUJITSU LIMITED
Migration Challenges
Design and Architecture challenges
Development effort required for successful database migration to PostgreSQL
Minimize development effort when migrating Applications
Copyright 2016 FUJITSU LIMITED
Migration Assessment
Infrastructure
Database Objects
Data Migration
PL/SQL Objects
Database Environment
Application
Copyright 2016 FUJITSU LIMITED
Infrastructure Migration
Copyright 2016 FUJITSU LIMITED
High Availability
Oracle Dataguard Broker
PostgreSQL Streaming Replication
Migration Impact
• More simpler to implement
• Supports all the protection modes in Oracle
• Supports cascading replication with some limitations
Copyright 2016 FUJITSU LIMITED
High Availability
Oracle Dataguard Observer
PostgreSQL pgPool-II
Master
Standby
Switches over when the
master crashes
pgPool-II Automatically promotes the
standby to be the new
master
pgPool-II Automatic failover disable
Application
Copyright 2016 FUJITSU LIMITED
High Availability
Role reversal – Limitations in PostgreSQL
Master
Standby (new master)
Streaming Replication
Standby can be promoted to Standalone new master
manually
Old master cannot be made standby. Needs to rebuilt
from new master’s backup
Copyright 2013 FUJITSU LIMITED
High Availability
PostgreSQL Streaming Replication – Limitations
Filesystem structure has to be identical
v
Master
Streaming Replication /data/tbs01
/data/tbs02
/data/tbs03
Standby
/data/tbs01
/data/tbs02
/data/tbs03
Copyright 2016 FUJITSU LIMITED
Tablespaces
Oracle Tablespaces
PostgreSQL Tablespaces
Migration Impact
• More OS dependent
• Re-Architect database disk architecture
• Performance testing and benchmarking
• Re-do Capacity Planning
Copyright 2016 FUJITSU LIMITED
Tablespaces
/disk1/datafile-1
/disk2/datafile-2
/disk3/datafile-3
Logical Structure
Logical Structure
Physical Structure
Physical Structure
/disk1/tbs01
Oracle Tablespace Structure PostgreSQL Tablespace Structure
Table 1 Table 2
Index 1
Table 1 Table 2
Index 1
Datafile 1 Datafile 2
Datafile 3
Copyright 2016 FUJITSU LIMITED
Tablespaces
PostgreSQL Disk Architecture
/disk1/tbs01
/disk2/tbs02
/disk3/tbs03
/disk4/tbs04
/disk5/tbs05
Each partition has to be an extendable partition for
better capacity management
A single partition over multiple disk spindles
(RAID)
/data/pgdata
Copyright 2013 FUJITSU LIMITED
Tablespaces
Limitations - PostgreSQL
Tablespaces alone cannot be backed up
Transportable tablespaces not possible
Tablespace restore/recovery is not quite possible
Copyright 2013 FUJITSU LIMITED
Databases
DB01
Instance 1
DB02
Instance 2
DB03
Instance 3
DB01
Instance 1
DB02
DB03
Oracle PostgreSQL
• Database Size
• Database performance
• Application
Architecture/behaviour
• Database operations
• Security
• Multitenancy
• Other business and
operational
constraints
Copyright 2016 FUJITSU LIMITED
Database Objects Migration
Copyright 2016 FUJITSU LIMITED
Schemas
Oracle Users
PostgreSQL Users & Schemas
Migration Impact
• Users and Schemas must be separately created in PostgreSQL
• A user can own multiple schemas
• Flexibility around changing ownerships
Copyright 2016 FUJITSU LIMITED
Database Objects
Oracle Objects PostgreSQL Objects
Materialized Views Materialized Views (limitations)
Index-Organized-Tables Cluster a Table (partial support)
Public Synonyms search_path
Global Temporary Tables Unlogged Tables
Copyright 2016 FUJITSU LIMITED
Data Types
Oracle Data type PostgreSQL Data Type
VARCHAR2 VARCHAR
NUMBER INTEGER
DATE TIMESTAMP
CLOB TEXT / JSON / JSONB
BLOB BYTEA / JSONB LO
Copyright 2016 FUJITSU LIMITED
Data Migration
Copyright 2016 FUJITSU LIMITED
Data Migration
BLOB
BYTEA (1 GB LIMIT)
LO
BLOB JSONB
Copyright 2016 FUJITSU LIMITED
Data Migration
Oracle BLOB
PostgreSQL BYTEA
(max limit 1 GB) Ora2pg (automatic)
Migration Impact
• Oracle BLOBs are automatically migrated to PostgreSQL bytea by
ora2pg
• Unstable Application performance can be expected which would hit the
benchmarking metrics
• Can accommodate more than 1 GB due to PostgreSQL compression
algorithm
• Triggers an Application code change
Copyright 2016 FUJITSU LIMITED
Application Impact (Example : JAVA)
getBytes() BYTEA
Migration Impact
• Only getBytes() function can be used against BYTEA data type in PostgreSQL
• getBytes() attempts to read the whole BYTEA into the memory at a time which will impose performance problems resulting in excessive usage of Memory and Network bandwidth
• Cannot stream BYTEA in small chunks
BYTEA Column
Size
Text File 10K
Text File 10K
Image 20M
Image 300M
Text file 100K
Data Migration
getBLOB() BLOB
Copyright 2016 FUJITSU LIMITED
Data Migration
Oracle BLOB
PostgreSQL LO Manual
Migration Impact
• Custom ETLs must be built
• Migration effort and time can be a challenge
• Application code change is expected
• Application functionality and Performance testing
Copyright 2016 FUJITSU LIMITED
Application Impact (Example : JAVA)
getBlob()
LO
Migration Impact
• getBlob() function is used against LO similar to Oracle
• getBlob() is the recommended approach for accessing LO (Large Objects)
datatypes from PostgreSQL
• Large Objects can be streamed
• BLOBs can be streamed in multiple chunks resulting in effective performance
Data Migration
BLOB
Copyright 2016 FUJITSU LIMITED
Data Migration
Oracle BLOB with JSON data
PostgreSQL JSONB Manual
Migration Impact
• Can result in an heavy Application design and code change
• Migration effort and time can be a challenge
• Heavy development can be on the cards
SELECT UTL_RAW.CAST_TO_VARCHAR2 select col->>‘value' FROM table;
Copyright 2016 FUJITSU LIMITED
Data Migration
Oracle CLOB with JSON data
PostgreSQL TEXT AUTOMATIC (Ora2pg)
Migration Impact
• Can result in an heavy Application design and code change
• Migration effort and time can be a challenge
PostgreSQL JSONB
Manual
Copyright 2016 FUJITSU LIMITED
PL/SQL Migration
Copyright 2016 FUJITSU LIMITED
PL/SQL Migration
PL/SQL Object PostgreSQL Alternative
Procedures Functions with RETURNS VOID
Functions Functions
Triggers Triggers
Packages Schemas + Temp tables + Functions
Copyright 2013 FUJITSU LIMITED
System Packages
SYSTEM PACKAGES PostgreSQL Alternative
DBMS* PACKAGES • ORAFCE external contrib module
• PostgreSQL alternatives • Custom functions
Copyright 2016 FUJITSU LIMITED
System Packages
Some of the system packages supported by PostgreSQL
PostgreSQL - ORAFCE
• DBMS_OUTPUT • DBMS_ALERT • DBMS_PIPE • DBMS_UTILITY • DBMS_ASSERT • DBMS_RANDOM
Copyright 2016 FUJITSU LIMITED
System Packages
Challenges in migrating DBMS_JOBS system package
Oracle PostgreSQL
• DBMS_JOBS • pg_agent
Copyright 2016 FUJITSU LIMITED
Packages
Packages
PACKAGES PostgreSQL Alternative
PACKAGE DEFINITION SCHEMA
GLOBAL VARIABLES TEMP TABLES
PACKAGE BODY FUNCTION(S)
Copyright 2016 FUJITSU LIMITED
Migrating Oracle Queries
Copyright 2016 FUJITSU LIMITED
Oracle Hints
Oracle Hints
PostgreSQL NO HINTS
Migration Impact
• HINTS are not supported in PostgreSQL - NOT REQUIRED
• Code changes required
• Functionality testing
• Performance testing and Benchmarking required
Copyright 2016 FUJITSU LIMITED
Oracle Outer joins
Oracle LEFT OUTER JOIN PostgreSQL LEFT OUTER JOIN
SELECT e.ename, d.dname 2 FROM emp e,
dept d 3 WHERE e.deptno = d.deptno(+);
SELECT e.ename, d.dname FROM emp e
LEFT OUTER JOIN dept d ON (e.deptno = d.deptno);
Migration Impact
• Application code change • Functionality testing
Copyright 2016 FUJITSU LIMITED
Oracle PostgreSQL SELECT sequencename.nextval from dual SELECT nextval(‘sequencename'); SELECT sequencename.currval from dual SELECT currval(‘sequencename’);
Sequences
Copyright 2013 FUJITSU LIMITED
Subqueries
Every subquery has to have an alias
Oracle PostgreSQL
select * from (select * from emp) select * from (select * from emp) as foo
Copyright 2016 FUJITSU LIMITED
Implicit Type Casting
Implicit type casting can be highly beneficial and can help reduced the need to change the application code to a greater extent
Increases the possibility usage of Indexes
Avoids the need to create function-based Indexes
Table PostgreSQL
select * from table where col1::int > 2;
create cast(varchar as integer) with inout as implicit;
select * from table where col1 > 2;
Oracle
select * from table where col1 > 2;
varchar varchar
2 Sydney
3 London
4 Singapore
3 London
4 Singapore
3 London
4 Singapore
Copyright 2016 FUJITSU LIMITED
Hierarchical Queries
Oracle Hierarchical Queries
PostgreSQL Hierarchical Queries
(There is no straight forward way)
Oracle PostgreSQL
• CONNECT BY, START WITH, SYS_CONNECT_BY_PATH and CONNECT_BY_ROOT
• WITH RECURSIVE • connectby() function part of tablefunc
contrib module • UNION and UNION ALL • pl-pgsql functions
Copyright 2016 FUJITSU LIMITED
Application Migration Challenges
Copyright 2016 FUJITSU LIMITED
JDBC Driver
Oracle JDBC Driver
PostgreSQL JDBC Driver
Migration Impact
AUTOCOMMIT OFF • Application behaviour is different in autocommit off mode • Multiple transactions will be automatically part of a transaction block, which means COMMIT ALL or
NONE • Heavy application code change may be required • Functionality testing • Performance testing • Changing legacy code might impose further more challenges
Copyright 2013 FUJITSU LIMITED
Monitoring
Automation
Integration
Database Environment
Copyright 2016 FUJITSU LIMITED
PostgreSQL Capabilities
Copyright 2016 FUJITSU LIMITED
Replication Slots
wal_keep_segments Earlier to 9.4
upgrade Replication Slots
9.4
WAL Optimization
PostgreSQL – 9.4, the amount of WAL data to be written has been reduced in-case of updates. This results in IO performance gain and reduction in the size of WAL files.
Example : When one column part of a 4 column row gets updated, only the updated part of the row is written to WAL files
Table WAL File (9.3)
Row 1
Row 2
Col 1 Col 2 Col 3 Col 4
Row 1
Row 2
WAL File (9.4 )
Row 1
Row 2
Copyright 2016 FUJITSU LIMITED
Copyright 2016 FUJITSU LIMITED
WAL Optimization
Table Size – 19GB
4 Columns
Updated 1 column
0
5
10
15
20
25
WALs (GB) in 9.3 WALs (GB) in 9.4
Series1
Copyright 2013 FUJITSU LIMITED
Questions ?
top related