brian marwick several projects undertaken by manager ... · reliability no of addresses % 1 (gps...

Brian Marwick

Manager

Spatial Information Services

Several Projects undertaken by

LogicaCMG using Oracle Locator

Overview of Presentation

� LogicaCMG

� Building and maintaining PSMA Australia’s

national databases

� G-NAF and Transport

� Integrating Oracle Spatial into Smallworld

environments

3

LogicaCMG - Global Presence

United Arab Emirates

United Kingdom

Netherlands

New Zealand

Switzerland

Sweden

United States

Saudi Arabia

Indonesia

Canada

Germany

Belgium

Italy

Czech Republic

Australia

Hong Kong

Malaysia

Japan

France

Luxembourg

India

Ireland

Brazil

Norway

• Formed in December 2002 through the merger of Logica and CMG, the company has over 40 years of combined experience in the IT services arena.

• Listed on the London (FTSE 250 &

techMARK 100) & Amsterdam (AEX)

Stock Exchanges.

• A major international force in IT services and wireless telecoms with a leading position in Europe.

• 28,000 people in 36 countries.

• A broad portfolio of offerings across key industry sectors.

• A worldwide client base of blue-chip organisations.

4

LogicaCMG - Australia

Projects & Services

• IT services with YVW, CWW, TXU, Aurora, Transend,

Hydro, Eraring Energy, Pacific Power, Integral Energy…..

• SMARTBuy and e Health, NSW

• Gas Retail Market NSW, ACT, SA and WA

• Metering for TCA and Infomet

• G-NAF and Roads for PSMA Australia

• Customer information & billing at IE, AE & YVW

• SAP implementation support for >35 companies

• GIS at over ten utilities ( S’world, Intergraph, ESRI )

• Vicmap Property data management

• Department of Education and Training, Victoria

• nemPOWER with six utilities

• SCADA & SMS in electricity, gas and water

• Victorian Gas Market with OEAM/Envestra

• Alinta Gas Networks and Sales

• Commenced operations in 1974

• Industry Sector Focus

• Financial Services

• Government

• Energy and Utilities

• Industry, Distribution and Transport

• Telecommunications

• Strategic Growth

• Acquired MITS in late 2000

• Acquired eGlobal in 2002

• Merger with CMG on 1/1/2003

• Local Staff about 850

• IT skills in

• Systems Integration

• Managed Services

• Application development and management

• Field & Equipment Maintenance

5

� PSMA Australia Limited (Public Sector Mapping Agencies Australia) is an unlisted public company, established under the Corporations Act (2001), wholly owned by the state, territory and Australian Governments.

� PSMA Aust. draws on the public sector’s extensive, geospatial data resources to co-ordinate the assembly of, and facilitate access to fundamental national spatial datasets that deliver economic, environmental and social benefits to Australia

� G-NAF / Transport / Cadastral / Administrative boundaries databases

Client - PSMA Australia Limited

6

PSMA Australia’s Transport dataset

PSMA Australia Transport Dataset

ROADNAME

ROAD

TYPE

PERSISTENT

ID

CREATE

DATE

RETIRE

DATE

EXPERSISTENT

ID

SURFACE

TYPE

GROUND

RELATIONSHIP LOCALITY_1 LOCALITY_2

RIVERINA HIGHWAY 301 NSW501565482 19981029 501565482 1 3 ALBURY EAST ALBURY

UNNAMED 305 NSW501565025 19981029 501565025 3 3 ALBURY EAST ALBURY

WILSON STREET 305 NSW501565321 19981029 501565321 1 3 ALBURY EAST ALBURY

DELANY STREET 305 NSW501563446 19981029 501563446 1 3 ALBURY GLENROY

NORTH STREET 301 NSW501565782 19981029 501565782 1 3 ALBURY NORTH ALBURY



EDMONDSON AVENUE 305 NSW501565595 19981029 501565595 1 3 ALBURY NOT REQUIRED

WILCOX STREET 305 NSW501565168 19981029 501565168 1 3 ALBURY NOT REQUIRED

THURGOONA STREET 305 NSW501565307 19981029 501565307 1 3 ALBURY NOT REQUIRED

DEAN STREET 305 NSW501565169 19981029 501565169 1 3 ALBURY NOT REQUIRED

KIEWA STREET 305 NSW501565657 19981029 501565657 1 3 ALBURY NOT REQUIRED

UNNAMED 305 NSW501565073 19981029 501565073 3 3 ALBURY NOT REQUIRED

CRISP STREET 305 NSW501565573 19981029 501565573 1 3 ALBURY NOT REQUIRED

QUEEN STREET 305 NSW501565801 19981029 501565801 1 3 ALBURY NOT REQUIRED

GOLF VIEW DRIVE 305 NSW501565834 19981029 501565834 1 3 ALBURY NOT REQUIRED

STANLEY STREET 305 NSW501565286 19981029 501565286 1 3 ALBURY NOT REQUIRED

THURGOONA STREET 305 NSW501565477 19981029 501565477 1 3 ALBURY NOT REQUIRED

AVONDALE DRIVE 305 VIC5897193 20041209 5897193 1 3 WODONGA NOT REQUIRED








AWBURN STREET 305 VIC5894068 20041209 5894068 1 3 WODONGA NOT REQUIRED

BAELON STREET 305 VIC5891540 20041209 5891540 1 3 WODONGA NOT REQUIRED

PSMA Australia: G-NAF

• G-NAF is Australia’s first authoritative index of locality, street, number and geographic coordinate (geocode)

• Addresses in G-NAF have been supplied by the States and Territories of Australia, AEC and AUSP

• G-NAF has been built and now maintained using a

methodology developed by PSMA and implemented by LogicaCMG and Geometry

• The methodology uses a range of spatial and aspatial validation techniques

• First released in March 2004

• Updated every three months – 7 updates completed

9

CONFIDENCE GNAF_PID

LEGAL_

PARCEL_ID

FLAT_

NUMBER

NUMBER_

FIRST

STREET_

NAME

STREET_

TYPE

LOCALITY_

NAME

STATE_

ABBREVIA

TION

DATE_GEOCODE

_CREATED LATITUDE LONGITUDE

2 GANSW705382383 DP/28//708269 4 ILUKA AVENUE MOAMA NSW 20040415 -36.10550 144.75700

2 GANSW705382384 DP/27//708269 5 ILUKA AVENUE MOAMA NSW 20040415 -36.10540 144.75600

2 GANSW705382385 26//708269 6 ILUKA AVENUE MOAMA NSW 20040415 -36.10530 144.75600

2 GANSW705382410 DP/3//801078 11 KULGOA AVENUE MOAMA NSW 20040415 -36.10410 144.75600

2 GANSW705374532 DP/1//195161 13 SIMMS STREET MOAMA NSW 20040415 -36.11480 144.76100

2 GANSW705374534 DP/5/29/758686 16 SIMMS STREET MOAMA NSW 20040415 -36.11420 144.76200



2 GANSW705374538 SP/1//19738 1 20 SIMMS STREET MOAMA NSW 20040415 -36.11430 144.76200



PSMA Australia : G-NAF

10

Statistics – G-NAF (Feb 2006 release)


Reliability No of Addresses %

1 (GPS derived address level) 0 0%

2 ( Within address site boundary) 10,662,127 87%

3 (gap geocoded ) 195,951 2%

4 (street Level) 1,278,035 10%

5 (locality level) 207,429 2%

6 (topo level) 1,817 0%

no geocode 0 0%Total 12,305,359 100%

SUMMARY OF GEOCODE RELIABILITY OF PRINCIPAL ADDRESSES

Confidence level No of Addresses %

Confidence of 0 3,925,756 32%

Confidence of 1 2,253,763 18%

Confidence of 2 6,125,840 50%12,305,359 100%

SUMMARY OF CONFIDENCE OF PRINCIPAL ADDRESSES

11

Statistics – PSMA Data used for G-NAF


STATE

Roads

(segments)

G-NAF

(Principal Addresses)

Total Addresses from

Contributors LOCALITIES

NSW 1,016,489 4,226,893 8,803,421 4646

VIC 685,484 2,862,919 6,741,026 2948

QLD 312,739 2,519,369 5,144,668 3404

WA 508,047 1,217,910 2,743,021 1684

SA 506,780 919,865 2,049,991 1610

TAS 261,325 293,303 710,765 1107

ACT 24,344 178,089 433,747 104

NT 36,084 85,465 182,506 113

OT 1,917 1,546 1,657 0

3,353,209 12,305,359 26,809,145 15616

12

Technology Environment

• Dell 2650

• Windows 2003 Server

• Dell 2650 • Dell 2650 • Dell 2850

• Oracle Locator

• G-NAF Java Application

FME AutoCAD Map AutoCAD Map

Applications

13

Example of Processing for PSMA Roads

14


Determining localities in which road segments are located.

15


• Caulfield South

Ormond

10mm

• Road centreline

• Locality Boundary

16


• Caulfield South

Ormond

• Road centreline

• Locality Boundary

Buffer added to each locality to allow inclusion of road segments not exactly aligned to the locality.

Currently investigating the options to enable vertical topology

10mm

17

Data Processing for G-NAF

Datasets used

• G-NAF (addresses)

• Roads (road segments)

• Localities (polygons)

• CadLite (land parcels)

• Place Names

Gazetteer (all gazetted

Places names in Aust.)

Oracle Tables

Pre-processing

phase primarily

using FME

Core G-NAF

processing using

java app.

Post processing

phase primarily

using FME

18

Overview of G-NAF Methodology

2. Locality Name

1. Initial Validation

Is the address complete and

does it comply with the required

format? Rectify if possible

Are there any known address

inconsistencies? Apply rules

19


2. Locality Name

3. Street Name







20


2. Locality Name

3. Street Name4. Street Number







21

Examples of Spatial processing

• PSMA roads database stores individual segments between road intersections

22


ROADNAME ROADTYPE PERSISTENTID CREATEDATE LOCALITY_1

RIVER ROAD 302 NSW500168676 19980702 GREENWICH












RIVER ROAD 302 NSW505193609 20010604 LANE COVE








RIVER ROAD 302 NSW505193593 20010604 NORTHWOOD

RIVER ROAD 302 NSW500168808 19980702 ST LEONARDS





RIVER ROAD 302 NSW505139166 20010501 WOLLSTONECRAFT









23


• Individual segments for the entire length of River road are joined to create PSMA Road line complete

• Road line complete is then segmented into the respective locality polygons in which exists

24


• Creation of Street Locality Geocode for Street within Locality

25


NSW

• Locality polygons – 4646

• Addresses – 2.9 million

• Road Segments – 1.1 million

Point in polygon of each Jurisdictional address to ensure each address has the correct locality

26


Many complex polygons as a locality boundaries following cadastral boundaries, road centrelines and coastlines

27


Example of locality fix

Use Oracle to sub-set addresses and then undertake point in polygon using FME for each locality polygon.

28

Summary – PSMA Datasets

• Maintenance of PSMA datasets involves considerable spatial data processing

• Considerably flexibility has been required to address various data quality issues as they have been identified.

• The size of the larger State datasets initially impacted significantly processing speeds however the spatial functionality of Oracle Locator together with FME has allow the development of cost effective solutions

• Oracle Locator has provided a reliable platform to build and maintain the G-NAF and Transport datasets for PSMA using a range of software applications of our choosing.

29

Oracle in a Smallworld Environment

Case 1: Replacing Smallworld VMDS with Oracle Schema

• Replacing a Smallworld native database with an Oracle Schema using Smallworld’s Spatial

Object Controller (SOC) to treat Oracle as if it were a native dataset

– Richer range of tools for manipulating data in an Oracle Database.

– More accessible skills in the Oracle development space.

– Data could be maintained in-house using standard PL/SQL skills.

Smallworld GIS

SOC VMDS SOC OracleSOC VMDS

Data to replace

(cadastre)

New Dataset

(cadastre)

Asset data

30

Requirements

• Create New Schema in Oracle for cadastral data

• Load up Oracle Schema with New version of data

• Re-configure Smallworld Spatial Object Controller for new Dataset

• Decommission old Smallworld VMDS data partition

• Smallworld Features to be retained

–Richer Geometry types for Points and Text Labels

–Fast Drawing within the Smallworld Application

• Smallworld Features not required in this instance

–Geometry Topology

–Version Management

Case 1 : Replacing Smallworld VMDS with Oracle Schema

31

Solution:

• Store properties as extra fields in table.

• Use extra code (style methods) to render SDO points in Smallworld.

• Smallworld Point

–Coordinate (x,y,z)

–Angle

–Scale

–Mirror

• Oracle SDO_Point


Modelling Smallworld Points in Oracle


Other Possibility:

•Extend SDO Object model to handle other attributes

–Not compatible with the Smallworld interface to Oracle Geometry. Only standard types supported.

32

Modelling Smallworld Text in Oracle

Solution:

• Store missing properties as extra fields in Oracle.

• Use extra code (style methods) to render SDO points in Smallworld using the extra fields.

• Smallworld Label


–Angle

–Scale

–Draw Direction

–Justification Point 25 point

–String



Text

TextText

Text


Other Possibility:

•Extend SDO Object model

33

Speed of Refresh in GIS• Challenge

– Time to perform a screen refresh of Oracle data in Smallworld GIS similar to native geometry in VMDS.

• Observations

– Time to perform spatial query in Oracle faster than same query in VMDS.

– Creating Materialised Views of restricted data to be normally displayed in GIS created a significant performance improvement.

– Recreated R-Tree after large updates.

– Unable to cache the Oracle data in the GIS as data could have changed with each redraw

• Result:

– Achieved performance close to drawing native data


34

Case 2: Smallworld Mirror in Oracle

• Requirement to have all corporate databases available in Oracle.

• Other Benefits:

– Able to perform comparative queries of data in GIS and other corporate systems more efficiently

– Able to use other tools to access GIS data (eg MapInfo)

– Some types of queries are more efficiently handled by Oracle.

– Data became more accessible to a wider range of users.

35

Technology

• InSync

– Connects to Oracle using a Smallworld Spatial Object Controller

– Uses a checkpoint to note the differences between the current database version and when the last Synchronisation occurred

• Checkpoints: similar to a save point in Workspace Manager

– A mapping database is created to describe a translation between Oracle and Smallworld, tables and fields

Smallworld GIS VMDS SOCOracle SOC

current synch

last synch

Inserts, Updates, Deletes


36

Data Modelling

• Differences between Oracle and Smallworld table naming

– 30 character name length, special characters, reserved words

• Hidden tables in Smallworld

– Intermediate Tables, Discrete Lists, Topology, Configuration, Datamodel metadata

– Auto-generated table names and fields (eg. int!miscellaneous_miscellaneous5)

• Differences between geometry engines

– self intersecting sectors and other geometry errors, topology, extra properties of text and point geometry


37

Modelling Smallworld Text and Points

Solution:

• Store missing properties as extra fields in Oracle using a convention that can be handled by code.

• InSync modified to update the extra property fields in Oracle.

• Smallworld Label


–Angle

–Scale

–Draw Direction

–Justification Point 25 point

–String



Text

TextText

Text

Case 2 : Smallworld Mirror in Oracle

Other Possibility:

• Extend SDO Object model to handle other properties (new geometry types)

–Not compatible with the Smallworld/Oracle Access

methods

38

Summary – Oracle in Smallworld Environment

• Organisations are successfully implementing a variety of solutions to enable either:

– improved access by corporate databases to their established GIS’s.

– the incorporation of externally maintained data

• Some issues do exist with regard the management of text and other model issues however once identified these can be resolved

39

The End

brian marwick several projects undertaken by manager ... · reliability no of addresses % 1 (gps...

Documents