spatial analysis in government lecture 2- data & information

Spatial Analysis in GovernmentLecture 2- Data & Information

By Guy Thigpen, M.Phil., MUSA

Purpose of Lecture

Review the Data and Information contained in GEODB2

Explore an example (ULRS) of the information available in the data warehouse

Examine the data structure of a parcel layer

Frameworks of Understanding

Spatial Analysis

GeographySystems Thinking

Cartography

Philadelphia’s Data Warehouse

Data and Information

282 feature files (tables, points, etc)

20 departments provide information

26 geodata folders with additional feature files

GEODATABASEA database

that is in some way referencedto location on the earth.

VECTOR DATAspatial data represented. as

points, lines and polygons

ATTRIBUTE DATAadditional information about each spatial feature housed

in tabular format.

RASTER DATAcell-based data such as aerial imagery and digital elevation

models

METADATA Documentation of GIS datasets.Contains information such as,

•the coordinate system, •when the data was created,•contact of who created it •definitions for any of the code attribute data.

ArcGIS Services

• Caching

One of the main characteristics that makes the server very capable of publishing maps over web is Caching. Additionally, we have more control over how the cache gets built and updated. Caching particular areas on the map and On Demand caching are the other main features.

• Security

Management of services with ArcGIS Server's role based security has been a major advantage. Also, the product supports token based security for web services and applications.

• Imagery

ArcGIS Server 10.1's Imagery capabilities enable us to handle large amount of GIS data and provides access to it enterprise wide. It also allows processing and analysis of images on the client side.

ESRI Arc GIS Server 10.1

Some of the services that we provide include, City Aerial Photography, City Hybrid Map, Parcel Layer, Building Footprint Layer, Septa Lines etc.

All these services can be overlaid upon each other according to the user's choice in any

web application or desktop around the city. 

Pictometry

Pictometry is the leading provider in digital, geo-referenced, oblique aerial imagery and innovative geospatial technologies.

ORTHO Shot straight down and oriented North-up. 

OBLIQUEshot at a 40~45 degree angle. Individual image is available from North, South, East or West.

CAPTURE AREA: The oblique imagery is captured for the City

of Philadelphia, including • center city area, • the entire Philadelphia International

Airport, which is located in Delaware County, and

• the Wyndmoor portion of Montgomery County.

With Pictometry Electronic Field Study (EFS) software, licensed users are allowed to view and work with the aerial images contained in the Pictometry Image Library

SPECIFICATION:RESOLUTION:  Pictometry offers two levels of resolution, “community” level and “neighborhood” level. The average resolution of oblique imagery at community level (shot from around 4,500 ft) is 1-foot per pixel, and at neighborhood level (shot from around 3,500 ft) is 4-inch per pixel.

DATE/ TIME STAMP : Each image is date and time stamped with the time of capture as reported by the time information transmitted from the Global Positioning System.

How to make a connection to GEODB2?

1) In ArcCatalog:

The easiest way to create a connection to the central data warehouse, is by using ArcCatalog and connect to the ArcSDE application server (aka. Geodb2). Geodb2 will broker all your operations to the database.

2) Connecting with Oracle Client:

The central warehouse is in essence an Oracle database, and for power users out there who perform more complex operations and queries, you can connect to the warehouse directly using any Oracle Client .

GIS Spatial Data Sharing and Distribution

• In general the staff and departments who have been in the forefront of the development of the City’s spatial data support the sharing of this data within the City and to the public where appropriate and where the City’s costs to provide this data access can be reimbursed.

• The interns at the Philadelphia redevelopment Authority are also provided access to the database .

Publically Available Data from GEODB2@http://maps11.phila.gov/EnterpriseGIS/Downloads.aspx

Philly GIS Forum • Bring together the GIS users in all the departments of City of Philadelphia.• The topics discussed here are expected to get attention from the concerned users

who may make posts pertaining to the discussion.• Threads can serve as future reference for various issues should the need arise.

Group structure with restrictions:• Threads in the forum pertaining to certain specific topics will be having access

limitations; consequently, the forum users will be divided into groups according to the topics that are discussed and these groups will be having access those threads. One user can be present in multiple groups depending on the topics he is engaged in. Another one will be the read only restriction for certain threads and for certain users.

• Solutions can be found soon without numerous email tags and phone call exchanges as the problems are viewed by the community.

• Betterment in speed, resource utilization and problem-solving of city-wide GIS analysis and development.

Data and Information Process

Regulation

ComplianceStandard

DistributionMedium

Resources Committed

Primary DataCapture

DataProcessing

DATA InformationKnow-ledge

Under-standing WISDOM

Data and Information The foundation of an analysis include DATA and INFORMATION.

The City of Philadelphia has solid foundation of data and information.

W = Wisdom U = Understanding K = Knowledge 

I = Information (URLS parcel layer and batch geocoder) D = Data (various shapefiles geospatially located) 

O CPS IOutputs CustomersProcessesSuppliers Inputs

COMPLIANCESTANDARD:

Programs/projects are required or encouraged by

federal, state, and local sources to

collect data

carried out by contractors and city agencies

DATA:

Programs and projects

produce data

DATAPROCESSING:

Data is uploaded, stored and

collated to be available as

data or information

PLATFORM:

Open Data Philly

PASDA

phila.gov/map

Public

Clients

City

Agencies

Data and Information Relates to Analysis

Data

InformationBasic

Analysis

Knowledge

Understanding

Wisdom

Parcel Polygons

ULRS: Information in GEODB2

A simple task – displaying an address spatiallyWhat do we know about an address?

- What are the sources for that address?- (process of standardization)

- What parcel from the Department of Records does this address belongs too?- (process of geocoding)

Address standardization

An ADDRESS_ID is any unique combination of

STREET_NUMBERSTREET_NUMBER_SUFFIXSTREET_PREFIXSTREET_NAMESTREET_TYPESTREET_SUFFIX

Incoming addresses are scrubbed to one or more ADDRESS_IDs

(~ similar to LOC field in REAL_ESTATE table)

!!! ADDRESS_ID is an artifact of the ULRS !!!

STREET_CENTERLINE + any other street known by OPA informs the “valid” streets

Parcel geocodingFor each ADDRESS_ID, assigned to active DOR parcel

2 methods- by direct reference to DOR: if the address is in the parcel layer- by estimation: if the address is not in the parcel layer, ULRS will perform “educated guess”

!! Not every ADDRESS_ID can be associated with a parcel !!

Relations in ULRS

PARCEL

145N140168

026S050256

068N220032

070N090134

...etc...

ADDRESSES

1234 MARKET ST

2700 SPRING GARDEN ST

1206 NAUDAIN ST

...etc….

BRT Tax Accounts

023150808

453137110

883333239

...etc...

LNI Hanssen Address Keys

141144

141145

141147

141148

..etc...

PWD WaterRev

757315026001

5614004537001

8088001300002

...etc...

NON-SPATIALSPATIAL

Process of address standardizationProcess of geocoding

ULRSDB

(“address switchboard”)

BRT

LNI Hanssen

Water Revenue

GIS web services

Other spatial data (e.g. police district boundaries, trash

pickup, …)

CityMaps

BRT Web

Zoning Archive

LegistarStreets

Workorder 311

DHS Facts 2

Map Plugin

CRS Online

LNI Workorder

311

Online Zoning Map

Batch Geocoder

BUILD

OHCD Prop lookup

FPC Treemanag

ement

Crime Map

Police PHA interface

PhiCAMS

PWD Doc managemernt

PWD Plumming

PWD workorder

311

ParcelExplorer

Conveyance Fraud Tracker

Revenue Online

RESA

Streets Snowcategory

Pothole Reporter

Address Explorer

Unified Land Records System

Authentication Geocoding

Reverse Geocoding

Land Record Extraction

ULRS system lookups

Service area lookups

Generate map thumbs

Address standardization &

verification

Published Data(~reports~)

MASTER_ADDRESSMASTER_ADDRESS_ALL

PARCEL_ULRSBRTREALESTATE_DORPARCELMASTER_ADDRESS_BRTPICT

….(upon request)

….

(monthly-ish)

(Weekly)

ULRS access?

Exposed through various means

» Webservices + their clients» Ready-made applications: AddressExtractor,

AddressExplorer» bulk data reports in the “GIS_ULRS_DATA” schema

Web Services

• = developer technology• Two flavors

– CityMaps Webservices• “generic GIS” functionality: • relies on ULRS for only two pieces: the address cleanup logic &

parcel-based geocoding

– ULRS WebServices• Main window in the ULRS.

– Allows you to extract any information that is stored in the ULRS

• Provides simple read access to other db systems.

Web Services

• Who uses the webservices?– Around 25 clients from various departments: lni, police, revenue, dot, brt, ohcd, dhs, water, sherrif’s

office,…– 2,000,000+ lookups against the web services per month– Supports web sites + batch processing– Majority of ULRS clients do not rely on “geocoding logic”

• mainly to resolve address input• switchboard to retrieve account information based on address

(!!! after migration to ArcGIS Server, some functionality was lost!!! )

Existing ApplicationsAddress Explorer

Replaces the ULRS Explorer – which was discontinued after June 30, 2009.

Why the rename? - address is central – not the parcel, not the BRT#,…- also, underline the fact that ULRS does not store any other information other than relationships between addresses & parcels on the one hand, and addresses & accounts on the other hand. (nothing more, nothing less)

Mimic three basic ways of retrieving information from the web services- by address- by parcel #- by account # (topic key)

~corresponds to 3 ways of organizing the data

Existing ApplicationsBatch Geocoder/Address Extractor

-Allows you to process address based data

- Standardize; clean up an address(= most basic operation)- Validate: clean up an address & check if imported by ULRS- Geocode: clean up an address + find location- Extract land records (~~~ topic pane from Address Explorer)- Extract DOR parcel info: retrieve parcel identifiers by address

Data Reports- bulk delivery of ulrs data under GIS_ULRS_DATA on geodb2 (on monthly basis)

- Replaces ulrs publisher: low usage + data not always meaningful

GOALS-organize data in meaningful way-secure core applications-flexible schema

- data added upon request: make sure that every report actually has a user- not recommended for application use – but not enforced either.

-SO FAR:-ALIAS_ADDRESS-MASTER_ADDRESS-MASTER_ADDRESS_ALL-MASTER_ADDRESS_BRT_PICT-PARCEL_ULRS-BRTREALESTATE_DORPARCEL

This Square

Exploring DATA: how it is generated

Legal Description and Survey Plan

Consistency - some legal descriptions do not indicate streets and just use bearings where others do indicate the streets with the bearings.

Understanding Bearings

Example- South 03 degrees 42 minutes 30 seconds East S03º42’30”E

• 1: First it gives the rotation of the line with respect to the geographical north in degrees, minutes and seconds. It is in the middle part between the directions.

• 2: Second it gives the direction of the line from the starting point to the end point in one of the four predefined ways - South to East, South to West or North to East, North to West. These directions are at the start & end.

Reading Bearings From a Legal Description

55º

15º

60º

20º

Bearing Definition

NENW

SESW

55º43’59” 15º21’45”

60º10’25” 20º46’12”

N

S

W E

O

g

b

W

L

LO = N 55º43’59” WOL = S 55º43’59” E

bL =?Lb =?

gL =?Lg =?

How to Define Bearings

Each Segment should have bearing

Angles in Bearings are never greater than 90º

Bearings are referenced from North or SouthFour predefined ways - South to East,

South to West

North to East, North to West

A Parcel Should have Bearings either in a clockwise or Anticlockwise Direction

Minutes & Seconds should not be greater than 60º

bL = S 15º21’45” WLb = N 15º21’45” E

gL = N 60º10’25” WLg = S 60º10’25” E

How to Check Bearing of Parcel

Example: Parcel 61B

Either – 74.417’Or – 74’5”

Missing Bearing

Two Bearing Segments Missing from Legal Description

Missing Segments Assumed At Right Angle

Bearing To Be Verified

2nd Deed

2nd Shape in identified parcel

Final Remainder

This Square

• ALL THAT CERTAIN lot or piece of ground Situate in the 6th Ward of the City of Philadelphia and described as follows:

• Beginning at a point of intersection of the Southeasterly side of 46th Street ( ) and the Southeasterly side of Fairmount Avenue (60 feet wide);

• Thence extending N86°17'30"E, along the Southeasterly side of Fairmount Avenue a distance of 243.417 feet;

• Thence extending S03°42'30"E, along a line at right angles to said Fairmount Avenue a distance of 249.000 feet;

• Thence extending S86°17'30"W, along a line parallel with said Fairmount Avenue, a distance of 243.417 feet to a point on the Southeasterly side of said 46th Street;

• Thence extending N03°42'30"W, along the said Southeasterly side of said 46th Street, a distance of 249.000 feet to the point of beginning.

• Containing 60,610.83 square feet or 1.3914 acres, more or less.

• BEING KNOWN AS:?

Remainder Legal Description

This Square

The resulting shape in GIS layer

Next week’s Topic: The Front Door

We will discuss the challenges and analysis that led to the implementation of the ‘Front Door’