Working with GIS DataWorking with GIS DataLecture 1 of 2Lecture 1 of 2

Class 3Class 3 GISG 110 GISG 110

ObjectivesObjectives

Working with Spatial DataWorking with Spatial Data

Geographic data reviewGeographic data review

Linking features and attributesLinking features and attributes

Data formatsData formats

Project and data managementProject and data management

Working with ArcCatalogWorking with ArcCatalog

MetadataMetadata

Representing geographic featuresRepresenting geographic features

Vector (discrete)Vector (discrete)

RasterRaster

Data components:Data components:– Geography (representation)Geography (representation)– Attributes (characteristics)Attributes (characteristics)– Behavior rules (instructions)Behavior rules (instructions)

Point, Line, PolygonPoint, Line, Polygon

Rows + Columns Rows + Columns Equally sized cells Equally sized cells

Feature classesFeature classes

Feature classFeature class

Groupings of features (points, lines, Groupings of features (points, lines, polygons) with the same type of geometry polygons) with the same type of geometry

Can create point, line, Can create point, line, oror area feature classes area feature classes

Example 1: Highways, primary streets, and Example 1: Highways, primary streets, and secondary streets = “Roads” (lines FC)secondary streets = “Roads” (lines FC)

Example 2: Seven territories that house Example 2: Seven territories that house identical wildlife species = “Habitat” identical wildlife species = “Habitat” (polygon FC)(polygon FC)

Linking features and attributesLinking features and attributes

Feature classes are tables that store spatial Feature classes are tables that store spatial datadata

Each feature has a record in the tableEach feature has a record in the table

Spatial Data FormatsSpatial Data Formats

ArcGIS can work with spatial data in multiple ArcGIS can work with spatial data in multiple formats:formats:– RasterRaster– CADCAD– ShapefileShapefile– CoverageCoverage– GeodatabaseGeodatabase– Internet Map Service (ArcIMS)Internet Map Service (ArcIMS)– TablesTables

Shapefile Data FormatShapefile Data FormatSingle feature classSingle feature class

– Points or lines or polygonsPoints or lines or polygons

Attributes stored as dBASE (.dbf) tableAttributes stored as dBASE (.dbf) table

Consists of at least three files:Consists of at least three files:– Shapefile name.shpShapefile name.shp– Shapefile name.shxShapefile name.shx– Shapefile name.dbfShapefile name.dbf– Others may include .sbn and .sbxOthers may include .sbn and .sbx

If coordinate system is defined, the spatial If coordinate system is defined, the spatial reference is also stored in shapefile:reference is also stored in shapefile:

– Shapefile name.prjShapefile name.prj

ShapefileShapefile

Display quicklyDisplay quickly

Fully editable (coordinate and tabular in Fully editable (coordinate and tabular in ArcGIS)ArcGIS)

Simple structureSimple structure– Polygons do not share bounding linesPolygons do not share bounding lines

Coverage data formatCoverage data format

A folder containing multiple feature classesA folder containing multiple feature classes

Can store point, line, polygon FC and moreCan store point, line, polygon FC and more

Attributes stored in a separate INFO tableAttributes stored in a separate INFO table

Commonly found format (due to ArcInfo Commonly found format (due to ArcInfo market dominance)market dominance)

Data model more complexData model more complex

Display more slowly in ArcGISDisplay more slowly in ArcGIS

Coordinate data not editable in ArcGISCoordinate data not editable in ArcGIS

Polymorphic (point/line/polygon/annotation)Polymorphic (point/line/polygon/annotation)

CoverageCoverage

Polygons share bounding linesPolygons share bounding lines

Same topological rules can be built in Same topological rules can be built in geodatabasegeodatabase

ArcInfo Coverage organizationArcInfo Coverage organization

Coverages must be stored in an ArcInfo workspaceCoverages must be stored in an ArcInfo workspace

Workspaces contain an Workspaces contain an infoinfo folder folder

The The infoinfo folder stores information about the folder stores information about the attribute tablesattribute tables

Feature classFeature class Feature attribute tableFeature attribute tablePointPoint <cover>.PAT Point attribute table<cover>.PAT Point attribute table

ArcArc <cover>.AAT Arc attribute table<cover>.AAT Arc attribute table

Node*Node* <cover>.NAT Node attribute table<cover>.NAT Node attribute table

PolygonPolygon <cover>.PAT Polygon attribute table<cover>.PAT Polygon attribute table

*Node – the beginning and ending points of an arc, topologically inked to all arcs

that meet there

ArcInfo Coverage organizationArcInfo Coverage organization

To manage coveragesTo manage coverages

Only ArcGIS tools can be usedOnly ArcGIS tools can be used

Files in both the Files in both the infoinfo and coverage folders and coverage folders are requiredare required

Operating systems are unaware of this link Operating systems are unaware of this link (trying to view or copy coverage in Windows (trying to view or copy coverage in Windows Explorer won’t work)Explorer won’t work)

Geodatabase data formatGeodatabase data format

Stores spatial features and their attributes in the Stores spatial features and their attributes in the same RDMSsame RDMS– Different than SHP and COV (file-based)Different than SHP and COV (file-based)

Geodatabases are sets of feature classesGeodatabases are sets of feature classes– World.mdb comprises a polygon FC for countries, World.mdb comprises a polygon FC for countries,

a polyline FC of rivers, and a point FC of citiesa polyline FC of rivers, and a point FC of cities– Each geodatabase feature class only stores a Each geodatabase feature class only stores a

single feature typesingle feature typeStorage format: Storage format: – PersonalPersonal– MultiuserMultiuser

Personal and multiuser gdbPersonal and multiuser gdb

PersonalPersonal

Designed to store small to medium amounts of data Designed to store small to medium amounts of data (up to two gigabytes)(up to two gigabytes)

Accessed by several users at the same time, but Accessed by several users at the same time, but only edited by one person at a timeonly edited by one person at a time

MultiuserMultiuser

Suitable for large workgroups or enterprisesSuitable for large workgroups or enterprises

Have no size limitHave no size limit

Permit simultaneous editing by different usersPermit simultaneous editing by different users

Work with RDMS (Informix, SQL Server, Oracle)Work with RDMS (Informix, SQL Server, Oracle)

Geodatabase validationGeodatabase validation

ValidationValidation

Geographic features can be made to allow certain Geographic features can be made to allow certain types of editing, display, or analysis, depending on types of editing, display, or analysis, depending on circumstances the user definescircumstances the user defines

Two typesTwo types– Spatial validation (ArcEditor)Spatial validation (ArcEditor)

Topology or Geometric NetworksTopology or Geometric Networks

– Attribute validationAttribute validationUsed with tabular dataUsed with tabular data

Required for maintaining data integrity and efficiency Required for maintaining data integrity and efficiency during management, display and editingduring management, display and editing

GeodatabaseGeodatabase

11stst preferred vector format in ArcGIS preferred vector format in ArcGIS

Display very quicklyDisplay very quickly

Fully editable (coordinate and tabular) in Fully editable (coordinate and tabular) in ArcGISArcGIS

Can store many files from many formatsCan store many files from many formats

Store multiple feature classesStore multiple feature classes

Ability to store labels as annotationAbility to store labels as annotation

Create domains for attributesCreate domains for attributes

CAD file data formatCAD file data format

Computer Aided Design files (DXF, DWG, DGN)Computer Aided Design files (DXF, DWG, DGN)Typically product of engineering softwareTypically product of engineering software– E.g., utilities and infrastructureE.g., utilities and infrastructure

Logical collectionLogical collection– Access one or all feature class(es) at a timeAccess one or all feature class(es) at a time

Coordinate data is not editable in ArcGISCoordinate data is not editable in ArcGIS– Edit after exporting to GDB FC, coverage, or Edit after exporting to GDB FC, coverage, or

shapefileshapefileFrequently contain “sloppy” dataFrequently contain “sloppy” dataNo enforced topologyNo enforced topologyGaps in dataGaps in dataFrequently contain little or no useful attribute dataFrequently contain little or no useful attribute data

Converting tabular locations to a Converting tabular locations to a point FCpoint FC

Table with raw coordinates New point FC

Table with addresses New point FC

Add data to your map as…

Images and gridsImages and grids

Rows and columns of Rows and columns of equal-sized cellsequal-sized cells– Each cell stores a valueEach cell stores a value– Detail depends on cell Detail depends on cell

sizesizeGrids (ESRI native raster Grids (ESRI native raster format)format)– VAT: report # of cellsVAT: report # of cellsImages (TIFF, BMP, SID, Images (TIFF, BMP, SID, JPG, ERDAS)JPG, ERDAS)

GridsGrids

ImagesImages

Data Types (or Models)Data Types (or Models)

Two methods of representing geographic Two methods of representing geographic data:data:– Vector (points, lines, polygons)Vector (points, lines, polygons)– Raster (images and grids)Raster (images and grids)

Common Data SourcesCommon Data Sources

LocalLocal

www.sandag.org

www.sangis.org

San Diego local San Diego local governmentsgovernments

NationalNational

www.census.gov

www.usgs.gov

www.nationalatlas.gov

GlobalGlobal

www.geographynetwork.com

http://www.manifold.net/download/freemaps.html

OtherOther

www.gisdata.blogspot.com

Yahoo or Google search: Yahoo or Google search: keywords “free GIS data”keywords “free GIS data”

Project and data managementProject and data management

GIS strengths and weaknessesGIS strengths and weaknesses

Integrates large amounts of data in different Integrates large amounts of data in different formats from different sourcesformats from different sources

Frequently creates data management Frequently creates data management nightmaresnightmares

Effective data management is absolutely Effective data management is absolutely essentialessential

Common concernsCommon concerns

Where are the source files? Where are the source files? What types of files are they? What types of files are they? What projection/coordinate system are they stored What projection/coordinate system are they stored in? in? Are there multiple copies of the files? Are there multiple copies of the files? – If so, which are the most current or correct? If so, which are the most current or correct?

Where are new files automatically placed? Where are new files automatically placed? How do I move ArcMap documents between How do I move ArcMap documents between computers? computers?

These types of questions These types of questions mustmust be answered  be answered definitively for every project, regardless of the size definitively for every project, regardless of the size of the projectof the project

General system file managementGeneral system file management

GIS is more file-intensive than most applicationsGIS is more file-intensive than most applications

Knowledge of operating system (OS) file system is Knowledge of operating system (OS) file system is criticalcritical

Disk, directory, and file management skillsDisk, directory, and file management skills– Disk spaceDisk space– Directory structure and nomenclatureDirectory structure and nomenclature– File nomenclature, file sizes, multiple-file data File nomenclature, file sizes, multiple-file data

setssets

Where (file system-wise) is every file in a current Where (file system-wise) is every file in a current project?project?

Setting the working directorySetting the working directory

Before starting a new projectBefore starting a new project

Plan, plan, and plan some morePlan, plan, and plan some more

Determine default location for new filesDetermine default location for new files

Set working directorySet working directory– Avoids major headache laterAvoids major headache later– Have no doubts about where new files will be Have no doubts about where new files will be

placedplaced

TIPSTIPS– Create a new directory for each projectCreate a new directory for each project– Set working directory to the new directorySet working directory to the new directory

Renaming and copyingRenaming and copying

Use caution when renaming and copying filesUse caution when renaming and copying filesEvery file of a multiple-file data source Every file of a multiple-file data source (.shp, .shx, .dbf) must be renamed/copied(.shp, .shx, .dbf) must be renamed/copiedFiles that are renamed/copied will not be found in Files that are renamed/copied will not be found in map documentsmap documentsFiles (not layers) to be copied/renamed cannot be in Files (not layers) to be copied/renamed cannot be in use (being edited) in current project use (being edited) in current project Data sources are frequently composed of multiple Data sources are frequently composed of multiple individual filesindividual files– Shapefiles = at least 3 filesShapefiles = at least 3 files– Image data = at least 1 file, at most 4Image data = at least 1 file, at most 4

Renaming and copyingRenaming and copyingInstead of Windows Explorer, use ArcCatalog Instead of Windows Explorer, use ArcCatalog to rename/copy and paste shapefilesto rename/copy and paste shapefiles

Renaming & CopyingRenaming & Copying

ArcCatalog functionality applies to any ArcCatalog functionality applies to any supportedsupported data setsdata sets

Other files can be copied/moved/renamed using OS Other files can be copied/moved/renamed using OS file managementfile management– ImagesImages– Text filesText files– dBase files (not associated with shapefiles)dBase files (not associated with shapefiles)– CAD filesCAD files

All supported vector data can be converted and All supported vector data can be converted and managed as shapefiles or geodatabasesmanaged as shapefiles or geodatabases

ArchivingArchiving

Why archive?Why archive?

Effectively manages your dataEffectively manages your data

Saves/stores large amounts of project dataSaves/stores large amounts of project data– No longer in useNo longer in use– Periodically in usePeriodically in use

Frees disk spaceFrees disk space

Un-clutters directory structureUn-clutters directory structure

ArchivingArchiving

3 ways to archive3 ways to archive

Create new and secure backup directoriesCreate new and secure backup directories– Assuming your agency does not have own Assuming your agency does not have own

back up locations or archival systemsback up locations or archival systems

Write stable media (CD, DVD, Zip)Write stable media (CD, DVD, Zip)

Use archiving tool, e.g., WinZipUse archiving tool, e.g., WinZip

Copying and moving ArcMap Copying and moving ArcMap documentsdocuments

ArcGIS project files only contain pointers to ArcGIS project files only contain pointers to data locationdata location

File locations are “hard coded” into project File locations are “hard coded” into project filesfiles

Copying and moving map documents (MXD) Copying and moving map documents (MXD) does NOT copy or move data setsdoes NOT copy or move data sets

Copying and moving ArcMap Copying and moving ArcMap documentsdocuments

Map documents can lose track of the source dataMap documents can lose track of the source data– Source data gets moved (Source data gets moved (!!))

Use Source tab to change the data source for a layerUse Source tab to change the data source for a layer– Shortcut: Right-click the layer > Data > Set Data Shortcut: Right-click the layer > Data > Set Data

SourceSource

Layers with Layers with misplaced datamisplaced data

Click here to Click here to change the data change the data sourcesource

Copying and moving ArcMap Copying and moving ArcMap documentsdocuments

Data sets must be moved, copied, or specifiedData sets must be moved, copied, or specified

Data structures must be completely Data structures must be completely recreated, orrecreated, or

Map documents that are copied or moved Map documents that are copied or moved must be altered for new file locationsmust be altered for new file locations

Absolute vs. relative pathsAbsolute vs. relative paths

Absolute (full) pathsAbsolute (full) paths

C:\GIS\Project1\Boundary.shpC:\GIS\Project1\Boundary.shp

Everyone who uses MXD mustEveryone who uses MXD must– Be on same computer ORBe on same computer OR– Have data on their computer in exactly the same Have data on their computer in exactly the same

folder structurefolder structure

Relative pathsRelative paths

\Project1\Boundary.shp

Specify the location of the data contained in MXD, relative to the current location of the MXD

Relative pathsRelative paths

Enable the MXD and its associated data to be moved without the MXD having to be repaired (as long as same directory structure is used at new location)

Allow users to share maps easily

Data referenced by a relative path can be in the same folder as the MXD or in a folder above or below the MXD (recommend below)

\Project1\finalproject.mxd

\Project1\Boundary.shp or

\Project1\Data\Boundary.shp

Exporting dataExporting data

Why export?Why export?

To share data with othersTo share data with others– ArcGIS usersArcGIS users– Users of other softwareUsers of other software

Integration with desktop publishing Integration with desktop publishing applications (e.g., MS Word)applications (e.g., MS Word)

ArchivingArchiving

Converting data to shapefilesConverting data to shapefiles

Any Any supportedsupported vector data source can be vector data source can be converted to shapefileconverted to shapefile

Polymorphic data sources need to be Polymorphic data sources need to be converted to several shapefilesconverted to several shapefiles– CAD filesCAD files– ArcInfo coveragesArcInfo coverages

Shapefiles can be moved, copied, or Shapefiles can be moved, copied, or archived easily and safely with OS archived easily and safely with OS (ArcCatalog recommended)(ArcCatalog recommended)

Exporting tablesExporting tables

Tables can be exported to “generic” formatsTables can be exported to “generic” formats– dBasedBase– ASCII tab-delimitedASCII tab-delimited– INFOINFO

Table subsets can be exportedTable subsets can be exported– Only selected records are exportedOnly selected records are exported

Formats readable by other applicationsFormats readable by other applications

Use in spreadsheet or statistics softwareUse in spreadsheet or statistics software

Exporting tablesExporting tables

How to exportHow to export

In ArcMap, open table and select OptionsIn ArcMap, open table and select Options

From window, select Export and name new file From window, select Export and name new file (assumes same format)(assumes same format)

Exporting mapsExporting maps

File File Export Map Export Map

Available in Data or Available in Data or Layout viewLayout view

Specify output type, Specify output type, location and namelocation and name

Exporting mapsExporting maps

Exported images can be imported to desktop Exported images can be imported to desktop publishing applicationspublishing applications

ArcCatalogArcCatalog

Using ArcCatalogUsing ArcCatalog

Database design & managementDatabase design & management– Manage data: Copy, Rename, DeleteManage data: Copy, Rename, Delete

Provides a uniform view of all of your dataProvides a uniform view of all of your data

Define or modify table and feature class Define or modify table and feature class definitionsdefinitions

Three ways to view dataThree ways to view data

ArcCatalogArcCatalog

ContentsContents

PreviewPreview

MetadataMetadata

The Contents tabThe Contents tab

ArcCatalogArcCatalog

Large iconsLarge icons

ListList

DetailsDetails

ThumbprintsThumbprints– DataData– LayersLayers– MapsMaps

The Preview tabThe Preview tab

ArcCatalogArcCatalog

Geography or Geography or Table ViewTable View– Choose from Choose from

dropdown listdropdown list– Build custom Build custom

viewsviews

The Metadata tabThe Metadata tab

ArcCatalogArcCatalog

Create and display metadata in a variety of Create and display metadata in a variety of waysways

Introducing MetadataIntroducing Metadata

What is metadata?What is metadata?– Descriptive information Descriptive information

about dataabout data

Metadata tabsMetadata tabs– DescriptionDescription– SpatialSpatial– AttributesAttributes

Create and editCreate and edit

Display using different Display using different formats: FGDC, ISO, XMLformats: FGDC, ISO, XML

Connecting to foldersConnecting to folders

ArcCatalogArcCatalog

Connect to foldersConnect to folders

Connect to networkConnect to network

Disconnect from Disconnect from foldersfolders

Setting viewable dataSetting viewable data

ArcCatalogArcCatalog

Tools Tools OptionsOptions

Turn all data Turn all data types on or offtypes on or off

Creating thumbnail imagesCreating thumbnail images

Used by Contents thumbnail viewUsed by Contents thumbnail view

2.) Preview tab2.) Preview tab

1.) Choose objectChoose object

3.) Zoom/Pan3.) Zoom/Pan 4.) Create Thumbnail Icon4.) Create Thumbnail Icon

ReviewReview

1.1. ArcGIS can work with spatial data in multiple formats. (T/F)ArcGIS can work with spatial data in multiple formats. (T/F)2.2. Groupings of features (points, lines, polygons) with the same type of Groupings of features (points, lines, polygons) with the same type of

geometry are called ______.geometry are called ______.3.3. A shapefile is a folder containing multiple feature classes. (T/F)A shapefile is a folder containing multiple feature classes. (T/F)4.4. Attributes in the coverage format are stored in an INFO table. (T/F)Attributes in the coverage format are stored in an INFO table. (T/F)5.5. Both shapefile and coverage formats are composed of multiple files. (T/F)Both shapefile and coverage formats are composed of multiple files. (T/F)6.6. It is best to manage your coverage data with your operating system. (T/F)It is best to manage your coverage data with your operating system. (T/F)7.7. The _____ is the 1The _____ is the 1stst preferred vector format in ArcGIS. preferred vector format in ArcGIS.8.8. What two types of validations are available for the geodatabase?What two types of validations are available for the geodatabase?9.9. List two reasons to archive.List two reasons to archive.10.10. What is metadata?What is metadata?11.11. What are the three ways to view data in ArcCatalog?What are the three ways to view data in ArcCatalog?12.12. What keywords would you search on to find GIS data?What keywords would you search on to find GIS data?13.13. Absolute paths allow users to share maps easily by Absolute paths allow users to share maps easily by moving your data without

the MXD having to be repaired (as long as same directory structure is used at new location). (T/F)

Remainder of ClassRemainder of Class

BreakBreak

Lab Exercises:Lab Exercises:– ESRI Chapter 4 ESRI Chapter 4 Exploring ArcCatalogExploring ArcCatalog – Exercise 1: Chapter 4 questions and Exercise 1: Chapter 4 questions and

SANDAG and Census Bureau data SANDAG and Census Bureau data assignment (due 9/6/07)assignment (due 9/6/07)

Exercise Questions 4aExercise Questions 4a

1.1. How many feature classes are contained in the world How many feature classes are contained in the world geodatabase (world.mdb)?geodatabase (world.mdb)?

2.2. List the feature class of each:List the feature class of each:CitiesCitiesCountriesCountriesDisapp_areaDisapp_areaWorld30World30

3.3. Using the ID tool, what is the population (POP_CNTRY) of:Using the ID tool, what is the population (POP_CNTRY) of:PeruPeruIraqIraqDenmarkDenmarkSouth AfricaSouth Africa

4.4. What is metadata?What is metadata?5.5. How many records are in the countries feature class? What is How many records are in the countries feature class? What is

the Definition Source? Data type?the Definition Source? Data type?

Exercise Questions 4bExercise Questions 4b

1.1. Geographical searches use _____ and _____ Geographical searches use _____ and _____ coordinates to find data.coordinates to find data.

2.2. Sealfloor.tif is a _____ data set.Sealfloor.tif is a _____ data set.

3.3. Why would sea floor elevation in East Oceania be Why would sea floor elevation in East Oceania be useful in a map of where Earhart disappeared?useful in a map of where Earhart disappeared?

4.4. In seafloor.lyr, depth of water is indicated by the In seafloor.lyr, depth of water is indicated by the shade of _____; darker is _____ . White areas are shade of _____; darker is _____ . White areas are _____._____.

5.5. Layer files do not store symbology information. Layer files do not store symbology information. (T/F)(T/F)

Exercise Questions 4cExercise Questions 4c

1.1. How do you open ArcMap from ArcCatalog?How do you open ArcMap from ArcCatalog?2.2. A map document can hae one or many data A map document can hae one or many data

frames. (T/F)frames. (T/F)3.3. Briefly describe the difference between the Briefly describe the difference between the

Data view and Layout view.Data view and Layout view.4.4. What are the two ways to add layers to What are the two ways to add layers to

ArcMap?ArcMap?5.5. To change the name of a Data frame, you To change the name of a Data frame, you

can simply left-click on the name and type can simply left-click on the name and type the new name. (T/F)the new name. (T/F)