ats-16: making data count, krista nordback

Bike-Ped PortalThe National Online Non-motorized Traffic

Count Archive

Krista Nordback, Ph.D., P.E., Kristin Tufte, Ph.D.Morgan Harvey, Nathan McNeil

March 14, 2016Oregon Active Transportation Summit

Thank you to our partners!

Funded by NITC, ODOT, Oregon MPOs, Cities of Boulder and Austin, and FHWABend Metropolitan Planning Organization (Bend, OR), Mid-Willamette Valley Council of Governments (Salem, OR)Rogue Valley Council of Governments (Medford, OR), Cycle Oregon or Oregon Community Foundation, Eugene

Oregon Community Foundation

Introduction to Bike-Ped Portal• Why?• What is Bike-Ped Portal?• How to use it

How many bike and walk?• Surveys

• National• Regional• Local• Intercept• GPS

• Counts• Permanent• Short duration

• Manual• Automated

Source: Community Cycles

Why aggregate bicycle and pedestrian count data?

National Archive

Stat

e DO

T

Stat

e DO

T

National Archive

Stat

e DO

T

Stat

e DO

TRegional Data

National DataAg

greg

ated

Dat

a

Perfo

rman

ce M

etric

sAADB

National Archive

Stat

e DO

T

Stat

e DO

T

What is Bike-Ped Portal?

Bike/Ped Portal

Upload DownloadStorage

Data Checking(QA/QC)

Data Visualization

Bike-Ped Portal• Online database bp.its.pdx.edu• 5 million records loaded for 5 states • Upload/download data

Uploaded Data• 5 states• 12 counties• 343 road or path segments (including 278 in Oregon)• 355 detectors (both human and machine)• 38 million people counted

Bicycle65%

Other9%

Pedestrian25%

MODES

Facility Types

Roadway21%

Path45%

Sidewalk7%

Crosswalk1%

Cycle track1%

Bike Lane24%

General Activity Count1%

Facility Types

How to use it

bp.its.pdx.edu

Search

Data Download

12:00:00 AM

3:00:00 AM

6:00:00 AM

9:00:00 AM

12:00:00 PM

3:00:00 PM

6:00:00 PM

9:00:00 PM0

100200300400500600700800

01

Hawthorne Bridge

WeekendWeekday

Annual Average Daily Bicyclists (AADB)

2012 2013 20140

1,000

2,000

3,000

4,000

5,000 4,438 4,659 4,682

AADB

Manual Count Data

PedestrianData

Compare Bicycle & Pedestrian Data

• 12% of non-motorized traffic is walking at peak hour on south sidewalk

OtherBridges

Bend

18

Source: GoogleMaps

OtherBridges

AADB Comparison

2012 2013 2014 20150500

1,0001,5002,0002,5003,0003,5004,0004,5005,000

HawthorneKey Bridge

Source: David Patton

Next Steps

Phase II: Add “Explore Data” Page

Home Page

Password protectedOpen to anyone

About Page Add Data Page(User Dashboard)

Create Segment Area

Create Facility

Create Flow

Create Detector

Create Flow- Detector

Upload Data

Explore Data Page

Questions?To get involved contactKrista [email protected]

bp.its.pdx.edubp-demo.its.pdx.edu

mailto:[email protected]

Extra Slides

Hawthorne Bridge

2012 2013 2014 20150

5001,0001,5002,0002,5003,0003,5004,0004,5005,000

Hawthorne

Change over time

Jan Feb Mar Apr May June July Aug Sep Oct Nov Dec -

1,000

2,000

3,000

4,000

5,000

6,000

7,000

Hawthorne Bridge Bicycle Counts

2012 2013 2014 2015

20% decrease in Hawthorne bicycle counts after Tilikum Crossing opened (700/day)

Many Formats

Bike-Ped Portal Data Format

Why measure walking & biking?

If we don’t count it, it doesn’t count.

Why measure walking & biking?

•Funding & policy decisions•To show change over time•Facility design•Planning (short-term, long-term, regional…)

•Economic impact•Public health•Safety

What good are counts?• Funding!• Facility Level

• Change Over Time• Planning and Design• Safety Analysis

• Validate Regional Models• Prioritize Projects• Bicycle Miles Traveled (BMT)• Signal Timing

Signal Timing

Vehicle Delay

Kothuri, S. M., Reynolds, T., Monsere, C. M., & Koonce, P. (2013). Testing Strategies to Reduce Pedestrian Delay at Signalized Intersections. A Pilot Study in Portland, OR. Paper presented at the 92nd Annual Meeting of the Transportation Research Board, Washington, D.C.

Pedestrian

What?

People actually bike here?

Yes! 200 per day

What? People actually walk here?

Yes!

596 in a day

Conclusions

Conclusions• Data sharing makes the most of the data we have• Bicycle and pedestrian count data are complex• Designed for compatibility• Connecting a “Detector” with a “Flow” via a table adds

versatility to the schema– Allows archive to handle mobile counters– Allows multiple counts of the same flow/time (as for validation

counts)• Minimizing data in count data table

– Saves memory– Improves performance/efficiency

Next Steps in Phase I

• User data input interface• Automated upload• User data output interface• Basic QA/QC

Phase II and beyond

• Future Phases (unfunded)– Enhanced QA/QC– Analysis tools

• Summary Statistics• AADT from short duration counts• Integrating with weather data

Use Case – Mobile counters

The same detector can be associated with multiple facilities and flows (at different times).

Use CaseValidation Counts – Manual counts checking automated counter

• Multiple counts of the same flow at the same time with different “detectors”

DETECTOR 1

DETECTOR 2

Schema

Schema Elements

• Segment Area• Facility• Flow• Detector• Count Descriptor• Count Data

Segment Area

Segment Area

A segment area is a stretch of transportation right-of-way over which the volume of non-motorized traffic is not expected to substantially change.

Example Segment Area

Example Facility

Example Flow

Measured Flow: Westbound Bicyclists

Unmeasured Flow: East- and westbound Pedestrians

Example Detector

Detector

Schema

Seg-ment Area

Detectors

Facilities Flows

Count Descriptor

Count DataCD IdStart TimeMeasure PeriodVolume

Many Formats

Progress

Highlights• Online database bp.its.pdx.edu

• Upload non-motorized counts• Download online

• 4.8 million records loaded for 5 states• Demo-site • API

Records by Year19

9819

9920

0020

0120

0220

0320

0420

0520

0620

0720

0820

0920

1020

1120

1220

1320

1420

1520

1620

17

0

500,000

1,000,000

1,500,000

2,000,000

2,500,000

3,000,000

Data in Bike-Ped Portal by Year

Number of Records Total Volume

Count Data Sources

Bike-Ped Portal Database

Bike-Ped Portal Web Site

Semi-automated ftp uploads

Data Uploaded via Web interface

Raw data

Validateddata

Meta-data

Email with approval link (automated uploads)

Bike-Ped PortalSystem Architecture

Visual Validation Interface

Data Upload Interface

Data Upload Script

Rejected Data Automated

QA/QC Checks

Data Upload Summary

Krista Nordback

Comparing Automated to Manual

< 5% error

Segment Areas

Colorado3%

Oregon81%

Texas1%

Virginia7%

Washington8%

Segment Areas: 343 Total

Modes of Travel

Bicycle65%

Equestrian0%

Off-Road Motor Vehicles

0%

Other9%

Pedestrian25%

MODES

Display Data Options

Examples

CDOT OTIS

Portal

Philly

Portland

Arlington, VABike Arlingtonwww.bikearlington.com

http://www.bikearlington.com/

http://www.bikearlington.com/pages/biking-in-arlington/counting-bikes-to-plan-for-bikes/counter-dashboard/

Weather

Sunday Saturday


Sunday Saturday


Wednesday

7pmnoon7am

CDOT 2011 Bike-Ped Interface (AVID)

ExampleShort Duration Sites: 200Permanent Sites: 20Count Records: 30,000Peak Hour: 1,500Peak Day: 15,000Max AADB: 3,000

Selected Year: 2014Selected County: Multnomah

Short Duration Sites: 200Permanent Sites: 20Count Records: 30,000Peak Hour: 1,500Peak Day: 15,000Max AADB: 3,000

2012 2013 20140

5001,0001,5002,0002,5003,0003,5004,0004,5005,000

AADB

High Volume Site (Peak Hour > 60)

High Volume Site (Peak Hour 20 to 60)

Low Volume Site (Peak Hour < 20)Selected Metric: AADB

Questions for TAC on “Explore Data Page”• What information should be displayed

immediately?• How should counts be aggregated?• For which detectors should data be displayed?

• All detectors in the archive? • Just the detectors with the most recent data?

• Map• Should there be a map?• What should be on the map?

Phase II

Phase IIFHWA funded• AADT estimation for new sites

NITC funded • Basic Data Quality

• Quality notes from data provider• Improved data warning flags• Communication with data provider

• Explore Data Page• Usability

• edit metadata

• Maintenance• Data Wrangling

Need funding for• Manual data user interface• Input interface

improvements:• Draw segment area as

polygon• Intersection schema design • Intersection schema

changes• QA/QC enhancements• TMG format output/input

What is our purpose?• Calculate monthly or annual average (AADT, etc)• Chart trends over time• Made data accessible• Promote consistent and reliable bike/ped data• Prioritizing ped/bike projects• Other Purposes from TAC:

• Counter Maintenance• Corridor Analysis

Use Cases1. Calculate monthly or annual average counts2. Chart trends over time (year over year change)3. Make data accessible4. Show decision-makers the data5. Evaluate the effects of new infrastructure6. Compare to other communities7. Understand the impact of weather8. Compare to NHTS/ACS data9. Prioritize projects10. Crash exposure measures11. Corridor analysis

Use Cases1. Calculate monthly or annual average counts2. Chart trends over time (year over year change)3. Make data accessible4. Show decision-makers the data5. Evaluate the effects of new infrastructure6. Compare to other communities7. Understand the impact of weather8. Compare to NHTS/ACS data9. Prioritize projects10. Crash exposure measures11. Corridor analysis

Addressed in Tier 1

Portal

Demonstration Site

Daily Count

8/25/2009 10/14/2009 12/3/2009 1/22/2010 3/13/2010 5/2/20100

50

100

150

200

250

300

350

400

450

500

Daily Count

Hourly Count

0:002:00

4:006:00

8:0010:00

12:0014:00

16:0018:00

20:0022:00

0

40

80

120

160

Total

Future Summaryplots

Weekday Average Hourly Counts

Weekend Average Hourly Counts

Future Summary Plots

Average Daily Counts by Day of Week

Average Daily Count by Month

Ideas from EcoVisio

Segment Area

FacilitiesFlowsDetectors

Conceptual diagram of the pieces of the schema.

Segment area is the largest rectangular region. (Think the Hawthorne Bridge in Portland, OR)

Facilities are the smaller green rectangular regions.(Think sidewalks or bike lanes, for example)

The lines represent flows. The large black dots represent detectors.

Seg-ment Areas

Name, State,

County, TMG

direction, Functional

class, Speed limit,

National highway?, Route & number,

Observed land use, Start/end

dateGeometry

DetectorsShort NameOrganizationJurisdictionDescriptionMake/Model/Serial#Automated?

FacilitiesDescriptionTypeSidePaved?WidthOver or Underpass?Sharrows?Bike Route Signs?Bike boulevard?For bike lanes and cycle tracks:

- Color- Placement of color- Buffer

FlowsDirectionsMode•Pedestrian•Bicycle•Equestrian•Off-road Vehicles•Motor Vehicle•Other

DataCD IdStart TimeMeasure PeriodVolume

Count DescriptorsDetector ID, Flow

ID, Start/end dates, Location

(latitude/longitude)

Schema

• A Segment Area represents a segment of roadway with multiple facilities (lanes) and multiple modes of travel

Segment Area ER Diagram

Seg-ment Areas

Name, State,

County, TMG


class, Speed limit,



dateGeometry









Schema

Facilities ExampleZooming in on a cross-section of the Hawthorne Bridge Segment area, we see the individual facilities that make it up. In our context, a facility is a demarcated portion of the roadway, such as a traffic lane, bike lane or sidewalk. On the Hawthorne Bridge, the facilities include a north side shared use path, two westbound traffic lanes, two east bound traffic lanes, and a south side shared use path.

Shared use pathWestbound traffic lanesEastbound traffic lanesShared use path

Google Maps

• A Facility represents a facility along which people travel • People may use multiple modes of travel along a facility (bikes, walking, horses)• An example of a facility is a sidewalk or a bike lane • Path_type can be : 'roadway’, 'path/trail’, 'sidewalk’, 'cycle track', 'bike lane’, 'general activity count'

ER Diagram

Seg-ment Areas

Name, State,

County, TMG


class, Speed limit,



dateGeometry









Schema

Flows and Detector Example

Measured Flow: Eastbound Bicyclists


Detector

•A Flow represents a flow of vehicles – might be bikes, or peds or horses …•A Flow may be bi-directional or single-dirctional

ER Diagram

Seg-ment Areas

Name, State,

County, TMG


class, Speed limit,



dateGeometry









Schema

• Detectors represents a physical detector – no location information• Detectorid is key• Serial_num also identifies detector• Handles mobile detectors well

ER Diagram

Seg-ment Areas

Name, State,

County, TMG


class, Speed limit,



dateGeometry









Schema

count_descriptor

• Count Descriptor connects detectors and flows • A Count Descriptor represents an installation of a detector at a particular location – note location information (geom)• Designed to handle permanently installed detectors and mobile detectors

count_descriptor_id

ER Diagram

Seg-ment Areas

Name, State,

County, TMG


class, Speed limit,



dateGeometry









Schema

• The Data• Simple and sweet• No unnecessary attributes!

count_descriptor_id INTEGER

ER Diagram

count_descriptorcount_descriptor_id

ER Diagram

count_descriptor

count_descriptor_id

count_descriptor_id

Top three lines are for REFERENCE ONLY

Document Name

Header Row – must read as shown

The “Date-Time” is the start time for the time period during which count occurred. Date-time must be in the following format:

YYYY-MM-DD HH:MM:SS

Duration is the length of the time period during which counts occurred. Duration is a time field in the following format:

HH:MM:SS

The count is the number of road users of the type defined as the traffic “flow” which are counted by the “detector” during the given “duration” after the given “Date-Time.”

Bike-Ped Portal Data Format

Reference Lines

The first 3 lines of the upload file can be any text you choose:

• There is a limit of 1024 characters per line.

• Only use Column A

• These can be any text. It will be saved in the archive with the record of the upload.

• These lines are for REFERENCE ONLY

NOTE: The count will be linked to the Count Descriptor selected during the web upload process. The Reference lines are NOT used to link the count to a location or detector

Ensure only 1 flow per CSV file

Save File as CSVIf you are editing the file in Excel, save it as a CSV by1. File/Save As2. Click “Yes” in the dialog below.

CSV file opened in Excel CSV file opened in Notepad

Document NameRules for document names (aka file names):

• The file name must not contain only letters, numbers and underscores.

• No spaces or special characters ($,/,-, ^…).

• The file must end in *.csv.

• Don’t use names longer than 200 characters.

Other than that, the name is completely up to the person supplying the data.

We suggest that documents be labeled with some indication of what detector/station it is and some indication of the facility and flow of traffic. For example, “Hawthorne_bike_NE.csv” indicates the location name (Hawthorne), the traffic flow counted (bike) and the facility (N), and direction of travel (E).

Header Row

4th row is the header row which must read as shown:

• Date-Time,Duration,Count

NOTE: The upload script must find these rows in order to properly upload the data. Do not include any spaces.

Date-Time Column

Below the Header Row, each row of the “Date-Time” column represent the START TIME of the count.

The Date-Time column must be in the following format:

YYYY-MM-DD HH:MM:SS

Converting to the Date-Time Column formatTo convert your Date-Time column to the correct format:1) Select the relevant cells2) Click “Home” > “Number” > “Custom” as shown below3) Copy and Paste “YYYY-MM-DD HH:MM:SS” (without quotes) into the box below the work

“Type:”4) Select “OK”

Converting to the Date-Time Column format

Wrong Format Correct Format

Computing DurationDuration is the length of the time period during which counts occurred. For example, if 25 cyclists were counted between 5:30 PM and 6:00 PM, the duration would be 00:30:00.

If you only have start times counts in your file, duration can be calculated in Excel by subtracting the Date-Time in the following row from the Date time in the current row as shown below.

Converting to the Duration Column formatTo convert your Duration column to the correct format:1) Select the relevant cells2) Click “Home” > “Number” > “Custom” as shown below3) Copy and Paste “HH:MM:SS” (without quotes) into the box below the work “Type:”4) Select “OK”

Save File as CSVIf you are editing the file in Excel, save it as a CSV by1. File/Save As

Schema Review

Segment Area Example

Google Maps

Facilities Example

Shared use pathWestbound traffic lanesEastbound traffic lanesShared use path

Google Maps

Flows and Detector Example

Measured Flow: Eastbound Bicyclists


Detector

Seg-ment Areas

Name, State,

County, TMG


class, Speed limit,



dateGeometry









Schema

Count Data Sources

Bike-Ped Portal Database

Bike-Ped Portal Web Site

Semi-automated ftp uploads

Data Uploaded via Web interface

Raw data

Validateddata

Meta-data

Email with approval link (automated uploads)

Bike-Ped PortalSystem Architecture

Visual Validation Interface

Data Upload Interface

Data Upload Script

Rejected Data Automated

QA/QC Checks

Metadata Input User Interface

Live Demo

Draft UserInterface

Data uploads

ats-16: making data count, krista nordback

Government & Nonprofit