Transportation leadership you can trust.

presented topresented to

Florida Model Task Force Florida Model Task Force Model Advancement CommitteeModel Advancement Committee

presented bypresented by

Robert G. Schiffer, AICPRobert G. Schiffer, AICPThomas F. RossiThomas F. RossiCambridge Systematics, Inc.Cambridge Systematics, Inc.

Yongqiang Wu, P.E. Yongqiang Wu, P.E. Florida Department of TransportationFlorida Department of Transportation

November 28, 2007November 28, 2007

FSUTMS-Cube Calibration StandardsDraft Guidelines and Standards

2

Presentation Overview

Literature Review

Default Guidelines and Standards(LRTP transit models)

• Checking Input Data

• Trip Generation

• Trip Distribution

• Mode Choice

• Trip Assignment

Validation for Other Model Applications

Discussion on Volume-over-Count Ratios

Next Steps

3

Project Overview

Follow-up to Phase I Study on model parameters• Prepared summary of NHTS statistics for potential model use

• Identified adjustable parameters and potential sourcesfor defaults

• Final report is available at the FSUTMSOnline web site − http://www.fsutmsonline.net/images/uploads/mtf-files/FSUTMS-

Cube_Parameters.pdf

Phase II Study on calibration standards includes 4 subtasks• Literature Review (complete)

• Model Calibration/Validation Guidelines and Standards (today)

• Best Practices for Model Calibration/Validation (next)

• Documentation – 1) Calibration and Validation Standards;2) Best Practices for Model Validation; 3) Guidelines forModel Application

4

Project Overview (continued)

Calibration vs. validation

• Calibration – process where models are adjusted to simulateor match observed travel behavior in the study area

• Validation – procedure used to adjust models to simulatebase-year traffic counts and transit ridership figures

Standards vs. guidelines/benchmarks

• Standards – desirable accuracy levels for comparing estimated versus observed metrics

• Benchmarks – documented statistical ranges from literature review, model outputs, NHTS, etc.

Literature review

• 60+ documents reviewed – specific modelsand reference reports

5

Default Guidelines and StandardsChecking Input Data

Socioeconomic data

• Visual comparisons

• Statistical comparisons

• Regionwide comparisons(below)− Persons per

DU (or HH)

− Employment/population ratio

− Autos/DU (or HH)

CountyCounty Census DataCensus Data NERPM DataNERPM Data

   20002000 20032003Percent Percent

DifferenceDifference 20002000 20052005Percent Percent

DifferenceDifference

Duval Duval 779,618779,618 817,480817,480 5%5% 762,674762,674 810,493810,493 6%6%

ClayClay 141,671141,671 157,502157,502 11%11% 139,036139,036 167,020167,020 20%20%

NassauNassau 57,90357,903 61,62561,625 6%6% 56,89756,897 64,69564,695 14%14%

St JohnsSt Johns 124,458124,458 142,869142,869 15%15% 120,738120,738 150,084150,084 24%24%

TotalTotal 1,103,6501,103,650 1,179,4761,179,476 7%7% 1,079,3451,079,345 1,192,2921,192,292 10%10%

StatisticStatistic

Benchmarks/SettingsBenchmarks/Settings

LowLow HighHigh

Regionwide Persons/DU (or HH)Regionwide Persons/DU (or HH) 2.02.0 2.72.7

Regionwide Employment/Population RatioRegionwide Employment/Population Ratio 0.450.45 0.750.75

Regionwide Autos/DU (or HH)Regionwide Autos/DU (or HH) 1.751.75 2.102.10

Approximate Population/TAZApproximate Population/TAZ N/AN/A 3,0003,000

6

Default Guidelines and StandardsChecking Input Data (continued)

Highway network data

Transit network data

Highway and transit speed data

• Hierarchy

• Balance highway and transit

Terminal times

• Hierarchy

• Phase I Report

7

Default Guidelines and StandardsTrip Generation

Aggregate trip rates

• Person trips/TAZ

• Person trips/person

• Person trips/DU (or HH)

• HBW person trips/employee

Total unbalanced attractions versus productions by purpose

• Preferred +/-10%; acceptable in some instances +/-50%

Percent external-external trips by zone/station

• Great variation expected (4-21 percent range documented)

Statistic

Benchmarksa

Low High

Person Trips/TAZ N/A 15,000

Person Trips/Person 3.3 4.0

Person Trips/DU (or HH) 8.0 10.0

HBW Person Trips/Employee 1.20 1.55

a Generally excludes nonmotorized trips; including motorized trips could increaseperson trips per DU up to 11.5.

8

Default Guidelines and StandardsTrip Generation (continued)

Percent trips by purpose

Statistic

Benchmarks

Low (Percent) High (Percent)

Percent Trips by Purpose – HBW 12 24

Percent Trips by Purpose – HBSH 10 20

Percent Trips by Purpose – HBSR 9 12

Percent Trips by Purpose – HBSC 5 8

Percent Trips by Purpose – HBOa 14 28

Percent Trips by Purpose – HBNWb 45 60

Percent Trips by Purpose – NHBc 20 33

a HBO includes a variety of special trip purposes depending on the model (e.g., airport, college, and shop).

b HBNW accounts for all home-based trip purposes except HBW.

c NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate.

9

Default Guidelines and StandardsTrip Distribution

Average trip length by purposeTrip length frequency distributions by purposeCoincidence ratios by purpose – measures the percent of areathat coincides for two triplength frequencies

Statistic

Benchmarks

Low High

Average Trip Length – HBW (minutes) 12 35

Average Trip Length – HBSH (minutes) 9 19

Average Trip Length – HBSR (minutes) 11 19

Average Trip Length – HBSC (minutes) 7 16

Average Trip Length – HBOa (minutes) 8 20

Average Trip Length – NHBb (minutes) 6 19

Average Trip Length – IE (minutes) 26 58

Statistic Standards

Mean Trip Length, Observed Total Trips +/-3%

Trip Length Frequency Distribution versus Observed

+/-5%

Coincidence Ratios by Purposec 70%a HBO includes a variety of special trip purposes, depending

on the model (e.g., airport, college, and school).

b NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate.

c Some lower coincidence ratios have been deemed acceptable for trip purposes that had relatively few tripsand therefore higher error rates.

8%8%

6%6%

4%4%

2%2%

0%0%

Percent of Total TripsPercent of Total Trips

Travel Time (in Minutes)Travel Time (in Minutes)0 10 20 30 40 50 60

Coincidence Ratio = 0.82Coincidence Ratio = 0.82

Estimated (ATL = 18.2 Min) Estimated (ATL = 18.2 Min)

Observed (ATL = 18.9 Min)Observed (ATL = 18.9 Min)

10

Default Guidelines and StandardsTrip Distribution (continued)

Percent intrazonal trips by purpose

Node-point charts

• Zone-based

• Number of trips

• Trip productions/attractionsby purpose

StatisticStatistic

BenchmarksBenchmarks

LowLow HighHigh

Percent Intrazonal – HBWPercent Intrazonal – HBW 1%1% 4%4%

Percent Intrazonal – HBSHPercent Intrazonal – HBSH 3%3% 9%9%

Percent Intrazonal – HBSRPercent Intrazonal – HBSR 4%4% 10%10%

Percent Intrazonal – HBSCPercent Intrazonal – HBSC 10%10% 12%12%

Percent Intrazonal – HBOPercent Intrazonal – HBOaa 3%3% 7%7%

Percent Intrazonal – NHBPercent Intrazonal – NHBbb 5%5% 9%9%

Percent Intrazonal – Total TripsPercent Intrazonal – Total Trips 3%3% 5%5%

StandardsStandards

StatisticStatistic AcceptableAcceptable PreferablePreferable

Percent Intrazonal, Observed Total TripsPercent Intrazonal, Observed Total Trips +/-3%+/-3% +/-5%+/-5%

a HBO includes a variety of special trip purposes, depending on the model (e.g., airport, college, and school).

b NHB includes combined purposes for NHB Work and NHB Nonwork, where appropriate.

11

Default Guidelines and StandardsMode Choice

Mode split targets (ideal)• Trip purpose• Mode• Auto ownership level• Geographic subarea

ModeModeZero-Vehicle Zero-Vehicle HouseholdsHouseholds

One-Vehicle One-Vehicle HouseholdsHouseholds

Two-Vehicle Two-Vehicle HouseholdsHouseholds

Three-Vehicle Three-Vehicle HouseholdsHouseholds

WalkWalk 5,0005,000 6,0006,000 4,0004,000 3,0003,000

BikeBike 2,0002,000 1,0001,000 500500 200200

Drive AloneDrive Alone -- 130,000130,000 350,000350,000 200,000200,000

Shared Ride 2 PersonsShared Ride 2 Persons 6,0006,000 15,00015,000 20,00020,000 10,00010,000

Shared Ride 3 PersonsShared Ride 3 Persons 1,0001,000 2,0002,000 4,0004,000 2,0002,000

Local Bus, WalkLocal Bus, Walk 6,0006,000 7,0007,000 4,0004,000 1,0001,000

Local Bus, PNRLocal Bus, PNR -- 5005002,0002,000 500500

Local Bus, KNRLocal Bus, KNR -- 200200

Express Bus, WalkExpress Bus, Walk 1,0001,000 1,0001,000 1,0001,000 500500

Express Bus, PNRExpress Bus, PNR -- 2,0002,000 4,0004,000 2,0002,000

Express Bus, KNRExpress Bus, KNR -- 200200 500500

LRT, WalkLRT, Walk 500500 1,0001,000 400400

LRT, PNRLRT, PNR-- 300300 500500

LRT, KNRLRT, KNR

12

Default Guidelines and StandardsMode Choice (continued)

Mode splits by observed calibration targets

Total area transit trips, estimatedversus observed

Transit trips between districts

• Tabular comparisons (CTPP)

• Desire lines

Mean trip length, estimated transit trips versus observed

Statistic

Standards

Low High

Total Area Transit Trips versus Observed +/-1% +/- 2%

Transit Trips between Districts Compare model trip table against CTPP or HH survey

Mean Trip Length Transit Trips versus Observed +/-5% +/-15%

Mode Splits by Observed Calibration Targets +/- 2% +/- 2%

Elasticity of Demand with Respect to LOS Variables -0.6 -0.1

13

Default Guidelines and StandardsTrip Assignment

Volume-over-count ratios

+/-1 lane percent error (recalculated per FDOT LOS Handbook)

Aggregate VMT

• VMT/HH (60-75)

• VMT/person (24-32)

• VMT/commercial vehicle (3-25%)

Statistic

Standards

Acceptable Preferable

Freeway Volume-over-Count +/- 7% +/- 6%

Arterial Volume-over-Count +/- 15% +/- 10%

Collector Volume-over-Count +/- 25% +/- 20%

Frontage Road Volume-over-Count +/- 25% +/- 25%

Freeway Peak Volume-over-Count 75% of links @ +/-20%; 50% of links @ +/-10%

Major Arterial Peak Volume-over-Count 75% of links @ +/-30%; 50% of links @ +/-15%

Assigned VMT-over-Count Areawide +/-5% +/-2%

Assigned VHT-over-Count Areawide +/-5% +/-2%

Assigned VMT-over-Count by FT/AT/NL +/- 25% +/- 15%

Assigned VHT-over-Count by FT/AT/NL +/- 25% +/- 15%

Statistic

Standards

Acceptable Preferable

Percent Error – LT 10,000 volume (2L road) 50% 25%

Percent Error – 10,000-30,000 (4L road) 30% 20%

Percent Error – 30,000-50,000 (6L road) 25% 15%

Percent Error – 50,000-65,000 (4-6L freeway)

20% 10%

Percent Error – 65,000-75,000 (6L freeway) 15% 5%

Percent Error – GT 75,000 (8+L freeway) 10% 5%

14

Default Guidelines and StandardsTrip Assignment (continued)

Screenline volume-over-count

RMSE by volume group

Transit assignment validation

StatisticStatistic

StandardsStandards

AcceptableAcceptable PreferablePreferable

RMSERMSE – LT 5,000 AADTLT 5,000 AADT 150%150% 45%45%

RMSERMSE – 5,000-9,999 AADT5,000-9,999 AADT 45%45% 35%35%

RMSERMSE – 10,000-14,999 AADT10,000-14,999 AADT 35%35% 27%27%

RMSERMSE – 15,000-19,999 AADT15,000-19,999 AADT 35%35% 25%25%

RMSERMSE – 20,000-29,999 AADT20,000-29,999 AADT 27%27% 15%15%

RMSERMSE – 30,000-49,999 AADT30,000-49,999 AADT 25%25% 15%15%

RMSERMSE – 50,000-59,999 AADT50,000-59,999 AADT 20%20% 10%10%

RMSERMSE – 60,000+ AADT60,000+ AADT 19%19% 10%10%

RMSE AreawideRMSE Areawide 45%45% 35%35%Statistic

Benchmarks

Low High

Estimated-over-Observed Transit Trips +/- 9% +/- 3%

Standards

Statistic Acceptable Preferable

Acceptable Error – Transit Screenlines +/-20% +/-10%

Transit Ridership – <1,000 Passengers/Day +/-150% +/- 100%

Transit Ridership – 1k-2k Passengers/Day +/- 100% +/- 65%

Transit Ridership – 2k-5k Passengers/Day +/- 65% +/- 35%

Transit Ridership – 5k-10k Passengers/Day +/- 35% +/- 25%

Transit Ridership – 10k-20k Passengers/Day +/- 25% +/- 20%

Transit Ridership – >20,000 Passengers/Day +/- 20% +/- 15%

15

Other Model ApplicationsLRTP Highway Only Models

Same default guidelines and standards except

• Replace mode choice checks with auto occupancy comparisons against NHTS and other surveys

• Commercial vehicle VMT checks not likely relevant

• No transit assignment validation

Auto Occupancy RatesAuto Occupancy Rates

PurposePurposeCurrentCurrentModelModel

19881988FLSWMFLSWM

20012001NHTS FLNHTS FL

20012001NHTS USNHTS US

HBW 1.10 1.30 1.06 1.10

HBShop 1.80 1.55 1.57 1.80

HBSR 1.94 2.27 1.79 1.94

HBO 1.70 1.50 1.90 1.70

NHB 1.71 1.58 1.82 1.71

Statistic

Benchmarks/Settings

Low High

Auto Occupancy Rates – HBW 1.05 1.10

Auto Occupancy Rates – HBSH 1.50 1.80

Auto Occupancy Rates – HBSR 1.70 1.90

Auto Occupancy Rates – HBOa 1.65 1.95

Auto Occupancy Rates – NHBb 1.60 1.90

a HBO includes a variety of special trip purposes, depending on the model (e.g., airport, college, and school).

b NHB includes combined purposes for NHB Work and NHB Non-Work,where appropriate.

16

Other Model ApplicationsFTA New Starts Models

Transit networks and pathbuilding checks

• Compare skim settingsto on-board surveys

Trip distribution checks

Mode choice calibration

Highway assignment checks

Transit assignment checks

• Assign on-board survey trip table and compare ridership

• SUMMIT diagnostics

StatisticStatistic

Acceptable Range of ValuesAcceptable Range of Values

LowLow HighHigh

Elasticity of demand with Elasticity of demand with respect to LOS variablesrespect to LOS variables

-0.10-0.10-0.70-0.70

IVT parameter – HBW*IVT parameter – HBW* -0.02-0.02 -0.03-0.03

IVT parameter – HBNW*IVT parameter – HBNW* 0.1 to 0.5*CIVT0.1 to 0.5*CIVTHBW tripsHBW trips

0.1 to 0.5*CIVT0.1 to 0.5*CIVTHBW tripsHBW trips

IVT parameter – NHB*IVT parameter – NHB* ~CIVT HBW trips~CIVT HBW trips ~CIVT HBW trips~CIVT HBW trips

RatioRatio – OVT/IVT OVT/IVT parameters – HBW*parameters – HBW*

2.02.03.03.0

RatioRatio – OVT/IVT OVT/IVT parameters – HBNW*parameters – HBNW*

2.02.03.03.0

RatioRatio – OVT/IVT OVT/IVT parameters – NHB*parameters – NHB*

2.02.03.03.0

Implied value of time –Implied value of time –Percent of incomePercent of income

25%25%33%33%

Implied value of time – Implied value of time – HBWHBW

$2.00$2.00$7.00$7.00

Implied value of time – Implied value of time – HBNWHBNW

$0.50$0.50$5.00$5.00

Implied value of time – Implied value of time – NHBNHB

$0.20$0.20$5.00$5.00

* FTA published guideline.

17

Other Model Applications Subarea Models

Prerequisite – approved regional model validation

Input data – focus on socioeconomic and network data

Trip generation – review and compare subarea versus regional model aggregate trip rates

Trip distribution – compare subarea versus regional average trip length and percent intrazonal trips by purpose

Mode choice – check subarea mode shares versus regional

Trip assignment – volume-over-count (v-o-c), percent error, VMT and VHT v-o-c, v-o-c by screenline/cutline, and RMSE

18

Other Model Applications Corridor Models

Same subarea modelvalidation checksInput data – focus on network details surrounding corridorTrip generation – review corridor productions and attractions by zoneTrip distribution –desire line analysesMode choice – review of mode shares within study areaTrip assignment – more stringent standards for v-o-c, v-o-c by screenline/cutline

StatisticStatistic

StandardsStandards

AcceptableAcceptable PreferablePreferable

Freeway Volume-over-CountFreeway Volume-over-Count +/- 6%+/- 6% +/- 5%+/- 5%

Arterial Volume-over-CountArterial Volume-over-Count +/- 10%+/- 10% +/- 7%+/- 7%

Collector Volume-over-CountCollector Volume-over-Count +/- 15%+/- 15% +/- 10%+/- 10%

Frontage Rd Volume-over-CountFrontage Rd Volume-over-Count +/- 20%+/- 20% +/- 15%+/- 15%

19

Other Model Applications Models for DRIs and Other Impact Studies

Input data

• SE data – site, nearby zone assumptions, pop/TAZ

• Networks – verify coding, path traces from site

• Transit – access coding, headways, stop locations near site

Trip generation – document trip rate assumptions

Trip distribution – district summaries

Mode choice – check ITE tripsversus model trips

Trip assignment – select zoneand select link, turn volumes

20

An Issue for the MTF to Discuss…

Discussion on volume-over-count ratios

Ratio of summed modeled volumes for group of linksand the sum of count volumes on the links(should be near 1.0)

This check does appear in the draft report at this time

In the opinion of some, this is mathematically erroneous because of double counting

It is somewhat duplicative of VMT checks

21

An Example

1 mile

V = 7,500 V = 7,500

C = 5,000

V = 5,000

C = 5,000 C = 5,000v/c =

(7,500+7,500+5,000)/(5,000+5,000+5,000)

= 1.33

22

An Example (continued)

1 mile

V = 5,000 V = 5,000

C = 5,000

V = 7,500

C = 5,000 C = 5,000v/c =

(5,000+5,000+7,500)/(5,000+5,000+5,000)

= 1.17

23

Proposed Solution

Use VMT check

• Example has same solution for both cases

VMT(m) / VMT(c) = 1.25

VMT is not double counted

Screenline/cutline checks should be retained since double counting should not be an issue

24

Next Steps

Take comments from MTF committee today

Revise draft guidelines and standards based on FDOT and MTF committee comments

Prepare technical report on best practices

Develop guidelines for model application work