source: nhi course on travel demand forecasting ( 152054a)
DESCRIPTION
Session 11: Model Calibration, Validation, and Reasonableness Checks. Source: NHI course on Travel Demand Forecasting ( 152054A). Objectives:. Identify and interpret trends affecting travel demand Explain difference between calibration and validation Identify critical reasonableness checks - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/1.jpg)
Source: NHI course on Travel Demand Forecasting (152054A)
Session 11: Model Calibration, Validation, and Reasonableness
Checks
![Page 2: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/2.jpg)
Objectives:
• Identify and interpret trends affecting travel demand • Explain difference between calibration and validation• Identify critical reasonableness checks– socioeconomic– travel survey – network – trip generation – mode split – trip assignment
![Page 3: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/3.jpg)
Terminology• Model Calibration• Model Validation– Reasonableness checks– Sensitivity checks
• Special generators• Screen lines (some modelers do not think this
is important)
Is the model sensitive to policy options?
![Page 4: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/4.jpg)
• Not enough attention on model evaluation and reasonableness checks
• Checks performed after each step– reduces error propagation
Errors can also “cancel”
Key Concepts
![Page 5: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/5.jpg)
![Page 6: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/6.jpg)
Planner responsibilities• Actively involve all participants
– Modelers– Planners– Decision makers– Public
• Fairly present all alternatives– Timely– Unbiased
• Identify (clearly) the decision making process– Who, when, and how– Allows input from all interested groups
• You must rely on the TDM– Therefore, must be validated– Accurate and easy to understand (documented)
![Page 7: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/7.jpg)
• Planners should monitor the following trends:– Demographics– Composition of the labor force– Immigration and emigration– Regional economic development– Modal shares– Vehicle occupancy– Average trip length– Freight transport
• Are trends consistent with assumptions made in the modeling process?
Must be aware of trends to ensure reasonable forecasts
Trends Affecting Travel Demand
![Page 8: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/8.jpg)
Information Requirements for Validation and Reasonableness
• Demographics and employment• Highway and transit networks• Model specification• Base year survey• Base year traffic counts
![Page 9: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/9.jpg)
Sources of Error
• Coding• Sampling• Computation (if done by
hand)• Specification• Data Transfer• Data aggregation
Improper structure of model, e.g., wrong variables
![Page 10: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/10.jpg)
Scrutinize these characteristics: • Data requirements• Hardware requirements• Logic of structure and conceptual appeal • Ease of calibration • Effectiveness of the model (accuracy, sensitivity)• Flexibility in application• Types of available outputs• Operational costs• Experience and successes to date• Public or private domain availability• Compatibility with other models and model types
How do you judge a model/recommend improvement?
![Page 11: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/11.jpg)
Reasonable?Methodology?
Source?
Current?Reasonable
?
Complete?Level of Detail?
Sensitive?Documentation of
calibration?Valid for base
year?
Evaluation and Reasonableness Checks Overview
Transportation Transportation systemsystem
(supply)(supply)►Network DataNetwork Data
Number and Number and location of location of
households and households and employment employment
(demand)(demand)►Socioeconomic Socioeconomic
DataData
TDFTDF►Model SpecificationModel Specification►Model validation and Model validation and calibrationcalibration
Travel Travel
survey datasurvey data
Transportation Transportation systemsystem
performanceperformance
![Page 12: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/12.jpg)
Model Calibration and Validation
Model CalibrationModel Calibration
Model ValidationModel Validation
Model ApplicationModel Application
Feedb
ack
Loop
![Page 13: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/13.jpg)
“Transportation Conformity Guidelines” (Air Quality) require model validated < 10 years ago
Model Validation
• Validation of new model– Model applied to complete model chain– Base year model compared to observed travel– Judgment as to model suitability, return to calibration
if not• Validation of a previously calibrated model– Compare to a new base year, with new …
• SE data• Special gen.• Network• Counts
![Page 14: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/14.jpg)
Socioeconomic Data: Check Reasonableness
• reviewSource for estimates and forecasts• Population and household size trends (graph 1950 to present
and check trend)• Household income trends (graph as far back as this goes …
1990?)• Check dollar values used in forecast (use constant dollars)• If used, check trend of automotive availability (S curve?)• Check distribution of employment by type (basic, retail,
service) over time• Plot and check trend of employees per household and per
capita … rate of increase is decreasing• Check future household and employment changes by zone
![Page 15: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/15.jpg)
Trav
el not
sensit
ive to
fuel
price?
http://www.eia.doe.gov/oiaf/aeo/pdf/trend_4.pdf
![Page 16: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/16.jpg)
Travel Survey Data Reasonableness Checks
• Determine source of travel survey data – Types of survey conducted– Year of survey
• If no survey (borrowed)– Check source of trip rates, lengths, TLFD– Is area similar
• Geographic area?• pop/HH/empl. characteristics?• Urban density and trans system?
• Compare to similar regions and to same region in earlier times:
– Person trip rates by trip purpose– Mean trip lengths by trip purpose
• HBW longest? HBO shortest?– TLFDs by trip purpose
![Page 17: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/17.jpg)
Network Data Reasonableness Checks
• Check Trees for 2-3 major attractions• Check coded facility types – how used (BPR?)?• Verify speed and capacity look-up table (what
LOS used for capacity?)• Significant transportation projects – narrative
included? Still viable?• Consistency with MTP• Plot (facility types, # lanes,
speeds, area types) to detect coding errors
![Page 18: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/18.jpg)
Trip Generation Reasonableness Checks
• Examine trip production and attraction models– Form?– sensitivity?
• Examine trip purposes used• External-through and external-local trips – how
modeled?• Truck trips – how modeled?• Person trip or vehicle trip rates used?• P&A balance (0.9-1.1 ok)• Special generators (check, and be consistent in
future model)
![Page 19: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/19.jpg)
Trip Generation CalibrationTypical Values
• Person trips per household: 8.5 to 10.5• HBW person trips per household: 1.7 to 2.3• HBO person trips per household: 3.5 to 4.8• NHB person trips per household: 1.7 to 2.9• HBW trips: 18% to 27% of all trips• HBO trips: 47% to 54% of all trips• NHB trips: 22% to 31% of all trips
![Page 20: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/20.jpg)
Scale survey for participation (relative participation)
Note: each
income class is
a purpos
e!
TRIP PURPOSES Scaling Factor
HBW low income 0.795
HBW low-middle income 0.823
HBW middle income 0.861
HBW upper middle income 0.908
HBW high income 0.936
HB elementary school 0.733
HB high school 1.991
HB university 0.895
HB shopping 0.698
HB social-recreation 0.945
HB other 0.875
NHB work-related 0.858
NHB other 0.820
Truck 0.985
Internal-external 0.591
Trip Generation Calibration
Colorado Springs 1996 Travel Demand Model Calibration
![Page 21: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/21.jpg)
Trip Generation CalibrationReasonableness checks – compare to other cities, check
future trends• Population 503,345• Households 201,116• Average Household Size 2.50• Basic employment 76,795 (33%)• Retail employment 50,465 (24%)• Service employment 101,697 (43%)• Military employment 42,800• Population per employee 1.81• Person trips per person 4.26• Person trips per household 10.65• HBW attractions per employee 1.44• HBW productions per household 1.74• HB shopping attractions per retail employee 5.99
Colorado Springs 1996 Travel Demand Model Calibration
![Page 22: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/22.jpg)
Trip Distribution Reasonableness Checks
Examine …• Mean trip length (increasing or decreasing?)• TLFDs• Treatment of friction factors (same?)• Treatment of terminal times (logic?)• Treatment of K factors• Comparison with JTW trip length• Comparison with JTW sector interchange volumes or
percentages.
![Page 23: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/23.jpg)
Calibrate
friction factors
1st iteratio
n
![Page 24: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/24.jpg)
Travel TimesTravel TimesRanges from Ranges from SkimsSkims
Observed Trip Observed Trip Expanded from Expanded from SurveysSurveys
Input Input Friction Friction FactorsFactors
Gravity Gravity Model TripsModel Trips
Adjustment Adjustment FactorFactorObservedObservedGravity Gravity ModelModel
New Friction New Friction FactorsFactorsFriction Friction AdjustmentAdjustmentFactor x Friction Factor x Friction FactorFactor
2.52.5 7,1007,100 30.030.0 8,2008,200 0.870.87 25.9825.98
5.05.0 14,95014,950 2.502.50 16,30016,300 0.920.92 2.292.29
7.57.5 17,85017,850 1.801.80 19,25019,250 0.930.93 1.671.67
10.010.0 16,00016,000 1.501.50 19,10019,100 0.840.84 1.261.26
12.512.5 15,50015,500 1.201.20 17,10017,100 0.910.91 1.091.09
15.015.0 15,90015,900 1.001.00 12,30012,300 1.291.29 1.291.29
17.517.5 16,40016,400 0.950.95 18,00018,000 0.910.91 0.870.87
20.020.0 15,15015,150 0.900.90 14,30014,300 1.061.06 0.950.95
22.522.5 13,50013,500 0.850.85 11,90011,900 1.131.13 0.960.96
25.025.0 11,00011,000 0.800.80 9,2509,250 1.191.19 0.950.95
27.527.5 9,5009,500 0.750.75 8,1008,100 1.171.17 0.880.88
30.030.0 9,1009,100 0.700.70 6,1006,100 1.491.49 1.041.04
32.532.5 5,7005,700 0.650.65 4,9004,900 1.161.16 0.760.76
…… …… …… …… …… ……
Calibrating a Gravity ModelAdjusting Friction Factors
![Page 25: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/25.jpg)
2nd iteratio
n
![Page 26: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/26.jpg)
Commute Length in Commute Length in MinutesMinutes PercentPercent
Journey-to-WorkJourney-to-WorkFlowsFlows PercentPercent
< 15< 15 27.8727.87 Central-CentralCentral-CentralCountyCounty
31.4931.49
15-2915-29 41.6341.63 Central-SuburbanCentral-SuburbanCountyCounty
7.487.48
30-3930-39 17.0417.04 Suburban-Central Suburban-Central CountyCounty
15.1315.13
40-5940-59 7.707.70 Within Suburban Within Suburban CountyCounty
32.9832.98
>60>60 3.003.00 To Other Suburban To Other Suburban CountyCounty
10.8110.81
Mean 21.44Mean 21.44 Work out of areaWork out of area 2.112.11
Trip Distribution Calibration and Validation
• Check modeled vs. household survey TLFD and mean trip lengths• Get HBW area-to-area flows from JTW
HBW 1990 JTW TLFD and Area-to-Area Flows for Kansas City
![Page 27: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/27.jpg)
![Page 28: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/28.jpg)
Mode Split Reasonableness Checks
• Automobile occupancy factors by trip purpose used?• Basis? • Constant?
• Mode split model? • Form?• Variables included in the utility functions?
• Coefficients logical?• Value of time assumptions• Parking cost assumptions
• How do mode shares change over time?• Mode share comparisons with other cities
![Page 29: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/29.jpg)
• Experienced planning consultant required …– Form of LOGIT model– Variables included in utility functions– Calibration of coefficients for utility function variables– Testing for IIA properties– Analysis of household survey data– Analysis of on-board transit survey data
• Calibration tasks we can do:• Compare highway and transit trips
• Total• By purpose
• Compare Ridership by route• CBD cordon line survey (if bus service is downtown only)
Mode Split Calibration and Validation
![Page 30: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/30.jpg)
• All-or-nothing assignment • study effect of increasing capacity• Compare to Equilibrium assignment
• Check volume delay equation (BPR parameters)• Compare
• screen line volumes• Cut line volumes
• Time-of-day assignments?• Source of factors• Peak spreading used for future?• If not, conversion factors source?
(peak hour to 24-hour) • Local VMT (% assigned to
intrazonals and centroid connectors All or
Nothing
Equil
ibrium
Trip Assignment Reasonableness Checks
![Page 31: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/31.jpg)
Assignment
calibration
performed last
Trip Assignment Calibration and Validation
Overall VMT or VHT check• 40 to 60 miles per day per HH in large metro areas• 30 to 40 miles per day per HH in medium metro• +/- 10% OK on screen lines• Sign is important
![Page 32: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/32.jpg)
Compute by …- volume group- facility type- transit assignments- time of day
![Page 33: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/33.jpg)
![Page 34: Source: NHI course on Travel Demand Forecasting ( 152054A)](https://reader035.vdocuments.site/reader035/viewer/2022062304/56813108550346895d9739bf/html5/thumbnails/34.jpg)
Other Factors Impacting Forecasted Travel Demand (use your noodle)
• Can be implied in travel surveys (but not explicit)– Telecommuting– Flexible work hours– HB business
• How to account for …– Aging population– Internet shopping– Roadway congestion (will it affect generation in the future)– New modes