tutorial: microscopic traffic simulation model calibration...

1

June 27, 2006June 27, 2006

Tutorial:

Microscopic Traffic Simulation Model Calibration & Validation

Tutorial:

Microscopic Traffic Simulation Model Calibration & Validation

Instructors: Byungkyu (Brian) Park, Ph.D.Jongsun Won

Instructors: Byungkyu (Brian) Park, Ph.D.Jongsun Won

AGENDAAGENDA

HandsHands--on Practice (CORSIM)on Practice (CORSIM)Demo (VISSIM)Demo (VISSIM)

1:00 ~ 2:201:00 ~ 2:20

2:30 ~ 3:502:30 ~ 3:50

ContentsContentsTimeTime

IntroductionIntroduction9:30 ~ 10:009:30 ~ 10:00

WrapWrap--up / Discussionup / Discussion4:00 ~ 4:304:00 ~ 4:30

LunchLunch11:45 ~ 1:0011:45 ~ 1:00

Calibration and Validation MethodCalibration and Validation Method10:10 ~ 11:4510:10 ~ 11:45

2

Lesson 1

Introduction9:30 ~ 10:00

Lesson 1

Introduction9:30 ~ 10:00

IntroductionIntroduction

• What is “Simulation”?

• Why “Calibration” is important?

• What to calibrate?

• What is “Simulation”?

• Why “Calibration” is important?

• What to calibrate?

3

OutputOutputSimulationSimulationInputInput

SimulationSimulationSimulation

What is “Simulation”?What is “Simulation”?

The technique of “imitating the behavior of some

situation or process” by means of “a suitably

analogous situation or apparatus” [Oxford Dictionary]


“Computer” Simulation““ComputerComputer”” SimulationSimulation

OutputOutputSimulationProgram

SimulationProgramInputInput

Process of …• Designing a computerized model of a system• Conducting experiments with this model• Understanding behavior of system of evaluating various

strategies for the operation of the system

4

OutputOutput


MacroscopicMacroscopic

MesoscopicMesoscopic

MicroscopicMicroscopic

InputInput

Traffic Simulation ModelTraffic Simulation Model

“Traffic” Simulation““TrafficTraffic”” SimulationSimulation

• Process of applying simulation for traffic applications• Depending on fidelity of modeling traffic, it can be

“Macro”, “Meso” or “Micro”-scopic models

OutputOutputGeometryGeometry

VolumeVolume

Control SystemControl System...

CORSIMCORSIM

VISSIMVISSIM

SimTrafficSimTraffic...

OutputOutputMicroscopic TrafficSimulation Model

Microscopic TrafficSimulation ModelInputInput

Travel TimeTravel Time

DelayDelay

Queue LengthQueue Length...


“Microscopic Traffic” Simulation““Microscopic TrafficMicroscopic Traffic”” SimulationSimulation• Fidelity of simulation system component is modeled

at an individual level• Each vehicle is simulation system entity for conducting

simulation.

5

Cost-

Effectiveness

Cost-

Effectiveness

SafetySafety

Time-

Efficiency

Time-

Efficiency

AAPP

PPLL

IICC

AATT

IIOO

NN BBEE

NNEE

FFII

TT

Microscopic TrafficSimulation Models

Microscopic TrafficSimulation Models

OperationsOperations

PlanningPlanning

ITS StrategiesITS Strategies


Simulation ModelsSimulation Models

Examples of Microscopic traffic simulation modelsExamples of Microscopic traffic simulation modelsExamples of Microscopic traffic simulation models

VISSIMVISSIMVISSIM CORSIMCORSIMCORSIM

6

• Developed by University of

Karlsruhe in Germany (1970s)

• Distributed by PTV (1993)

• Currently being used by

State DOTs, Consultants,

and Research Institutes

Simulation Models -VISSIMSimulation Models -VISSIM

BackgroundBackgroundBackground


• Microscopic, Time-step

based simulation model

• Simulate traffic operations

in urban streets and

freeways

• Emphasize multi-modal

transportations (Bus, LRT,

Heavy Rail, etc.)

OverviewOverviewOverview

7


Traffic Flow ModelTraffic Flow ModelTraffic Flow Model Signal Control ModelSignal Control ModelSignal Control Model

VISSIMVISSIMVISSIM


• Various measures of

effectiveness(e.g., Delay, Travel Time, Queue

Length, etc.)

• 2D & 3D animations

OutputOutputOutput

8

Simulation Models -CORSIMSimulation Models -CORSIM

• Developed & funded by

FHWA

• Widely adopted by many

entities in the U.S.

BackgroundBackgroundBackground


TRAFED

TRAFED TRAFV

U

TRAFV

U

CORSIMCORSIM

TSISTSISTraffic Software

Integrated SystemsTraffic Software

Integrated Systems

9

• Detailed stochastic &

Microscopic simulation

model

• Analysis of freeways,

arterials, and basic transit

operations

• NETSIM & FRESIM


OverviewOverviewOverview

• Various measures of

effectiveness(e.g., Link Travel Time, Queue

Length, Delay, etc)

• 2D animation (TRAFVU)


OutputOutputOutput

10

Why calibration & validation are important?Why calibration & validation are important?

Calibration & ValidationCalibration & ValidationCalibration & Validation

The process of evaluating software or model at the end of the development process to ensure compliance with requirements

The action or process of determining the correct position, value, capacity, etc.

ValidationCalibration

Microscopic Traffic Simulation Model Calibration & Validation WorkshopMicroscopic Traffic Simulation Model Calibration & Validation Workshop


Traffic Simulation Model Calibration & ValidationTraffic Simulation ModelTraffic Simulation Model Calibration & ValidationCalibration & Validation

Process of ensuring calibrated traffic simulation model can be trusted for representing untried field condition

Process of determining appropriate parameters such that simulation model can represent field condition

ValidationCalibration

11

•• Uses Uses ““DefaultDefault”” parametersparameters•• Modifies parameter base on Modifies parameter base on ““EngineerEngineer’’s judgments judgment””•• Due toDue to……

-- The The limitationslimitations on time, cost, etc.on time, cost, etc.-- Lack of available and formal calibration procedureLack of available and formal calibration procedure

Currently…CurrentlyCurrently……


IssuesIssuesIssues

SolutionSolutionSolution

•• No No single model single model (i.e., a set of calibration parameters) can (i.e., a set of calibration parameters) can

describe describe various field conditionsvarious field conditions perfectlyperfectly

•• Manual adjustment could be either Manual adjustment could be either unrealistic or erroneousunrealistic or erroneous

““Garbage In Garbage OutGarbage In Garbage Out””

•• Uncalibrated Simulation Model could mislead the results.Uncalibrated Simulation Model could mislead the results.



IssuesIssuesIssues


12

•• Conducting Conducting ““Calibration ProcedureCalibration Procedure”” prior to the applicationprior to the application

of simulation modelof simulation model



IssuesIssuesIssues


The importance of calibration- Example case study -

The importance of calibrationThe importance of calibration-- Example case study Example case study --


13

Field Travel TimeField Travel TimeField Travel Time


0

5

10

15

20

25

30

35

40

375 400 425 450 475 500 525 550 575 600 625 650 675 700 725 750 775 800Travel Time

Frequency

• Two video cameras were installed at each end of the segment• Each travel time data is extracted• Frequency of each group is plotted

• Multiple run result for both “Calibrated” and “Uncalibrated”VISSIM network are compared with field result

• Multiple run result for both “Calibrated” and “Uncalibrated”VISSIM network are compared with field result

Field vs. Calibrated

0

5

10

15

20

25

30

35

40

300 325 350 375 400 425 450 475 500 525 550 575 600 625 650 675 700 725 750 775 800

Travel Time

Frequency

Field Calibrated VISSIM

Field vs. Uncalibrated

0

5

10

15

20

25

30

35

40

300 325 350 375 400 425 450 475 500 525 550 575 600 625 650 675 700 725 750 775 800

Travel Time

Frequency

Field Uncalibrated VISSIM

Field vs. Uncalibrated ModelField vs. Uncalibrated ModelField vs. Uncalibrated Model Field vs. Calibrated ModelField vs. Calibrated ModelField vs. Calibrated Model

Comparison of simulation model outputComparison of simulation model outputComparison of simulation model output


14

Result with “Uncalibrated” modelResult with Result with ““UncalibratedUncalibrated”” modelmodel Result with “Calibrated” modelResult with Result with ““CalibratedCalibrated”” modelmodel

Comparison of timing plan optimization resultComparison of timing plan optimization resultComparison of timing plan optimization result

Travel Time (seconds)

Frequency

0

5

10

15

20

25

30

35

40

400 425 450 475 500 525 550

Current T7F Synchro

Travel Time (seconds)

Frequency

0

5

10

15

20

25

30

35

400 450 500 550 600 650 700 750

Current T7F Synchro


• Effect of signal optimization can be overstated• Effect of signal optimization can be overstated

What to calibrate?What to calibrate?

•• ““Is the parameter difficult to collect from the field?Is the parameter difficult to collect from the field?””Difficulties and limitations on field data collectionDifficulties and limitations on field data collection

•• ““Is it critical to the output of the simulation model?Is it critical to the output of the simulation model?””

Importance of parameter to the specific simulation modelImportance of parameter to the specific simulation model

How to select calibration parametersHow to select calibration parametersHow to select calibration parameters

15

What to calibrate?What to calibrate?

• Driving behavior parameters- Lane changing behavior- Desired speed - Car-following

• Other parameters defined by user manual

• Example parameters- Gap, Headway, Desired speed, Acceleration rate, etc

Available calibration parametersAvailable calibration parametersAvailable calibration parameters

Lesson 2

Calibration andValidation Method

10:10 ~ 11:45

Lesson 2

Calibration andValidation Method

10:10 ~ 11:45

16

Calibration ProcedureCalibration Procedure

Calibration and

Validation Concept

Calibration and

Validation Concept

Calibration and

Validation Overview

Calibration and

Validation OverviewCalibration MethodCalibration Method

Validation MethodValidation Method

Concept of calibrationConcept of calibration

ConceptConceptConcept

Probability [|“Reality” – Prediction| < δ] > αδ = tolerable differenceα = level of assuranceSource: NISS Technical Report 144, July 2004.Source: NISS Technical Report 144, July 2004.

Assurance level>Prob. of the difference less than toleranceHow “Certain” ?

Tolerance<Difference of real and predicted valueHow “Close” ?

17

How to calibrate?How to calibrate?

Systematic Calibration Procedure (Simplified)Systematic Calibration Procedure (Simplified)Systematic Calibration Procedure (Simplified)

• Multiple runs with defaultparameters

• Comparison of simulationoutputs & field data

Usefulness of default Value

• Simulation runs with differentparameter combinations

• Comparison of simulationoutputs & field data

• Calibrating parameters by using

“Optimization method”

Parameter Range Determination

Parameter Calibration

• Verify calibration result withuntried data set

Parameter Validation

Passed?

End

No

Yes

SatisfiedUnsatisfied

No

Yes

Satisfied

Unsatisfied

Calibration Procedure Flow ChartCalibration Procedure Flow ChartCalibration Procedure Flow Chart

Passed?

Simulation Model SetupSimulation Model SetupSimulation Model Setup

Initial EvaluationInitial EvaluationInitial Evaluation

Experimental DesignExperimental DesignExperimental Design

Feasibility TestFeasibility TestFeasibility Test

Adjust Key Parameter RangesAdjust Key Parameter RangesAdjust Key Parameter RangesParameter Calibration Using

Genetic Algorithm Parameter Calibration Using Parameter Calibration Using

Genetic Algorithm Genetic Algorithm

Evaluation of calibrated parameter set

Evaluation of calibrated Evaluation of calibrated parameter setparameter set

Model Validation & Visualization

Model Validation & Model Validation & VisualizationVisualization


18

1. Simulation Setup1. Simulation Setup1. Simulation Setup

Conditions for Conditions for ““performance measureperformance measure”” selection.selection.

•• Does simulation model provides Does simulation model provides corresponding outputcorresponding output??

•• Is it possible to collect Is it possible to collect from the fieldfrom the field??

•• Does the selected measure reflects Does the selected measure reflects traffic conditionstraffic conditions directly?directly?

e.g. Travel Time, Queue Length, Delay, Speed, etc.e.g. Travel Time, Queue Length, Delay, Speed, etc.

① Determination of Performance Measure①① Determination of Performance MeasureDetermination of Performance Measure

Preparing simulation modelPreparing simulation modelPurposePurposePurpose


•• Required data for network coding which can be obtained Required data for network coding which can be obtained

from the field (e.g., Signal settings, Volume, Geometry, etc)from the field (e.g., Signal settings, Volume, Geometry, etc)

Fundamental Input DataFundamental Input DataFundamental Input Data


② Field Data Collection②② Field Data CollectionField Data Collection


19

•• Following data are desirable to consider variability Following data are desirable to consider variability

-- Multiple days (dayMultiple days (day--toto--day variability)day variability)

-- Multiple types Multiple types

•• Data collection location should be same as the point in Data collection location should be same as the point in

simulation model to get more accurate resultsimulation model to get more accurate result

Performance Measure DataPerformance Measure DataPerformance Measure Data




•• New performance measure data needs to be collected New performance measure data needs to be collected



Validation DataValidation DataValidation Data

•• Different types or surveying dateDifferent types or surveying date

•• Before/After the implementation of operation strategiesBefore/After the implementation of operation strategies

New performance measure dataNew performance measure dataNew performance measure data


20

•• All fundamental inputs need to be coded in a simulation modelAll fundamental inputs need to be coded in a simulation model

-- Fundamental Inputs: Geometry, Volume, Fundamental Inputs: Geometry, Volume,

•• Data collection points and/or segments of simulation model Data collection points and/or segments of simulation model

which are corresponding to the field measured location shouldwhich are corresponding to the field measured location should

be installedbe installed


③ Network Coding③③ Network CodingNetwork Coding


Performance of Default Parameter SetPerformance of Default Parameter SetPerformance of Default Parameter Set

No

YesMultiple runs with default parameter setMultiple runs with Multiple runs with

default parameter setdefault parameter set

Extract simulation outputs (Performance

measure)

Extract simulation Extract simulation outputs (Performance outputs (Performance

measure)measure)

Performance measure data comparison

(Field and Simulation)

Performance measure Performance measure data comparisondata comparison

(Field and Simulation)(Field and Simulation)Calibration

requiredCalibration Calibration

requiredrequired

Close match found?Close match found?Close match found?Use default

parameter set for further analysis

Use default Use default parameter set for parameter set for further analysisfurther analysis


2. Initial Evaluation2. Initial Evaluation2. Initial Evaluation

Check Check the validitythe validity of default simulation modelof default simulation modelPurposePurposePurpose

21

0

10

20

30

40

50

60

70

80

400 420 440 460 480 500 520 540 560 580 600 620 640

Travel Time

Frequency

Field Data

0

2

4

6

8

10

12

590 600 610 620 630 640 650 660 670 680 690 700 710 720 730 740 750 760 770 780 790 800 810

Travel Time

Frequency

Field Data


What is close match?What is close match?What is close match?


Unacceptable CaseUnacceptable CaseUnacceptable Case Close MatchClose MatchClose Match

Distribution of simulation outputdoes not include field data

Distribution of simulation outputdoes include field data

. . . . . . . . . . .


Consideration of Multiple Performance MeasuresConsideration of Multiple Performance MeasuresConsideration of Multiple Performance Measures


Close MatchClose MatchClose Match• Each dot on the X-Y plot represents each output data point that correspond to selected performance measures.• If the combination of field collected data includes certain number of points (say 10%), it can be considered as a close match

Field Value

22

1) 1) Identify all available Identify all available calibration parameterscalibration parameters with the selected with the selected

simulation model simulation model

2) Categorize each parameter with its 2) Categorize each parameter with its characteristicscharacteristics

3) Consider the 3) Consider the relevancerelevance of each category for given siteof each category for given site

4) Determine the 4) Determine the acceptable rangeacceptable range of each parameterof each parameter

Identification of Key ParametersIdentification of Key ParametersIdentification of Key Parameters


3. Experimental Design3. Experimental Design3. Experimental Design

Sample the combinations of parameters to trySample the combinations of parameters to try--ononPurposePurposePurpose

Experimental Design for CalibrationExperimental Design for CalibrationExperimental Design for Calibration

•• It is impossible to try out every single combinationsIt is impossible to try out every single combinations((e.g., 10 parameters, 5 levels e.g., 10 parameters, 5 levels 551010 = 9,765,625 combinations)= 9,765,625 combinations)

Why Experimental DesignWhy Experimental DesignWhy Experimental Design

• Optimization-basedexperimental design method - Minimize pair-wise correlations- Maximally covers surface space

•• OptimizationOptimization--basedbasedexperimental design experimental design method method -- Minimize pairMinimize pair--wise correlationswise correlations-- Maximally covers surface spaceMaximally covers surface space

Latin Hypercube Sampling (LHS) methodLatin Hypercube Sampling (LHS) methodLatin Hypercube Sampling (LHS) method

Parameter RangeParameter RangeParameter Range

Freq

uenc

yFr

eque

ncy

Freq

uenc

y

20% 20% 20% 20% 20%



23


ExampleExampleExample

Latin Hypercube SamplingLatin Hypercube SamplingLatin Hypercube SamplingRandom SamplingRandom SamplingRandom Sampling


1

2

34

5X1

X2

61

2

34

5X1

X2

6

Example parameter setExample parameter setExample parameter set

5.04

6.71

4.77

3.39

2.40

7.15

3.51

8.00

5.73

2.92

V5

6.553.99 7.32 3.7210

8.504.11 2.311.779

6.303.034.221.988

9.66 3.54 8.18 3.177

7.424.984.844.55 6

7.174.71 8.894.05 5

8.333.70 5.584.92 4

8.843.39 3.35 2.52 3

9.344.56 9.57 1.28 2

7.67 4.22 6.17 2.991

V4V3V2V1


0.052-0.053-0.096-0.071V5

-0.0220.0420.062V4

0.0370.097V3

0.080V2

V4V3V2V1

Correlation of each parameter setCorrelation of each parameter setCorrelation of each parameter set

“Low Correlation” betweenparameters““Low CorrelationLow Correlation”” betweenbetween

parametersparameters


24

•• If the field collected value falls withinIf the field collected value falls withinthe 90 percent area of simulation the 90 percent area of simulation output distribution, then it considersoutput distribution, then it considersto be feasible.to be feasible.(e.g. : Feasible(e.g. : Feasible

: Not Feasible): Not Feasible)

5%

50%

95 %

Acceptable Region

Feasible?Feasible?Feasible?


4. Feasibility Test4. Feasibility Test4. Feasibility Test

To check whether the calibration data is reasonablyTo check whether the calibration data is reasonablycovered by the simulation output distributioncovered by the simulation output distributionPurposePurposePurpose

Consideration of Multiple Performance MeasuresConsideration of Multiple Performance MeasuresConsideration of Multiple Performance Measures


4. Feasibility Test4. Feasibility Test4. Feasibility Test

90% Confidence interval range90% Confidence interval range90% Confidence interval range

Field collected performance measure value range

Field collected performance Field collected performance measure value rangemeasure value range

Two boxes should overlap to be recognized as a feasible range as shown in the figure on the left side.

Two boxes should overlap to be Two boxes should overlap to be recognized as a feasible range as recognized as a feasible range as shown in the figure on the left side.shown in the figure on the left side.

25

Key parameter can be identified by using following two methodsKey parameter can be identified by using following two methods•• XX--Y PlotY Plot•• Analysis of Variance (ANOVA) testAnalysis of Variance (ANOVA) test

• Finding Key Parameters•• Finding Key ParametersFinding Key Parameters

•• Understand the changing pattern of corresponding output asUnderstand the changing pattern of corresponding output asthe parameter value altersthe parameter value alters

•• Check the reality of some measures (i.e. saturation flow rate)Check the reality of some measures (i.e. saturation flow rate)

• Adjust key parameters•• Adjust key parametersAdjust key parameters


5. Parameter Adjustment5. Parameter Adjustment5. Parameter Adjustment

•• Plot different Plot different parameter levelsparameter levelsversusversus simulation outputssimulation outputsat each parameter levelat each parameter level

The trend of simulation outputsThe trend of simulation outputsalong different parameter levelsalong different parameter levelscan be observedcan be observed

• X-Y Plot•• XX--Y PlotY Plot

Example of X-Y PlotExample of XExample of X--Y PlotY Plot

400

600

800

1000

1200

1400

1600

1 2 3 4 5

Additive Part of desired safety distance

TRAVEL TIME

400

600

800

1000

1200

1400

1600

1 2 3 4 5

Additive Part of desired safety distance

TRAVEL TIME

Parameter levelsParameter levelsParameter levels

Perf

orm

ance

Mea

sure

Perf

orm

ance

Mea

sure

Perf

orm

ance

Mea

sure



26

•• Analysis of VarianceAnalysis of Variance•• A series of statistical procedures for examining differences inA series of statistical procedures for examining differences inmeans and for partitioning variancemeans and for partitioning variance

•• Many different statistical software provides ANOVA functionMany different statistical software provides ANOVA function(i.e. SAS, SPSS, Excel, Minitab, etc.)(i.e. SAS, SPSS, Excel, Minitab, etc.)

•• Significant source (parameter) can be determined with its Significant source (parameter) can be determined with its ““PP--ValueValue””

• What is ANOVA?•• What is ANOVA?What is ANOVA?



*Note: Significance value is less than 0.05.

• ANOVA table•• ANOVA tableANOVA table



333.70165,339.20Error140.699.82

F0

1855,563.35Total

0.24746,948.05146,948.05Variable 2< 0.0013,276.1013,276.10Variable 1

P-ValueMean SquareDegrees of Freedom

Sum of Squares

Source of Variation

27

0.1870.187Parameter 4Parameter 4

0.0070.007Parameter 5Parameter 5

0.0370.037Parameter 3Parameter 30.4350.435Parameter 2Parameter 20.9440.944Parameter 1Parameter 1

Significance Value Significance Value (p(p--value)value)

*Note: Significance value is less than 0.05.

•• If If ““pp--valuevalue”” is smaller thanis smaller thanprepre--determined criteria, thendetermined criteria, thenthose parameters arethose parameters areconsideredconsidered to have significantto have significanteffect to theeffect to the MOEMOE

• ANOVA test•• ANOVA testANOVA test



-3D Contour Plot- Combined effect of parameters

to the performance measurevalue

Enhancement and Evaluation ofPreviously Developed Simulation ModelCalibration and Validation Procedure


- Interval Plot- Effect of individual parameter

to the performance measurevalue

Parameter vs. Performance MeasureParameter vs. Performance Measure Interaction of ParametersInteraction of Parameters



28

•• A method of implementing the action of A method of implementing the action of ““NaturalNaturalSelection and EvolutionSelection and Evolution””

•• Widely applied in Widely applied in ““OptimizationOptimization”” areaarea•• Why GA was chosen?Why GA was chosen?

StochasticStochasticGlobal SearchGlobal Search

•• Genetic Algorithm (GA)Genetic Algorithm (GA)


6. Parameter Calibration6. Parameter Calibration6. Parameter Calibration

• Genetic Algorithm (GA) Process•• Genetic Algorithm (GA) ProcessGenetic Algorithm (GA) Process

StartStartCreate Initial Population

of Calibration Parameters

Create Initial Population Create Initial Population of Calibration of Calibration

ParametersParameters

Extracting Simulation

Results

Extracting Extracting Simulation Simulation

ResultsResults

Calculation of Fitness Value

Calculation of Calculation of Fitness ValueFitness Value

Stopping Criterion?Stopping Criterion?

Simulation Runs

Simulation Simulation RunsRuns

Generate Next Populations

Generate Next Generate Next PopulationsPopulations

EndEndYes

No

Ranges of Ranges of each each

parameterparameter

InputInputInput

Calibrated Calibrated value of each value of each

parameterparameter

OutputOutputOutput



29

• Major steps in GA•• Major steps in GAMajor steps in GA

SelectionSelectionSelection CrossoverCrossoverCrossover MutationMutationMutation


InitialPopulation

InitialInitialPopulationPopulation

NewPopulation

NewNewPopulationPopulationEvaluationEvaluationEvaluationSolutionSolutionSolution


• Three main steps in GA•• Three main steps in GAThree main steps in GA

•• Select specificSelect specificsettings insettings inpopulation for nextpopulation for nextstepstep

•• As it gives low As it gives low fitness function fitness function value, it has more value, it has more chances to be chances to be selectedselected

SelectionSelectionSelection

•• Randomly select a Randomly select a part of the each part of the each string string

•• Exchange selectedExchange selectedpart of settingspart of settings

CrossoverCrossoverCrossover

•• Randomly flips Randomly flips some digits in the some digits in the string for the string for the settingssettings

MutationMutationMutation



30

•• Relative errorRelative error• Fitness Value•• Fitness ValueFitness Value

•• GA stops, if one of the following conditions are satisfiedGA stops, if one of the following conditions are satisfied--““Fitness ValueFitness Value”” meets premeets pre--defined thresholddefined threshold-- PrePre--defined defined ““Max. number of generationMax. number of generation”” has reachedhas reached

• Stopping Criterion•• Stopping CriterionStopping Criterion

Field

SimulationField

PMPMPM

FV−

=

•• FVFV: Fitness Value: Fitness Value•• PMPMFieldField: Performance Measure collected from the : Performance Measure collected from the

fieldfield•• PMPMSimulationSimulation: Simulation Outputs: Simulation Outputs

WhereWhere



Applying Additional Performance MeasuresApplying Additional Performance MeasuresApplying Additional Performance Measures

Constraints Insertion Method

Constraints Constraints Insertion Insertion MethodMethod

Log Transformation

Method

Log Log Transformation Transformation

MethodMethod



31

(1) Constraint Method(1) Constraint Method

Objective FunctionObjective Function



Constraint Method ConceptConstraint Method Concept

AcceptableRange2nd Performance Measure

Fitn

ess V

alue Primary Performance Measure

- Fitness Value Calculation

Additional Performance Measures- Constraint Method



(2) Log Transformation Method(2) Log Transformation Method



Real vs. Log Transformed ValueReal vs. Log Transformed Value

1.215150Travel Time (sec)

2.32002,000Volume(veh/hr)

Log Trans.

10% Alteration

Average Value

Data Type

185 1.1



32

• Convergence•• ConvergenceConvergence



Number of generations

Fitness

0

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0 1 2 3 4 5 6 7

Best Ave.

• As the generation progresses, fitness value for both average value and the best value of each generation converges to “0”

•• Check the reliability of each parameter and compare theCheck the reliability of each parameter and compare theparameter setting of each casesparameter setting of each cases..

(1) Make 100 runs for each (1) Make 100 runs for each ““CalibratedCalibrated”” and and ““DefaultDefault”” models.models.(2) Extract 100 outputs for each model.(2) Extract 100 outputs for each model.(3) Draw a histogram and compare with the field result.(3) Draw a histogram and compare with the field result.


7. Evaluation of calibrated parameter set7. Evaluation of calibrated parameter set7. Evaluation of calibrated parameter set

•• Every simulation model generates a different result to explain Every simulation model generates a different result to explain the variability such as the variability such as ““dayday--toto--day variability.day variability.””

Why 100 runs?Why 100 runs?Why 100 runs?

33

Default

0

5

10

15

20

25

30

35

40

45

20 30 40 50 60 70 80Travel Time (sec)

Frequency

Field: 46.51 sec Calibrated


7. Evaluation of calibrated parameter set7. Evaluation of calibrated parameter set7. Evaluation of calibrated parameter set

•• As a result, As a result, ““Calibrated modelCalibrated model”” can mimic the can mimic the field condition. However, default model could not.field condition. However, default model could not.

•• Conduct another multiple runsConduct another multiple runs•• Compare Compare ““Untried dataUntried data”” with the distribution of simulationwith the distribution of simulationoutputsoutputs

•• Untried data can be either same or different type of dataUntried data can be either same or different type of datathat has been collected on different daythat has been collected on different day

• Model Validation•• Model ValidationModel Validation


•• Verifying calibrated simulation model with untried dataVerifying calibrated simulation model with untried data

• Purpose•• PurposePurpose

8. Model Validation and Visualization8. Model Validation and Visualization8. Model Validation and Visualization

34

•• Conduct additional simulation runs and watch animation Conduct additional simulation runs and watch animation •• Check whether unacceptableCheck whether unacceptableanimation occurs or not.animation occurs or not.

•• If If unrealistic animationsunrealistic animationswere observed, go back towere observed, go back toGA optimization step andGA optimization step andconduct new calibrationconduct new calibrationprocedureprocedure

• Visualization Check•• Visualization CheckVisualization Check

Example of Unrealistic Animation in VISSIM Example of Unrealistic Animation in VISSIM

Some vehicles on right-hand side

lane are trying to make left turn

Some vehicles on Some vehicles on rightright--hand side hand side

lane are trying to lane are trying to make left turnmake left turn



0

5

10

15

20

25

30

10 20 30 40

Max imum Queue (Vehicles)

Fre

qu

en

cy

• Example of validated data•• Example of validated dataExample of validated data



24 Vehicles•• Calibrated with TravelCalibrated with TravelTime dataTime data

•• Validation data wasValidation data wasselected as a selected as a ““Max.Max.queue lengthqueue length””

•• Validation was satisfiedValidation was satisfiedin this casein this case

35

Questions?Questions?

Lesson 3

Exercise 1:00 ~ 3:50

Lesson 3

Exercise 1:00 ~ 3:50

36

ExerciseExercise

Working on the proposed calibration and Working on the proposed calibration and validation procedure by using automated validation procedure by using automated

calibration computer programcalibration computer program

GoalGoalGoal

Isolated Actuated Signalized IntersectionIsolated Actuated Signalized Intersection(Zion Crossroads, VA)(Zion Crossroads, VA)

Selected SiteSelected SiteSelected Site

ExerciseExercise

3. Exercise – CORSIM / Site 153. Exercise 3. Exercise –– CORSIM / Site 15CORSIM / Site 15

4. Demo Presentation – VISSIM / Site 154. Demo Presentation 4. Demo Presentation –– VISSIM / Site 15VISSIM / Site 15

1. Site Description – Site 151. Site Description 1. Site Description –– Site 15Site 15

2. Preparation to Calibration2. Preparation to Calibration2. Preparation to Calibration

37

ExerciseExercise




2. Preparation to Calibration2. Preparation to Calibration2. Preparation to Calibration

ExerciseExercise - 1. Site Description- 1. Site Description

Where is it?Where is it?Where is it?

We are here!

Rt. 250

Rt. 15

Site 15

• Located between

Charlottesville

and Richmond

• Junction of

Rt. 15 and Rt. 250

• Closely located to

Interstate 64

Exit # 136

38

• Fully actuated signal system

• 4-leg intersection with 2 lanes (1 shared lane + exclusive Right-turn lane on each approach

Site 15 OverviewSite 15 OverviewSite 15 Overview

ExerciseExercise - 1. Site Description- 1. Site Description

ExerciseExercise




2. Preparation for the Calibration2. Preparation for the Calibration2. Preparation for the Calibration

39

ExerciseExercise - 2. Prepare for the Calibration- 2. Prepare for the Calibration

Required Types of DataRequired Types of DataRequired Types of Data

GeometryGeometryGeometry

Lane AlignmentLane AlignmentLane Alignment

Lane UtilizationLane UtilizationLane Utilization

Signal SettingSignal SettingSignal Setting

Actual Signal DataActual Signal DataActual Signal Data

Controller SetupController SetupController Setup

Traffic DataTraffic DataTraffic Data

Traffic VolumeTraffic VolumeTraffic Volume

Heavy Vehicle %Heavy Vehicle %Heavy Vehicle %

Turning %Turning %Turning %

PerformanceMeasure DataPerformancePerformanceMeasure DataMeasure Data

Calibration DataCalibration DataCalibration Data

Validation DataValidation DataValidation Data


Data CollectionData CollectionData Collection

•• Prior to the field data collectionPrior to the field data collection•• Geometry InformationGeometry Information•• Signal Setting Signal Setting

Preliminary Data CollectionPreliminary Data CollectionPreliminary Data Collection

•• Apr 15, 22, May 13 (Tue) and Apr 15, 22, May 13 (Tue) and Jun 5 (Thu) in 2003(p.m. peak 5:00~ 6:00 p.m.)5:00~ 6:00 p.m.)

•• Signal SettingSignal Setting•• Performance Measure DataPerformance Measure Data

Field Data CollectionField Data CollectionField Data Collection

• Traffic Data• Other Complimenting Data

40


Preliminary Data CollectionPreliminary Data CollectionPreliminary Data Collection

•• Aerial photo was used as a background imageAerial photo was used as a background image•• Geometry information was confirmed by preliminary site visitGeometry information was confirmed by preliminary site visit

Geometry InformationGeometry InformationGeometry Information

•• Traffic Signal Timing Data obtained from VDOT personnel.Traffic Signal Timing Data obtained from VDOT personnel.•• The Traffic Signal Timing Data Manipulated to be suitable for tThe Traffic Signal Timing Data Manipulated to be suitable for the he

simulation modelsimulation model

Signal SettingSignal SettingSignal Setting



Traffic DataTraffic DataTraffic Data

• Apr. 15, 22, May 13 (Tue) and June 5 (Thu) in 2003(PM peak 5:00~ 6:00 p.m.)

To account for day-to-day variability

Traffic VolumeTraffic VolumeTraffic Volume

Heavy Vehicle %Heavy Vehicle %Heavy Vehicle %

Turning %Turning %Turning %

Recorded by Video Camera

mounted on the STV

Recorded by Recorded by Video Camera Video Camera

mounted on mounted on the STVthe STV

Extracted by watching

recorded video

Extracted by Extracted by watching watching

recorded videorecorded video

41



•• To obtain actual signal timing data (e.g., average green time)To obtain actual signal timing data (e.g., average green time)

Signal Setting DataSignal Setting DataSignal Setting Data


Recorded by Video CameraRecorded by Video CameraRecorded by Video Camera Extracted by watching recorded video

Extracted by watching Extracted by watching recorded videorecorded video

TravelTravelTimeTimeDataData

Reflects the LOS of the signalized intersection

Easy to collect both from the field and the model

Simple calculation method



•• Travel Time on South Bound ApproachTravel Time on South Bound Approach

–– Heavy traffic during PM peak periodHeavy traffic during PM peak period


Recorded the Recorded the

License Plate License Plate

Number from EntryNumber from Entry

Camera 1Camera 1Camera 1

Recorded the Recorded the

License Plate License Plate

Number from ExitNumber from Exit

Camera 2Camera 2Camera 2Time Difference

Travel TimeTravel TimeTravel Time

42




April 15, 2003Tues.

p.m. 5:00 ~ 6:00

April 22, 2003Tues.

p.m. 5:00 ~ 6:00

May 13, 2003Tues.

p.m. 5:00 ~ 6:00

June 5, 2003Thur.

p.m. 5:00 ~ 6:00

Calibration DataCalibration DataCalibration Data Validation DataValidation DataValidation Data

Travel Time Data

N

E

Route 15

Route 250

AA

BB

P1

P2

P3P4

P5

STV

Smart Travel Van (STV)Smart Travel Van (STV)-- A : Vehicle movementA : Vehicle movement-- B : Turning Movement andB : Turning Movement and

Vehicle ClassificationsVehicle ClassificationsVideo CameraVideo Camera-- License Plate (Travel Time)License Plate (Travel Time)P2, P3, P4P2, P3, P4 : Queue Length: Queue LengthP5P5 : Displayed green times: Displayed green times



43

Travel Time DataTravel Time DataTravel Time Data

Travel Time (sec)

No. of Vehicles

0

20

40

60

80

100

120

140

0 20 40 60 80 100 120 140 160 180 200

4/22/2003 5/13/2003 5/20/2003 6/5/2003

ExerciseExercise


- 2. Prepare for the Calibration- 2. Prepare for the Calibration



Travel Time DataTravel Time DataTravel Time Data

April 15, 2003Tues.

p.m. 5:00 ~ 6:00

April 22, 2003Tues.

p.m. 5:00 ~ 6:00

May 13, 2003Tues.

p.m. 5:00 ~ 6:00

June 5, 2003Thur.

p.m. 5:00 ~ 6:00

Travel Time Data

Calibration Data

70.4 sec 53.3 sec 46.5 sec51.5 sec

Validation Data

56.8 sec

44

ExerciseExercise

•• Network was drawn based on the aerial photoNetwork was drawn based on the aerial photo

•• Field collected data was reduced and applied to the modelField collected data was reduced and applied to the model

•• Calibration and Validation Data Collection points were installeCalibration and Validation Data Collection points were installed d

on the exactly same locationson the exactly same locations

•• Multiple test runs were made to check the validity of the Multiple test runs were made to check the validity of the

simulation networksimulation network

•• Network was built with CORSIM and VISSIMNetwork was built with CORSIM and VISSIM

•• Site 15 CORSIM network fileSite 15 CORSIM network file

Network CodingNetwork CodingNetwork Coding


ExerciseExercise

Network CodingNetwork CodingNetwork Coding


2

5

1 3

6

4

Rt.

15

Rt. 250

N

Subject Section: Subject Section: Rt. 15 SB approachRt. 15 SB approach

Travel Time Travel Time measurement sectionmeasurement section-- 960 ft distance960 ft distance-- From node # : 6From node # : 6-- To node # : 2To node # : 2-- Link : ( 6, 2)Link : ( 6, 2)

45

ExerciseExercise





ExerciseExercise - 3. CORSIM / SITE 15- 3. CORSIM / SITE 15

Calibration Program Set UpCalibration Program Set UpCalibration Program Set Up

Enabling MS Excel Macro FunctionEnabling MS Excel Macro FunctionEnabling MS Excel Macro Function

1.1. Load MSLoad MS--Excel ProgramExcel Program

2.2. Tools Tools Macro Macro SecuritySecurity

46


Starting UpStarting UpStarting Up

Simulation Model Simulation Model SelectionSelection

Click on CORSIMClick on CORSIM


Starting UpStarting UpStarting Up

Enables to conductEnables to conduct

Evaluation of Evaluation of ““DefaultDefault”” ModelModel

Run DefaultRun DefaultRun Default

Enables to conductEnables to conduct

Calibration ProcedureCalibration Procedure

Run CalibrationRun CalibrationRun Calibration

47


Default Model TestingDefault Model TestingDefault Model Testing

Number of Multiple runs you want to make

with default model

Defining the location of simulation input file(e.g., CORSIM: *.trf

VISSIM: *.inp)

Defining the location of executable

simulation program

①

②

③

④


Pull down to select your link of interest

5. ( 6, 2)

①

Click on “View Histogram” Button to display the travel time

output distributionCheck out the travel time output value of

each runs

②③

④

After looking at the histogram, click on

“Start Calibration” to Calibrate the model


48


①

②

③

Histogram of 100 Travel Time outputs

Click “Exit” or “Back”button to go back

Check on “yes” and type the field

measured travel time value.

Then click the button on the right side



Calibration Parameter SelectionCalibration Parameter SelectionCalibration Parameter Selection

Let’s try to select the CORSIM

calibration parameters with

“Your own” judgment !!!

Refer to the “worksheet” and

“copy of CORSIM manual”

5~10 minutes

49


Calibration Parameter SelectionCalibration Parameter SelectionCalibration Parameter Selection

Select all the parameters that need to be calibrated by

click on the checkbox of each parameter

①

②

Click on “OK” button to go to next step


Parameter Range DeterminationParameter Range DeterminationParameter Range Determination

Let’s try to determine the range of selected calibration

parameters with “Your own”judgment !!!

Refer to the “worksheet” and “copy of CORSIM manual”

5~10 minutes

Activated text boxindicates selected parameter

50



Type in determined minimum and maximum value for each

selected parameter( For record type 145 and 149, unless you select any specific type, it uses same value for two parameters

①

For the parameters that take distribution format, just use the value for driver type 1

②③

Determine the number of samples that you want to test


Experimental DesignExperimental DesignExperimental Design

Number of Multiple runs you want to make

with default model

Defining the location of simulation input file(e.g., CORSIM: *.trf

VISSIM: *.inp)

Defining the location of executable

simulation program

①

②

③

④

51



Pull down to select your link of interest

5. ( 6, 2)

①

Click on “View Histogram” Button to check out the travel

time output distribution

②③



①

②

③

Histogram of 200 Average Travel Time

outputs

Click “Exit” or “Back”button to go back

Check on “yes” and type in the field

measured travel time value.

Then click the button on the right side

52



Is the set of ranges

acceptable?

YesNo

Adjust

Ranges

Start

Calibration


Parameter Range AdjustmentParameter Range AdjustmentParameter Range Adjustment

①

②

Copy the location information

Click on “Statistical Analysis” button to

start parameter range adjustment

53


Click on “Start Button”

!! If you cannot load the Excel program, please

check your security level again

Remember! It has to be “LOW”.



①

②

③

Paste the location

information that you

copied from last window

Select the simulation

model name that

you are using (e.g.,

CORSIM)

If you are all set, click “Run”


54


Showing the

histogram of

travel time data

Presenting field

travel time value

on the histogram

Creating X-Y plots of travel

time data and each

parameter value



Select the calibration parameters that you want

to generate X-Y plot(e.g., X (Travel time),Y (Parameter Value))

①

②

Click on “Run” button to generate X-Y plots


55


In order to start calibration procedure, you

need to specify...

1. Number of Generations

2. Field Travel Time

3. Link identification number

4. Number of Populations

Type in all those information whenever asked

on the black window

Parameter CalibrationParameter CalibrationParameter Calibration


Look at two files in “bin” folder

result.datresult.datresult.dat Save_parameters.datSave_parameters.datSave_parameters.dat


56



Calibrated Parameter Values

(e.g., “0” means that parameter was not selected

to be calibrated

Click on “Test Parameters” to evaluate

the calibrated model


EvaluationEvaluationEvaluationPull down to select your link of interest

5. ( 6, 2)

①

Travel Time Outputs from multiple runs with

calibrated model

②

Check on the level of Travel Time to

conduct visualization

③

Then, click on “Run percentile” button

④

57


Next Steps…Next StepsNext Steps……

– Watch animation with the calibrated parameters set to maker sure it

is acceptable. If not, go back to one of previous steps…

– Conduct validation of calibrated parameters set using untried

dataset.

ExerciseExercise





58

- 4. VISSIM / SITE 15 Demo Presentation- 4. VISSIM / SITE 15 Demo PresentationExerciseExercise

VIDEOVIDEO

ExerciseExercise

QUESTIONS or

COMMENTS?

QUESTIONS or

COMMENTS?

59

Lesson 4

Discussion4:00 ~ 4:30

Lesson 4

Discussion4:00 ~ 4:30

SummarySummary

• Emphasized the importance of microscopic simulation

model calibration and validation

• Reviewed two commonly used microscopic simulation

models, VISSIM and CORSIM

• Provided microscopic calibration and validation

Procedure

• Demonstrated calibration and validation can be achieved

using a case study of an actuated signalized intersection

•• Emphasized the importance of microscopic simulation Emphasized the importance of microscopic simulation

model calibration and validationmodel calibration and validation

•• Reviewed two commonly used microscopic simulation Reviewed two commonly used microscopic simulation

models, VISSIM and CORSIMmodels, VISSIM and CORSIM

•• Provided microscopic calibration and validation Provided microscopic calibration and validation

ProcedureProcedure

•• Demonstrated calibration and validation can be achieved Demonstrated calibration and validation can be achieved

using a case study of an actuated signalized intersection using a case study of an actuated signalized intersection

60

RecommendationsRecommendations

• Microscopic traffic simulation models under default

parameters shall be used with caution

• Microscopic traffic simulation models should be

calibrated and validated before being used for

evaluating alternatives

• Microscopic traffic simulation models should be

calibrated and validated on the basis of the proposed

procedure

•• Microscopic traffic simulation models under default Microscopic traffic simulation models under default

parameters shallparameters shall be used with caution be used with caution

•• Microscopic traffic simulation models should be Microscopic traffic simulation models should be

calibrated and validated before being used for calibrated and validated before being used for

evaluating alternativesevaluating alternatives

•• Microscopic traffic simulation models should be Microscopic traffic simulation models should be

calibrated and validated on the basis of the proposed calibrated and validated on the basis of the proposed

procedureprocedure

Thank you !Contact Information:Byungkyu (Brian) Park, Ph.D.

Department of Civil Engineering

University of Virginia

Ph: 434-924-6347

Email: [email protected]

Thank you !Contact Information:Byungkyu (Brian) Park, Ph.D.

Department of Civil Engineering

University of Virginia

Ph: 434-924-6347

Email: [email protected]

tutorial: microscopic traffic simulation model calibration...

Documents