Analysis of Driving simulator Analysis of Driving simulator validation for speed researchvalidation for speed research
Stuart T. Godley, Thomas J. Triggs, Brian N. Stuart T. Godley, Thomas J. Triggs, Brian N. FildesFildes
1.Purpose of Research ideas/question1.Purpose of Research ideas/question
To evaluate countermeasures for mean speed using behavioral validation of an advanced driving simulator
Using behavioral validation : relative validity, absolute validity and interactive validity:
to validate the Monash University Accident Research Centre (MUARC) driving simulator.
2. Background support2. Background support
Blaauw(1982) proposed two levels of Blaauw(1982) proposed two levels of validity:validity:
Physical validity is correspondence of Physical validity is correspondence of simulator components, layout, simulator components, layout, dynamics with real cardynamics with real car
Behavioral Validity is correspondence Behavioral Validity is correspondence between simulator and car in the between simulator and car in the way the human operator behavesway the human operator behaves
Rel val. is necessary unlike absoluteRel val. is necessary unlike absolute
Background support: Absolute Vs Relative ValidityBackground support: Absolute Vs Relative Validity
3. Theoretical Basis/ Hypothesis3. Theoretical Basis/ Hypothesis Analysis of behavioral validation, absolute Analysis of behavioral validation, absolute
validity and relative validity is the validity and relative validity is the theoretical basis.theoretical basis.
Beh. Val. :comparison between simulator Beh. Val. :comparison between simulator and real car drivingand real car driving
Absolute Validity: comparison of Absolute Validity: comparison of performance difference between performance difference between experimental conditions in sim. and carexperimental conditions in sim. and car
Relative Validity: difference between Relative Validity: difference between experimental conditions of identical experimental conditions of identical magnitudemagnitude
4. Practical Contribution4. Practical Contribution
Simulator provides experimental control, Simulator provides experimental control, efficiency, expenses, safety, ease of data efficiency, expenses, safety, ease of data collection which may/may not be provided collection which may/may not be provided by real car by real car
Driving patterns in the simulator observed Driving patterns in the simulator observed to improve future carsto improve future cars
5. Theoretical Contribution5. Theoretical Contribution
For the purpose of effective For the purpose of effective countermeasure, speed is validated to be countermeasure, speed is validated to be dependent variable for research using dependent variable for research using simulator.simulator.
Validity of speed covers research Validity of speed covers research investigating the relative differences investigating the relative differences between treatment and control roads.between treatment and control roads.
6. Appropriate Methodologies6. Appropriate Methodologies Driver’s speed response to rumble strips was experimental Driver’s speed response to rumble strips was experimental
stimuli, simulator adjustedstimuli, simulator adjusted Two experiments conducted; driving instrumented car and Two experiments conducted; driving instrumented car and
simulatorsimulator VariablesVariables Instrumented car : 12 males, 12 females, average age 29.8 Instrumented car : 12 males, 12 females, average age 29.8
years years Three sites; approach to stop sign, right curve, left curveThree sites; approach to stop sign, right curve, left curve 2 Treatment sites for simulator and real road: Consisted of 2 Treatment sites for simulator and real road: Consisted of
rumble stripsrumble strips 2 control sites for simulator and real road: without rumble 2 control sites for simulator and real road: without rumble
stripsstrips Treatment and control matched for on road off road detailsTreatment and control matched for on road off road details
Methodologies...Methodologies... Rumble strips : width 60 cm, preceded Rumble strips : width 60 cm, preceded
intersection by 30-50 m, spaced at decreasing intersection by 30-50 m, spaced at decreasing distancesdistances
Two experimental routes: half male/female on Two experimental routes: half male/female on each route, had vehicles parked on themeach route, had vehicles parked on them
MethodMethod Instructed to drive normally, experimenter in Instructed to drive normally, experimenter in
front seat, technician at backfront seat, technician at back First a practice drive:11 min, then experimental First a practice drive:11 min, then experimental
drive: ~40-50 min.drive: ~40-50 min.Data collectionData collectionInstantaneous speed recorded at each intersectionInstantaneous speed recorded at each intersection
Methodologies...Methodologies...
Some participants excluded from data pool for Some participants excluded from data pool for individual treatment-control analysis: unrestricted individual treatment-control analysis: unrestricted drive to intersection, slow moving car, school drive to intersection, slow moving car, school children walking, etcchildren walking, etc
No participant removed from all analysisNo participant removed from all analysis
Validity of measuresValidity of measures Participants had to sign a consent form approved Participants had to sign a consent form approved
by Monash University Ethics Committeeby Monash University Ethics Committee
Methodologies...Methodologies...
22ndnd method: Simulator Experiment method: Simulator Experiment
VariablesVariables Simulator car : 12 males, 8 females, average age Simulator car : 12 males, 8 females, average age
26.4 years 26.4 years Projected through computer :resolution 640x 480 Projected through computer :resolution 640x 480
pixels, rate of 30 Hz. 180 deg forward, 60 deg pixels, rate of 30 Hz. 180 deg forward, 60 deg horizontal, 45 deg vertical viewhorizontal, 45 deg vertical view
Rest all variables are sameRest all variables are sameMethod and data collection were the same; except there was Method and data collection were the same; except there was
no hindrance while driving; hence collection of datano hindrance while driving; hence collection of data
7. Statistical Analysis7. Statistical Analysis A modified correlation analysis based on A modified correlation analysis based on
canonical correlation was used.canonical correlation was used. Correlation of each participant’s data not feasible: Correlation of each participant’s data not feasible:
separate participants used in each expseparate participants used in each exp Measures of effect size in ANOVA are measures of Measures of effect size in ANOVA are measures of
degree of association between effect and degree of association between effect and dependent variable (ex: main effect, linear dependent variable (ex: main effect, linear contrast, an interaction)contrast, an interaction)
Two one-way ANOVA calculated: one for Two one-way ANOVA calculated: one for treatment, other for controltreatment, other for control
Omega squared and interclass correlation Omega squared and interclass correlation estimate degree of association in presentationestimate degree of association in presentation
Statistical Analysis...Statistical Analysis...
Non significant results validate simulator, but Non significant results validate simulator, but may arise from inadequate statistical power may arise from inadequate statistical power rather than genuine absence of difference.rather than genuine absence of difference.
Small omega square reflects genuine non-Small omega square reflects genuine non-differences; large value suggests a difference differences; large value suggests a difference exists, but insufficient sample size.exists, but insufficient sample size.
Omega square at or below 0.01 meaninglessOmega square at or below 0.01 meaningless
•F- test used to interpret the significant differences in
experiments
•Alpha = 0.05
•95% confident that differences are significant
•Correlation used to support interactive relative validity
•Size effect verify if the non-significant result can be
conclusive
8.Presentation of Results8.Presentation of Results
Results : Stop Sign ApproachResults : Stop Sign Approach
Results : Stop Sign ApproachResults : Stop Sign ApproachSignificant difference between mean speed at treatment site relative to the control site ( P<0.001 )
Very small size effect (0.002) to support this non-significant result
Average relative validity was established
The pattern of speeds for the treatment site and control site similar in both experiments, supported by a significant correlation (0.40)
Interactive relative validity established
Speeds for the two sites converged as they approach the intersection in the simulator
This pattern was not observed in the data for the on-road experiment
Absolute validity was not established
Results:Right Curve Approach ApproachResults:Right Curve Approach Approach
Results:Right Curve Approach ApproachResults:Right Curve Approach ApproachCar: No significant difference between the mean speed Simulator: Significant difference between the mean speed Significant interaction between 2 experiments (P<0.01)Average relative validity was not established Speed pattern similar for the first three-quarters of the measurement areaSignificant correlation (0.52) supported interactive relative validityRight curve treatment site speeds not statistically different between the car and simulator experiments (P = 0.590)Small effect size(0.007) supports this non-significant result.Control site speed slower for instrumented car experiment compared to simulator experiment (P<0.05)Absolute validity established for treatment sites, but not control sites
Results:Left Curve Approach ApproachResults:Left Curve Approach Approach
Results:Left Curve Approach ApproachResults:Left Curve Approach Approach
Significant difference between mean speed at treatment site relative to the control site (P<0.001) Significant speed differences between the two experiments (P<0.001)Averaged relative validity not establishedSame speed pattern for the first three quarters of data collection areaSignificant correlation (0.50) supported interactive relative validity Significant speed differences in treatment site for instrumented car and simulator experiments (P<0.001)Control site speeds not significantly different, but produced a size effect close to medium Non-significant result cannot be considered as a conclusive. Absolute validity was not established.
Relative validation: Treatment speeds started to reduce relative to
their control sites, before the rumble strips Significant correlations in relation to interactive
relative validity. Rumble strips led to an average slower
approach speed to the stop sign intersection in both the simulator and on-road trials
Relative validity well established for mean speed
Absolute Validation:Absolute speed values were generally different in the two experimentsGeneral trend: simulator induce slower speeds than instrumented cars
Results: Relative and Absolute ValidationResults: Relative and Absolute Validation
9. Conclusion : Relative Validity9. Conclusion : Relative Validity
Simulator produced larger average speed differences between its treatment and control sites during the curve approaches
Two reasons: 1. On-Road : Speeds initially faster at the treatment relative to the
control sites ; Non Perfect Site Matching
Wider lane widths conducive to faster driving Treatment site roads may have geographical characteristic that
encouraged faster speeds
2. Treatment site speeds started to reduce further back before the first of the rumble strips in the simulator experiment
main difference between the simulator and the on-road experiments
Without the two experimental differences, average relative validation for the curves may have been established.
Absence of G-force motion cues in simulator
Intentionally short practice section may have contributed to simulator participants to slow earlier when they perceive the rumble strips
Converging speed pattern could only be determined when the vehicle stopped for on-road trial, rather than when the front of the simulator car crossed the stop line
Simulator may have slightly enhanced the speed differences
Address limitations and these differences should be able to be minimized.
Simulator does seem to be a valid tool for generating and generalizing relative speed results for experiments involving road based speeding countermeasures aiming to influence decelerating.
9. Conclusion : Relative Validity9. Conclusion : Relative Validity
9. Conclusion : Absolute Validity9. Conclusion : Absolute Validity
Different absolute speed values in two experiments not particularly negative because simulator experiments only concerned speed differences between road environments
Experiments did not attempt to establish the numerical speeds at which the investigated road manipulations encourage drivers to drive
9. Conclusion :Overall9. Conclusion :Overall Evidence to conclude that speed is a valid measure to use for
the experiments on driving simulator involving road based speeding countermeasures.
Speed profiles found indicate a speed reduction relative to control roads or other roads.
Speed has been clearly validated as a dependent variable for research using the simulator.
Validity of speed only covers research investigating the relative differences between road treatments and control roads.
Inconsistencies between the two experiments account for:
1. Differences in characteristics of the road and road environment between the treatment and control sites in the on-road experiments hindered the validation.
2. Procedural methods with the simulator, notably practice, may have also contributed.
10. Future Work/Research Directions:10. Future Work/Research Directions:Reduce the inconsistencies between the two experiments.
Include investigating absolute numerical speed values for validity of speed
More similar initial speeds for the on-road curve sites, so that a more similar treatment-control site speed difference to the simulator experience may be found.
Conduct experiment with same participants for the on-road and simulator trials so as to observe any difference in the results.
Experiments can be modified so that converging patterns can appear for both on-road and simulator data.
Attempt to establish the numerical speeds at which the investigated road manipulations encourage drivers to drive
Validity of speed to investigate the absolute differences between road treatments and control roads.