simulation of driver behavior with agent-based back ...web.mit.edu/linsenc/www/simulation of driver...

76

Click here to load reader

Upload: duongbao

Post on 27-Aug-2018

214 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

44

but does require sufficient real traffic and action data to capture theunderlying relationship between states and actions. Therefore, ANNmodels estimate actions on the basis of the real state–action mappingof natural behavior.

Data from the naturalistic driving database of the NaturalisticTruck Driving Study (NTDS) (1), collected by the Virginia TechTransportation Institute and Blanco et al. (2), are used to find thereal causalities and responses of truck drivers in car-following situ-ations. In this paper, first the process of calibrating the well-knownGazis–Herman–Rothery (GHR) car-following model for an individualdriver is described. Then, a back-propagation (BP) neural networkis constructed to train agents that represent individual drivers. Bothmethods use the same naturalistic data set.

CAR-FOLLOWING MODELS

Brief Review

Many car-following models have been developed in the past 50 yearsto represent longitudinal driver behavior, including safety distancemodels, collision avoidance models, psycho-physical action pointmodels, and models based on fuzzy logic (3). Most well-knowncar-following models have been embedded in microsimulationsoftware, such as the Pipes model in CORSIM (4), the Gipps modelin AIMSUN (5), the Fritzsche model in Paramics (6), and theWiedemann model in VISSIM (7 ).

Car-following models assume that the following vehicle reactsaccording to observed stimulus from its leader according to predefinedrules. The models mentioned above require specific defined functionsto relate stimuli that the following vehicle observes to the reactionit takes. For example, the GHR model uses speed difference andspace headway as stimuli to determine acceleration of the followingvehicle. The Wiedemann model divides headway and speed differencespace into several driving regimes with predefined thresholds, wherethe following vehicle reacts differently each regime. The Wiedemannmodel uses the differences between actual and desired followingdistances as a stimulus in the closing-in regime, a calibrated accel-eration in the following regime, and a desired speed as the drivingobjective in the free-driving regime (7). The Gipps model uses vehicledynamics as constraints and derives acceleration of the followingvehicle from estimated deceleration of the leading vehicle (5).

Driver Behavior Simulation

The calibration of a car-following model is an important process torepresent driver behavior and simulate vehicle trajectory. Calibrationparameters are considered to be driver dependent and to remain

Simulation of Driver Behavior with Agent-Based Back-Propagation Neural Network

Linsen Chong, Montasir M. Abbas, and Alejandra Medina

Two microscopic simulation methods are compared for driver behavior:the Gazis–Herman–Rothery (GHR) car-following model and a proposedagent-based neural network model. To analyze individual driver charac-teristics, a back-propagation neural network is trained with car-followingepisodes from the data of one driver in the naturalistic driving databaseto establish action rules for a neural agent driver to follow under perceivedtraffic conditions during car-following episodes. The GHR car-followingmodel is calibrated with the same data set, using a genetic algorithm. Thecar-following episodes are carefully extracted and selected for modelcalibration and training as well as validation of the calibration rules.Performances of the two models are compared, with the results showingthat at less than 10-Hz data resolution the neural agent approach out-performs the GHR model significantly and captures individual driverbehavior with 95% accuracy in driving trajectory.

The simulation of driver actions in traffic is an important part ofmodeling microscopic driver behavior. A driver action indicatesdriver behavior in terms of causalities and responses to traffic flow.Microscopic car-following models provide many powerful simula-tion tools to study individual driver behavior, interactions betweenleading and following vehicles, and cumulative macroscopic trafficphenomena. The performance of car-following models relies on theparameters of individual drivers that can represent unique drivingbehavior. Parameter calibration becomes a necessary process beforecar-following models can be applied to a simulation environment.

Driver actions in car-following models are defined by predefinedrules. These rules are mostly interpreted by relating the traffic statethat a driver observes to the response or action that the driver takes.Different car-following models consider different criteria as causal-ities that stimulate a driver’s reactions. However, in reality, theserules specified by car-following models might not capture naturaldriving behavior because of the complexity and instability of thehuman decision-making process.

In the proposed approach, instead of using predefined drivingrules from car-following models, a reactive-structure artificial neuralnetwork (ANN) is used to relate traffic states to driver actions. AnANN does not require a function to connect traffic states to actions

L. Chong, 301-D Patton Hall, and M. M. Abbas, 301-A Patton Hall, Charles Via, Jr.,Department of Civil and Environmental Engineering; and A. Medina, Virginia TechTransportation Institute, 3500 Transportation Research Plaza; Virginia Polytech-nic Institute and State University, Blacksburg, VA 24061. Corresponding author:M. M. Abbas, [email protected].

Transportation Research Record: Journal of the Transportation Research Board,No. 2249, Transportation Research Board of the National Academies, Washington,D.C., 2011, pp. 44–51.DOI: 10.3141/2249-07

Page 2: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

Chong, Abbas, and Medina 45

constant during all car-following situations (7 ). Therefore, thecalibration process becomes an optimization problem with theobjective of searching for the best car-following model parametersto minimize the deviation of the estimated car-following trajectoryfrom the naturalistic vehicle trajectory. Accordingly, an optimalparameter set is considered to best represent real driver behavior.

Shortcomings

Both macroscopic and microscopic data have been used in the cal-ibration of car-following models in much of the literature. Macro-scopic traffic data provided by loop detectors has been used to calibratemacroscopic parameters such as free-flow speed, jam density, andspeed at capacity. Parameters in car-following models can be rep-resented by steady-state macroscopic parameters of traffic flow (8).However, loop detector data includes all the vehicles that have passedby, which results in calibration parameters representing an averagebehavior of all the drivers rather than individual drivers.

Microscopic calibration uses vehicle trajectory data (speed, dis-tance, and so on) to calibrate the unique parameters of individualdrivers. Microscopic data collection requires specialized sensor-equipped instrumented vehicles or above-road video observation ofvehicles (9, Appendix A), which means that collection efforts areexpensive and time-consuming. However, technological advancesin recent years in methods of microscopic data collection havefueled an increase in studies with trajectory data in the calibrationof car-following models.

Although car-following models can describe and explain driverbehavior to some extent, logical and statistical errors reduce thereliability of these models. From real-world experience, the drivertakes action on the basis of the interactions of all observed stimuliin his or her environment. Unfortunately, most car-following modelsconsider only some stimuli in the causality of driver response, whichmay result in a biased conclusion. In addition, the parameters maynot be constant for different stimuli. Studies have shown that reactiontime while accelerating is longer than while decelerating (10). Inaddition, the sensitivity parameter of speed difference is 10% greaterin deceleration than in acceleration (11).

Statistically, the chosen calibration method causes biases. As Ossenand Hoogendoorn point out, measurement errors can create consid-erable bias in estimation results (12). Parameters that minimize theobjective function do not necessarily capture the following dynam-ics best, and measurement errors substantially reduce the sensitivityof the objective function and consequently reduce the reliability ofthe results. Brockfeld et al. tested the validity of 10 car-followingmodels after calibration and found that no model appeared to besignificantly better than any other model; all models shared the sameproblem with particular data (13). The authors concluded that eventhough some models had more calibration parameters, they did notprovide better results in general.

AGENT-BASED MODEL

An agent that can learn driver behavior from previous actions takenin the observed traffic environment will adopt driver behavioralpolicy and may act intelligently as its clone, thus avoiding theabove-mentioned problems. The simulator observes all traffic statestimuli, makes judgments on the basis of critical stimulus or stimuli,and tries to replicate driver reactions.

Agent-based modeling (ABM) is a relatively new paradigm forexploring complex system behavior (14). Within the transportation

domain, ABM is particularly appropriate for modeling humandecision making and action systems. Driver behavior studies withABM include driver response to incidents, driver interaction betweenpassenger vehicles and trucks, and driver behavior when approachinga work zone (15–17 ). Bonabeau suggests that ABM is best appliedto simulations when agents are heterogeneous; interactions of agentsare complex, nonlinear, or discontinuous; and agents exhibit complexbehavior, including learning and adaptation (18).

The ABM system in this paper uses an agent simulator to approx-imate a driver and car-following situations from the driver as trafficstate scenarios experienced by the agent. An ANN is used in agenttraining. The agent receives supervised training based on the drivingcharacteristics and state–action pairs as extracted from the naturalistictrajectory data. After training, the neural agent should be able toreplicate the action selection policy.

Neural networks have been applied in car-following model design(19, 20). Hongfei et al. applied a neural car-following model withtest data collected with a five-wheel system (19). Speed of the fol-lowing vehicle, relative speed, relative distance, and desired speedwere selected as four inputs from test data and acceleration of thefollowing vehicle as output. Three types of drivers (risky drivers,ordinary drivers, and conservative drivers) were classified by desiredspeed as the fourth input to represent driver heterogeneity. Theclassification of different drivers based on desired speed alone causesthe loss of other driver-dependent characteristics. Furthermore, theirstudy did not validate performance of the neural network.

Panwai and Dia developed a neural car-following model andimplemented it in the AIMSUN simulation software (20). Themodeling data included speeds and distance headways of leader andfollower, based on the premise that the driver would select individualspeeds and maintain a desired headway. Driving conditions weredivided into five regimes by distance headway and speed difference.The developed model was interfaced with AIMSUN in validationand compared with the default Gipps model in AIMSUN. Results ofthe BP neural network were 20% better than those of the Gipps modelembedded in AIMSUN. Although this novel model approximatedfield data well, the five regimes seemed arbitrary and the output isspeed, which is a state (not an action decision).

In this paper, the performance of ANNs is tested by using longi-tudinal driver action acceleration (deceleration) as the driver action.A neural agent is trained to replicate the action decision processof a real driver. According to the nature of ANN, this neural agentshould be able to adapt to changes in driver behavior and insufficienttrajectory data.

GHR MODEL

The GHR model is a general form of earlier car-following models.Driver action (acceleration) is considered to be a function of speed v,speed difference, and spacing x. The model formulation is

where

an = acceleration of vehicle n at time t;Δx and Δv = relative spacing and speed, respectively, between

the leader vehicle n − 1 and the following vehicle n;T = driver’s perception–reaction time; and

c, m, l, and T = four calibration parameters.

a t cv tv t T

x t Tn n

ml

( ) = ( ) Δ −( )Δ −( )

( )1

Page 3: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

46 Transportation Research Record 2249

How well the GHR model simulates driver behavior depends on thecalibration parameters. As mentioned previously, many parametersdo not necessarily have a better approximation on vehicle trajectory.Therefore, the four-parameter GHR model seems to be an acceptablecar-following model in this study.

The minimum sum squared error (SSE) between the estimated andreal speeds is used as the objective function. The optimization tooladjusts the car-following model parameters to make the best matchbetween the GHR estimated speed vest and the field-measured speed vf.

Genetic algorithms were used as the optimization tool for searchingthe parameter space. To make the optimization reasonable, constraintswere applied to the parameter space for minimum and maximumvalues from past empirical studies and calibration results (3): m wasrestricted to (0, 1.5), l was restricted to (0, 2.5), and T was restrictedto (0, 2).

NEURAL AGENT MODEL

In the proposed neural agent model, the driver agent observes thetraffic state (its environment) and reacts, which is similar to the stateperception and action mapping of a real-world driver. This approachis based on a reactive structure using an ANN. Unlike car-followingmodels, an ANN does not need a predefined function or an equationto associate traffic state and actions. Instead, an ANN extracts trafficstate and action mapping rules from naturalistic driving states andactions. An ANN receives training from drivers’ state–action pairsin naturalistic driving data sets and establishes state–action mappingrules for drivers to follow. With a limited set of input and outputtraining data, an ANN can provide state–action mapping rules forthe entire state space, even when some state patterns are missingfrom the training data.

A BP neural network is applied as the state–action mapping rulefor estimating action in the proposed approach. As the term implies,BP is a propagation of error that requires an agent to have basicknowledge of the desired output action from the training data. A BPneural network calculates the error between the desired output andthe actual BP output to propagate error back to each neuron in thenetwork. Network weights between layers are updated by traininguntil the error propagation becomes relatively small and weight valuesconverge. A BP neural network follows learning rules associatedwith the gradient descent algorithm to gradually approximate driveractions from input data and target data from the training episodes.

A BP neural network is composed of an input layer, hidden layer(s),and an output layer. The kth hidden-layer vector s(k) is computed fromits upstream layer input vector s(k − 1). A weighted sum of input andbias is calculated, and results are transformed by a transfer function:

S

S

S

h wk

k

m k

i1,

2,

.

.

.

,

⎢⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎥⎥⎥⎥⎥⎥⎥⎥⎥⎥

=

,, ,

, ,

.

.

.

1 1 11

1

1k

i kk

i

n

i mk

i k

s b

h w s

k

−=

+⎡

⎣⎢

⎦⎥

++⎡

⎣⎢

⎦⎥

⎪⎪⎪⎪⎪⎪

⎪⎪⎪⎪⎪⎪

⎪⎪⎪⎪⎪⎪

=

∑ bmk

i

nk

1

1

. ⎭⎭

⎪⎪⎪⎪⎪⎪

( )3

SSE est= −( )=∑ v vft

N2

0

2( )

where

sm,k = value of the mth hidden neuron,h(�) = transfer function,

nk = number of neurons in the kth layer,w k

i,m = weight connecting the ith input and the mth hidden neuronsin the kth layer, and

bkm = bias for the mth hidden neuron in the kth layer.

A nonlinear sigmoid transfer function is used here. This functiontakes the value from the summation results and turns them intovalues between 0 and 1:

Similarly, the output layer vector y(k) is calculated as

where woi,l is the weight connecting the ith hidden and the lth output

neurons and bl is bias for the lth hidden neuron. The structure of theneural network is illustrated in Figure 1.

The BP learning algorithm is divided into two phases: propagationand weight update. The propagation phase transfers training inputforward through a neural network to generate the propagation’s outputactivations. Then, BP of output activations is transferred through aneural network using the target output to generate the gradient of alloutput and hidden neurons. In the weight update phase, output deltaand input activation are multiplied to get the gradient of weight.Weights are brought in the opposite direction of the gradient by sub-tracting a ratio from the weight learning rate. One iteration of BPlearning can be written as

where

Wk = vector of current weights and biases,αk = learning rate, andgk = current gradient.

The agent receives the traffic environment information as the inputlayer of the neural network. Each input is weighted with an appro-priate weight w. The sum of weighted inputs and bias becomes inputto the transfer function in the hidden layer(s). The neurons of the lastlayer are the output of the transform function.

To compare the driver agent results with the GHR car-followingmodel, the same traffic state, vehicle speed, relative distance, andspeed were used as variables in the input layer and driver actionacceleration as the single output in the output layer during driver agenttraining. Because the BP training process requires input and outputbehavioral examples, numerous car-following episode trajectoriesand corresponding accelerations from the naturalistic driving datawere used in the training process.

W W gk k k k+ = −1 6α ( )

y

y

y

h w s

l

io

i k

1

2

.

.

.

, ,⎡

⎢⎢⎢⎢⎢⎢⎢⎢⎢

⎥⎥⎥⎥⎥⎥⎥⎥⎥

=

−1 11 11

11

+⎡

⎣⎢

⎦⎥

+

=

−=

b

h w s b

o

i

n

i lo

i k lo

i

n

k

k

.

.

.

, ,

⎡⎡

⎣⎢

⎦⎥

⎪⎪⎪⎪⎪⎪

⎪⎪⎪⎪⎪⎪

⎪⎪⎪⎪⎪⎪

⎪⎪⎪⎪⎪⎪.

( )5

h ze

( ) =+

−−

2

11 4

2z( )

Page 4: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

Chong, Abbas, and Medina 47

NATURALISTIC DRIVING DATA

NTDS Data Collection

For this research, NTDS data collected by Virginia Tech Transporta-tion Institute were used to test the proposed method. In contrast totraditional epidemiological and experimental or empirical approaches,this in situ process used drivers who operated vehicles equippedwith specialized sensor, processing, and recording equipment. Ineffect, the vehicle was a data-collection device. Drivers operatedand interacted with these vehicles during their normal driving routines,and the data-collection equipment continuously recorded numerousitems of interest during the entire time.

Naturalistic data-collection methods require a sophisticatednetwork of sensor, processing, and recording systems. They providea diverse collection of on-road driving and driver (participant,nondriving) data, including driver input and performance (e.g., laneposition, headway), four camera views, and driver activity data. Thisinformation may be supplemented by subjective data, for example,from a questionnaire.

Four companies and 100 drivers participated in the NTDS study(2). The NTDS data-acquisition system collected three forms ofdata: video, dynamic performance, and audio. Participants wererecruited from four trucking fleets across seven terminals, and oneto three trucks in each fleet were instrumented (nine trucks total).Each participant in this on-road study was observed for approximatelyfour consecutive work weeks. After 4 weeks of data were collectedfor a given participant, another participant started driving the instru-mented truck. The NTDS study collected data for approximately14,500 driving data hours, covering 735,000 mi traveled in thenine instrumented trucks.

In the proposed test, the following vehicle is the instrumentedvehicle. The measurements of the following vehicle include speed,longitudinal and lateral accelerations, yaw angle, heading, and turnsignal indications. For leading-vehicle information, the range, rangerate, and azimuth were collected by instrumented forward-viewingradar from the following vehicle. Both leading- and following-vehicledata were recorded at 10 Hz. Speed (collected from the speedometer)and range and range rate (from radar) were used as inputs in both GHR

model calibration and neural agent training. Videos were observedto confirm trajectory data.

Extraction and Selection of Car-Following Episodes

Car-following situations were automatically extracted from the data-base to analyze the behavior of following drivers. In the iterativefiltering process, initial values and conditions were used to flag events,then events were compared with video data and values adjusted tominimize noise. Visual inspection of the first subsets created revealedsome false positives (i.e., non-car-following events flagged as car-following events), so additional filters were applied to remove suchevents from the database.

Car-following episodes were extracted according to the followingconditions:

• Radar target ID > 0 (eliminates points in time without a radartarget detected),

• Radar range ≤ 120 m [represents 4 s of headway at 70 mph(112 km/h)],

• −1.9 m < range * sin (azimuth) < 1.9 m (restricts data to onlyone lane in front of the lead vehicle),

• Speed ≥ 20 km/h (minimizes the effect of traffic jams butleaves the influence of congestion),

• Rho-inverse ≤ 1/610 m−1 (limits roadway curvature such thatvehicles were not misidentified as being in the same lane as thesubject vehicle), and

• Length of car following ≥ 30 s while range ≤ 61 m.

The automatic extraction process was verified with video analysis.For the random sample of 400 episodes, 392 were valid.

Car-following episodes selected for neural network trainingshould cover a wide range of traffic states. A neural agent shouldbe able to choose proper actions that are from states the agent hasnot experienced but are within the training range. A neural networkinterpolates the action for the missing state. In most car-followingepisodes, the driver travels at a high speed on a highway or inter-state. To avoid missing low speeds in the traffic states, low-speed

Speed V

Spaceheadway

ΔX

Speeddifference

ΔV

Acceleration A

Input Layer

HiddenLayer

Outputlayer

Data fromspeedometer

Data from radar

FIGURE 1 Neural agent structure.

Page 5: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

48 Transportation Research Record 2249

episodes also are used in training and calibration. Choosing thegood episodes is not easy.

Although 1,133 car-following episodes were ready to use in thepresent study, using all of them might have caused redundancy intraining. To validate the neural agent model, episodes exhibitingstable driving behavior are desired. Driver actions in some episodeswere inconsistent, and using them would generate more noise. In thepresent paper, 10 car-following episodes with 10,732 traffic statesand actions were chosen for the test.

EXPERIMENT

In previous research, the present authors used one car-followingepisode (64 s) in BP ANN training and GHR model calibration (21).Trajectory data at 1- and 10-Hz resolutions were used to compareagent and GHR performances and to test the contribution of high-resolution data to calibration and training results because 10-Hz datahave 10 times more traffic states in training. At both resolutions, theneural agent model outperformed GHR. With a 1-Hz resolution ofnaturalistic data in training and calibration, the neural agent modelhas 30.5% less SSE (Figure 2). With a 10-Hz data resolution, theneural agent model significantly improved the degree of accuracy, butthe GHR model did not (Figure 3). Therefore, a 10-Hz resolutioncan capture driver actions well in neural network training.

Even though a neural network can work perfectly on one episode,using only one episode for training is insufficient for capturing stabledriver behavior. As mentioned previously, training data should covera large range for state–action mapping rules. However, using multiplecar-following episodes may result in unstable actions because driveractions are not completely consistent. More data might reduce bias,but the calibration and training results might not work as well as withone episode.

In this paper, 10 episodes from one driver were selected from1,133 episodes of naturalistic driving data available from the NTDSdatabase, including 10,732 traffic state data points for use in GHR

model calibration, neural agent training, and validation of calibrationrules. The R2 value was used to measure the degree of accuracybetween speeds from naturalistic data and estimated speeds frommodels in validation.

The naturalistic data points were randomly divided into three sets;60% were used to train the network, 20% were used to validate howwell the network generalized, and 20% provided an independent testof network generalization to data that the neural agent had never seen.The performance function of the neural agent training data using allof the available data points is illustrated in Figure 4. Starting at alarge value, the performance function decreases to a smaller valueover the training process, showing that the neural agent is learning.Training stopped when the number of iterations, the performancefunction, the magnitude of the gradient, and training time weregreater than or less than the predefined threshold values. Validationstopped when the validation error increased. Training continued aslong as validation errors decreased. According to Figure 4, trainingstopped after 68 iterations, when the network generalized the best forvalidation. This result is reasonable because test error and validationerror have similar characteristics and no significant overfitting hasoccurred.

One car-following episode (48 s) is illustrated as an example ofvalidation performance for both models. The neural agent model hasmilder accelerations than the field data and simulates driver behaviorwell, with an R2 of 95% (Figure 5). However, validation indicates thatthe GHR model does not simulate driver behavior well, even withthe optimal calibrated parameter sets; R2 is 57% (Figure 6). Also,results of the neural agent model look more continuous and probablyare a good approximation of realistic driver actions.

CONCLUSIONS AND FUTURE RESEARCH

A GHR car-following model calibration method using a geneticalgorithm was proposed. Then, an agent-based BP neural networkmodeling approach to simulate driver behavior was described.

0 10 20 30 40 50 60 704

6

8

10

12

14

16

18

Time (s)

Spe

ed (

m/s

)

Naturalistic DataAgentGHR model

FIGURE 2 Speeds of neural agent and GHR model (1 Hz).

Page 6: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

Chong, Abbas, and Medina 49

Naturalistic DataAgentGHR model

0 10 20 30 40 50 60 704

6

8

10

12

14

16

18

Time (s)

Spe

ed (

m/s

)

FIGURE 3 Speeds of neural agent and GHR (10 Hz).

FIGURE 4 Neural agent model training, validation, and test [performance(mean squared error between actual and ANN model output) � 0.017359,goal � 0].

Naturalistic microscopic vehicle trajectory data were used in bothneural agent training and GHR model calibration. Validation wasperformed on both models, and results indicated that the neural agentcould capture driver behavior well with properly selected trainingdata. The neural agent model performs better than the GHR model.

Because robust calibration has been reported for the Gipps modeland intelligent driver model of car following (indicating that thesemodels are more robust than the GHR model), future research mayfocus on testing optimal parameters for these models and comparison

with a BP neural network. Also other types of ANNs will be testedin driver action estimation.

The neural agent is a successful method for modeling driverbehavior in car-following scenarios. Because a neural agent canlearn from driver reactions in different traffic environments, theheterogeneity of actions that a driver when facing various stimuli(which current car-following models cannot handle) should bestudied. For example, it will be interesting to study the executionand duration of lateral lane-changing behavior. In the car-following

Page 7: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

50 Transportation Research Record 2249

0 5 10 15 20 25 30 35 40 45 50-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

Time (s)

Acc

eler

atio

n (m

/s2)

Naturalistic Data

Agent

FIGURE 5 Neural agent acceleration action in validation.

25

25.5

26

26.5

27

27.5

28

28.5

Spe

ed (

m/s

)

Naturalistic Data

Agent

GHR model

0 5 10 15 20 25 30 35 40 45 50

Time (s)

FIGURE 6 Validation example for neural agent and GHR model.

episodes used, especially in congested traffic environments, driverstypically made longitudinal (acceleration and deceleration) andlateral (swerving) maneuvers simultaneously. Because more data areavailable with car-following episodes for different drivers—includingsafety-critical events in which actions are more complex—it isfeasible to use lateral trajectory information in training input and tosimulate actions based on steering wheel motion.

This paper focused mainly on the longitudinal behavior of indi-vidual drivers; however, lateral behavior such as lane changing canbe analyzed by using a similar approach. Whereas in this research

scope the car-following episodes used were from the same driver,future research could be done to test the robustness of a neural agentfor other drivers.

ACKNOWLEDGMENTS

This material is based on work supported by the Federal HighwayAdministration. The authors thank C. Y. David Yang, FHWA agree-ment manager, for support and guidance during this project. They

Page 8: Simulation of Driver Behavior with Agent-Based Back ...web.mit.edu/linsenc/www/Simulation of Driver Behavior with Agent... · teristics, a back-propagation neural network is trained

Chong, Abbas, and Medina 51

also thank individuals at Virginia Polytechnic Institute and StateUniversity and the Virginia Tech Transportation Institute whocontributed to the study in various ways: Bryan Higgs, Zain Adam,Greg Fitch, Shane McLaughlin, Brian Daily, and Rebecca Olson.

REFERENCES

1. Olson, R., R. J. Hanowski, J. S. Hickman, and J. L. Bocanegra. DriverDistraction in Commercial Vehicle Operations. Federal Motor CarrierSafety Administration, U.S. Department of Transportation, 2009.

2. Blanco, M., J. S. Hickman, R. L. Olson, J. L. Bocanegra, R. J. Hanowski,A. Nakata, et al. Investigating Critical Incidents, Driver Restart Period,Sleep Quantity, and Crash Countermeasures in Commercial VehicleOperations Using Naturalistic Data Collection. Federal Motor CarrierSafety Administration, U.S. Department of Transportation, in press.

3. Brackstone, M., and M. McDonald. Car-Following: A Historical Review.Transportation Research Part F, Vol. 2, No. 4, 1999, pp. 181–196.

4. Pipes, L. A. Operational Analysis of Traffic Dynamics. Journal ofApplied Physics, Vol. 24, No. 3, 1953, pp. 274–281.

5. Gipps, P. G. A Behavioural Car-Following Model for Computer Simula-tion. Transportation Research Part B, Vol. 15, No. 2, 1981, pp. 105–111.

6. Fritzsche, H.-T. A Model for Traffic Simulation. Transportation Engi-neering and Control, Vol. 35, No. 5, 1994, pp. 317–321.

7. Wiedemann, R., and U. Reiter. Microscopic Traffic Simulation: TheSimulation System MISSION, Background and Actual State. ProjectICARUS (V1052), Final Report. Coopération Education Culture,Brussels, Belgium, 1992.

8. Rakha, H. A., C. C. Pecker, and H. B. B. C. Cybis. Calibration Proce-dure for Gipps Car-Following Model. In Transportation ResearchRecord: Journal of the Transportation Research Board, No. 1999, Trans-portation Research Board of the National Academies, Washington,D.C., 2007, pp. 115–127.

9. Ossen, S., and S. P. Hoogendoorn. Car-Following Behavior Analysisfrom Microscopic Trajectory Data. In Transportation Research Record:Journal of the Transportation Research Board, No. 1934, TransportationResearch Board of the National Academies, Washington, D.C., 2005,pp. 13–21.

10. Forbes, T. W. Human Factor Considerations in Traffic Flow Theory. In Highway Research Record 15, HRB, National Research Council,Washington, D.C., 1963, pp. 60–66.

11. Gazis, D. C., R. Herman, and R. B. Potts. Car-Following Theory ofSteady-State Traffic Flow. Operations Research, Vol. 7, No. 4, 1959,pp. 499–505.

12. Ossen, S., and S. P. Hoogendoorn. Validity of Trajectory-Based Cali-bration Approach of Car-Following Models in Presence of MeasurementErrors. In Transportation Research Record: Journal of the Transporta-tion Research Board, No. 2088, Transportation Research Board of theNational Academies, Washington, D.C., 2008, pp. 117–125.

13. Brockfeld, E., R. D. Kuhne, and P. Wagner. Calibration and Validationof Microscopic Traffic Flow Models. In Transportation Research Record:Journal of the Transportation Research Board, No. 1876, TransportationResearch Board of the National Academies, Washington, D.C., 2004,pp. 62–70.

14. Bernhardt, K. L. S. Agent-Based Modeling in Transportation. InTransportation Research Circular E-C113: Artificial Intelligence inTransportation, Transportation Research Board of the National Acade-mies, Washington, D.C., 2007, pp. 72–80.

15. Peeta, S., W. Zhou, and P. Zhang. Modeling and Mitigation of Car–Truck Interactions on Freeways. In Transportation Research Record:Journal of the Transportation Research Board, No. 1899, TransportationResearch Board of the National Academies, Washington, D.C., 2004,pp. 117–126.

16. Hidas, P. Modelling Lane Changing and Merging in Microscopic TrafficSimulation. Transportation Research Part C, Vol. 10, No. 5–6, 2002,pp. 351–371.

17. Louisell, C. A Simple Algorithm for Quantifying the Impact of DriverBehavior on Traffic Flow and Safety During Forced Merges in WorkZones. Presented at 85th Annual Meeting of the TransportationResearch Board, Washington, D.C., 2006.

18. Bonabeau, E. Agent-Based Modeling: Methods and Techniques forSimulating Human Systems. Proc., National Academy of Sciences of theUnited States of America, Vol. 99, No. 3, 2002, pp. 7280–7287.

19. Hongfei, J., J. Zhicai, and N. Anning. Develop a Car-Following ModelUsing Data Collected by “Five-Wheel System.” Proc., Intelligent Trans-portation Systems, IEEE, Vol. 1, 2003, pp. 346–351.

20. Panwai, S., and H. Dia. Neural Agent Car-Following Models. IEEE Trans-actions on Intelligent Transportation Systems, Vol. 8, No. 1, 2007, p. 11.

21. Abbas, M., L. Chong, and A. Medina. Neural-Agent Driver BehaviorModeling Using Naturalistic Data. Proc., 2010 13th International IEEEConference on Intelligent Transportation Systems, ITSC ’10, Sept. 19–22,2010, Funchal, Madeira Island, Portugal.

Any opinions, findings, and conclusions or recommendations expressed in thispublication are those of the authors and do not necessarily reflect the view of theFederal Highway Administration.

The Traffic Flow Theory and Characteristics Committee peer-reviewed this paper.