Testing and Evaluation Methods for ICT-based Safety Systems
Collaborative Project
Grant Agreement Number 215607
Deliverable D3.2
Final Testing Protocols
Confidentiality level: Public
Status: Final
Executive Summary This report is the final document summarizing the inspection and testing protocols of the
eVALUE project. It describes principles, inspection protocols and testing protocols for
performance testing of ICT-based safety systems. The inspection protocols (published earlier
in D2.2) and the testing protocols introduced in D3.1 are replaced by the ones in D3.2. The
older versions are obsolete and should be disregarded.
The inspection protocols cover the definition of the test vehicle, HMI aspects, environmental
conditions, and functional safety. The inspection protocols are used to prepare for the
physical tests as well as evaluating the performance of the vehicle.
The testing protocols address longitudinal, lateral, and stability-oriented traffic scenarios. The
longitudinal scenarios include a pedestrian crossing the road in front of the vehicle, or the
situation where a driver approaches a stationary queue of cars. Involuntarily lane departures
and cars in the blind spot during a lane change are situations covered by the lateral
scenarios. Exiting a highway, avoiding an obstacle, and braking on a partially ice-covered
road surface are examples of traffic scenarios related to stability.
Deliverable D3.2
eVALUE 2 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Document Name
eVALUE-101031-D32-V20-FINAL.doc
Version Chart
Version Date Comment
0.1 2010-10-08 First draft version
0.2 2010-10-15 Second draft released for cluster 3 comments
0.3 2010-10-25 Final draft for internal review
1.0 2010-11-05 Final version
1.1 2010-11-11 Minor corrections
1.2 2010-12-22 Updates after final review, addition of Annex C
2.0 2011-01-18 Fully revised version
Authors
The following participants contributed to this deliverable:
Name Company Chapters
Jan Jacobson, Henrik Eriksson, Jacques Hérard SP all
Susanna Leanderson-Olsson, Lars Nordström, Rafael Basso,
Krister Fredriksson VTEC all
Josep Maria Dalmau, Andrés Aparicio, Sebastien Baures, Oscar
Muñoz IDIADA all
Micha Lesemann, Adrian Zlocki, Jörn Lützow, Felix Fahrenkrog ika all
Fredrik Bruzelius, Håkan Andersson, Mattias Hjort VTI all
Daniel Westhoff SICK all
Mauro Vesco, Isabella Camuffo, Silvano Marenco, Vincenzo
Murdocco, CRF all
Javier Sanchez, Lucía Isasi RBTK all
Coordinator
Dipl.-Ing. Micha Lesemann
Institut für Kraftfahrzeuge – RWTH Aachen University
Steinbachstraße 7, 52074 Aachen, Germany
Phone: +49-241-8027535
Fax: +49-241-8022147
E-Mail: [email protected]
Copyright
© eVALUE Consortium 2010
Deliverable D3.2
eVALUE 3 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Table of Contents
1 Introduction .................................................................................................................... 9
2 Performance testing..................................................................................................... 10
2.1 The task of performance testing ............................................................................. 10
2.2 Scenarios .............................................................................................................. 11
2.3 ICT-Based safety functions .................................................................................... 12
3 Physical testing ............................................................................................................ 14
3.1 The approach for physical testing .......................................................................... 14
3.2 Testing protocols for longitudinal functions ............................................................ 14
3.3 Testing protocols for lateral functions..................................................................... 15
3.4 Testing protocols for stability functions .................................................................. 15
4 Inspection .................................................................................................................... 16
4.1 Aims and goals of Inspections ............................................................................... 16
4.2 Inspection - Definition of Subject Vehicle ............................................................... 17
4.3 Inspection - Environmental Conditions ................................................................... 18
4.4 Inspection - HMI .................................................................................................... 19
4.5 Inspection - Functional Safety ................................................................................ 19
5 Safety performance indicators ..................................................................................... 21
5.1 Longitudinal functions ............................................................................................ 22
5.2 Lateral functions .................................................................................................... 23
5.3 Stability functions ................................................................................................... 25
6 The eVALUE Performance Testing .............................................................................. 30
7 Literature ..................................................................................................................... 33
Annex A Inspection protocols .................................................................................... 34
A.1 Inspection protocol - Definition of the subject vehicle ......................................... 35
A.1.1 Scope ...................................................................................................... 35
A.1.2 References .............................................................................................. 35
A.1.3 Definitions ................................................................................................ 37
A.1.4 Inspection ................................................................................................ 38
A.1.4.1 Principle .............................................................................................. 38
A.1.4.2 Information required from the manufacturer ......................................... 39
A.1.4.3 Inspection procedure ........................................................................... 39
A.1.4.4 Report ................................................................................................. 39
A.1.5 Checklist - Type of vehicle ....................................................................... 41
A.1.6 Checklist - Longitudinal functionality ........................................................ 43
A.1.7 Checklist - Lateral functionality ................................................................ 45
A.1.8 Checklist - Stability functionality ............................................................... 48
A.1.9 Checklist - External communication ......................................................... 50
A.2 Inspection protocol - Environmental conditions................................................... 52
A.2.1 Scope ...................................................................................................... 52
A.2.2 References .............................................................................................. 52
A.2.3 Definitions ................................................................................................ 52
A.2.4 Inspection ................................................................................................ 53
Deliverable D3.2
eVALUE 4 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.2.4.1 Principle .............................................................................................. 53
A.2.4.2 Information required from the manufacturer ......................................... 53
A.2.4.3 Inspection procedure ........................................................................... 53
A.2.4.4 Report ................................................................................................. 54
A.2.5 Checklist - Environmental conditions ....................................................... 55
A.3 Inspection Protocol - Human-machine interface (HMI) ....................................... 58
A.3.1 Scope ...................................................................................................... 58
A.3.2 References .............................................................................................. 58
A.3.3 Definitions ................................................................................................ 59
A.3.4 Inspection ................................................................................................ 59
A.3.4.1 Principle .............................................................................................. 59
A.3.4.2 Information required from the manufacturer ......................................... 59
A.3.4.3 Inspection procedure ........................................................................... 59
A.3.4.4 Report ................................................................................................. 60
A.3.5 Checklist - HMI ........................................................................................ 61
A.4 Inspection Protocol - Functional safety ............................................................... 68
A.4.1 Scope ...................................................................................................... 68
A.4.2 References .............................................................................................. 68
A.4.3 Definitions ................................................................................................ 69
A.4.4 Inspection ................................................................................................ 71
A.4.4.1 Principle .............................................................................................. 71
A.4.4.2 Information required from the manufacturer ......................................... 71
A.4.4.3 Inspection procedure ........................................................................... 71
A.4.4.4 Report ................................................................................................. 72
A.4.5 Checklist - Functional safety .................................................................... 73
Annex B Testing Protocols ........................................................................................ 75
B.1 General conditions for eVALUE testing protocols ............................................... 75
B.1.1 Scope ...................................................................................................... 75
B.1.2 References .............................................................................................. 75
B.1.3 Definitions ................................................................................................ 76
B.1.4 Test conditions and environment ............................................................. 79
B.1.4.1 Test track conditions ............................................................................ 80
B.1.4.2 Temperature conditions ....................................................................... 80
B.1.4.3 Visibility conditions .............................................................................. 80
B.1.4.4 Wind conditions ................................................................................... 80
B.1.5 Subject vehicle preparation and conditioning ........................................... 80
B.1.5.1 Data collection systems ....................................................................... 81
B.1.5.2 Configuration of subject vehicle ........................................................... 81
B.2 Avoidance or mitigation of rear-end collision, open loop test .............................. 82
B.2.1 Scope ...................................................................................................... 82
B.2.2 References .............................................................................................. 82
B.2.3 Definitions ................................................................................................ 82
B.2.4 Test procedure - Avoidance or mitigation of rear-end collision ................. 83
B.2.4.1 Principle .............................................................................................. 83
Deliverable D3.2
eVALUE 5 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4.2 Test objectives .................................................................................... 83
B.2.4.3 Drivers ................................................................................................. 84
B.2.4.4 Equipment ........................................................................................... 84
B.2.4.4.1 Target vehicle ............................................................................... 84
B.2.4.5 Testing environment ............................................................................ 84
B.2.4.6 Information required for the test ........................................................... 84
B.2.4.7 Subject vehicle preparation ................................................................. 85
B.2.4.8 Test procedure and data processing ................................................... 85
B.2.4.8.1 Tests ............................................................................................. 85
B.2.4.8.2 Test 1: Rear-end collision – passive driver .................................... 87
B.2.4.8.3 Test 2: Rear-end collision – driver brakes strongly ........................ 88
B.2.4.8.4 Test 3: Rear-end collision – driver brakes mildly ........................... 89
B.2.4.9 Uncertainty .......................................................................................... 90
B.2.4.10 Result ................................................................................................ 90
B.3 Avoidance of collision with transversally moving target, open loop test .............. 91
B.3.1 SCOPE .................................................................................................... 91
B.3.2 References .............................................................................................. 91
B.3.3 Definitions ................................................................................................ 91
B.3.4 Test procedure - Avoidance of collision with transversally moving target . 91
B.3.4.1 Principle .............................................................................................. 91
B.3.4.2 Test objectives .................................................................................... 92
B.3.4.3 Drivers ................................................................................................. 92
B.3.4.4 Equipment ........................................................................................... 92
B.3.4.4.1 Target vehicle ............................................................................... 93
B.3.4.4.2 Pedestrian target ........................................................................... 93
B.3.4.5 Testing environment ............................................................................ 93
B.3.4.6 Information required for the test ........................................................... 93
B.3.4.7 Subject vehicle preparation ................................................................. 93
B.3.4.8 Test procedure and data processing ................................................... 94
B.3.4.8.1 Tests ............................................................................................. 94
B.3.4.8.2 Test 1: Transversally moving target – passive driver ..................... 95
B.3.4.8.3 Test 2: Transversally moving target - driver brakes strongly.......... 96
B.3.4.8.4 Test 3: Transversally moving target – driver brakes mildly ............ 97
B.3.4.9 Uncertainty .......................................................................................... 98
B.3.4.10 Result ................................................................................................ 98
B.4 Lane departure, open loop test ........................................................................... 99
B.4.1 Scope ...................................................................................................... 99
B.4.2 References .............................................................................................. 99
B.4.3 Definitions ................................................................................................ 99
B.4.4 Test procedure - Avoidance of lane departure ......................................... 99
B.4.4.1 Principle .............................................................................................. 99
B.4.4.2 Test objectives .................................................................................. 100
B.4.4.3 Drivers ............................................................................................... 100
B.4.4.4 Equipment ......................................................................................... 100
Deliverable D3.2
eVALUE 6 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.4.4.5 Testing conditions and environment .................................................. 100
B.4.4.6 Information required for the test ......................................................... 100
B.4.4.7 Subject vehicle preparation and conditioning ..................................... 101
B.4.4.8 Test procedure and data processing ................................................. 101
B.4.4.8.1 Tests ........................................................................................... 101
B.4.4.8.2 Test 1: Lane departure on a straight road ................................... 103
B.4.4.8.3 Test 2: Lane departure in a curve................................................ 105
B.4.4.8.4 Test 3: Lane departure just before a curve .................................. 107
B.4.4.9 False alarms in Tests 1-3 .................................................................. 108
B.4.4.10 Uncertainty ...................................................................................... 108
B.4.4.11 Result .............................................................................................. 109
B.5 Avoidance of lane change collision on a straight road, open loop test .............. 110
B.5.1 Scope .................................................................................................... 110
B.5.2 References ............................................................................................ 110
B.5.3 Definitions .............................................................................................. 110
B.5.4 Test procedure - Avoidance of lane change collision on a straight road . 110
B.5.4.1 Principle ............................................................................................ 110
B.5.4.2 Test objectives .................................................................................. 111
B.5.4.3 Drivers ............................................................................................... 111
B.5.4.4 Equipment ......................................................................................... 111
B.5.4.4.1 Target vehicle ............................................................................. 111
B.5.4.5 Testing conditions and environment .................................................. 112
B.5.4.6 Information required for the test ......................................................... 112
B.5.4.7 Subject vehicle preparation ............................................................... 113
B.5.4.8 Test procedure and data processing ................................................. 113
B.5.4.8.1 Tests ........................................................................................... 113
B.5.4.8.2 Test 1: Lane change collision avoidance on a straight road ........ 114
B.5.4.9 False alarms in Test 1 ....................................................................... 116
B.5.4.10 Uncertainty ...................................................................................... 116
B.5.4.11 Result .............................................................................................. 116
B.6 Emergency braking on a μ split, open loop test ................................................ 117
B.6.1 Scope .................................................................................................... 117
B.6.2 References ............................................................................................ 117
B.6.3 Definitions .............................................................................................. 117
B.6.4 Test procedure - Emergency braking on a μ split ................................... 117
B.6.4.1 Principle ............................................................................................ 117
B.6.4.2 Test objectives .................................................................................. 117
B.6.4.3 Drivers ............................................................................................... 118
B.6.4.4 Equipment ......................................................................................... 118
B.6.4.5 Testing environment .......................................................................... 118
B.6.4.6 Information required for the test ......................................................... 118
B.6.4.7 Subject vehicle preparation ............................................................... 119
B.6.4.8 Test procedure and data processing ................................................. 119
B.6.4.8.1 Tests ........................................................................................... 119
Deliverable D3.2
eVALUE 7 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.6.4.8.2 Test 1: -split braking .................................................................. 122
B.6.4.9 Uncertainty ........................................................................................ 122
B.6.4.10 Result .............................................................................................. 123
B.7 Emergency braking on a μ-split, closed loop test.............................................. 124
B.7.1 Scope .................................................................................................... 124
B.7.2 References ............................................................................................ 124
B.7.3 Definitions .............................................................................................. 124
B.7.4 Test procedure -Emergency braking on a μ-split .................................... 124
B.7.4.1 Principle ............................................................................................ 124
B.7.4.2 Test objectives .................................................................................. 124
B.7.4.3 Drivers ............................................................................................... 125
B.7.4.4 Equipment ......................................................................................... 125
B.7.4.5 Testing environment .......................................................................... 125
B.7.4.6 Information required for the test ......................................................... 125
B.7.4.7 Subject vehicle preparation ............................................................... 126
B.7.4.8 Test procedure and data processing ................................................. 126
B.7.4.8.1 Tests ........................................................................................... 126
B.7.4.8.2 Test 1: -split braking .................................................................. 128
B.7.4.9 Uncertainty ........................................................................................ 129
B.7.4.10 Result .............................................................................................. 129
B.8 Obstacle avoidance .......................................................................................... 130
B.8.1 Scope .................................................................................................... 130
B.8.2 References ............................................................................................ 130
B.8.3 Definitions .............................................................................................. 130
B.8.4 Test procedure - Obstacle avoidance .................................................... 130
B.8.4.1 Principle ............................................................................................ 130
B.8.4.2 Test objectives .................................................................................. 130
B.8.4.3 Drivers ............................................................................................... 131
B.8.4.4 Equipment ......................................................................................... 131
B.8.4.5 Testing environment .......................................................................... 131
B.8.4.6 Information required for the test ......................................................... 131
B.8.4.7 Subject vehicle preparation ............................................................... 131
B.8.4.8 Test procedure and data processing ................................................. 131
B.8.4.9 Uncertainty ........................................................................................ 133
B.8.4.10 Result .............................................................................................. 133
B.9 Highway exit, open loop test............................................................................. 134
B.9.1 Scope .................................................................................................... 134
B.9.2 References ............................................................................................ 134
B.9.3 Definitions .............................................................................................. 134
B.9.4 Test procedure - Highway exit ............................................................... 134
B.9.4.1 Principle ............................................................................................ 134
B.9.4.2 Test objectives .................................................................................. 134
B.9.4.3 Drivers ............................................................................................... 135
B.9.4.4 Equipment ......................................................................................... 135
Deliverable D3.2
eVALUE 8 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.9.4.5 Testing environment .......................................................................... 135
B.9.4.6 Information required for the test ......................................................... 135
B.9.4.7 Subject vehicle preparation ............................................................... 135
B.9.4.7.1 Characterization of the lateral dynamics...................................... 136
B.9.4.8 Test procedure and data processing ................................................. 136
B.9.4.8.1 Tests ........................................................................................... 136
B.9.4.8.2 Test 1 Initial run .......................................................................... 138
B.9.4.8.3 Test 2 Successive runs ............................................................... 138
B.9.4.9 Uncertainty ........................................................................................ 138
B.9.4.10 Result .............................................................................................. 139
Annex C Resolved open issues ............................................................................... 140
Deliverable D3.2
eVALUE 9 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
1 Introduction
The eVALUE project develops test methods for Information and Communication
Technologies (ICT)-based safety systems in road vehicles. Previous deliverables of the
project have described the state-of-the-art, scenarios for testing, concepts of testing,
inspection protocols, a testing matrix, and testing protocols. Testing protocols for
performance testing exist for longitudinal, lateral, and stability scenarios. Inspection protocols
cover the definition of the test vehicle, Human-Machine Interface (HMI) aspects,
environmental aspects, and functional safety. This deliverable is the final document
summarizing the eVALUE Test Programme. It is based on definitions, results, and
conclusions from other eVALUE deliverables.
This chapter describes the purpose and the contents of the document. Chapter 2 gives the
basics of performance testing of ICT-based safety functions in road vehicles and summarizes
the scenarios previously selected for performance testing.
Chapter 3 describes physical testing and testing protocols.
Chapter 4 describes inspections for the definition of the test vehicle, the inspection of the
Human-Machine Interface (HMI), the environmental, and the functional safety aspects. (The
―inspection‖ was earlier in the project named ―design review‖. ―Inspection‖ has been chosen
since it is considered as the most appropriate name for this activity.)
Safety performance indicators are explained and listed in Chapter 5.
Chapter 6 summarises the eVALUE performance testing process.
Annex A contains the inspection protocols to be used.
Annex B contains the testing protocol describing the physical tests.
The inspection and testing protocols have been published in other eVALUE deliverables and
have been reviewed by external stakeholders for this final version.
Annex C contains a discussion on open issues.
Deliverable D3.2
eVALUE 10 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
2 Performance testing
2.1 The task of performance testing
Evaluation of the functional performance of a preventive safety system considers the
technical performance of the function as well as the overall safety effects; evaluation that the
function does what it was designed for. Technical performance testing aims at investigating
whether a safety function meets technical requirements and specifications on what the
function shall do [eVALUE D3.1].
Performance testing in terms of the overall safety effects aims at investigating that the
function fulfils its purpose. Interpretation of the testing result is complex because of the need
for considering driver’s response and quantifying the safety effects in terms of indicator
values derived from measurable data.
In passive safety system evaluation, safety effects are quantified by evaluating the decrease
of biomechanical injuries with e.g. airbags in crash tests. In evaluation of preventive safety
systems, intended to help avoiding or mitigating the effect of accidents, the desired safety
effects are connected to the ability to avoid an accident and to increase the driver’s control of
the situation. For such scenarios the driver’s behaviour plays an important role. Driver
behaviour modelling is very complex compared to biomechanical movements during short
lapses of time as in a crash situation. However the efforts that were invested in the eVALUE
project were to be focussed on predictable and repeatable tests that could fit within the
amount of work and time allocated to the project.
Performance testing of preventive safety systems is a way to demonstrate the performance
of the safety functions of a road vehicle. The result will be depending on the safety systems
installed in the vehicle and on the behaviour of the driver. The traffic scenarios and the test
cases chosen will give, at a reasonable test effort, an overview of the performance of the
safety functions provided by the vehicle under test. The test result is intended to be
communicated to the buyers of a vehicle but also gives information to the developer.
Development testing of preventive safety systems at a vehicle OEM or at a component
supplier is different from performance testing. During the development process an extensive
series of test cases in many different traffic scenarios need to be conducted. Thus the effort
required for development testing will be far greater than the effort required for performance
testing.
Performance testing has to be performed with limited efforts. This will also put requirements
on the efforts needed to install measuring and control equipment in the vehicle under test. It
may be time-consuming to connect to the internal data buses of the vehicle and to interpret
the messages recorded. In many cases it will also be impossible for the test engineer to have
access to the OEM-internal instructions describing the data bus communication of the
vehicle. The most efficient way to equip a vehicle would be to install a stand-alone measuring
system to record data such as position, speed, acceleration, yaw rate and distance to the
Deliverable D3.2
eVALUE 11 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
target. Additional sensors may have to be added to the subject vehicle to record impact with
the target vehicle. There must also be techniques to record when warning signals are issued.
The testing protocols which describe the test procedures and the test cases must be robust
to variations of test conditions. The same test results shall be obtained on different test
tracks, with different test drivers, different measurement equipment and sensors, different
target objects, at slight variations in the environmental conditions, and for different individual
vehicles of the same type with the same equipment.
A large amount of data will be recorded during the performance testing. This data has to be
possible to process and interpret in an efficient way. The logged raw data must be
transformed into a log file with measured data. The measured data will then be used to
calculate safety performance indicators describing the safety performance of the safety
function. Post-processing of measured data should be possible to automate and represent in
a clear format.
A test procedure shall fulfil the following requirements:
Cover all safety categories
High repeatability of the manoeuvre
Driver independent
Clear metric available for safety assessment
Accurate results
Reasonable test and evaluation effort
Neutral for different vehicle categories
As neutral as possible for different weather and track condition
Motivate OEM for new system introduction and improvement
Accident representative
Promote the use of the available safety features
2.2 Scenarios
Physical tests are based on scenarios that enable the evaluation of preventive safety
functions provided by a vehicle [eVALUE D1.2].
The scenarios define the context in which the testing procedures take place. They have been
selected because they permit to apply test procedures for the evaluation of the overall safety
functions in vehicles without addressing specific safety systems. The choice of scenarios is
directly based on road accident statistics. The testing procedures associated to each
scenario specify a sequence of manoeuvres from which information is derived.
The testing procedures associated to each scenario, include a description of the:
- conditions under which the tests are executed
- type of vehicle(s) involved in the tests
Deliverable D3.2
eVALUE 12 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
- safety indicators that quantify the overall safety performance of the current subject
vehicle
- need for measurement equipment
The preventive safety functions associated to the current scenarios are classified in three
clusters, namely:
- Cluster 1 for longitudinal assistance
- Cluster 2 for lateral assistance
- Cluster 3 for yaw/stability assistance
The final scenarios chosen by eVALUE are listed in Table 1.
Table 1 Scenarios chosen by eVALUE
Cluster Scenario
1 Scen-C1-1 Rear end collision on a straight road
Scen-C1-2 Rear end collision on a curved road
Scen-C1-3 Collision with a transversally moving target
2 Scen-C2-1 Lane departure on a straight road
Scen-C2-2 Lane departure in a curve
Scen-C2-3 Lane departure on a straight road just before a curve
Scen-C2-4 Lane change collision avoidance on a straight road
3 Scen-C3-1 Emergency braking on a μ-split
Scen-C3-2 Obstacle avoidance
Scen-C3-3 Highway exit
2.3 ICT-Based safety functions
Within the scope of eVALUE, the longitudinal domain cluster includes Forward Collision
Warning (FCW), Collision Mitigation by Braking (CMbB) and Adaptive Cruise Control (ACC).
All these functions are based on sensors which monitor the frontal environment of the vehicle
and warn, support the driver, or intervene in the case of a dangerous situation caused by an
oncoming obstacle or a pedestrian crossing the path of the subject vehicle. The proposed
test procedures are oriented to these frontal situations and emphasize the capabilities of the
sensors in this field.
The lateral domain includes Lane Departure Warning (LDW), Lane Keeping Assistant (LKA)
and Blind Spot Detection (BSD). These systems are based on sensors monitoring the side of
the vehicle and will warn, support, or intervene in case of a dangerous situation against side
obstacles or lane/road departures. For this cluster, the testing protocols try to ensure that the
safety functions are able to handle all situations where the driver lane keeping ability is
Deliverable D3.2
eVALUE 13 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
unintentionally lost, or when a vehicle is approaching from another lane during a lane
change.
The stability cluster evaluates the response of a vehicle in situations where the dynamics are
compromised. Electronic Stability Control (ESC) and Anti-lock Braking System (ABS) are the
systems included in this cluster.
Deliverable D3.2
eVALUE 14 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
3 Physical testing
3.1 The approach for physical testing
The general approach of eVALUE is based on real accident scenarios. This approach is
emphasized in physical testing, where there is a clear implementation of real traffic
scenarios. The purpose of this type of test is to assess the complete vehicle’s performance.
In other words, the approach is not to test one particular ICT-based safety system, but to
validate the whole vehicle’s safety performance under different scenarios, i.e. specific real
driving situations, which are relevant regarding the functions of the considered ICT-based
safety systems. [eVALUE D3.1]
It shall be possible to perform the complete set of eVALUE tests within maximum one week
of testing on a proving ground. Additional time will be needed for preparations, inspections
and instrumentation of the subject vehicle. The analysis of the measured data and the
reporting of the test will also require additional time. The cost of preparations, testing,
analysis and reporting is aimed to be comparable to the cost of NCAP passive safety testing
of a vehicle.
For every testing protocol there are one or more safety performance indicators. The safety
performance indicators are the key parameters which allow the evaluation of a safety
function in an objective way. They are the values which will enable the comparison of
different vehicles and show which are the strengths and weaknesses of each one, by
establishing a ranking among them, see Chapter 5. These safety performance indicators
should be consistent for different situations, and they shall be calculated from the measured
data collected during the tests.
3.2 Testing protocols for longitudinal functions
There are two testing protocols defined for the longitudinal domain:
- Avoidance of rear-end collision
- Avoidance of collision with a transversally moving target
Test procedures for rear-end collision evaluate the performance of the longitudinal safety
functions of a vehicle approaching another vehicle which is moving slower in the same
direction and lane, decelerating, or being stationary on a straight or curved road. Test
procedures for a vehicle approaching a transversally moving target address a passenger
vehicle, or a pedestrian.
Both testing protocols are open loop tests, i.e. the natural response and feedback from the
driver are not considered. Open loop tests enable the fulfilment of eVALUE requirement for
physical testing, ensuring high repeatability and driver independent procedures. Different test
cases are proposed to cover the scenarios identified for cluster 1, see Table 1.
Deliverable D3.2
eVALUE 15 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
3.3 Testing protocols for lateral functions
There are two testing protocols defined for the lateral domain:
- Avoidance of lane departure
- Avoidance of lane change collision on a straight road
The test procedure for avoidance of lane departure on a straight or curved road addresses
the safety of different types of subject vehicles (car, bus or truck). Lane change collision is
tested by simulating manoeuvres at lane change by a vehicle.
Both testing protocols are open loop tests, i.e. the natural response and feedback from the
driver is not expected. Different test cases are proposed to cover the scenarios identified for
cluster 2, see Table 1.
3.4 Testing protocols for stability functions
There are four testing protocols defined for the longitudinal domains:
- Emergency braking on µ-split, open loop
- Emergency braking on µ-split, closed loop
- Obstacle avoidance
- Fast driving into a curve
The emergency braking is performed both as open loop and closed loop. The obstacle
avoidance and fast driving into a curve are open loop tests. Different test cases are proposed
to cover the scenarios identified for cluster 3, see Table 1.
Deliverable D3.2
eVALUE 16 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
4 Inspection
4.1 Aims and goals of Inspections
An inspection is a systematic, comprehensive and documented analysis to determine the
capability and adequacy of the ICT-based safety functions. Most parts of the inspection are
done studying the documentation and further information provided by the manufacturer. The
remaining parts of the inspection might be done by investigating the vehicle [eVALUE D2.2].
An inspection may be performed with one out of three different objectives, see Figure 1. The
first objective of an inspection would be to familiarize with the subject vehicle. Efficient
performance testing will require the test engineer to understand the safety functions of the
vehicle. The design principles of the ICT-based safety systems should also be understood,
even if the performance testing does not explicitly address the performance of individual
systems. Such a ―get-to-know‖ inspection would include a definition of the subject vehicle, an
overview of functions provided for longitudinal and lateral assistance as well as an overview
of functions related to stability and external communication needs The second objective of an
inspection would be to prepare for the tests. The test cases of the performance testing will
partly be chosen depending on potential weaknesses of the system and on the already
performed tests by the manufacturer. Such an inspection concerns system performance in
different environmental conditions (rain, snow, ice, darkness, etc.). The performance in
different environmental conditions may be reviewed and assessed instead of thoroughly
tested during the performance testing. The manufacturer must be able to show the efforts
made during development testing of user behaviour and environmental conditions. ―Normal
conditions‖ (dry asphalt or concrete, flat road, approx 20°C) would then suffice at the
performance testing.
The third objective of an inspection will be to actually perform an evaluation reaching a pass
or a fail judgement. The inspection will be part of the safety evaluation. The interface to the
driver (human-machine interface, HMI) will be reviewed and assessed by a small number of
experts. A time-consuming HMI test with a large number of drivers cannot be managed
during performance testing. The focused HMI inspection by experts will conclude if the HMI
will ―pass‖ or ―fail‖ in the performance testing.
Deliverable D3.2
eVALUE 17 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Figure 1: Objectives to perform an inspection (INSP)
An issue related to Inspections is the need of having a close contact with the vehicle
manufacturer. Some of the information required can be obtained by simply inspecting or
driving the vehicle. However, to get further information, it is necessary to contact the
manufacturer. At some points, it might not be easy to get information, because the
information could be sensitive, confidential, or because the manufacturer would not find the
necessity of sharing it.
4.2 Inspection - Definition of Subject Vehicle
It is important to clearly identify the vehicle under test (i.e. the subject vehicle), especially if
the vehicle is a prototype vehicle. All major characteristics of the vehicle influencing the
performance testing must be noted. The challenge of identifying the vehicle under test is
greater at performance testing than at passive safety testing. Even small design
modifications in the active safety systems may severely influence the performance.
A certain model of a road vehicle is normally produced in several different types. There will
be differences in engine, body, brakes etc. Many of these differences will not be important for
the performance testing. It will in many cases be possible to choose one type of vehicle for
the performance testing, and to assume that the test results are valid also for other types of
the same vehicle.
INSP
=
get to
know
the
subject
vehicle
INSP = evaluation
INSP
=
prepare
test
Physical testing
Deliverable D3.2
eVALUE 18 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Evaluation of longitudinal, lateral, and stability safety functions require information about their
characteristics and limits, e.g. the kind of sensor used and limits of its operational range. If
some tests have been carried out by the manufacturer, it is helpful to receive information
about these tests to ensure that they are adapted to the system tested and to the related
standards.
The safety performance dependence on reliable external communication will also be
reviewed. External communication includes positioning, vehicle-to-vehicle (V2V), and
vehicle-to-infrastructure (V2I) communication. Transmission anomalies, such as distortion
and interference, can have a negative effect on the overall performance and it is
consequently addressed in the inspection.
The inspection protocol describes how to perform an inspection to identify the type of vehicle,
its safety functions, and possible dependence on external communication. The inspection is
necessary to characterise the vehicle under test and all other types of the vehicle model
intended to be covered by the performance test. It is also a necessary step for the test
engineers familiarising with the subject vehicle and its safety functions.
The results of the inspection cannot be a pass or fail judgement, but simply a definition of the
subject vehicle and its safety functions.
4.3 Inspection - Environmental Conditions
Development of ICT-based safety systems requires extensive testing to understand how the
vehicle is influenced by its environment. Temperature, light conditions, pollution,
precipitation, or road characteristics may influence the performance of the vehicle. Sensors,
controllers, and actuators might be negatively affected by adverse environmental conditions.
Development testing by the manufacturer will address all identified hazards.
Performance testing on the test track is often performed in normal environmental conditions.
Due to the limited time and resources allocated to performance testing, an inspection could
be used to validate the environmental conditions that the vehicle can handle.
The objective is to verify that the design fulfils the requirements on operation during different
environmental conditions. The important environmental influence factors must be identified,
and the vehicle and its ICT-based safety functions must be verified during different
environmental conditions.
The inspection carried out shall determine:
- if environmental conditions are expected to influence the performance of the design to be tested
- if the developer has specified requirements for environmental influence - if the developer has performed adequate tests of the environmental influence - if normal environmental conditions can be applied at performance testing on the test
track
Deliverable D3.2
eVALUE 19 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
4.4 Inspection - HMI
Development of ICT-based safety systems requires extensive testing to understand how the
vehicle and driver are influenced by the HMI design. The interaction between the vehicle and
the driver relates to the level of assistance provided by the vehicle which includes one or
more of the following: warning, support, or intervention.
Performance testing on the test track is often performed using robots or professional drivers.
Due to the limited time and resources allocated to performance testing, an inspection could
be used to validate the suitability of the HMI design of the vehicle.
The HMI requirements specification and the user manual for the subject vehicle will be
analysed. Status indication, visual warnings, visual information, haptic warnings and auditory
warnings will be examined for the longitudinal, the lateral, and the stability safety functions.
Also the intervention and combinations of warnings/interventions will be assessed.
The inspection carried out shall determine:
- if the HMI design is expected to influence the performance of the design to be tested - if the developer has specified requirements for HMI design - if the developer has performed adequate tests of the HMI design - if simulation experiments of the HMI design is necessary to complement performance
testing on the test track
4.5 Inspection - Functional Safety
Introduction of additional safety functions in vehicles increases the complexity in safety
requirements. Preventive measures to avoid faulty states and module failures have to be
considered. Functional safety turns out to be one of the key issues as new functions as
ADAS (Advanced Driver Assistance System), dynamics control, and additional safety
systems emerge.
Faults in ICT-based safety functions may cause critical failures. The inspection of the
functional safety shall address the safety principles applied, and the safety measures
implemented to avoid hazardous system states or operation modes. The development of
ICT-based functions must be based on a requirement specification set-up adapted to the
hazardous situation attributed to systematic faults, component failures, or driver mistakes.
Due to the limited time and resources allocated to performance testing, an inspection could
be used to verify how the vehicle handles different types of failure.
The objective of the inspection is to verify that the specified requirements are fulfilled when
the vehicle is operating in the occurrence of hazardous events. The important faults/failures
that may influence the behaviour of the vehicle must be identified, and the vehicle and its
ICT-based safety functions must be verified for different failure modes.
Deliverable D3.2
eVALUE 20 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Failures in ICT-based safety functions are unlikely to occur during the limited time of
performance testing. Thus, an inspection is required to verify the vehicle behaviour at fault.
The inspection shall check if a standard for functional safety has been used. After
examination of the safety functions it shall be shown that:
- a situation analysis and hazard identification has been carried out - a classification of the hazards has been made - a safety integrity level is assigned to each safety function
The result of this inspection is to present an opinion on the safety-related aspects of the ICT-
based safety functions.
Deliverable D3.2
eVALUE 21 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
5 Safety performance indicators
Adequate safety performance indicators are essential to characterise the behaviour of the
tested vehicle according to the concept adopted in eVALUE. The number of selected
indicators of safety performance must be limited in order to reduce the complexity of the
assessment of ICT-based safety functions. A safety performance indicator shall reflect a real
impact on road safety and should not be confused with test conditions or measured values.
Test conditions are prerequisites for the test procedure e.g. the speed of the target vehicle.
Measured values are logged during the test e.g. the position of the subject vehicle. The
concept is to select the most important safety performance indicators where a real impact on
road safety can be expected [eVALUE D3.1].
An assessment of the most representative safety performance indicators is made to quantify
the overall safety performance of the vehicles. Many of the selected safety performance
indicators can be found in previous research projects or literature concerning automotive
safety development.
The approach for developing safety performance indicators involved three steps:
- The first step involved consultation of the available standards and results from
previous research projects and available assessment programme on traffic safety.
- The second step involves identifying the relevant scenarios that are representative for
the tests in which the vehicles safety functions can prevent accidents or at least
reduce their gravity.
- The third step involves the identification and development of appropriate safety
indicators. Statistics for traffic accidents involving human casualties and fatalities
have had a leading role in the final choice of safety indicators.
The safety performance indicators have been chosen to:
- Characterize the safety performance associated to the sequence of events that take
place in the current test.
- Provide information about the tested vehicle to the developer.
- Quantify the test results for comparison with a threshold value.
The approach applied, when executing the manoeuvres specified in the testing protocols, is
chosen between one of the two options; open loop or closed loop.
An open loop test enables the verification of the safety performance of the current vehicle,
without considering natural response and feedback from the driver. A professional driver or a
driving robot is used for triggering an actuation from the vehicle.
A closed loop test enables the evaluation of the safety performance of the current vehicle by
considering the driver response to eventual system activation.
Deliverable D3.2
eVALUE 22 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
5.1 Longitudinal functions
Two safety performance indicators are proposed for longitudinal functions, see Table 2.
Table 2 Safety performance indicators for longitudinal functions
Safety
performance
indicator
Definition/Motivation
collision speed Definition: Collision speed is the speed, at which the subject vehicle
collides into the target vehicle. It is measured at the moment of the
collision.
The relationship between the collision speed and the kinetic energy is
given by the following formula:
2
2
1collisionkin mvE
Motivation: The kinetic energy released at the moment of the collision
determines the severity of the impact between the subject vehicle and
the target vehicle.
time-to-collision
(TTC) at
warning
Definition: the point of time of the warning generation to the subject
vehicle driver. A warning signal is issued when some part of the target
vehicle lies within the alert zone or when the target vehicle is predicted
to be in the alert zone within a specified time delay after the warning
signal onset. An alert zone shall define the position at which the target
vehicle can provoke a warning signal.
A warning signal that does not fulfil this requirement is a False alarm.
Motivation: A large number of false warnings will cause the driver to
mistrust the system and ultimately the driver might turn the system off.
This safety performance indicator is introduced in deliverable D3.2 as a
result of gained experience during testing.
Deliverable D3.2
eVALUE 23 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
5.2 Lateral functions
Two safety performance indicators are proposed for lateral functions, see Table 3.
Table 3 Safety performance indicators for lateral functions
Safety
performance
indicator
Definition/Motivation
time-to-line
crossing (TLC)
at warning
Definition: the point of time of the warning generation to the driver. A
warning signal is issued when some part of the subject vehicle lies
within the alert zone or when the subject vehicle is predicted to be in the
alert zone within a specified time delay after the warning signal onset.
An alert zone typically defines the position at which the issuing of a lane
departure warning is allowed. A warning signal that does not fulfil this
requirement is a False alarm.
Motivation: The objective of the warning signal is to support the driver
in his manoeuvres to avoid a lane departure.
A large number of false warnings will cause the driver to mistrust the
system and ultimately the driver might turn the system off. This safety
performance indicator is introduced in deliverable D3.2 as a result of
gained experience during testing.
Deliverable D3.2
eVALUE 24 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Safety
performance
indicator
Definition/Motivation
time until target
enters Collision
Risk Zone at
warning
Definition: the time to entering Collision Risk Zone is based on the
vehicles position and speed at the time of the warning. It depends also
on the geometry of the vehicles and the Collision Risk Zone.
The Collision Risk Zone may be stated by the OEM/system
manufacturer in terms of an official performance specification or chosen
according to an applicable standard such as ISO 17387.
The value of the time to entering Collision Risk Zone is obtained by
dividing the distance between the Collision Risk Zone boundary and the
leading or trailing edge of the target vehicle by the overtaking speed.
All warnings shall occur within an allowed warning zone (cf. ISO 17387),
which is an extension of the Collision Risk Zone. A warning signal that
does not fulfil this requirement is a False alarm.
Motivation: The objective of the warning signal is to support the driver
in his manoeuvres to avoid a lane change collision.
A large number of false warnings will cause the driver to mistrust the
system and ultimately the driver might turn the system off. This safety
performance indicator is introduced in deliverable D3.2 as a result of
gained experience during testing.
Deliverable D3.2
eVALUE 25 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
5.3 Stability functions
Twelve safety performance indicators are proposed for stability functions, see Table 4.
Table 4 Safety performance indicators for stability functions
Safety
performance
indicator
Definition/Motivation
mean
longitudinal
deceleration
Definition: the mean longitudinal deceleration is expressed by the
following formula
where
T0 is the time when driver starts to act on the brake pedal
T2 is the time when vehicle reaches the 10 km/h velocity drop off
Vx and ax follow the definitions in the glossary, expressed in [m/s] and [m/s2].
Ψ is the yaw rate [rad/sec]
Motivation: is the basis to evaluate the longitudinal performance i.e.
stopping distance. This safety performance indicator is introduced in
deliverable D3.2 as a result of gained experience during testing.
equivalent
deceleration
Definition: a parameter combining one performance measure for
stability and one performance measure for the ability to stop. It is able
to quantify the stability and the control done by the vehicle’s systems of
the available adherence of the different wheels in µ-split scenario.
2
0
2
0110 T
T
x
T
T
X
FED
dta
dtV
aa
(See variable definition listed in mean longitudinal deceleration)
Motivation: is used to evaluate the trade-off between stability and
braking performance. This safety performance indicator is introduced in
deliverable D3.2 as a result of gained experience during testing.
2
002
10
1T
T
xF dtaTT
a
Deliverable D3.2
eVALUE 26 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Safety
performance
indicator
Definition/Motivation
equivalent
deceleration on
different tracks
Definition:
(See variable definition listed in mean longitudinal deceleration)
ηHIGH and ηLOW are respectively ratios between the longitudinal
decelerations means on high and low adherence surfaces with respect
to gravity acceleration.
Motivation: aims to be a representative parameter of the vehicle
behaviour on µ split surfaces and it should be quite stable to the track
change. This safety performance indicator is introduced in deliverable
D3.2 as a result of gained experience during testing.
use of adherence Definition: The use of adherence (ε) is calculated as the quotient
between the theoretical stopping distance and the stopping distance
using the deceleration at μ-split.
Where
theorSD is the theoretical stopping distance calculated using the
average deceleration between high and low µ initial braking
manoeuvres.
µsplitSD is the stopping distance using the deceleration of the µ-split
braking manoeuvre.
Motivation: The maximum theoretical value ε=100% would represent
a very high use of adherence. The theoretical value ε=0% would
represent a very poor use of adherence. A high use of adherence
corresponds to a short stopping distance. This safety performance
indicator is introduced in deliverable D3.2 as a result of gained
experience during testing.
dtV
TTg
aa
T
T X
LOWHIGHLOWHIGH
F 2
002
10 1111
Deliverable D3.2
eVALUE 27 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Safety
performance
indicator
Definition/Motivation
stability Definition: the stability is expressed by the following formula;
high
lowSWRSWAStability
maxmax
Where
maxSWA is the maximum steering wheel angle achieved during the
braking manoeuvre
maxSWR is the maximum steering wheel rate achieved during the
braking manoeuvre
low is the maximum deceleration achieved on homogeneous braking
on low µ surface
high is the maximum deceleration achieved on homogeneous braking
on high µ surface
Motivation: Stability indicates the effort exerted by the driver to keep
the vehicle on the intended course. This safety performance indicator
is introduced in deliverable D3.2 as a result of gained experience
during testing.
yaw rate ratio at
COS+1s and
COS+1.75s
Definition: yaw rate is the rotation around the z axis of the vehicle.
Motivation: yaw rate describes the stability of the vehicle. This safety
performance indicator is introduced in deliverable D3.2 as a result of
gained experience during testing.
lateral
displacement at
1.07s
Definition: the lateral displacement of the vehicle’s centre of gravity
with respect to its initial straight path during the portion of the sine with
dwell manoeuvre prior to beginning of the steering dwell.
Motivation: lateral displacement quantifies the vehicle
responsiveness. This safety performance indicator is introduced in
deliverable D3.2 as a result of gained experience during testing.
Deliverable D3.2
eVALUE 28 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Safety
performance
indicator
Definition/Motivation
driver intention
following
Definition: relates the difference between the real and the expected
yaw response.
time
SW
Atime
yaw
R
time
ay
time
Bra
ke
pre
ss.
time
SW
A
time
yaw
R
time
ay
time
Bra
ke p
ress.
Response gain = Response peak / SWA peak
delay
SWA peak
Response peak
time
SW
A
time
ya
wR
time
ay
time
sl R
EF
time
Bra
ke
pre
ss
time
ya
wR
diff
time
ay d
iff
time
vx
Response
control
(ay)
Response
control
Response
stability
expected responseactual responseexpected response
actual response
expected
expected - realDIF
0 expected yaw response
> 0 over responsive yaw motion
< 0 under responsive yaw motion
Motivation: expresses how well the movements of the vehicle follow the acting of the driver at the steering wheel. This safety performance indicator is introduced in deliverable D3.2 as a result of gained experience during testing.
1st steering
wheel torque
peak
Definition: corresponds to the first steering wheel torque peak
measured during the manoeuvre.
Motivation: a very high torque indicates a low possibility to control the
vehicle. This safety performance indicator is introduced in deliverable
D3.2 as a result of gained experience during testing.
wheel lift (WL) Definition: This represents the rollover condition: if both inner wheels
are lifted more than 5 cm from the ground and for more than 20 ms.
Motivation: The wheel lift is used to assess roll stability of the vehicle.
Deliverable D3.2
eVALUE 29 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Safety
performance
indicator
Definition/Motivation
relative radius
Rrel
Definition: The relative radius (Rrel) is the difference between the trajectory radius in the test run (Ri) and the trajectory radius in the initial test run (R1). Rrel = Ri – R1
Where R1 is the final radius value at the closing curve.
Motivation: A small relative radius value corresponds to a good
understeer control.
slip angle
(SlipCOG)
Definition: slip angle at the centre of gravity (SlipCOG) of the vehicle. Motivation: The slip angle (SlipCOG), is an indicator of the oversteer
during the test manoeuvres. A small slip angle value corresponds to a
good oversteer control. This safety performance indicator is introduced
in deliverable D3.2 as a result of gained experience during testing.
Deliverable D3.2
eVALUE 30 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
6 The eVALUE Performance Testing
Performance testing must be made with limited resources. Scenarios, test cases and the
number of trials have to be carefully selected. Each scenario will require a number of test
cases. Each test case will have to be performed several times to get statistically significant
data. Performance testing must concentrate on high output from a limited number of trials.
The development testing performed by vehicle manufacturers and system suppliers will be
far more time-consuming [eVALUE D3.1].
The complete eVALUE performance testing process will comprise inspections and physical
testing for the three clusters, see Figure 2. The performance testing is started by performing
two inspections.
Figure 2: The eVALUE performance testing process
The first inspection is to define the subject vehicle as a preparation for the coming tests. A
certain vehicle model may exist in several types with different level of assistance. The
inspection is to identify the type of vehicle as well as to analyse performance and
characteristics of ICT-based safety targeting longitudinal, lateral, and stability assistance.
Additionally, the safety performance demand on external communication is reviewed. The
inspection is necessary to specify the vehicle under test (the subject vehicle) and all other
types of the vehicle model for which the result of the performance test will be regarded valid.
eVALUE
Performance Testing
Inspection
Define the Subject Vehicle
Inspection
Environmental Conditions
Physical testing
Cluster 1
Longitudinal
Physical testing
Cluster 2
Lateral
Physical testing
Cluster 3
Stability
Inspection
HMI
Assessment
Test Report
Inspection
Functional Safety
Deliverable D3.2
eVALUE 31 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The second inspection analyses how environmental factors influence the performance of a
vehicle. The inspection shall confirm if performance testing on the test track can be made at
normal environmental conditions.
Five actions are then taken to assess the performance. The test procedures for physical
testing in cluster 1, 2 and 3 can be performed in any order or even in parallel. Inspections for
HMI and functional safety can also be performed at the same time. The results of all five
activities are then used as input for the assessment of the ICT-based safety functions.
The eVALUE performance testing can also be described by showing the components of the
test programme, see Figure 3. Each testing protocol describes a test procedure which
contains test cases which shall be performed (each test case shall also be performed a
certain number of times, trials). The output from the testing protocols is the test results which
are used in the assessment protocols where the performance of the subject vehicle in the
current test is determined.
Testing
Protocols
Assessment
ProtocolsTest Reports
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE 1
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE 2
...
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE n
Test Results 1
Test Results 2
Test Results n
...
CLUSTER 1
defined in
WP3
defined in
WP4
obtained in the
execution of the
tests defined by
the project
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE 1
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE 2
...
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE n
Test Results 1
Test Results 2
Test Results n
...
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE 1
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE 2
...
Test
Case 1
Test
Case 2
Test
Case 3
...
TEST
PROCEDURE n
Test Results 1
Test Results 2
Test Results n
...
each test
procedure is
developed in
WP2
CLUSTER 2 CLUSTER 3
Scenarios
compiled in
WP1
eVALUE Test Programme
Figure 3: The components of the eVALUE performance testing
Deliverable D3.2
eVALUE 32 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The testing protocol guides the test engineer by specifying: how to prepare the subject
vehicle and test track for the tests, how to conduct the actual tests, and which test cases to
perform. The prioritisation of test cases has been made by studying and interpreting accident
data. Based on this study, assumptions concerning test conditions; vehicle speeds,
departure rates, target vehicles, etc have been made. For a test program, a reasonable
amount of test cases is necessary but shall be realisable with limited resources.
Consequently the prioritisation of test cases is important and the most representative test
cases should be selected.
Deliverable D3.2
eVALUE 33 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
7 Literature
[eVALUE D1.1] eVALUE.
Project deliverable D1.1 State of the Art and eVALUE scope
2008
[eVALUE D1.2] eVALUE.
Project deliverable D1.2 Concepts definition
2008
[eVALUE D2.2] eVALUE.
Project deliverable D2.2 Specifications list
2009
[eVALUE D3.1] eVALUE.
Project deliverable D3.1 Testing protocols, first version
2010
Deliverable D3.2
eVALUE 34 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Annex A Inspection protocols
In this annex the four inspection protocols for the eVALUE inspections are presented. The
four inspections are:
1. Definition of the subject vehicle
2. Environmental conditions
3. Human-Machine Interface (HMI)
4. Functional safety
The inspection protocols have a similar structure. The first three sections are Scope,
References, and Definitions. The content of these chapters are rather self-explanatory:
Scope deals with the applicability of the inspection protocol, References lists useful
references, and Definitions defines term which are used throughout the inspection protocol.
Following that, there is a section which describes the inspection procedure, i.e. information
needed as input and how the result from the inspection shall be reported. All inspection
protocols contain one or several checklists which support the execution of the inspection.
Deliverable D3.2
eVALUE 35 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.1 Inspection protocol - Definition of the subject vehicle
A.1.1 Scope
It is important to clearly identify the vehicle under test (i.e. the subject vehicle), especially if
the vehicle is a prototype vehicle. All major characteristics of the vehicle influencing the
performance testing must be noted. The challenge of identifying the vehicle under test is
greater at performance testing than at testing of passive safety. Even small design
modifications in the active safety systems may severely influence the performance.
A certain model of a road vehicle is normally produced in several different types. It may exist
differences e.g. in engine, body and brakes. However some differences may not be important
for the performance testing. It will in many cases be possible to choose one type of vehicle
for the performance testing and assume that the test results are valid also for other types of
the same vehicle.
Evaluation of longitudinal, lateral, and stability safety functions require information about their
characteristics and limits, e.g. the kind of sensor used and limits of its operation range. If the
manufacturer has already done tests it is helpful to receive information about these tests to
ensure that they are adapted to the safety functions of the vehicle under test and to the
related standards.
The safety performance dependence on reliable external communication will also be
inspected. External communication includes positioning, vehicle-to-vehicle (V2V), and
vehicle-to-infrastructure (V2I) communication. Transmission anomalies, such as distortion
and interference, can have a negative effect on the overall performance and it is
consequently addressed in the inspection.
This testing protocol describes how to perform an inspection to identify the type of vehicle, its
safety functions, and possible dependence on external communication. The inspection is
necessary to specify the vehicle under test and all other types of the vehicle model intended
to be covered by the performance test. It is also a necessary step for the test engineers
familiarizing with the subject vehicle and its safety functions.
The inspection will be applicable for cars, trucks and buses. It shall be performed before any
physical testing or laboratory testing is performed. The results of the inspection cannot be a
pass or fail judgement, but simply a definition of the subject vehicle and its safety functions.
A.1.2 References
Longitudinal systems (e.g. ACC, FCW, and CM):
[15622] International standard ISO 15622:2002
Transport information and control systems – Adaptive Cruise Control Systems
Performance requirements and test procedures
International Standardisation Organisation, 2002
Deliverable D3.2
eVALUE 36 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
[15623] International standard ISO 15623:2002
Transport information and control systems – Forward vehicle collision warning
systems –Performance requirements and test procedures
International Standardisation Organisation, 2002
[J2399] SAE standard J2399
Adaptive Cruise Control (Acc) Operating Characteristics and User Interface
Society of Automotive Engineers, 2003-12
[FMC05b] Houser, A., Pierowicz, J., and McClellan, R.
Concept of Operations and Voluntary Operational Requirements for Forward
Collision Warning Systems (CWS) and Adaptive Cruise Control (ACC)
Systems On-Board Commercial Motor Vehicles.
FMCSA-MCRR-05-007, Federal Motor Carrier Safety Administration, 2005
[J2400] SAE standard J2400
Human Factors in Forward Collision Warning Systems: Operating
Characteristics and User Interface Requirements
Society of Automotive Engineers, 2003-08
Lateral systems (e.g. BSD, LDW, and LKA):
[17361] International standard ISO 17361:2007
Intelligent transport systems – Lane departure warning systems –
Performance requirements and test procedures
International Standardisation Organisation, 2007
[17387] International standard ISO 17387:2008
Intelligent transport systems – Lane change decision aid systems (LCDAS)–
Performance requirements and test procedures
International Standardisation Organisation, 2007
[FMC05a] Houser, A., Pierowicz, J., and Fuglewicz, D.
Concept of Operations and Voluntary Operational Requirements for Lane
Departure
Warning Systems (LDWS) On-Board Commercial Motor Vehicles.
FMCSA-MCRR-05-005, Federal Motor Carrier Safety Administration, 2005
[810757] Development of Crash Imminent Test Scenarios for Integrated Vehicle-Based
Safety Systems (IVBSS)
Deliverable D3.2
eVALUE 37 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Publication DOT 810 757
U.S. Department of Transportation, National Highway Traffic Safety
Administration
April 2007
Download at
http://www.itsa.org/itsa/files/pdf/IVBSS%20Crash%20Imminent%20Test%20Scenario
%20Report%20-%20DOT%20HS%20810%20757.pdf
[J2808] SAE standard J2808
Road/Lane Departure Warning Systems: Information for the Human Interface
Society of Automotive Engineers, 2007-08
Stability systems (e.g. ABS and ESC):
[21994] International standard ISO 21994:2007
Passenger cars – Stopping distance at straight-line braking with ABS – Open-
loop test method
International Standardisation Organisation, 2007
[FMVSS126] FMVSS No. 126
Laboratory Test Procedure for Electronic Stability Control Systems
U.S. Department of Transportation
National Highway Traffic Safety Administration
April, 2007
Download at
http://www.nhtsa.dot.gov/staticfiles/DOT/NHTSA/Vehicle%20Safety/Test%20Procedur
es/Associated%20Files/TP126-00.pdf
A.1.3 Definitions
Model – The model of a vehicle is the basic model used as a trade name by the OEM (e.g.
Citroen C5 Tourer 2.0 16V)
Type – The type of a vehicle is one specific configuration of the vehicle model characterized
e.g. by body, engine, ICT based safety systems and brakes
Subject vehicle: Vehicle equipped with the system, which should be tested and which is
related to the topic of discussion
Target vehicle: The target vehicle reproduces the characteristics of the required vehicle
according to sensor’s technology ant the test protocol requirements.
Relative speed: The relative speed rv is the difference between the target vehicle’s speed vt
and the subject vehicle’s speed vs; rv = vs - vt
Clearance: Distance from target vehicle’s trailing surface to the subject vehicle’s leading
surface
Deliverable D3.2
eVALUE 38 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Time gap: Time interval for travelling the clearance between consecutive vehicles. The time
gap is related to vehicle speed v and clearance c by t = c/v
Time to collision: Time interval before an impact between subject vehicle and target
vehicle, if the velocities of the subject and target vehicle are kept constant. The time to
collision is related to relative speed rv and clearance c by: tc = c/rv
Normal environmental conditions: Conditions normally applied at performance testing on
the test track:
- Test location shall be on a flat, dry asphalt or concrete surface. - Temperature 20 ˚C ±20˚C. - Horizontal visibility shall be greater than 1 km.
[Standard ISO 15622, clause 7.1]
Botts’ dots: Round non-reflective raised pavement markers
Rumble strips: Also known as audio tactile profiled markings are a road safety feature that
alerts drivers to potential danger by causing a tactile vibration and audible rumbling,
transmitted through the wheels into the car body.
A.1.4 Inspection
A.1.4.1 Principle
The vehicle OEM will supply information concerning the model of the vehicle, the type of the
vehicle and all other information on factors influencing the performance testing.
The vehicle OEM or safety system supplier will supply information about their ICT-based
safety functions. The information provided must contain results of previous tests and
simulations with an analysis on how these systems contribute to decrease accidents and
their influence on the driver. They will also supply an overview concerning the development
testing of the safety systems under different driving conditions.
The vehicle OEM or safety system supplier will provide their analysis with regard to the
communication influence on the operation of the ICT-based safety functions installed in the
vehicle. They will also supply information concerning the development testing of the safety
systems under poor transmission conditions
The inspection will include a study of the documentation provided, and contacts with
representatives of the manufacturer. The inspection is supported by checklists. The result will
be summarized in a written report.
The inspection only addresses the type of vehicle and its ICT-related safety functions.
Aspects regarding the preparation of the vehicle (load, fuelling, driving robots etc) are
specified in the testing protocols describing physical testing.
Deliverable D3.2
eVALUE 39 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.1.4.2 Information required from the manufacturer
The following information will be necessary for the inspection:
name, prototype status, body, engine, brakes, tyres
ICT-based safety functions o description of the system, its characteristics, limitations and expected
capabilities and to which standards it complies o description of test set-ups (targets, lane markings etc) o description of external communication channels used
A.1.4.3 Inspection procedure
Following aspects shall be inspected:
name/designation (e.g. Volvo FH16)
prototype vehicle
body (e.g. sedan, coupé, cabriolet, station wagon, bus, truck)
engine (e.g. diesel D16G: 540, 600, 700 hp)
brakes
tyres (e.g. winter tyres Michelin X-ice 205/60R15 )
ICT-based safety functions (e.g. ABS, ESC) o test documentation of the OEM o test conditions o system characteristics o related standards o accuracy and repetitiveness of the test o accuracy of sensors used during the test
external communication requirements
The inspection will study the documentation provided by the manufacturer. At least one
meeting with representatives of the manufacturer should be held. Working notes shall be
taken.
The inspection is supported by checklists (see appendix 1-5) listing specific aspects. For
each ICT-based safety function the checklist in appendix 5 shall be considered. The
inspection will end when the type of vehicle and the safety functions have been identified and
a report has been written.
A.1.4.4 Report
The result of the inspection will be:
- a clear identification of the subject vehicle. The identification shall include a list of all active safety functions included.
- a list of all other types of the vehicle model for which the performance testing results will be regarded valid.
- if the characteristics of the system complies with the related standard - if the inspection protocol used matches with the requirements of the standard - if the inspection protocol used by the OEM provides repeatable test results
Deliverable D3.2
eVALUE 40 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
- influence of the availability of communication systems on the performance of the subject vehicle
- definition of safe-state in the occurrence of communication failures/faults - performed test cases for communication shortcomings by the manufacturer
The result will be documented in a report based on the questions in the checklists in
appendices A.1.5 to A.1.9.
Deliverable D3.2
eVALUE 41 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.1.5 Checklist - Type of vehicle
n.a. = Not applicable question (some questions may not be applicable for all vehicles).
No Question Answer
Name/designation
A What trade name is used for the subject vehicle?
B Are there other trade names for the same vehicle?
C What is the date of production (model year) of the vehicle?
Prototype
D Is the vehicle a prototype vehicle? If so, please identify all active safety functions including electronic hardware and software versions extra carefully. (See question Q.)
Body
E Which body type is used for the subject vehicle? (sedan, station wagon, etc.)
F Are there other body types for which the test results should be considered valid?
G If the vehicle is a truck; what trailer is used at the performance testing?
Engine
H Which engine is used in the subject vehicle?
I Are there other engines for which the test results should be considered valid?
Brakes
J Which brakes are used in the subject vehicle?
K Are there other brakes for which the test results should be considered valid?
Deliverable D3.2
eVALUE 42 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Suspension
L Which suspension is used in the subject vehicle?
M Is there other suspension for which the test results should be considered valid?
Tyres
N Which tyres (type and dimension) are used in the subject vehicle?
O Are there other tyres (types and dimensions) for which the test results should be considered valid?
ICT-based safety systems
P Which ICT-based safety functions are used in the subject vehicle?
Q Have all ICT-based safety systems been adequately identified (including electronic hardware and software versions)?
Supplemental equipment
R Is there other supplemental equipment at the subject vehicle which is standard when the vehicle is marketed?
Identification numbers
S Is there already a type approval number for the vehicle type? (The number shall be written down in inspection protocol.)
T Is there a vehicle identification number of the vehicle? (The number shall be written down in inspection protocol report.)
Deliverable D3.2
eVALUE 43 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.1.6 Checklist - Longitudinal functionality
n.a. = Not applicable question (some questions may not be applicable for all vehicles).
No Question n.a. Yes No Comment
Capabilities of the subject vehicle
A1 Does the subject vehicle warn the driver when a target is detected?
A2 Does the subject vehicle warn the driver when a dangerous situation is detected?
A3 Does the safety function act on any component of the vehicle (brakes, brake lights, steering…)?
A4 Does the safety function advise the driver before actuating any component?
A5 Is the longitudinal safety function active in a specified speed range? (Or at all speeds?)
Was the subject vehicle tested with different target vehicles?
B1 Real target?
B1a Car?
B1b Truck?
B1c Motorbike?
B1d Bus?
B2 Dummy target?
B2a Car?
B2b Truck?
B2c Motorbike?
B2d Bus?
Does the subject vehicle detect different targets?
C1 Real vehicle?
C1a Car?
C1b Truck?
C1c Motorbike?
C1d Bus?
C2 Dummy Vehicle?
C3 Car?
C4 Truck?
C5 Motorbike?
C6 Bus?
C7 Pedestrian?
Which data was obtained from the tests?
D1 Speed?
D2 Relative speeds (longitudinal and lateral)?
Deliverable D3.2
eVALUE 44 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
D3 Acceleration lateral and longitudinal?
D4 Positions X,Y,Z of target and subject vehicle?
D5 Time?
D6 Brake light activation?
D7 Brake fluid pressure?
D8 Clearances (longitudinal and lateral)?
D9 Time gap?
D10 Time to collision?
D11 Off-board video?
D12 On-board video?
Was the subject vehicle tested with different road characteristics?
E1 Straight road?
E2 Curve road?
E3 Longitudinal slopes – ramps (hills)?
E4 Transversal slopes (in bends)?
E5 Guard rails?
E6 Signs?
E7 Lamp posts?
E8 Tunnel?
E9 Urban region (town)?
Was the subject vehicle tested with different scenarios?
F1 Following the target?
F2 Overtaking the target?
F3 Overtaken by the target?
F4 Target braking?
F5 Target accelerating?
F6 Lane change of the target?
F7 Lane change of the subject vehicle?
F8 Without target?
Test driver information
G1 Professional driver (e.g. test driver or development engineer)?
G2 Normal driver (i.e. a driver without special training for test driving)?
Test equipment information
H1 Is position information provided by a GPS or differential GPS equipment with suitable accuracy?
H2 Is speed information provided by a speed sensor with suitable accuracy?
H3 Is information provided at suitable data acquisition frequency?
Deliverable D3.2
eVALUE 45 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.1.7 Checklist - Lateral functionality
n.a. = Not applicable question (some questions may not be applicable for all vehicles).
No Question n.a. Yes No Comment
Capabilities of the subject vehicle
A1 Does the subject vehicle warn the driver when a lane departure is detected?
A2 Does the subject vehicle support the driver when a road departure is detected?
A3 Does the safety function act on the steering of the vehicle?
A4 Does the safety function act on the braking of the vehicle?
A5 Does the safety function advise the driver before actuating any component?
A6 Is the lateral safety function active in a specified speed range? (Or at all speeds?)
Was the subject vehicle tested with different target vehicles?
B1 Real vehicle?
B1a Car?
B1b Truck?
B1c Motorbike?
B1d Bus?
B2 Dummy Vehicle?
B2a Car?
B2b Truck?
B2c Motorbike?
B2d Bus?
Does the subject vehicle detect different targets?
C Real vehicle?
C1a Car?
C1b Truck?
C1c Motorbike?
C1d Bus?
C2 Dummy Vehicle?
C2a Car?
C2b Truck?
C2c Motorbike?
C2d Bus?
Which data was obtained from the tests?
D1 Speed?
D2 Lateral relative speeds?
Deliverable D3.2
eVALUE 46 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
D3 Lateral acceleration?
D4 Positions X,Y,Z of target and subject vehicle?
D5 Time?
D6 Turn indicator actuation?
D7 Steering torque?
D8 Lateral clearances?
D9 Time gap?
D10 Time to collision?
D11 Off board video?
D12 On board video?
Was the subject vehicle tested with different types of lane marking?
E1 Circular reflectors?
E2 Rumble strips?
E3 Botts’ dots?
E4 Solid lines?
E5 Dashed lines?
E6 Dots?
E7 Yellow lane marks?
E8 White lane marks?
E9 Wide lane marks?
E10 Narrow lane marks?
Was the subject vehicle tested with different road characteristics?
F1 Straight road?
F2 Curve road?
F3 Hills?
F4 Guard rails?
F5 Signs?
F6 Lamp posts?
F7 Tunnel?
F8 Urban region (town)?
Was the subject vehicle tested with different scenarios?
G1 1) Lane departure?
G2 2) Road departure?
G3 3) Lane change with target vehicle overtaking?
Test driver information
H1 Driving robot?
H2 Professional driver (e.g. test driver or development engineer)?
H3 Normal driver (i.e. a driver without special training for test driving)?
Test equipment information
Deliverable D3.2
eVALUE 47 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
I1 Is position information provided by GPS equipment with suitable accuracy?
I2 Is speed information provided by a speed sensor with suitable accuracy?
I3 Is lane mark information provided by a camera with suitable accuracy?
I4 Is lane mark information provided by a laser scanner with suitable accuracy?
I5 Is lane mark information provided by infrared sensor with suitable accuracy?
I6 Is lateral object information provided by a camera with suitable accuracy?
I7 Is information provided at suitable data acquisition frequency?
Deliverable D3.2
eVALUE 48 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.1.8 Checklist - Stability functionality
n.a. = Not applicable question (some questions may not be applicable for all vehicles).
No Question n.a. Yes No Comment
Capabilities of the subject vehicle
A1 Does the subject vehicle warn the driver when it intervenes?
A2 Has the safety function been proved to increase the stability of the vehicle?
A3 Has the safety function showed undesired interventions during tests?
Have the interactions of the safety function with other control systems been tested?
B1 Driver steering recommendation?
B2 Active front steering?
B3 Active rear steering?
B4 Active differential?
B5 Roll stability control?
B6 Trailer stability assist?
B7 Others (specify)?
Was the subject vehicle tested for different trucks configurations? (only for trucks)
C1 Tractor only?
C2 Tractor with semitrailer?
C3 Tractor with trailer?
Was the subject vehicle tested for different vehicle load conditions?
D1 Driver only?
D2 Full load condition?
D3 Other load conditions (specify)?
Was the subject vehicle tested on different road surface?
E1 Dry asphalt?
E2 Wet asphalt?
E3 Snow / ice?
E4 μ-split?
Was the subject vehicle tested for different dynamic manoeuvres?
F1 Emergency braking in straight line?
F2 Emergency braking in curve?
F3 Slow ramp steer?
F4 Step steer?
F5 Swept-sine steer?
F6 Single/double lane change?
F7 Sine with Dwell?
Deliverable D3.2
eVALUE 49 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
F8 ATI reversed steer?
F9 Other (specify)?
Which data was obtained from the tests?
G1 Speed?
G2 Lateral acceleration?
G3 Longitudinal acceleration?
G4 Vehicle X,Y positions?
G5 Yaw rate?
G6 Sideslip angle?
G7 Corner pressures?
G8 Steering wheel angle?
Driver information
H1 Driving robot?
H2 Professional driver (e.g. test driver or development engineer)?
H3 Normal driver (i.e. a driver without special training for test driving)?
Test equipment information
I1 Is system sensors information available (acquisition via CAN bus)?
I2 Is information provided at suitable data acquisition frequency?
Deliverable D3.2
eVALUE 50 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.1.9 Checklist - External communication
n.a. = Not applicable question (some questions may not be applicable for all vehicles.)
No Question n.a. Yes No Comment
Requirements for external communication
A Is the safety performance of the subject vehicle dependent of external communication?
B Are requirements to availability and integrity of information/data specified?
Type of information/data
C Is the subject vehicle safety dependent on positioning information?
D Have the effect of different communication errors been analysed by the manufacturer/developer?
1) corruption 2) unintended repetition 3) incorrect sequence 4) loss 5) unacceptable delay 6) insertion 7) masquerade 8) addressing
E Has a safe state in the occurrence of communication failures been defined?
Deliverable D3.2
eVALUE 51 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
-
Vehicle-to-infrastructure communication
F Is the subject vehicle safety dependent of vehicle-to-roadside information?
G Have the effect of different communication errors been analysed by the manufacturer/developer?
1) corruption 2) unintended repetition 3) incorrect sequence 4) loss 5) unacceptable delay 6) insertion 7) masquerade 8) addressing
H Has a safe-state in the occurrence of communication failures been defined?
Vehicle-to-Vehicle communication
I Is the subject vehicle safety dependent of vehicle-to-vehicle information?
J Have the effect of different communication errors been analysed by the manufacturer/developer?
1) corruption 2) unintended repetition 3) incorrect sequence 4) loss 5) unacceptable delay 6) insertion 7) masquerade 8) addressing
K Has a safe-state in the occurrence of communication failures been defined?
Deliverable D3.2
eVALUE 52 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.2 Inspection protocol - Environmental conditions
A.2.1 Scope
Development of ICT based safety systems requires extensive testing to understand how the
vehicle is influenced by its environment. Temperature, light conditions, pollution, precipitation
or road characteristics may influence the performance. Sensors, controllers, and actuators
might be negatively affected by adverse environmental conditions. Development testing by
the manufacturer will address all identified hazards.
Performance testing on the test track is often performed at normal environmental conditions.
Due to the limited time and resources during performance testing, an inspection could be
used to validate which environmental conditions the vehicle can handle.
The objective is to verify that the design fulfils the requirements on operation during different
environmental conditions. The important environmental influence factors must be identified,
and the vehicle and its ICT-based safety systems must be verified during different
environmental conditions.
The inspection will be applicable for cars, trucks and buses. The results of the inspection will
be input to physical testing and lab testing in all clusters. If not already tested by the OEM,
the laboratory tests will be used to identify how environmental conditions affect vehicle
performance and in which way. These laboratory tests will establish the range (or limits) of
the subsequent physical tests.
A.2.2 References
[15622] International standard ISO 15622:2002 Transport information and control systems –
Adaptive Cruise Control Systems – Performance requirements and test procedures
[15623] International standard ISO 15623:2002 Transport information and control systems –
Forward vehicle collision warning systems – Performance requirements and test procedures
[17361] International standard ISO 17361:2007 Intelligent transport systems – Lane
departure warning systems – Performance requirements and test procedures
A.2.3 Definitions
normal environmental conditions : the conditions normally applied at performance testing
on the test track:
- test location shall be on a flat, dry asphalt or concrete surface. - temperature 20 ˚C ±20˚C. - horizontal visibility shall be greater than 1 km.
[standard ISO 15623, clause 6.2]
Deliverable D3.2
eVALUE 53 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.2.4 Inspection
A.2.4.1 Principle
The vehicle OEM or safety system supplier will supply their analysis how different
environmental conditions are expected to influence the safety function. They will also supply
information concerning the development testing of the safety systems during different
environmental conditions.
The inspection will include a study of the documentation provided, and an interview with
representatives of the manufacturer. The inspection is supported by a checklist. The
parameters to be analysed depend on the design validated. For example, lane markings
affect the performance of lateral safety functions but not stability safety functions.
A.2.4.2 Information required from the manufacturer
Following information will be necessary for the inspection:
- information on which environmental parameters were identified to influence the performance
- information on how environmental influence was tested by the manufacturer Following information is optional for the inspection:
- test protocols for environmental stress tests - references to standards
A.2.4.3 Inspection procedure
Following environmental parameters shall be inspected:
temperature
light conditions
pollution
precipitation
road characteristics
miscellaneous parameters The inspection will study the documentation provided by the manufacturer. At least one
meeting with representatives of the manufacturer should be held. Working notes shall be
taken. It may also be helpful to drive the vehicle, if possible at different environmental
conditions.
The review is supported by a checklist (see annex) listing specific aspects associated to
environmental parameters. The inspection will end when conclusions have been drawn and a
report has been written.
Deliverable D3.2
eVALUE 54 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.2.4.4 Report
The result of the inspection will be four conclusions:
- If environmental conditions are expected to influence the performance of the design to be tested.
- If the developer has specified requirements for environmental influence. - if the developer has performed adequate tests of the environmental influence. - if normal environmental conditions can be applied at performance testing on the test
track. The result will be documented in a report based on the checklist in Appendix A.2.5.
Deliverable D3.2
eVALUE 55 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.2.5 Checklist - Environmental conditions
n.a. = Not applicable question (Some questions may not be applicable for all vehicles.)
No Question n.a. Yes No Comment
Environmental requirements specification
A Will the performance of the vehicle be affected by environmental conditions?
B Are environmental condition parameters addressed in the requirements specification of the design?
Temperature
C Is the performance dependent on temperature conditions?
D Has the manufacturer tested the performance in the
appropriate temperature range?
1) 10 ± 30 ºC [17361]
2) 20 ºC ± 20 ºC [15623] [15622]
Light conditions
E Is the performance dependent on lighting conditions?
F Has the manufacturer tested the performance under lighting performance such as:
F1 Daylight?
F2 Night?
F3 Dusk or dawn?
G Has the manufacturer tested the performance under different sun orientation?
G1 Sun in the front?
G2 Sun in the back?
G3 Sun at left?
G4 Sun at right?
Pollution
H Is the performance affected by pollution?
I Has the manufacturer tested the performance in traffic conditions affected by pollution such as:
I1 Mud?
I2 Salt?
I3 Water spray?
I4 Smoke?
Precipitation
Deliverable D3.2
eVALUE 56 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
J Will the performance of the vehicle be affected by precipitation?
K Has the manufacturer tested the performance in different precipitation conditions such as:
K1 Rain?
K2 Snow?
K3 Hail?
K4 Fog?
L Has the manufacturer tested the performance on road/track with different friction such as:
L1 Ice?
L2 Wet road?
L3 Water puddle?
Wind conditions
M Will the performance of the vehicle be affected by wind conditions?
N Has the manufacturer tested the performance at different wind conditions (wind directions and wind speeds)?
Road characteristics
O Will the performance of the vehicle be affected by the road characteristics?
P Has the manufacturer tested the performance in the presence of parts of the road infrastructure such as:
P1 Signs? P2 Guard rails? P3 Lamp posts? P4 Tunnel? Q Has the manufacturer tested the performance on
different road surface materials such as:
Q1 Concrete? Q2 Asphalt? Q3 Gravel? Q4 Sand? R Has the manufacturer tested the performance at
different road inclinations?
S Has the manufacturer tested the performance at different road curvatures?
T Has the manufacturer tested the performance at different types of lane marking?
U Has manufacturer tested the performance at different types of crossing, such as:
U1 Y-crossing? U2 X-crossing? U3 T-crossing?
Deliverable D3.2
eVALUE 57 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
U4 roundabouts?
Miscellaneous
V Has the manufacturer identified other environmental conditions affecting the performance?
Deliverable D3.2
eVALUE 58 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.3 Inspection Protocol - Human-machine interface (HMI)
A.3.1 Scope
Development of ICT-based safety systems requires extensive testing in order to understand
how the vehicle and driver are influenced by the HMI design. Depending on the level, the
driver will be warned, supported, and/or the system intervenes in the vehicle behaviour.
Hence the interaction between the vehicle and the driver is paramount.
Physical testing on the test track is often performed using robots or professional drivers. Due
to the limited time and resources during performance testing, an inspection could be used to
validate the appropriateness of the HMI design of the vehicle.
The inspection will be applicable for cars, trucks and buses. The results of the inspection will
be input to physical testing and lab testing, simulation, in all clusters. If not already tested by
the OEM, simulations will be used to identify how HMI affect vehicle performance and in
which way. These simulations will serve as input to the subsequent physical tests.
A.3.2 References
[2575] ISO 2575: Road Vehicles – Symbols for Controls, Indicators and Telltales
[4040] ISO 4040: Road Vehicles – Location of Hand Controls, Indicators and telltales
in Motor Vehicles
[16352] ISO 16352: Road Vehicles – Ergonomic Aspects of In-Vehicle Presentation for
Transport Information and Control Systems: Warning Systems
[2402] SAE J2402: Road Vehicles – Symbols for Controls, Indicators and Telltales
[7000] ISO 7000: Graphical symbols for use on equipment
[17287] ISO 17287: Road vehicles – Ergonomic aspects of transport information and
control systems – Procedure for assessing suitability for use while driving
[9355] ISO 9355: Ergonomic requirements for the design of displays and control
actuators
[11428] ISO 11428: Ergonomics – Visual danger signals - General requirements,
design and testing
[15005] ISO 15005: Dialogue principles
Deliverable D3.2
eVALUE 59 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
[15622] ISO 15622: Transport information and control systems -- Adaptive Cruise
Control Systems -- Performance requirements and test procedures
[15623] ISO 15623: Transport information and control systems -- Forward vehicle
collision warning systems -- Performance requirements and test procedures
[17361] ISO 17361: Intelligent transport systems -- Lane departure warning systems --
Performance requirements and test procedures
[17387] ISO 17387: Intelligent transport systems -- Lane change decision aid systems
(LCDAS) -- Performance requirements and test procedures
A.3.3 Definitions
HMI: Human Machine Interface
A.3.4 Inspection
A.3.4.1 Principle
The vehicle OEM or safety system supplier will supply their analysis how HMI design is
expected to influence the safety function. They will also supply information concerning the
development testing of the HMI of the safety systems.
The inspection will include a study of the documentation provided, and an interview with
representatives of the manufacturer. The inspection is supported by a checklist. The
parameters to be analysed depend on the design validated.
A.3.4.2 Information required from the manufacturer
Following information will be necessary for the inspection:
- information on which HMI parameters were identified to influence the performance - information on how HMI design was tested by the manufacturer. -
Following information is optional for the inspection:
- test protocols for HMI design - references to standards
A.3.4.3 Inspection procedure
For each cluster (longitudinal, lateral, and stability), the following HMI design aspects shall be
inspected:
Deliverable D3.2
eVALUE 60 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
warnings o visual o auditory o haptic
driver support
intervene o system/driver override o competition between systems
workload
system status indication
The inspection will study the documentation provided by the manufacturer. At least one
meeting with representatives of the manufacturer should be held. Working notes shall be
taken. It may also be helpful to examine and drive the vehicle.
The inspection is supported by a checklist (see annex) listing specific aspects associated to
HMI design. The inspection will end when conclusions have been drawn and a report has
been written.
A.3.4.4 Report
The result of the inspection will be four conclusions:
- If HMI design are expected to influence the performance of the design to be tested. - If the developer has specified requirements for HMI design. - If the developer has performed adequate tests of the HMI design. - If simulation experiments of the HMI design is necessary to complement performance
testing on the test track. The result will be documented in a report based on the checklist in Appendix A.3.5.
Deliverable D3.2
eVALUE 61 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.3.5 Checklist - HMI
n.a. = Not applicable question (Some questions may not be applicable for all vehicles.)
No Question n.a. Yes No Comment
HMI requirements specification
A Will the performance of the subject vehicle be affected by the HMI design?
B Is HMI design addressed in the requirements specification of the subject vehicle?
Manual
C Is the HMI well-explained in the manual of the subject vehicle?
Longitudinal Functionality
General
D Is the subject vehicle equipped with longitudinal safety functions?
Status Indication
E1 Are the modes of operation of the longitudinal safety functions presented to the driver? (E.g. activated - deactivated)
E2 Is there a possibility to turn off longitudinal safety functions?
E3 Is there a risk to turn the longitudinal safety functions off by mistake?
E4 Are the conditions of the longitudinal safety functions presented to the driver? (E.g. faulty – fault-free, capable/incapable)
Visual warnings
F1 Are there visual warnings related to longitudinal safety functions?
F2 Are standardized symbols used for visual warnings? (E.g. ISO 2575, SAE J2402)
F3 Is the size of the symbols adequate? (in relation to other warning symbols)
F4 Is the colour of the symbols adequate? (has a appropriate warning colour been selected)
F5 Is the luminance of the symbols adequate? (in relation to other warning symbols)
F6 Is the contrast of the symbols adequate? (in relation to other warning symbols)
F7 Is the viewing angle of the symbols adequate?
F8 Is the timing of symbol presentation adequate? (not too late or too early)
Deliverable D3.2
eVALUE 62 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
F9 Is the interval at which symbols are presented adequate?
F10 Is the location of the symbols adequate? (is head movement necessary)
F11 Is there an intensification of the warnings? (E.g. caution -> warning)
F12 Is the intensification performed adequately?
Visual information
G1 Is there visual information related to longitudinal safety functions? (e.g. menus)
G2 Is the choice of graphics/representational features suitable for what they represent?
G3 Are graphics/representational features grouped where possible?
G4 Is the information presented legible? (i.e. size, contrast, brightness, illumination, image stability, resolution, colour, etc.)
Auditory warnings
H1 Are there auditory warnings related to longitudinal safety functions?
H2 Is the auditory output (warning info) appropriate for the information to be conveyed?
H3 Is the volume of auditory warnings adequate?
H4 Is the volume adjustable by the driver?
H5 Is the tone of auditory warnings adequate? (in relation to other warning sounds)
H6 Is the timing of auditory warnings adequate? (not too late or too early)
H7 Is the interval at which auditory warnings are presented adequate?
H8 Is the location of auditory warnings adequate? (threat from one side gives warning from the same side)
H9 Is there an intensification of the warnings? (E.g. caution -> warning)
H10 Is the intensification performed adequately?
Haptic warnings
I1 Are there haptic warnings related to longitudinal safety functions?
I2 Is the intensity used for haptic warnings adequate? (possible to feel during normal driving conditions)
I3 Is the timing of haptic warnings adequate? (not too late or too early)
I4 Is the interval at which haptic warnings are presented adequately?
I5 Is the location of haptic warnings adequate? (threat from one side gives warning from the same side)
I6 Is there an intensification of the warnings? (E.g. caution -> warning)
Deliverable D3.2
eVALUE 63 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
I7 Is the intensification performed adequately?
Interventions
J1 Are there interventions related to longitudinal safety functions?
J2 Can the driver override the safety functions if needed?
J3 Can the safety functions override the driver?
Lateral Functionality
General
K Is the subject vehicle equipped with lateral safety functions?
Status Indication
L1 Are the modes of operation of the lateral safety functions presented to the driver? (E.g. activated - deactivated)
L2 Is there a possibility to turn off lateral safety functions?
L3 Is there a risk to turn the lateral safety functions off by mistake?
L4 Are the conditions of the lateral safety functions presented to the driver? (E.g. faulty – fault-free, capable/incapable)
Visual warnings
M1 Are there visual warnings related to lateral safety functions?
M2 Are standardized symbols used for visual warnings? (E.g. ISO 2575, SAE J2402)
M3 Is the size of the symbols adequate? (in relation to other warning symbols)
M4 Is the colour of the symbols adequate? (has a appropriate warning colour been selected)
M5 Is the luminance of the symbols adequate? (in relation to other warning symbols)
M6 Is the contrast of the symbols adequate? (in relation to other warning symbols)
M7 Is the viewing angle of the symbols adequate?
M8 Is the timing of symbol presentation adequate? (not too late or too early)
M9 Is the interval at which symbols are presented adequate?
M10 Is the location of the symbols adequate? (is head movement necessary)
M11 Is there an intensification of the warnings? (E.g. caution -> warning)
M12 Is the intensification performed adequately?
Deliverable D3.2
eVALUE 64 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
Visual information
N1 Is there visual information related to lateral safety functions? (e.g. menus)
N2 Is the choice of graphics/representational features suitable for what they represent?
N3 Are graphics/representational features grouped where possible?
N4 Is the information presented legible? (i.e. size, contrast, brightness, illumination, image stability, resolution, colour, etc.)
Auditory warnings
O1 Are there auditory warnings related to lateral safety functions?
O2 Is the auditory output (warning info) appropriate for the information to be conveyed?
O3 Is the volume of auditory warnings adequate? (possible to hear during normal driving conditions)
O4 Is the volume adjustable by the driver?
O5 Is the tone of auditory warnings adequate? (in relation to other warning sounds)
O6 Is the timing of auditory warnings adequate? (not too late or too early)
O7 Is the interval at which auditory warnings are presented adequate?
O8 Is the location of auditory warnings adequate? (threat from one side gives warning from the same side)
O9 Is there an intensification of the warnings? (E.g. caution -> warning)
O10 Is the intensification performed adequately?
Haptic warnings
P1 Are there haptic warnings related to lateral safety functions?
P2 Is the intensity used for haptic warnings adequate? (possible to feel during normal driving conditions)
P3 Is the timing of haptic warnings adequate? (not too late or too early)
P4 Is the interval at which haptic warnings are presented adequately?
P5 Is the location of haptic warnings adequate? (threat from one side gives warning from the same side)
P6 Is there an intensification of the warnings? (E.g. caution -> warning)
P7 Is the intensification performed adequately?
Interventions
Q1 Are there interventions related to lateral safety functions?
Deliverable D3.2
eVALUE 65 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
Q2 Can the driver override the safety functions if needed?
Q3 Can the safety functions override the driver?
Stability Functionality
General
R Is the subject vehicle equipped with stability safety functions?
Status Indication
S1 Are the modes of operation of the stability safety functions presented to the driver? (E.g. activated - deactivated)
S2 Is there a possibility to turn off stability safety functions?
S3 Is there a risk to turn the stability safety functions off by mistake?
S4 Are the conditions of the stability safety functions presented to the driver? (E.g. faulty – fault-free, capable/incapable)
Visual warnings
T1 Are there visual warnings related to stability safety functions?
T2 Are standardized symbols used for visual warnings? (E.g. ISO 2575, SAE J2402)
T3 Is the size of the symbols adequate? (in relation to other warning symbols)
T4 Is the colour of the symbols adequate? (has a appropriate warning colour been selected)
T5 Is the luminance of the symbols adequate? (in relation to other warning symbols)
T6 Is the contrast of the symbols adequate? (in relation to other warning symbols)
T7 Is the viewing angle of the symbols adequate?
T8 Is the timing of symbol presentation adequate? (not too late or too early)
T9 Is the interval at which symbols are presented adequate?
T10 Is the location of the symbols adequate? (is head movement necessary)
Visual information
U1 Is there visual information related to stability safety functions? (e.g. menus)
U2 Is the choice of graphics/representational features suitable for what they represent?
Deliverable D3.2
eVALUE 66 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
U3 Are graphics/representational features grouped where possible?
U4 Is the information presented legible? (i.e. size, contrast, brightness, illumination, image stability, resolution, colour, etc.)
Auditory warnings
V1 Are there auditory warnings related to stability safety functions?
V2 Is the auditory output (warning info) appropriate for the information to be conveyed?
V3 Is the volume of auditory warnings adequate? (possible to hear during normal driving conditions)
V4 Is the volume adjustable by the driver?
V5 Is the tone of auditory warnings adequate? (in relation to other warning sounds)
V6 Is the timing of auditory warnings adequate? (not too late or too early)
V7 Is the interval at which auditory warnings are presented adequate?
V8 Is there an intensification of the warnings? (E.g. caution -> warning)
V9 Is the intensification performed adequately?
Haptic warnings
W1 Are there haptic warnings related to stability safety functions?
W2 Is the intensity used for haptic warnings adequate? (possible to feel during normal driving conditions)
W3 Is the timing of haptic warnings adequate? (not too late or too early)
W4 Is the interval at which haptic warnings are presented adequately?
W5 Is the location of haptic warnings adequate? (threat from one side gives warning from the same side)
W6 Is there an intensification of the warnings? (E.g. caution -> warning)
W7 Is the intensification performed adequately?
Intervention
X1 Are there interventions related to stability safety functions?
X2 Can the driver override the safety functions if needed?
X3 Can the safety functions override the driver?
Combinations
Warnings
Y1 Could there be combination of warnings? (from two or more safety functions)
Deliverable D3.2
eVALUE 67 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
No Question n.a. Yes No Comment
Y2 Are priorities of warnings handled adequately? (i.e. priority is given to the warning related to the function of highest safety relevance)
Y3 Can warnings be discriminated from each other?
Y4 Is there a risk that the workload of the driver becomes too high? (i.e. safety function(s) present(s) excessively distracting information)
Interventions
Z1 Could there be combination of interventions? (from two or more safety functions)
Z2 Are priorities of interventions handled adequately?
Deliverable D3.2
eVALUE 68 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.4 Inspection Protocol - Functional safety
A.4.1 Scope
More and more vehicle safety functions are implemented using ICT-based systems. As a
consequence, functional safety requirements, preventive actions to avoid faulty states, and
failures of those systems become key issues.
A failure in ICT-based safety functions is a potential source of hazards. The inspection of the
functional safety shall address the safety principles applied and the safety measures
implemented to avoid hazardous system states or operation modes.
The inspection addresses three aspects:
- Identification of safety functions - Independent evaluation according to the ISO 26262 or IEC 61508 standards - Hazard analysis
Due to the limited time and resources allocated to performance testing, an inspection could
be used to verify how the vehicle handles different types of failure.
The objective is to verify that functional safety was considered during the development. The
important faults/failures that may influence the behaviour of the vehicle must be identified,
and the vehicle and its ICT-based safety functions must be verified for different failure
modes.
Failures in ICT-based safety functions are unlikely to occur during performance testing. Thus,
an inspection is required to verify the vehicle behaviour at fault.
The inspection will be applicable for cars, trucks and buses.
A.4.2 References
[26262] ISO/DIS 26262 Road vehicles – Functional Safety, Rev.1 – Part 1 to Part 9,
2008-02-29
[26262-1] ISO/DIS 26262-1 Road vehicles – Functional Safety – Part 1: Glossary
[26262-2] ISO/DIS 26262-2 Road vehicles – Functional Safety – Part 2: Management of
functional safety.
[26262-3] ISO/DIS 26262-3 Road vehicles – Functional Safety – Part 3: Concept phase
[26262-4] ISO/DIS 26262-4 Road vehicles – Functional Safety – Part 4: Product
development system level
Deliverable D3.2
eVALUE 69 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
[IEC 61508] IEC 61508:2010 Functional safety of electrical/electronic /programmable
electronic safety-related systems
A.4.3 Definitions
ASIL: the Automotive Safety Integrity Levels are determined by a systematic evaluation of
potentially hazardous operational situations. The ASIL-Level shall be determined for each
hazardous event using the estimation parameters severity (S), probability of exposure (E)
and controllability (C) in accordance with Table 4. [26262-3]
Controllability (C): The controllability shall be assigned to one of the controllability classes
C0, C1, C2 and C3 in accordance with Table 3. If a situation is regarded as simply distracting
or disturbing but as controllable in general, the class C0 may be assigned. If a hazard is
assigned to controllability class C0, no ASIL assignment is required. [26262-3]
Element: sub-units of items including system, subsystem, component. [26262-3]
E/E system: system that consists of electrical and/or electronic elements, including
programmable electronic elements, power supplies, sensors and other input devices, data
highways and other communication paths, and actuators and output devices. [26262-3]
Functional safety: absence of unacceptable risk due to hazards caused by mal-function
behaviour of E/E systems. [26262-3]
Functional safety concept: specification of the functional safety requirements, their
allocation to architectural elements and their interaction necessary to achieve the safety
goals, and information associated with these requirements.
Functional safety requirements: specification of implementation-independent safety-
related behaviour or a safety measure including its safety-related attributes. [26262-3]
NOTE 1 Safety requirement implemented by a safety-related E/E system or by a safety-related system of other
technologies in order to achieve or maintain a safe state for the item taking into account a determined hazardous
event.
NOTE 2 The functional safety requirements are specified independent of the used technology in the concept
phase of product development. They are detailed into technical safety requirements after concept phase.
Hazard classification: the hazard classification scheme comprises the determination of the
severity (S), the probability of exposure (E) and the controllability (C) associated with the
considered hazard of the item. For a given hazard, this classification will result in one or
more combinations of S, E and C classes. Each such combination represents an estimate of
a potential harm in a particular driving situation, with the severity determined by the potential
harm and the exposure determined by the situation. The controllability rates how easy or
difficult it is for the driver or other road traffic participant to avoid the considered accident type
in the considered situation. [26262-3]
Deliverable D3.2
eVALUE 70 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Probability of exposure (E): The probability of exposure of the operational situations shall
be estimated. The probability of exposure shall be assigned to one of the probability classes
E1, E2, E3 and E4 in accordance with Table 2. [26262-3]
Risk: combination of the probability of occurrence of harm and the severity of that harm
[ISO/IEC Guide 51:1999, definition 3.2]
Safety: Freedom from unacceptable risk of physical injury or of damage to the health of
people, either directly or indirectly as a result of damage to property or to the environment.
Situation analysis and hazard identification: The operational situations and operating
modes in which the item is able to trigger hazards shall be described for cases when
correctly used, when incorrectly used in a foreseeable way, or in case of item failure. [26262-
3]
Severity (S): The severity of potential harm shall be estimated. The severity shall be
assigned to one of the severity classes S0, S1, S2 or S3 in accordance with Table 1. [26262-
3]
Situation analysis and hazard identification: The operational situations and operating
modes in which the item is able to trigger hazards shall be described for cases when
correctly used, when incorrectly used in a foreseeable way, or in case of item failure. [26262-
3]
Table 5 Classes of severity
Class S0 S1 S2 S3
Description No injuries
Light and moderate injuries
Severe and life-threatening injuries (survival probable)
Life-threatening injuries (survival uncertain), fatal injuries
Table 6 Classes of probability of exposure regarding operational situations
Class E1 E2 E3 E4
Description Very low probability
Low probability Medium probability
High probability
Table 7 Classes of controllability
Class C0 C1 C2 C3
Description
Controllable in general
Simply controllable
Normally controllable
Difficult to control or uncontrollable
Deliverable D3.2
eVALUE 71 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
C1 C2 C3
S1
E1 QM QM QM
E2 QM QM QM
E3 QM QM A
E4 QM A B
S2
E1 QM QM QM
E2 QM QM A
E3 QM A B
E4 A B C
S3
E1 QM QM A
E2 QM A B
E3 A B C
E4 B C D
Figure 1 ASIL determination
A.4.4 Inspection
A.4.4.1 Principle
The inspection is based on a study of the documents provided by the vehicle OEM or safety
system suppliers, and/or an interview with representatives of the manufacturer. The
inspection is supported by a checklist.
A.4.4.2 Information required from the manufacturer
The documents required to carry out the inspection, shall provide the following information:
- identification of the safety functions - description on how the safety functions are integrated in the vehicle - validation according to applicable standards
A.4.4.3 Inspection procedure
The present inspection comprises two phases. The first phase addresses the identification of
the safety functions. In the second phase, the fulfilment of existing automotive standard
requirements is verified.
Questions A to D support the identification of the safety functions and their integration into
the vehicle.
Questions E and F verifies the application of relevant safety development standard. Two
standards are applicable:
- ISO 26262 and
Deliverable D3.2
eVALUE 72 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
- IEC 61508.
Safety functions for which an independent evaluation according to one of the above
mentioned standards has been performed, no supplementary verification is required in this
inspection.
In the absence of an independent evaluation according to a specific standard, questions G to
U permit to verify how the management process and the safety requirements have been
handled.
The different safety aspects are verified as follows in the checklist:
- Implementation of the situation analysis and hazard identification, questions G to L.
- Hazard classification, questions M to P
- ASIL determination, questions Q and R
- Fault detection capacities, questions: S and T.
- False alarms, question U.
Safety principles, hardware design and software design are not parts of this inspection.
A.4.4.4 Report
The inspection shall provide information about the examination of the safety functions and
validate that an independent evaluation based on the ISO 26262 or IEC 61508 has been
conducted:
- An independent evaluation has been carried out.
- A classification of the hazards has been made
The result of this inspection is to deliver a statement on the functional safety of the ICT-
related systems.
Deliverable D3.2
eVALUE 73 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
A.4.5 Checklist - Functional safety
n.a. = Not applicable question (Some questions may not be applicable for all vehicles.)
Safety functions and independent evaluation
No Question n.a. Yes No Comment
A Are ICT-based safety functions implemented in the vehicle?
B Is each ICT-based safety function developed according to specific international or European standard?
C Are all ICT-based safety functions identified and documented?
D Are the test results of the ICT-based function integration at system and vehicle level documented?
E Have the safety functions been subject to an independent evaluation according to the ISO 26262 development standard?
F Have the safety functions been subject to an independent evaluation according to the IEC 61508 development standard?
If independent evaluation is missing
Situation analysis and hazard identification
No Question n.a. Yes No Comment
G Are the hazard analysis and risk assessment fully documented with regard to situation analysis and hazard identification?
H Have the potential unintended functional states that could lead to a hazard event been identified?
I Has foreseeable driver use and misuse been considered in the situation analysis and hazard identification?
J Has the impact of high speed driving been considered in the situation analysis and hazard identification?
K Has urban driving been considered in the situation analysis and hazard identification?
L Has the interaction between operational systems been considered in the situation analysis and hazard identification?
Deliverable D3.2
eVALUE 74 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Hazard classification
M Is the hazard analysis and risk assessment fully documented with regard to hazard classification?
N Has the severity (S) associated with the considered hazard been determined?
O Has the probability of exposure (E) associated with the considered hazard been determined?
P Has the controllability (C) associated with the considered hazard been determined?
ASIL determination
Q Is the Hazard analysis and Risk assessment fully documented with regard to ASIL determination?
R Has the ICT-based safety function been assigned a safety integrity level (ASIL)?
Fault detection capabilities
S Has the manufacturer identified the failure modes of the system?
T Has the manufacturer identified the causes and effects of such failures?
False alarms
U Has the manufacturer tested and analysed the identified situations where a false alarm occurs?
Deliverable D3.2
eVALUE 75 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Annex B Testing Protocols
B.1 General conditions for eVALUE testing protocols
B.1.1 Scope
This part addresses general conditions for eVALUE tests and includes the following sections:
References
Definitions
Test conditions & environment
Subject vehicle preparation & conditioning
B.1.2 References
[ECE 384] European agreement on main international traffic arteries, ECE/TRANS/SC.1/384,
Economic Commission for Europe Inland Transport Committee, 2008.
[NHTSA ESC] Electronic Stability Control NCAP Confirmation Test, March 2010, NHTSA.
[NHTSA FCW] Forward Collision Warning System NCAP Confirmation Test, March 2010,
NHTSA.
[NHTSA LDW] Lane Departure Warning System NCAP Confirmation Test, March 2010,
NHTSA.
[ISO 8855] Road vehicles – Vehicle dynamics and road-holding - Vocabulary (ISO
8855:1991)
[15037-1] Road vehicles – Vehicles dynamics test methods – Part 1: General conditions for
passenger cars (ISO 15037-1:2006, IDT)
[15037-2] Road vehicles — Vehicle dynamics test methods – Part 2: General conditions for
heavy vehicles and busses (ISO 15037-2:2007).
[15622] Transport information and control systems – Adaptive cruise control systems -
Performance requirements and test procedures (ISO 15622:2002)
[15623] Transport information and control systems – Forward vehicle collision warning
systems – Performance requirements and test procedures (ISO 15623:2002)
[17361] Intelligent transport systems – Lane departure warning system (LDWS) –
Performance requirements and test procedures (ISO 17361:2007, IDT)
[17387] Intelligent transport systems – Lane change decision aid systems – Performance
requirements and test procedures (ISO 17387:2008)
Deliverable D3.2
eVALUE 76 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
[21994] Passenger cars – Stopping distance at straight-line braking with ABS – Open-loop
test method (ISO 21994:2007, IDT)
[4138] Passenger cars — Steady-state circular driving behaviour – Open-loop test methods
(ISO 4138:2004).
B.1.3 Definitions
Clearance – Distance from the target vehicle’s trailing surface to the subject vehicle’s
leading surface [15622].
Closed loop – The closed loop method is based on the interaction of a human driver with
the vehicle. In such case, the response from the driver to the actuation from the subject
vehicle and the actual influence of the HMI design is considered as well.
Closing Speed – The closing speed of a target vehicle is defined as the difference between
the target vehicle’s speed and the subject vehicle’s speed. This definition applies to target
vehicles in the rear zone only. A positive closing speed indicates that the target vehicle is
closing on the subject vehicle on the rear [17387].
Collision speed – Collision speed is the relative speed at which the subject vehicle collides
into the target vehicle. The vehicle speeds are measured at the moment of the collision.
Departure – situation in which the outside of one of the front wheels of a vehicle or of the
leading part of an articulated vehicle – or, in the case of a three wheeled vehicle, the outside
of one of the wheels on the axle with the widest track – is crossing a specified line [17361].
Driver intention following – This parameter represents the performance of a vehicle in
terms of how closely it responds to the driver input.
Driver steering input – It will analyse the steering wheel angle, speed, and torque done by
the driver in order to quantify the amount of driver activity.
Electronic stability control (ESC) – ESC is a computerized technology that improves the
safety of a vehicle's stability by detecting and minimizing skids. When ESC detects loss of
steering control, it automatically applies the brakes to help "steer" the vehicle where the
driver intends to go. Braking is automatically applied to individual wheel, such as the outer
front wheel to counter oversteer or the inner rear wheel to counter understeer. Some ESC
systems also reduce engine power until control is regained. ESC does not improve a
vehicle's cornering performance; it rather helps minimize the loss of control.
Lane boundary – borderline of the lane, situated at the centre of a visible lane marking or, in
the absence of a visible lane marking, determined by incidental visible road features or other
means such as GPS, magnetic nails, etc. [17361].
Lane departure – point of departure across the lane boundary [17361].
Deliverable D3.2
eVALUE 77 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Lateral acceleration – The lateral acceleration is the acceleration value measured in the y
direction at the centre of gravity.
Lateral clearance – the lateral clearance of a target vehicle is defined as the lateral distance
between the side of the subject vehicle and the near side of a target vehicle [17387].
Lateral displacement – The lateral displacement is the lateral distance difference between
the start and end points of the manoeuvre.
Local time reference – The local time corresponds to the time reference of the data logger
used during testing.
Longitudinal acceleration – The longitudinal acceleration is the acceleration value
measured in the x direction at the center of gravity.
Longitudinal speed – Speed measured in the x direction at the centre of gravity. Can also
be calculated as the numeric derivative of the local x position with respect to the time.
Open loop – The open loop method is a test without considering the natural response and
feedback from the driver. The open loop test may be performed with a professional driver or
a robot driver.
Oversteer – A condition in which the vehicle’s yaw rate is greater than the yaw rate that
would occur at the vehicle’s speed as result of the Ackerman Steer Angle.
Peak lane deviation – The peak lane deviation is the maximum magnitude of lateral position
as measured from a point on the centreline of the vehicle projected downward to the
pavement and subtracted from lane centreline or other fixed reference.
Rate of departure – Subject’s vehicle approach speed at a right angle to the lane boundary
at the warning issue point [17361].
Rollover condition – Corresponds to the situation where both inner wheels lift more than 5
cm from the ground and for more than 20 ms.
Sideslip or side slip angle – The arctangent of the lateral speed of the centre of gravity of
the vehicle divided by the longitudinal speed of the centre of gravity.
Sideslip angle peaks – They are the minimum and maximum values of sideslip angles
during a manoeuvre. They are stability parameters directly related to the oversteer tendency
of the vehicle.
Steering activity – The steering activity is the integral of the steering wheel angle during the
manoeuvre.
Steering wheel angle average – The steering wheel angle average is the mean value of the
steering wheel angle during the manoeuvre.
Deliverable D3.2
eVALUE 78 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Steering wheel torque peak – The steering wheel torque peak is the absolute value and
sign of the steering wheel torque peak during the manoeuvre.
Standard deviation of lane position – During the manoeuvre, the middle of the lane is
considered as optimum and deviations from the optimum are measured each time step.
Then, the standard deviation of these measures is calculated.
Stopping distance – The stopping distance is the distance travelled by the vehicle during
the braking manoeuvre until complete stop.
Time to collision (TTC) – The time to collision gives clear information on the remaining time
before a collision.
It is calculated by: TTC = longitudinal distance / relative speed
Time to line crossing (TLC) – This value represents the time remaining before crossing the
line. The objective is to know how much time before (or after) crossing the line the system
warns or acts.
TLC = lateral distance to lane / lateral speed
Trial – A single execution of a test case.
Understeer – A condition in which the vehicle’s yaw rate is less than the yaw rate that would
occur at the vehicle’s speed as a result of the Ackerman Steer Angle.
Use of adherence – A parameter that quantifies the characteristics of the vehicle for the
available adherence in μ-split scenario.
Validation – Validation describes the process of evaluating the system impact e. g. on
safety. That is, validation checks and tests whether the system "does what it was designed
for", e.g. increase traffic safety by increasing headway, by avoiding impacts and so on. For
this purpose the driver needs to be in the loop. Driver in the loop testing is required.
Vehicle reference coordinate system – According to the common convention for the
vehicle coordinates system (ISO 8855), the x axis points to the front of the vehicle, the y axis
points to the left and the z axis points upwards (right-hand orthogonal system).
Vehicle speed variation – It is the difference in speed between the start and the end of the
manoeuvre. Depending on the function/manoeuvre and together with sideslip angles peaks,
it gives an idea about how much the ESC has to work in order to keep the vehicle stable.
Verification – Verification describes the test of a system/ function against its requirements,
that is, whether it fulfils its requirements.
Warning issue point – measured location and time at which a warning starts to be issued.
Deliverable D3.2
eVALUE 79 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Warning threshold – location where the warning is issued on the road, which corresponds
to a warning trigger point set in the system. The warning threshold shifts depending on the
rate of departure. The warning threshold is placed within the warning threshold placement
zone [17361] (see Figure B-1).
Warning threshold placement zone – zone between the earliest and the latest warning
lines within which the warning threshold is placed. There is a warning threshold placement
zone around the left lane boundary and one around the right lane boundary (see Figure B-1).
14
53
2
2
6
1 lane boundary 4 latest warning line
2 warning threshold placement zone 5 no warning zone
3 earliest warning line 6 warning threshold (for reference only)
Figure B-1 Concept of warning thresholds and warning threshold placement zone [17361]
Yaw response – The yaw response is the rotation around the z at the centre of gravity.
Yaw rate – The rate of change of the vehicle’s heading angle measured in degree/second of
rotation about the vertical axis through the vehicle’s centre of gravity.
Yaw rate ratios – They are the ratios between the yaw rate at certain times after the end of
the steering wheel actuation and the yaw rate peak. They are the stability parameters
suggested by NHTSA ESC NCAP Confirmation Test.
B.1.4 Test conditions and environment
The environmental conditions shall not change during the course of the test. Allowed
variations for temperature, visibility, and wind are stated below.
Deliverable D3.2
eVALUE 80 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.1.4.1 Test track conditions
All tests shall be carried out on a smooth, clean, dry (except for µ-split tests) and uniform
paved road surface with local lane markings. The friction coefficient of the road surface shall
not differ from standard road types1. The gradient of the paved test surface to be used sall
not exceed 2% (recommended 1.5%) in any direction when measured over any distance
interval between that corresponding to the vehicle track and 25 m [15037].
The track shall have an area where lanes with a radius of curvature of up to 500 m ±10% are
provided.
In testing protocols where lane width is important, a minimum lane width of 3.5 m shall be
considered [ECE 384].
Dashed lane markings shall be used during the lane departure tests. Different lane marking,
in terms of e.g. white coverage, applies to different European countries. So far no
standardized dashed lane marking have been decided for lane departure testing
B.1.4.2 Temperature conditions
The ambient temperature shall be between 0 ºC and 40 ºC and its variation during a
sequence of measurements shall be below 10 ºC.
The test track surface temperature shall be between 0 ºC and 50 ºC and its variation during a
sequence of measurement shall be below 10 ºC.
B.1.4.3 Visibility conditions
All tests are to be conducted during daytime. The horizontal visibility shall be greater than 1
km [15623].
B.1.4.4 Wind conditions
The ambient wind speed (regardless of wind direction) shall either not exceed 3 m/s or, if the
wind speed ranges between 3 m/s and 5 m/s maximum, an equal number of measurement
specified shall be carried out in both driving directions [21994].
B.1.5 Subject vehicle preparation and conditioning
The condition of the subject vehicle shall be in accordance with the vehicle manufacturer’s
specifications [21994].
1 Concerning the split tests, the friction coefficient of the high test surface shall be between 0.8 and 1.0 and its variation
shall not exceed ±5% over the length of the test surface. The friction coefficient of the low test surface shall be between 0.10
and 0.2, and its variation shall not exceed ±5% over the length of the test surface.
Deliverable D3.2
eVALUE 81 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
For cluster 1 and cluster 3 tests, tyres and brakes shall be conditioned according to ISO
21994 [21994].
B.1.5.1 Data collection systems
The subject vehicle is to be equipped with measurement equipment in order to record
measures and to warranty their precision and to fulfil the repetitiveness requirements for the
tests. The onboard measurement system shall provide a data collection from which it is
possible to extract the measures needed for computation of safety performance indicators.
The IMU (inertial measurement unit) sensor shall be located as close to the centre of gravity
of the vehicle as possible. In any case measured signals must be compensated of sensor
position effects and carried to the centre of mass.
Optionally, video recorders for collecting video sequences of the road scene and driver shall
be employed. Preferably, the video recording shall be synchronised with the data collecting
system. Driver and lane video sequences shall, if possible, be sampled at the same
frequency as other sensors (≥100 Hz) but 30 Hz might be sufficient.
B.1.5.2 Configuration of subject vehicle
The combined mass (of driver and test equipment) should not exceed 150 kg [15037].
The fuel tank shall be full and, in the course of the measurement sequence, the indicated fuel
level should not drop below ―half-full‖ [21994].
If the safety systems provide different settings, the default setting shall be used.
The tires used shall be standard ones as recommended by the OEM and with the
recommended pressure.
Deliverable D3.2
eVALUE 82 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2 Avoidance or mitigation of rear-end collision, open loop test
B.2.1 Scope
This chapter is the testing protocol that describes test procedures for testing the performance
of the longitudinal safety functions in a scenario where a vehicle is approaching another
vehicle which is moving slower in the same direction, decelerating, or being stationary on a
straight or curved road. The present test procedure addresses the safety of different types of
subject vehicles (passenger vehicle, bus or commercial vehicle).
The different tests shall represent different driver behaviour in longitudinal accident-related
scenarios.
The test procedures consider physical tests for evaluating the safety performance of the
vehicle by using professional drivers and/or driving robots that control the vehicle with
predefined manoeuvres. Warning, support, and mitigation functions are expected to be
activated during the manoeuvres.
The test procedure determines the safety performance indicators during the driving
manoeuvres.
Subject vehicle Target vehicle
Wt
at , vtas, vs
Subject vehicle
Target vehicle vt
as , vs
Figure B-2 Rear-end straight and curved road scenarios
B.2.2 References
References are listed in Chapter B.1.
B.2.3 Definitions
Definitions are listed in Chapter B.1.
Deliverable D3.2
eVALUE 83 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4 Test procedure - Avoidance or mitigation of rear-end collision
B.2.4.1 Principle
The test is based on the observation of the subject vehicle behaviour when executing the
manoeuvres specified in respective test. The open loop tests are focusing on the vehicle’s
technical performance.
The manoeuvres on the test tracks shall resemble real accident-prone traffic situations.
B.2.4.2 Test objectives
The objective of an open loop test is to test the technical performance of the vehicle, without
considering natural response and feedback from an arbitrary driver. A professional driver or a
driving robot is used for triggering an action from the vehicle. The timing sequence of the
open loop tests is presented in Figure B-3. There are different open loop tests depending on
the type of action from the professional driver or driving robot (no action, mild brake after
warning, and strong brake after warning). In test 1, the driver is passive, whereas in tests 2
and 3 the driver brakes strongly and mildly, respectively. The outcome of the tests will
depend on the level of assistance from the subject vehicle (warning, support, and/or
intervention).
t0 t1 t3 t4 t5 t6
Initial conditions for
subject vehicle and target
vehicle velocity and
clearance fulfilled
Pre-stabilisation
period
Warning Support
Collision
No Collision
Start of test
Intervention
Collision
No Collision
Collision
No Collision
Strong
braking action
Mild braking
action
Typical driver
reaction time
tr
Test 1
Test 2
Test 3
t2
Possible brake
by target vehicle
Passive
Figure B-3 Time sequence for open loop tests
Deliverable D3.2
eVALUE 84 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4.3 Drivers
Professional drivers or preferably driving robots are needed.
B.2.4.4 Equipment
Besides the equipment used to log positions, speeds, and accelerations (e.g. GPS with IMU)
of the subject and target vehicle respectively, there is a need to synchronously log warnings
provided from the vehicle to the driver. The warning signals might be obtained by tapping the
CAN bus, but preferably an external sensor should be used (e.g. a photo-sensitive device for
visual warnings). The collision instant can either by registered by a pressure-sensitive sensor
mounted at the front of the subject vehicle, or by registering the instant at which the distance
(clearance) between the two vehicles becomes zero. The distance between subject and
target vehicle can be calculated from synchronized high-precision GPS position data or be
directly measured using a RADAR or LASER distance sensor.
B.2.4.4.1 Target vehicle
To facilitate a possible collision, the target vehicle is simulated by a vehicle dummy similar to
ordinary vehicles with regard to physical dimensions and detection characteristics. The
following properties are important1:
- Size (width, height, and shape)
- Radar Cross Section (RCS)
- Reflectivity
- Size, contrast, and colour range
B.2.4.5 Testing environment
Test conditions and environment are described in Chapter B.1.
B.2.4.6 Information required for the test
The resulting reports from the following inspections are used.
- Definition of subject vehicle including the targeted safety functions
- Environmental conditions
These reports are necessary to understand if normal conditions are applicable during the
physical tests. Also these inspections help to get an understanding of the subject vehicle and
its safety functions.
1 Target vehicle dimensions and characteristics are still under discussion; eVALUE will not define a general target
vehicle. However, the generic target being developed in the ASSESS project might be a suitable candidate.
Deliverable D3.2
eVALUE 85 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4.7 Subject vehicle preparation
Subject vehicle preparation and conditioning are described in Chapter B.1.
B.2.4.8 Test procedure and data processing
B.2.4.8.1 Tests
The three tests in this testing protocol are:
Test 1: Rear-end collision – passive driver
Test 2: Rear-end collision – driver brakes strongly
Test 3: Rear-end collision – driver brakes mildly
For each of the three tests there a number of test cases. The test cases represent different
combinations of subject vehicle speed as well as target vehicle speed and deceleration.
Additionally the test cases consider: straight road, left curve, or right curve.
During all the tests, the measurements listed in Table B-1 shall be logged. From these
measurements, the safety performance indicators can be directly determined or calculated.
Table B-1 Measurements to be recorded during the tests
Measure Typical range Recommended
maximal error
local time reference 10 ms samples
local position of both vehicles 0 m to 250 m ± 0.50 m [21994]
speed of both vehicles 0 km/h to 100 km/h ± 0.5 km/h [21994]
longitudinal deceleration of both vehicles 1 m/s2 to 12 m/s2
0.1 g to 1.2 g
± 0.15 m/s2 [15037]
± 0.015 g
longitudinal distance between both vehicles
0 m to 250 m ± 1% (≤ 50 m)
± 0.50 m (> 50 m)
[21994]
lateral distance between vehicles 30 to + 30 m ± 0.05 m
warning instant N/A video: ±0.05s
audio: ±0.05s
collision instant (if there is any) N/A ±0.05s
brake pedal actuation force 0 – 1000 N ±2% [21994]
Tests 1 to 3 presented in the time sequence diagram in Figure B-3 shall be performed
whenever relevant for the function considered. Regardless of test, the subject vehicle follows
the target vehicle in the same driving lane. During the entire test, the alignment of both
vehicles’ longitudinal axis shall not deviate by more than ± 0.25 m.
Deliverable D3.2
eVALUE 86 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
After the pre-stabilisation period, t1, the initial speeds (and clearance) has been established
by the use of professional drivers in the subject and target vehicles. Depending on the test
case, the target vehicle may initiate a robot-controlled braking at t2 (t2- t1 = 3 s).
Subsequently, typical driver action is simulated by doing nothing (passive driver) when the
warning is issued or by a robot-controlled braking after a typical reaction time has elapsed.
The tests progress until a collision occurs or when the speed of the subject vehicle is equal
or lower than that of the target vehicle, i.e. no collision in Figure B-3.
Technical requirements and safety performance indicators
Measurement data is collected during the tests to verify that the test was correctly executed
(e.g. that initial speed was within tolerances) or computing the values of the safety
performance indicators. The technical requirement and safety performance indicators,
reflecting the performance of the vehicle, are presented in Table B-2.
The safety performance indicator with a direct and strong connection to safety is the collision
speed. A reduced collision speed reduces the collision energy and consequently causes less
harm to the subject vehicle occupants. The lower collision speed, the safer it is.
A second safety performance indicator, which indirectly affects safety, is the TTC at warning
(for TTC calculations, constant target speed or deceleration is assumed). If the TTC is too
large, false alarms will be the result, and the driver might deactivate the safety system, which
has an adverse effect on safety. Thus there is an upper bound for this safety performance
indicator. This bound is dependent on the vehicle speed.
Table B-2 Technical requirements and safety performance indicators for open loop tests
Technical requirement Safety performance indicator
The subject vehicle shall avoid a collision
or reduce the collision speed at a potential
rear-end collision
collision speed1
The subject vehicle shall not produce
nuisance alarms
time-to-collision at warning
1 The collision speed must be related/adjusted to the maximal deceleration available for the particular
track-tyres combination.
Deliverable D3.2
eVALUE 87 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4.8.2 Test 1: Rear-end collision – passive driver
Description
In this test a completely passive driver is simulated. After the initial speeds and clearance
have been established, the subject vehicle driver does nothing until the end of the test
(collision or no collision).
The test shall be performed with the initial speeds, clearances, decelerations according to
the values listed in Table B-3. The curve radius should be 250 m and the inner edge of the
curve shall be shielded with a fence or similar to limit the field of view.
Table B-3 Test cases – passive driver
Test
case
Initial
subject
vehicle
speed
[km/h]
Initial1
target
vehicle
speed
[km/h]
Initial clearance
between
subject and
target vehicle
[m]
Target vehicle
deceleration
[m/s2]
Road
topology
1.1 50±2 0 150±0.5 0±0.5 straight
1.2 70±2 0 150±0.5 0±0.5 straight
1.3 70±2 30±2 100±0.5 0±0.5 straight
1.4 70±2 50±2 100±0.5 0±0.5 straight
1.5 70±2 70±2 40±0.5 3±0.5 straight
1.6 70±2 70±2 40±0.5 5±0.5 straight
1.7 50±2 0 150±0.5 0±0.5 left curve
1.8 50±2 0 150±0.5 0±0.5 right curve
1.9 70±2 30±2 100±0.5 0±0.5 left curve
1.10 70±2 30±2 100±0.5 0±0.5 right curve
1.11 70±2 70±2 40±0.5 5±0.5 left curve
1.12 70±2 70±2 40±0.5 5±0.5 right curve
The number of trials for each test case is ≥1.2
1 For the test cases with a slower moving target vehicle (TC1.3-1.4 and 1.9-1.10) the initial speed of
the target vehicle shall be kept within the limits throughout the test manoeuvre.
2 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 88 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4.8.3 Test 2: Rear-end collision – driver brakes strongly
Description
In this test a driver who brakes strongly is simulated. After the initial speeds and clearance
have been established, the brake action is controlled by a brake robot to warrant
repeatability. The brake action is triggered by the warning signal from the vehicle. A typical
reaction time of 1 s is waited before the robot applies a brake force to the brake pedal. In this
case a pedal force of 700 N should be applied in 0.2 s and kept during the braking action.
The test shall be performed with the initial speeds, clearances, decelerations according to
the values listed in Table B-4. The curve radius should be 250 m and the inner edge of the
curve shall be shielded with a fence or similar to limit the field of view.
Table B-4 Test cases – driver brakes strongly
Test
case
Initial
subject
vehicle
speed
[km/h]
Initial1
target
vehicle
speed
[km/h]
Initial clearance
between
subject and
target vehicle
[m]
Target vehicle
deceleration
[m/s2]
Road
topology
2.1 50±2 0 150±0.5 0±0.5 straight
2.2 70±2 0 150±0.5 0±0.5 straight
2.3 70±2 30±2 100±0.5 0±0.5 straight
2.4 70±2 50±2 100±0.5 0±0.5 straight
2.5 70±2 70±2 40±0.5 3±0.5 straight
2.6 70±2 70±2 40±0.5 5±0.5 straight
2.7 50±2 0 150±0.5 0±0.5 left curve
2.8 50±2 0 150±0.5 0±0.5 right curve
2.9 70±2 30±2 100±0.5 0±0.5 left curve
2.10 70±2 30±2 100±0.5 0±0.5 right curve
2.11 70±2 70±2 40±0.5 5±0.5 left curve
2.12 70±2 70±2 40±0.5 5±0.5 right curve
The number of trials for each test case is ≥1.2
1 For the test cases with a slower moving target vehicle (TC2.3-2.4 and 2.9-2.10) the initial speed of
the target vehicle shall be kept within the limits throughout the test manoeuvre.
2 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 89 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4.8.4 Test 3: Rear-end collision – driver brakes mildly
Description
In this test a driver who brakes strongly is simulated. After the initial speeds and clearance
have been established, the brake action is controlled by a brake robot to warrant
repeatability. The brake action is triggered by the warning signal from the vehicle. A typical
reaction time of 1.5 s is waited before the robot applies a brake force to the brake pedal. In
this case a pedal force of 350 N should be applied in 0.4 s and kept during the braking
action.
The test shall be performed with the initial speeds, clearances, decelerations according to
the values listed in Table B-5. The curve radius should be 250 m and the inner edge of the
curve shall be shielded with a fence or similar to limit the field of view.
Table B-5 Test cases – driver brakes mildly
Test
case
Initial
subject
vehicle
speed
[km/h]
Initial1
target
vehicle
speed
[km/h]
Initial clearance
between
subject and
target vehicle
[m]
Target vehicle
deceleration
[m/s2]
Road
topology
3.1 50±2 0 150±0.5 0±0.5 straight
3.2 70±2 0 150±0.5 0±0.5 straight
3.3 70±2 30±2 100±0.5 0±0.5 straight
3.4 70±2 50±2 100±0.5 0±0.5 straight
3.5 70±2 70±2 40±0.5 3±0.5 straight
3.6 70±2 70±2 40±0.5 5±0.5 straight
3.7 50±2 0 150±0.5 0±0.5 left curve
3.8 50±2 0 150±0.5 0±0.5 right curve
3.9 70±2 30±2 100±0.5 0±0.5 left curve
3.10 70±2 30±2 100±0.5 0±0.5 right curve
3.11 70±2 70±2 40±0.5 5±0.5 left curve
3.12 70±2 70±2 40±0.5 5±0.5 right curve
The number of trials for each test case is ≥1.2
1 For the test cases with a slower moving target vehicle (TC3.3-3.4 and 3.9-3.10) the initial speed of
the target vehicle shall be kept within the limits throughout the test manoeuvre.
2 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 90 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.2.4.9 Uncertainty
Uncertainty in the measurement data shall be stated.
B.2.4.10 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
Deliverable D3.2
eVALUE 91 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.3 Avoidance of collision with transversally moving target, open loop test
B.3.1 SCOPE
This testing protocol describes the test procedure for testing the performance of the
longitudinal safety functions of a vehicle approaching a transversally moving target. The
moving target is a passenger vehicle, a motorcycle or a pedestrian. The present test
procedure addresses the safety of different types of subject vehicles (car or specific types of
commercial vehicles).
The different tests shall represent different driver behaviours in longitudinal accident-related
scenarios.
The test procedures consider physical tests for evaluating the safety performance of the
vehicle by using professional drivers and driving robots that control the vehicle with
predefined manoeuvres. Warning, support, and mitigation functions are expected to be
activated during the manoeuvres.
The test procedure determines the safety performance indicators during the driving
manoeuvres.
Subject vehicle
Target vehicle
vt
vs
Figure B-4 Transversally moving target scenario
B.3.2 References
References are listed in Chapter B.1.
B.3.3 Definitions
Definitions are listed in Chapter B.1.
B.3.4 Test procedure - Avoidance of collision with transversally moving target
B.3.4.1 Principle
The test is based on the observation of the subject vehicle behaviour when executing the
manoeuvres specified in respective test. The open loop tests are focusing on the vehicle’s
technical performance.
Deliverable D3.2
eVALUE 92 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The manoeuvres on the test tracks shall resemble real traffic situations.
B.3.4.2 Test objectives
The objective of an open loop test is to test the technical performance of the vehicle, without
considering natural response and feedback from an arbitrary driver. A professional driver or a
driving robot is used for triggering an action from the vehicle. The timing sequence of the
open loop tests is presented in Figure B-5. There are different open loop tests depending on
the type of action from the professional driver or driving robot (no action, mild brake after
warning, and strong brake after warning). In test 1, the driver is passive, whereas in tests 2
and 3 the driver brakes strongly and mildly, respectively. The outcome of the tests will
depend on the level of intervention from the subject vehicle (warning, support, and/or
intervention).
t0 t1 t3 t4 t5 t6
Initial conditions for
subject vehicle and target
vehicle velocity and
clearance fulfilled
Pre-stabilisation
period
Warning Support
Collision
No Collision
Start of test
Intervention
Collision
No Collision
Collision
No Collision
Strong braking action
Mild braking action
Typical driver
reaction time
tr
Test 1
Test 2
Test 3
Passive
Figure B-5 Time sequence for open loop tests
B.3.4.3 Drivers
Professional drivers or preferably driving robots are needed.
B.3.4.4 Equipment
Besides the equipment used to log positions, speeds, and accelerations (e.g. GPS with IMU)
of the subject and target vehicle respectively, there is a need to synchronously log warnings
provided from the vehicle to the driver. The warning signals shall preferably be detected by
an external sensor (e.g. a photo-sensitive device for visual warnings). If provided by the
Deliverable D3.2
eVALUE 93 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
OEM, warning signals may be obtained via the CAN bus. The collision instant can either by
registered by a pressure-sensitive sensor mounted at the front of the subject vehicle, or by
registering the instant at which the distance (clearance) between the two vehicles becomes
zero. The distance between subject and target vehicle can be calculated from synchronized
high-precision GPS position data.
B.3.4.4.1 Target vehicle
To facilitate a possible collision, the target vehicle is simulated by a vehicle dummy similar to
ordinary vehicles with regard to physical dimensions and detection characteristics. The
following properties are important1:
- Size (length, height, and shape)
- Radar Cross Section (RCS)
- Reflectivity
- Size, contrast, and colour range
B.3.4.4.2 Pedestrian target
The dummy target shall have characteristics similar to a real person in size and movement
and movement speed.
B.3.4.5 Testing environment
Test conditions and environment are described in Chapter B.1.
An urban environment where the lateral field-of-view is restricted by buildings shall be used.
B.3.4.6 Information required for the test
The resulting reports from the following inspections are used.
- Definition of subject vehicle including the targeted safety functions
- Environmental conditions
These reports are necessary to understand if normal conditions are applicable during the
physical tests. Also these inspections help to get an understanding of the subject vehicle and
its safety functions.
B.3.4.7 Subject vehicle preparation
Subject vehicle preparation and conditioning are described in Chapter B.1.
1 Target vehicle dimensions and characteristics are still under discussion; eVALUE will not define a general target
vehicle. However, the generic target being developed in the ASSESS project might be a suitable candidate.
Deliverable D3.2
eVALUE 94 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.3.4.8 Test procedure and data processing
B.3.4.8.1 Tests
The three tests in this testing protocol are:
Test 1: Transversally moving target – passive driver
Test 2: Transversally moving target– driver brakes strongly
Test 3: Transversally moving target– driver brakes mildly
For each of the three tests there a number of test cases. The test cases represent different
combinations of subject and target vehicle speeds as well as initial distances. Also there are
test cases for different target objects: passenger vehicle and pedestrian.
During all the tests, the measurements listed in Table B-6 shall be logged. From these the
safety performance indicators can be directly determined or calculated.
Table B-6 Measurements to be recorded during the tests
Measure Typical range Recommended
maximal error
local time reference 10 ms samples
local position of subject vehicle and target object
0 m to 250 m ± 0.50 m [21994]
speed of subject vehicle and target object
0 km/h to 100 km/h ± 0.5 km/h [21994]
longitudinal deceleration 1 m/s2 to 7 m/s2
0.1 g to 0.7 g
± 0.15 m/s2 [15037]
± 0.015 g
longitudinal distance between subject vehicle and target object
0 m to 300 m ± 0.05 m
lateral distance between subject vehicle and target object
-150 m to + 150 m ± 0.05 m
warning instant N/A video: ±0.05s
audio: ±0.05s
collision instant (if there is any) N/A ±0.05s
brake pedal actuation force 0 – 1000 N ±2% [21994]
Tests 1 to 3 presented in the time sequence diagram in Figure B-5 shall be performed. The
subject vehicle enters an intersection and a target vehicle/object crosses its path by a
transversal movement.
To successfully perform the tests, the positions and speeds of the subject vehicle and target
object must be controlled with very small errors. Communication between the subject and
target is also necessary to synchronize their movements.
Deliverable D3.2
eVALUE 95 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
After the pre-stabilisation period, t1, the initial speeds and distances has been established by
the use of robot drivers in the subject and target vehicles.
Subsequently, typical driver action is simulated by doing nothing (passive driver) when the
warning is issued or by a robot-controlled braking after a typical reaction time has elapsed.
The tests progress until a collision occurs, or when the target vehicle/object is out of the path
of the subject vehicle, i.e. no collision in Figure B-5.
Technical requirements and indicators
Measurement data is collected during the tests to verify that test was correctly executed (e.g.
that initial speeds were within tolerances), or computing the values of the safety performance
indicators. The technical requirement and safety performance indicators, reflecting the
performance of the vehicle, are presented in Table B-7.
The safety performance indicator with a direct and strong connection to safety is the collision
speed. A reduced collision speed reduces the collision energy and consequently causes less
harm to the subject vehicle occupants. The lower collision speed, the safer it is.
A second safety performance indicator, which indirectly affects safety, is the TTC at warning
(for TTC calculations, constant target speed or deceleration is assumed). If the TTC is too
large, false alarms will be the result, and the driver might deactivate the safety system, which
has an adverse effect on safety. Thus there is an upper bound for this safety performance
indicator. This bound is dependent on the vehicle speed.
Table B-7 Technical requirements and safety performance indicators for open loop tests
Technical requirement Safety performance indicator
The subject vehicle shall avoid a collision
or reduce the collision speed with a
transversally moving target.
collision speed1
The subject vehicle shall not produce
nuisance alarms
time-to-collision at warning
B.3.4.8.2 Test 1: Transversally moving target – passive driver
In this test a completely passive driver is simulated. After the initial speeds and distances
have been established, the subject vehicle driver does nothing until the end of the test
(collision or no collision).
1 The collision speed must be related/adjusted to the maximal deceleration available for the particular
track-tyres combination.
Deliverable D3.2
eVALUE 96 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The test shall be performed with the initial speeds and distances according to the values
listed in Table B-8. Figure B-6 shows the definitions of initial speeds and distances. Both a
passenger vehicle and pedestrian target shall be used.
V T
LS
Targetvehicle
V S
Subject
vehicle
T L
Subject
vehicle
VS
LS
LT
Pedestrian
target
VT
Figure B-6 Initial conditions
Table B-8 Test cases – passive driver
Test
case
Initial
subject
vehicle
speed
[km/h]
Initial
target
object
speed
[km/h]
Initial
subject
vehicle
distance (LS)
[m]
Initial
target
object
distance
(LT)
[m]
Target object
1.1 30±2 15±2 30±1 15±0.25 passenger vehicle
1.2 30±2 30±2 30±1 30±0.25 passenger vehicle
1.3 50±2 30±2 50±1 30±0.25 passenger vehicle
1.4 15±2 5±0.5 15±1 5±0.25 pedestrian
1.5 30±2 5±0.5 30±1 5±0.25 pedestrian
1.6 50±2 5±0.5 50±1 5±0.25 pedestrian
The number of trials for each test case is ≥1.1
B.3.4.8.3 Test 2: Transversally moving target - driver brakes strongly
In this test a driver who brakes strongly is simulated. After the initial speeds and distances
have been established, the brake action is controlled by a brake robot to warrant
repeatability. The brake action is triggered by the warning signal from the vehicle. A typical
1 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 97 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
reaction time of 1 s is waited before the robot applies a brake force to the brake pedal. In this
case a pedal force of 700 N in 0.2 s should be applied.
The test shall be performed with the initial speeds and distances according to the values
listed in Table B-9. Both a passenger vehicle and pedestrian target shall be used.
Table B-9 Test cases – driver brakes strongly
Test
case
Initial
subject
vehicle
speed
[km/h]
Initial
target
object
speed
[km/h]
Initial
subject
vehicle
distance (LS)
[m]
Initial
target
object
distance
(LT)
[m]
Target object
2.1 30±2 15±2 30±1 15±0.25 passenger vehicle
2.2 30±2 30±2 30±1 30±0.25 passenger vehicle
2.3 50±2 30±2 50±1 30±0.25 passenger vehicle
2.4 15±2 5±0.5 15±1 5±0.25 pedestrian
2.5 30±2 5±0.5 30±1 5±0.25 pedestrian
2.6 50±2 5±0.5 50±1 5±0.25 pedestrian
The number of trials for each test case is ≥1.1
B.3.4.8.4 Test 3: Transversally moving target – driver brakes mildly
In this test a driver who brakes strongly is simulated. After the initial speeds and distances
have been established, the brake action is controlled by a brake robot to warrant
repeatability. The brake action is triggered by the warning signal from the vehicle. A typical
reaction time of 1.5 s is waited before the robot applies a brake force to the brake pedal. In
this case a pedal force of 350 N in 0.4 s should be applied.
The test shall be performed with the initial speeds and distances according to the values
listed in Table B-10. Both a passenger vehicle and pedestrian target shall be used.
1 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 98 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Table B-10 Test cases – driver brakes mildly
Test
case
Initial
subject
vehicle
speed
[km/h]
Initial
target
object
speed
[km/h]
Initial
subject
vehicle
distance (LS)
[m]
Initial
target
object
distance
(LT)
[m]
Target object
3.1 30±2 15±2 30±1 15±0.25 passenger vehicle
3.2 30±2 30±2 30±1 30±0.25 passenger vehicle
3.3 50±2 30±2 50±1 30±0.25 passenger vehicle
3.4 15±2 5±0.5 15±1 5±0.25 pedestrian
3.5 30±2 5±0.5 30±1 5±0.25 pedestrian
3.6 50±2 5±0.5 50±1 5±0.25 pedestrian
The number of trials for each test case is ≥1.1
B.3.4.9 Uncertainty
Uncertainty in the measurement data shall be stated.
B.3.4.10 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
1 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 99 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.4 Lane departure, open loop test
B.4.1 Scope
This testing protocol describes the procedure for evaluating the performance of the safety
functions for avoidance of lane departure on a straight or curved road. The present test
procedure addresses the safety of different types of subject vehicles (passenger car, bus or
truck).
The test procedures define physical tests for evaluating the safety performance of the vehicle
by using professional drivers or driving robots that control the vehicle with predefined
manoeuvres.
This testing protocol describes open loop tests.
vt
vt
Figure B-7 Illustration of lane departure scenarios on a straight road.
B.4.2 References
References are listed in Chapter B.1.
B.4.3 Definitions
Definitions are listed in Chapter B.1.
B.4.4 Test procedure - Avoidance of lane departure
B.4.4.1 Principle
The test is based on the observation of the subject vehicle behaviour when executing the
manoeuvres specified in the respective tests 1 to 3, corresponding to scenarios C2-1 to C2-3
as stated in Chapter 2.2. The open loop tests address the vehicle’s technical performance.
For the present tests no distinction is made between lane departure and road departure, i.e.
the former is considered to cover the latter.
Deliverable D3.2
eVALUE 100 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The test itself aims at determining the subject vehicle’s performance by warning the driver to
prevent the occurrence of a lane departure. The following safety performance indicator for
the subject vehicle is determined:
- time to line crossing (TLC) at warning
B.4.4.2 Test objectives
This test protocol defines tests for open loop testing where a professional test driver or
driving robot is used.
The objective of an open loop test is to test the technical performance of the vehicle, without
considering natural response and feedback from an arbitrary driver. A professional driver or a
driving robot is used to drive the car in such way that a warning should be triggered. The
timing sequence of the open loop test is presented in Figure B-8.
t0 t1 t3 t4t2
Pre-
stabilisation
period
Create a lane
drift
Await
system
activation
(warning)
Start of test Lane drift established
and stable
Warning End of test
Figure B-8 Timing sequence for open loop testing of lane departure
B.4.4.3 Drivers
A professional driver or a driving robot is needed in the subject vehicle.
B.4.4.4 Equipment
The equipment used for the test procedure shall provide for the collection of appropriate
measures and their processing into safety performance indicators. A positioning system shall
be able to determine the position of the subject vehicle throughout the entire test. The time
for issuing warning signals in the subject vehicle must be accessible and possible to
synchronize with the subject vehicle position.
B.4.4.5 Testing conditions and environment
Test conditions and environment are described in Chapter B.1.
B.4.4.6 Information required for the test
The resulting reports from the following inspections shall be used:
Deliverable D3.2
eVALUE 101 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
- Definition of subject vehicle including the targeted safety functions.
o In particular, it is important to determine the activation speed of the safety
system and whether its design and implementation can be considered
symmetric, such that equal performance can be expected for both sides of the
vehicle.
- Environmental conditions
These reports are necessary to understand if e.g. normal conditions can be used during the
physical tests and to get an understanding of the subject vehicle and its safety functions.
An accurate position reference of the lane markings is needed. The accuracy should be at
least equal to the subject vehicle position accuracy, cf. Table B-11. In addition, evaluation of
line crossing requires that the vehicle dimensions as well as the placement of the antenna (or
similar) of the positioning system are known.
B.4.4.7 Subject vehicle preparation and conditioning
Subject vehicle preparation and conditioning are described in Chapter B.1.
B.4.4.8 Test procedure and data processing
B.4.4.8.1 Tests
The test procedure addresses lane (and road) departure on a straight or curved road
according to the following tests:
Test 1 Lane departure on a straight road (Figure B-9)
Test 2 Lane departure in a curve (Figure B-10)
Test 3 Lane departure on a straight road just before a curve (Figure B-11)
During the course of the tests, the measures listed in Table B-11 are to be recorded.
Deliverable D3.2
eVALUE 102 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Table B-11 Measurements during lane departure tests
Measure Typical range Recommended
maximal error
position of subject vehicle
0 m to 750 m X,Y: ± 0.05m
speed of the subject vehicle
0 km/h to 110 km/h ± 0.5 km/h [21994]
rate of departure - 5 m/s to 5 m/s ± 0.05 m/s
local time reference
shall cover all test duration, from stabilization
until lane avoidance or crossing, (typically 20 s)
and apply to all measured quantities.
video: ±0.01s
haptic: ±0.01s
audio: ±0.01s
subject vehicle
deviation from
the lane
boundary
0 m to 5 m X,Y: ± 0.05m
point of departure
across the lane
boundary
0 m to 250 m X,Y: ± 0.05m
warning signal (to the driver) issuing time
extracted from driver/dashboard video for visual
signals, audio warnings will be detected by using
microphones, haptic warnings by appropriate
sensors (ex accelerometers)
video: ±0.05s
haptic: ±0.05s
audio: ±0.05s
The measured values are logged during the driving manoeuvres until termination of the test.
Instructions about the manoeuvres to be carried out are given to the test driver of the subject
vehicle. At the end of the pre-stabilisation period, the subject vehicle reaches the initial speed
and moves along the centre line of the lane with the lateral safety function in operational
mode. The subject vehicle is guided to drift and eventually depart from the lane in a
predefined manner, either programmed into a steering robot or specified as a deviation zone
on the track indicating to the driver the course to follow while maintaining the specified
speed. The test driver is not allowed to correct the path before a warning is given. When a
warning is issued, the end of the test is reached.
Deliverable D3.2
eVALUE 103 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Technical requirements and safety performance indicators
Measurements are carried out to monitor the time for and type of warning provided by the
vehicle given certain predefined driving manoeuvres. The assessment of the safety
performance of the vehicle is based on the time of warning.
The technical requirements and safety performance indicators are presented in Table B-12.
Table B-12 Technical requirements and safety performance indicators for open loop tests
Technical requirements Safety performance indicator
The system shall warn the driver when the
warning condition is fulfilled [17361]
No warning shall occur between the two
earliest warning lines as described in
Section B.1.3 [17361]
time to line crossing (TLC) at warning
The safety performance indicator ―Time to Line Crossing― is determined based on the vehicle’s position and rate of departure (lateral speed) at the time of warning, as well as on the vehicle geometry and the position reference of the lane marking. It is computed in the following way:
- The shortest distance between the lane marking and the vehicle, divided by the rate of departure.
B.4.4.8.2 Test 1: Lane departure on a straight road
Description
vt
vt
Figure B-9 Illustration of lane departure scenarios on a straight road.
The timing sequence for open loop tests is presented in Figure B-8.
Deliverable D3.2
eVALUE 104 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
1. During the pre-stabilisation period (t0 - t1), the subject vehicle reaches the predefined
initial speed while moving along the centre line of the lane with the lateral safety
function activated.
2. Keeping the initial longitudinal speed, the driver of the subject vehicle is at a time
between t1 and t2 forced to deviate from the centre line.
3. The test is concluded when
a) the warning is issued or
b) half the vehicle has crossed the line into the adjacent lane (regarded as a
complete failure)
The tests shall be carried out with initial speed, rate of departure and departure direction
according to Table B-13.
Test implementation
The subject vehicle moves in a straight course of 100 m at the specified speed along the
centre of the lane. The lane departure test may be implemented using e.g. a path following
function or in terms of a predefined steering wheel angle, to obtain the predefined rate of
departure (lateral speed) stated in Table B-13 below.
When the pre-stabilization period is finished, the subject vehicle creates a lane drift with the
specified lateral speed.
Once the test is concluded, the lane change manoeuvre should be aborted by the driver.
The following considerations are needed in order to define longitudinal speed, rate of
departure and departure direction, based on inspection of the subject vehicle mentioned in
Section B.4.4.6:
Initial longitudinal speed. A low and a high longitudinal speed for testing shall be
selected to ensure that the lane departure warning system is operational.
o Low longitudinal speed = Minimum system activation speed + 5 km/h, no lower
than 72 km/h.
o High longitudinal speed = Low longitudinal speed + 30 km/h, not exceeding
110 km/h.
Rate of departure. A low and a high departure rate shall be determined based on the
chosen longitudinal speeds in the following way:
o Low departure rate interval = 1% of prescribed longitudinal speed ± 0.1 m/s
o High departure rate interval = 3% of prescribed longitudinal speed ± 0.1 m/s
Departure direction. If it is concluded from the inspection that the system is expected
to perform equally on both sides of the subject vehicle, it is considered sufficient to
choose either left or right as departure direction for all tests. If, in contrast, the system
Deliverable D3.2
eVALUE 105 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
cannot be considered as essentially symmetric, the testing of each case needs to be
carried out in both directions.
Table B-13 Test cases for lane departure on a straight road
Test
case
Initial
subject
vehicle
speed
Rate of departure Departure
direction
Lane marking
1.1 low low left/(right) dashed white
1.2 low high left/(right) dashed white
1.3 high low left/(right) dashed white
1.4 high high left/(right) dashed white
The number of trials for each test case is ≥1.1
B.4.4.8.3 Test 2: Lane departure in a curve
Description
vt
R
Figure B-10 Illustration of a lane departure scenario in a curve
These tests should be carried out in a curve of radius between 300 and 600 m.
The timing sequence for open loop tests is presented in Figure B-8.
1. During the pre-stabilisation period (t0 - t1), the subject vehicle reaches the predefined
initial speed while moving along the centre line of the lane with the lateral safety
function activated
1 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 106 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
2. Keeping the initial longitudinal speed, the driver of the subject vehicle is at a time
between t1 and t2 forced to deviate from the centre line.
3. The test is concluded when:
a. the warning is issued
b. half the width of the vehicle leading edge has crossed over the line into the
adjacent lane (regarded as complete failure)
The test shall be carried out with initial longitudinal speed and rate of departure according to
Table B-14.
Longitudinal speeds and rates of departure are equal to those defined for Test 1 in Section
B.4.4.8.2.
Test implementation
The subject vehicle moves along the centre line of the lane. The lane departure test may be
implemented using e.g. a path following function or in terms of a predefined steering wheel
angle, to obtain the predefined rate of departure (lateral speed) stated in Table B-14 below.
When the pre-stabilization period is finished, the subject vehicle creates the lane drift with the
specified lateral speed.
Once the test is concluded, the lane change manoeuvre should be aborted by the driver.
If it is concluded from the inspection that the system is expected to perform equally on both
sides of the subject vehicle, it is considered sufficient to choose either left or right curve for
all tests. If, in contrast, the system cannot be considered as essentially symmetric, the testing
of each case needs to be carried out in both types of curves.
Table B-14 Test cases for lane departure in a curve
Test
case
Initial
subject
vehicle
speed
Rate of
departure
Curve
direction
Departure
direction
Lane marking
2.1 low low left/(right) left/(right) dashed white
2.2 low high left/(right) left/(right) dashed white
2.3 low low left/(right) right/(left) dashed white
2.4 low high left/(right) right/(left) dashed white
The difficulty associated with high precision driving in curves implies two differences
compared to lane departure testing on a straight road:
1. It is recommended to use a steering robot to obtain adequate driving precision.
Deliverable D3.2
eVALUE 107 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
2. For safety reasons, curve testing is only carried out at low longitudinal speed
The number of trials for each test case is ≥11.
B.4.4.8.4 Test 3: Lane departure just before a curve
Description
vs
R
Figure B-11 Illustration of lane departure scenario on a straight lane just before a curve.
These tests should be carried out in a curve of radius between 300 and 600 m.
The timing sequence for open loop tests are presented in Figure B-8.
1. During the pre-stabilisation period (t0 - t1), the subject vehicle reaches the predefined
initial speed while moving along the centre line of the lane with the lateral safety
function activated
2. Keeping the initial longitudinal speed, the driver of the subject vehicle drives straight
into the curve.
3. The test is concluded when:
a. the warning is issued
b. half the width of the vehicle leading edge has crossed the line (regarded as
complete failure)
The test shall be carried out with the initial speed listed in Table B-15.
The longitudinal speed is equal to that defined for Test 1 in Section B.4.4.8.2.
1 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 108 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Test implementation
The subject vehicle moves along the centre line of the lane at the predefined speed. The
vehicle continues to go straight ahead when it enters the curve. Once the test is concluded,
the lane departure should be aborted by the driver.
If it is concluded from the inspection that the system is expected to perform equally on both
sides of the subject vehicle, it is considered sufficient to choose either left or right curve for
all tests. If, in contrast, the system cannot be considered as essentially symmetric, the testing
of each case needs to be carried out in both types of curves.
Table B-15 Test case for lane departure on a straight road just before a curve
Test case Initial subject
vehicle speed
Curve direction Lane marking
3.1 low left/(right) dashed white
3.2 low left/(right) dashed white
The number of trials for each test case is ≥1.1
B.4.4.9 False alarms in Tests 1-3
There are no specific tests dedicated to false alarms. However, in accordance with ISO
17361, a no-warning zone should be defined, within which any issued alarm is considered as
false.
All false alarms occurring during Tests 1-3 shall be reported.
The no-warning zone may be defined by the OEM or system manufacturer, in which case it
will be stated in the inspection protocol. If no other information is available, a no-warning
zone extending to 30 cm inside the lane boundaries on both sides will respect the conditions
for most light and heavy vehicles driving in the centre of a highway lane.
B.4.4.10 Uncertainty
Uncertainty in the measurement data shall be stated.
1 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 109 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.4.4.11 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
Deliverable D3.2
eVALUE 110 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.5 Avoidance of lane change collision on a straight road, open loop test
B.5.1 Scope
This testing protocol describes the procedure for evaluating the performance of the safety
functions for avoidance of lane change collisions. The present test procedure addresses the
safety of different types of subject vehicles (passenger car, bus or truck).
The test procedures define physical tests for evaluating the safety performance of the vehicle
by using professional drivers or driving robots that control the vehicle with predefined
manoeuvres.
This testing protocol describes open loop tests.
Subject vehicle
Target vehiclevt
vs
Figure B-12 Illustration of lane change collision scenario
B.5.2 References
References are listed in Chapter B.1.
B.5.3 Definitions
Definitions are listed in Chapter B.1.
B.5.4 Test procedure - Avoidance of lane change collision on a straight road
B.5.4.1 Principle
The test is based on the observation of the subject vehicle behaviour when executing the
manoeuvres specified in Test 1 corresponding to scenario C2-4 as stated in Chapter 2.2.
The open loop tests address the vehicle’s technical performance.
The test itself aims at determining the subject vehicle’s performance by warning/supporting
the driver, to prevent a lane change collision. The following safety performance indicator is
determined for the subject vehicle:
- Time until target enters Collision Risk Zone at warning
Deliverable D3.2
eVALUE 111 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.5.4.2 Test objectives
This testing protocol defines tests for open loop testing where professional test drivers or
driving robots are used.
The objective of an open loop test is to test the technical performance of the vehicle, without
considering natural response and feedback from the driver. Professional driver or driving
robots are used to drive the vehicles in such a way that a warning should be triggered by the
subject vehicle.
The timing sequence of open loop tests in lane change scenarios is presented in Figure B-
13.
t0 t1 t3t2
Pre-
stabilisation
period
Start of test Warning End of test
t5t4
Perfrorm
actions to
activate safety
functionality
Target enters
Collision Risk
Zone
Target leaves
Collision Risk
Zone
Figure B-13 Timing sequence of open loop tests for lane change collision
B.5.4.3 Drivers
Professional drivers or driving robots shall be used in the subject and target vehicles.
B.5.4.4 Equipment
The equipment used for the test procedure shall provide for the collection of appropriate
measures and their processing into safety performance indicators. Positioning systems shall
be able to determine the position of both vehicles throughout the entire test. Time stamps or
other means of time synchronization of the vehicle positions must be available. The time for
issuing warning signals must be accessible and possible to synchronize with subject vehicle
position.
B.5.4.4.1 Target vehicle
The target may either be a real vehicle or simulated by a vehicle dummy similar to an
ordinary vehicle with regard to physical dimensions and detection characteristics.
Deliverable D3.2
eVALUE 112 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The following properties are important1:
- Size (width, height, and shape)
- Radar Cross Section (RCS)
- Reflectivity
- Size, contrast, and colour range
B.5.4.5 Testing conditions and environment
Test conditions and environment are described in Chapter B.1.
B.5.4.6 Information required for the test
The resulting reports from the following inspections:
- Definition of subject vehicle including the targeted safety functions.
o In particular, it is important to determine the activation speed of the safety
system and whether its design and implementation can be considered
symmetric such that equal performance can be expected for both sides of the
vehicle.
o A definition of actions required to activate the subject vehicle’s applicable lane
change support (for instance activation of turn indicator and/or attempt a lane
change).
o Definition of a Collision Risk Zone, within which the system is expected to
generate a warning. The Collision Risk Zone may be stated by the OEM/
system manufacturer in terms of an official performance specification, or
chosen according an applicable standard such as ISO 17387.
- Environmental conditions
These reports are necessary to understand if e.g. normal conditions can be used during the
physical tests and to get an understanding of the subject vehicle and its safety functions.
The definition of the Collision Risk Zone may be chosen differently depending on the nature
of the safety system. For instance, for lane change support systems based on blind spot
detection, a Collision Risk Zone based on the ISO 17387 definition of blind spot zone may be
chosen.
Evaluation of performance may require knowledge of subject and target vehicle geometries
as well as placement of antennas (or similar) of the positioning systems.
1 Target vehicle dimensions and characteristics are still under discussion; eVALUE will not define a general target
vehicle.
Deliverable D3.2
eVALUE 113 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.5.4.7 Subject vehicle preparation
Subject vehicle preparation and conditioning are described in Chapter B.1.
B.5.4.8 Test procedure and data processing
B.5.4.8.1 Tests
The test procedure addresses lane change collision avoidance on a straight road according
to the following test:
Test 1 Lane change collision avoidance on a straight road (Figure B-12)
During the course of this test, the measures listed in Table B-16 shall be recorded.
Table B-16 Measurements during lane change collision warning tests
Measure Typical range Recommended
maximal error
position of subject vehicle 0 m to 750 m X,Y: ± 0.05m
position of target vehicle 0 m to 750 m X,Y: ± 0.05m
absolute distance between subject vehicle and target vehicle, if available
0 m to 250 m X,Y: ± 0.1m
relative lateral distance between subject vehicle and target vehicle, if available
0 m to 250 m X,Y: ± 0.1m
speed of the subject vehicle
0 km/h to 110 km/h ± 0.5 km/h
[21994]
speed of the target vehicle
0 km/h to 110 km/h ± 0.5 km/h
[21994]
local time reference
shall cover all test
duration and applies to all
measured quantities.
video: ±0.01s
haptic: ±0.01s
audio: ±0.01s
warning signal (to the driver) issuing time
extracted from
driver/dashboard video for
visual signals, audio
warnings will be detected
using microphones, haptic
warnings e.g.
accelerometers
video: ±0.05s
haptic: ±0.05s
audio: ±0.05s
The measured values are logged during the driving manoeuvres until termination of the test.
Deliverable D3.2
eVALUE 114 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Instructions about the manoeuvres to be carried out are given to the test driver of the subject
vehicle. At the end of the pre-stabilisation period the subject vehicle reaches the initial speed
and moves along the centre line of the lane taking such actions that the lateral safety
function is activated, while the target vehicle is manoeuvred to pass through the pre-defined
Collision Risk Zone of the subject vehicle.
Technical requirements and safety performance indicators
Measurements are carried out to monitor the time at which a warning is issued by the subject
vehicle.
The technical requirements and safety performance indicators for open loop tests are
presented in Table B-17.
Table B-17 Technical requirements and safety performance indicators
Technical requirements Safety performance indicator
When the safety function is enabled the system shall
detect the target vehicle and issue a warning to
prevent a lane change collision [17387].
time until target enters Collision
Risk Zone (at instance of warning)
The safety performance indicator ―Time to entering Collision Risk Zone― is determined based on the vehicle positions and speeds at the time of warning, as well as geometry of vehicles and Collision Risk Zone. It is computed in the following way: - The distance between the Collision Risk Zone boundary and the leading or trailing edge of the target vehicle, divided by the overtaking speed.
B.5.4.8.2 Test 1: Lane change collision avoidance on a straight road
Description The lane change open loop tests imply that the subject vehicle is overtaking a target vehicle
in the parallel, adjacent lane at speeds specified in the test cases, or that the target vehicle is
overtaking the subject vehicle in the parallel adjacent lane, either to the right or to the left.
The timing sequence for open loop test is presented in Figure B-13.
1. During the pre-stabilisation period (within t0 and t1) the subject vehicle and the target
vehicle reach their predefined initial speeds while moving along the centre line of their
respective lanes..
2. Keeping the initial speed, the driver of the subject vehicle shall take the necessary
actions to activate the safety function (for instance activate the turn indicator and/or
start a lane change attempt).
Deliverable D3.2
eVALUE 115 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
3. The vehicles are driven in such a way that the target vehicle enters the Collision Risk
Zone.
4. The test is concluded when the target vehicle has first entered and then completely
exited the Collision Risk Zone.
It shall be noted that some safety systems will require a lane change attempt in order to be
activated. Such systems require extra safety precautions due to the increased collision risk,
e.g. inflatable bump protection or rubber shielding of the target vehicle.
The test shall be carried out with the initial speeds according to the values listed in Table B-
18.
Test implementation
The subject and target vehicles move along the centre of their respective lanes and maintain
their predefined speed according to the test case definition in Table B-18.
If it is concluded in the inspection that the system is expected to perform equally on both
sides of the vehicle, it is considered sufficient to choose either left or right as the side of
target vehicle approach for all tests. If, in contrast, the system cannot be considered as
essentially symmetric, the testing of each case needs to be performed on both sides.
Table B-18 Test cases lane change collision warning
Test case Target vehicle
approach
Subject vehicle
speed
Target vehicle speed
1.1 from behind left/(right) side 65±2 km/h 75±2 km/h
1.2 from behind left/(right) side 65±2 km/h 90±2 km/h
1.3 from front left/(right) side 75±2 km/h 65±2 km/h
1.4 From front left/(right) side 90±2 km/h 65±2 km/h
The number of trials for each test case is ≥1.1
1 The number of trials has not yet been decided. More testing and evaluation of the testing protocols
are needed. Please refer to eVALUE deliverable D4.2 for a discussion on this issue.
Deliverable D3.2
eVALUE 116 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.5.4.9 False alarms in Test 1
There are no specific tests dedicated to false alarms. However, in accordance with ISO
17387, a no-warning zone shall be defined. When the entire target vehicle is located in this
zone, any issued alarm is considered as false.
All false alarms occurring during Test 1 shall be reported.
The following no-warning zones are devised by ISO 17387.
1. In front of a line extending from the subject vehicle’s leading edge
2. Behind a line located 30 m rear of, and parallel to, the subject vehicle’s trailing edge
B.5.4.10 Uncertainty
Uncertainty in the measurement data shall be stated.
B.5.4.11 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
Deliverable D3.2
eVALUE 117 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.6 Emergency braking on a μ split, open loop test
B.6.1 Scope
This testing protocol describes the test procedure for testing the safety performance of the
subject vehicle during a braking manoeuvre on a μ-split surface. The μ-split surface is such
that the left wheels of the subject vehicle are exposed to a significantly different coefficient of
friction ( than the right wheels. The present test procedure addresses the safety of different
types of subject vehicles (car or different types of commercial vehicles). The test is an open
loop test.
vs
vs
High µ
Low µ
High µ
Low µ
amax
amax
Figure B-14 μ-split braking scenario
B.6.2 References
References are listed in Chapter B.1.
B.6.3 Definitions
Definitions are listed in Chapter B.1.
B.6.4 Test procedure - Emergency braking on a μ split
B.6.4.1 Principle
The test is based on the observation of the subject vehicle behaviour when executing the
manoeuvres specified in scenario C3-1: emergency braking on a μ split as stated in Chapter
2.2.
The test aims at determining the performance for the subject vehicle with respect to stability
and stopping distance.
B.6.4.2 Test objectives
This testing protocol defines tests for open loop testing where a professional test driver or
driving robot is used.
Deliverable D3.2
eVALUE 118 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The objective of an open loop test is to evaluate the technical performance of the vehicle,
without considering natural response and feedback from the driver. A professional driver or a
driving robot is used to trigger an actuation from the vehicle.
Timing sequences of open loop tests in the emergency braking on a μ split scenario is
presented in Figure B-15.
Start of test
Velocity
stabilized
Braking manoeuvre
started
Stabilization function active
End of test
t0 t3 t1 t4 t2
10 km/h
drop-off
Figure B-15 Timing sequence of open loop test
B.6.4.3 Drivers
A professional driver or driving robot shall be used in the subject vehicle.
B.6.4.4 Equipment
Besides the data collection equipment described in Chapter B.1, the following equipment is
necessary during test or test preparations:
- brake temperature sensor
- track surface temperature sensor
B.6.4.5 Testing environment
Test conditions and environment are described in Chapter B.1.
B.6.4.6 Information required for the test
The resulting reports from the following inspections are used:
- Definition of subject vehicle including the targeted safety functions
- Environmental conditions
These reports are necessary to understand if e.g. normal conditions can be used during the
physical tests. Also these inspections help to get an understanding of the subject vehicle and
its safety functions.
Deliverable D3.2
eVALUE 119 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.6.4.7 Subject vehicle preparation
Subject vehicle preparation and conditioning are described in Chapter B.1.
B.6.4.8 Test procedure and data processing
B.6.4.8.1 Tests
The test procedure addresses μ-split braking according to the following test:
Test 1 μ-split braking (Figure B-16)
The braking manoeuvre consists of braking from a speed of 501 km/h to 0. The steering
wheel is kept at 0˚ during the manoeuvre. The scenario is shown in Figure B-16.
vs
vs
High µ
Low µ
High µ
Low µ
amax
amax
Figure B-16 μ-split braking
The end of the test occurs when the vehicle no longer has two wheels on the low adherence
surface.
During the tests, the parameters of Table B-19 shall be measured.
1 This speed is yet to be finally determined. Please see eVALUE deliverable D4.2 for a discussion on
this subject.
Deliverable D3.2
eVALUE 120 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Table B-19 Measurements
Measure Typical range Recommended
maximal error
distance 0 to 250 m ±1% (≤ 50 m) ±0.50 m (> 50m)
speed 0 km/h to 110 km/h ±0.5 km/h
position 0 m to 250 m X,Y: ± 0.50m
longitudinal acceleration -15 m/s2 to 15 m/s2 ±0.15 m/s2
lateral acceleration -15 m/s2 to 15 m/s2 ±0.15 m/s2
steering wheel angle -360º to 360º ±1 º (<50 º) ±2 º (>50 º and <180 º) ±4 º (>180 º)
yaw rate -50º/s to 50º/s ±0.3 º/s (<20 º/s) ±1 º/s (>20 º/s)
brake force trigger ≤ 10 N (trig point) ±5 N
brake friction material temperature 0-1000° C ±5°C
Technical requirements and safety performance indicators
The technical requirements and safety performance indicators for open loop tests are
presented in Table B-20.
Table B-20 Technical requirements and safety performance indicators
Technical requirements Safety performance indicator
The stopping distance, which depends on the
longitudinal deceleration, shall be short in a μ-split
braking scenario
mean longitudinal deceleration
The trade-off between stability and stopping distance
is a critical in a μ-split braking scenario
equivalent deceleration
equivalent deceleration on different
tracks
Taking as reference the first 10 km/h speed drop off for the µ-split braking, the following three
safety performance indicators are determined:
Deliverable D3.2
eVALUE 121 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The mean longitudinal deceleration:
2
102
10
1t
t
xF dtaTT
a
The equivalent deceleration:
2
1
2
1110 t
t
x
t
t
X
FED
dta
dtV
aa
The equivalent deceleration on different tracks:
dtV
ttg
aa
t
tX
LOWHIGHLOWHIGH
F 2
112
10 1111
Where:
t1 is the time when driver starts to act on the brake pedal
t2 is the time when vehicle reaches the 10 km/h speed drop off
Vx and ax follow the definitions contained into the glossary, expressed in [m/s] and [m/s2].
Ψ is the yaw rate [rad/sec]
ηHIGH and ηLOW are respectively ratios between the longitudinal decelerations means on high
and low adherence surfaces (obtained from the test described in Test Case paragraph) with
respect to gravity acceleration.
The first safety performance indicator may be the basis to evaluate the longitudinal
performance whereas the second one may be used to evaluate the trade-off between
stability and braking performance.
The last safety performance indicator aims to be a representative parameter of the vehicle
behaviour on µ split surfaces and it should be quite stable for different tracks.
Deliverable D3.2
eVALUE 122 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.6.4.8.2 Test 1: -split braking
Initial conditions
Drive: gear in neutral position or clutch disengaged (panic brake situation).
Braking manoeuvre
Fast brake application (Fpedal: from 0 to more than 50 daN in 0.15 s)
Brake pedal force is kept higher than 50 daN during the braking manoeuvre.
Steering wheel: the steering wheel is kept at 0 º ( 3º)
Yaw rate: minimum -0.75 º/s, maximum 0.75 º/s
The following applies for the lateral position of the vehicle during the -split braking test case:
the: vehicle longitudinal centreline projection must coincide with the longitudinal borderline
between surfaces. (The wheels on the right side of the vehicle must not enter the left track
surface, and vice versa)
Table B-21 Test cases -split braking
Test case Initial speed [km/h] Friction
1.1 1001 high
1.2 1001 low
1.3 501 split
Each test case shall be performed five times (a valid trial shall fulfil the initial conditions
stated in the respective test). Measures from the trials shall be harmonized (according to ISO
21994). If the test track permits, five additional trials shall be performed in the opposite
direction.
B.6.4.9 Uncertainty
Uncertainty in measurement data shall be stated.
1 This speed is yet to be finally determined. Please see eVALUE deliverable D4.2 for a discussion on
this subject.
Deliverable D3.2
eVALUE 123 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.6.4.10 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
Deliverable D3.2
eVALUE 124 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.7 Emergency braking on a μ-split, closed loop test
B.7.1 Scope
This testing protocol describes the test procedure for testing the safety performance of the
stability control of the subject vehicle when it is exposed to -split conditions. split means
that the left hand side wheels of the subject vehicle is experiencing a significantly other
coefficient of friction ( than the right hand side wheels. The present test procedure
addresses the safety of different types of subject vehicles (car or different types of
commercial vehicles). The test is a closed loop test.
vs
vs
High µ
Low µ
High µ
Low µ
amax
amax
Figure B-17 emergency braking on a -split scenario
B.7.2 References
References are listed in Chapter B.1.
B.7.3 Definitions
Definitions are listed in Chapter B.1.
B.7.4 Test procedure -Emergency braking on a μ-split
B.7.4.1 Principle
The test is based on the observation of the subject vehicle behaviour when executing the
manoeuvres specified in scenario C3-1: emergency braking on a μ split as stated in Chapter
2.2.
The test aims at determining the performance for the subject vehicle with respect to stability
and stopping distance.
B.7.4.2 Test objectives
This testing protocol defines tests for closed loop testing where a professional test driver is
used.
Deliverable D3.2
eVALUE 125 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
The objective of a closed loop test is to evaluate the overall performance of the vehicle when
considering natural response and feedback from the driver. A driver is used to trigger an
actuation from the vehicle.
Timing sequence of closed loop tests in the emergency braking on a μ split scenario is
presented in Figure B-18.
Start of test
Velocity
stabilized
Braking manoeuvre
started
Stabilization function active
End of test
t0 t2 t1 t3
Figure B-18 Timing sequence of closed loop test
B.7.4.3 Drivers
A professional driver shall be used in the subject vehicle.
B.7.4.4 Equipment
Besides the data collection equipment described in Chapter B.1, the following equipment is
necessary during test or test preparations:
- brake temperature sensor
- track surface temperature sensor
B.7.4.5 Testing environment
Test conditions and environment are described in Chapter B.1.
B.7.4.6 Information required for the test
The resulting reports from the following inspections are used.
- Definition of subject vehicle including the targeted safety functions
- Environmental conditions
These reports are necessary to understand if e.g. normal conditions can be used during the
physical tests. Also these inspections help to get an understanding of the subject vehicle and
its safety functions.
Deliverable D3.2
eVALUE 126 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.7.4.7 Subject vehicle preparation
Subject vehicle preparation and conditioning are described in Chapter B.1.
B.7.4.8 Test procedure and data processing
B.7.4.8.1 Tests
The test procedure addresses μ-split braking according to the following test:
Test 1 μ-split braking (Figure B-19)
During the tests, the parameters in Table B-22 shall be measured.
Table B-22 Measurements
Measure Typical range Recommended maximal
error
distance 0 to 250 m ±1% (≤ 50 m) ±0.50 m (> 50m)
speed 0 km/h to 110 km/h ±0.5 km/h
position 0 m to 250 m X,Y: ± 0.50m
longitudinal acceleration -15 m/s2 to 15 m/s2 ±0.15 m/s2
steering wheel angle -360º to 360º ±1 º (<50 º) ±2 º (>50 º and <180 º) ±4 º (>180 º)
steering wheel torque -30 Nm to 30 Nm ±0.1 Nm (< 10 Nm) ±0.3 Nm (> 10 Nm)
yaw rate -50º/s to 50º/s ±0.3 º/s (<20 º/s) ±1 º/s (>20 º/s)
brake force trigger ≤ 10 N (trig point) ±5 N
brake friction material temperature 0-1000°C ±5 °C
The braking manoeuvre consists of braking from a speed of 100 km/h1 to 0. The driver acts
on the steering wheel to try to make the vehicle run in a straight line. The scenario is shown
in Figure B-19.
1 This speed is yet to be finally determined. Please see eVALUE deliverable D4.2 for a discussion on
this subject.
Deliverable D3.2
eVALUE 127 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
vs
vs
High µ
Low µ
High µ
Low µ
amax
amax
Figure B-19 μ-split braking
Technical requirements and safety performance indicators
The technical requirements and safety performance indicators for open loop tests are
presented in Table B-23.
Table B-23 Technical requirements and safety performance indicators
Technical requirements Safety performance indicator
The stopping distance is shall be short in a μ-split
braking scenario
use of adherence
The stability shall be maintained in a μ-split braking
scenario
stability
The test itself aims at determining the following safety indicators for the subject vehicle: use
of adherence (ε) and stability.
The use of adherence (ε) is calculated as the quotient between the theoretical stopping
distance and the stopping distance using the deceleration at μ-split:
where theorSD is the theoretical stopping distance calculated using the average
deceleration between high and low μ initial braking manoeuvres
µsplitSD is the stopping distance using the deceleration of the µ-split braking
manoeuvre.
The maximum theoretical value ε=100% would represent a very high use of adherence. The
theoretical value ε=0% would represent a very poor use of adherence.
The stability is calculated as:
Deliverable D3.2
eVALUE 128 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
high
lowSWRSWAStability
maxmax
where maxSWA is the maximum steering wheel angle achieved during the braking
manoeuvre
maxSWR is the maximum steering wheel rate achieved during the braking
manoeuvre
low is the maximum deceleration achieved on homogeneous braking on low
my surface
high is the maximum deceleration achieved on homogeneous braking on high
my surface
B.7.4.8.2 Test 1: -split braking
An initial test on high surface is performed to determine the stopping distance at high
friction. The stopping distance will then be used to calculate the safety performance indicator
―use of adherence‖.
Initial conditions
Drive: gear in neutral position or clutch disengaged (panic brake situation).
Yaw rate: minimum -0.75 º/s, maximum 0.75 º/s
Steering angle: minimum -3 º, maximum 3 º
Procedure to satisfy the constraint about yaw rate (<0.75 °/s during the 0.5 s before braking
action):
• Maintain a velocity higher than manoeuvre reference velocity
• Disengage the clutch
• Approach to the -split surface
• Control the vehicle (in order to stabilize yaw rate)
• Start braking from the reference velocity
Braking manoeuvre
Fast brake application (Fpedal: from 0 to more than 50 daN in 0.15 s)
Brake pedal force is kept higher than 50 daN during the braking manoeuvre.
Deliverable D3.2
eVALUE 129 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Steering wheel: Driver may act on the steering wheel in order to keep a straight trajectory
and stability. (Driver correction shall be as smooth as possible reducing vehicle’s yaw rate
and without anticipation to vehicle reaction).
The following applies for the lateral position of the vehicle during the -split braking test case.
The vehicle longitudinal centreline projection must coincide with the longitudinal borderline
between surfaces. (The wheels on the right side of the vehicle must not enter the left track
surface, and vice versa)
Table B-24 Test cases -split braking
Test case Initial speed [km/h] Friction
1.1 1001 high
1.2 1001 low
1.3 1001 split
Each test case shall be performed five times (a valid trial shall fulfil the initial conditions
stated in the respective test). Measures from the trials shall be harmonized (according to ISO
21994). If the test track permits, five additional trials shall be performed in the opposite
direction.
B.7.4.9 Uncertainty
Uncertainty in measurement data shall be stated.
B.7.4.10 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
1 This speed is yet to be finally determined. Please see eVALUE deliverable D4.2 for a discussion on
this subject.
Deliverable D3.2
eVALUE 130 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.8 Obstacle avoidance
B.8.1 Scope
This testing protocol describes the test procedure for evaluating the safety performance of a
vehicle in an obstacle avoidance manoeuvre. This eVALUE testing protocol requires extra
safety performance indicators to be evaluated during the well-established sine-with-dwell
manoeuvre. However, the manoeuvre itself is performed exactly as described in the ECE
R13-H regulation or in the NHTSA FMVSS126 conformation test.
Figure B-20 Obstacle avoidance scenario
B.8.2 References
References are listed in Chapter B.1.
B.8.3 Definitions
Definitions are listed in Chapter B.1.
B.8.4 Test procedure - Obstacle avoidance
B.8.4.1 Principle
The objective of the test is to verify the vehicle behaviour during an obstacle avoidance
manoeuvre. The test is based on the observation of the subject vehicle behaviour when
executing the obstacle avoidance manoeuvre. The test procedure is based on the ESC test
in the ECE R13-H regulation.
B.8.4.2 Test objectives
This testing protocol defines tests for open loop testing where a steering robot is used.
The objective of an open loop test is to test the technical performance of the vehicle, without
considering natural response and feedback from the driver. A steering robot is used to trigger
an actuation from the vehicle.
Timing sequence of open loop test in the obstacle avoidance scenario is presented in Figure
B-21.
Deliverable D3.2
eVALUE 131 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Start of test
Velocity
stabilized
Steering manoeuvre
started
Stabilization function active
End of test
t0 t2 t1 t3
Figure B-21 Timing sequence of open loop test
B.8.4.3 Drivers
A steering robot shall be used in the subject vehicle.
B.8.4.4 Equipment
The necessary equipment is specified in the ESC test in the ECE R13-H regulation.
B.8.4.5 Testing environment
Test conditions and environment are described in Chapter B.1.
B.8.4.6 Information required for the test
The resulting reports from the following inspections are used.
- Definition of subject vehicle including the targeted safety functions
- Environmental conditions
These reports are necessary to understand if e.g. normal conditions can be used during the
physical tests. Also these inspections help to get an understanding of the subject vehicle and
its safety functions.
B.8.4.7 Subject vehicle preparation
Subject vehicle preparation and conditioning are described in Chapter B.1.
B.8.4.8 Test procedure and data processing
The procedure specified in the ECE R13-H regulation shall be performed.
Besides the measures specified in the ECE R13-H regulation, the following measure shall be
recorded:
Deliverable D3.2
eVALUE 132 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Table B-25 Measurement
Measure Typical range Recommended maximal
error
steering wheel torque -125 Nm to 125 Nm Accuracy 0.3 Nm
Technical requirements and safety performance indicators
The technical requirements and safety performance indicators for open loop tests are
presented in Table B-26.
Table B-26 Technical requirements and safety performance indicators
Technical requirements Safety performance indicator
Vehicle yaw stability shall be maintained in an
obstacle avoidance scenario
yaw rate ratio at COS+1s and
COS+1.75s
(see ECE R13-H for definition)
Vehicle shall be responsive (lateral displacement) in
an obstacle avoidance scenario
lateral displacement at 1.07s
(see ECE R13-H for definition)
The steering effort required to drive the manoeuvre
shall be within human capabilities in an obstacle
avoidance scenario
1st steering-wheel torque peak
The vehicle yaw response shall follow the driver’s
intentions in an obstacle avoidance scenario
driver intention following
Vehicle roll-over shall be prevented in an obstacle
avoidance scenario
wheel lift
In the ECE R13-H regulation, the following parameters are evaluated: yaw rate ratio and
lateral displacement. These are used to assess the yaw stability and the lateral
responsiveness of the vehicle, respectively. These two are also used as safety performance
indicators in the eVALUE testing protocol.
Besides these two safety performance indicators, three additional are proposed: steering
wheel torque, driver intention following, and wheel lift.
Deliverable D3.2
eVALUE 133 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Steering wheel torque is measured to describe the effort of the driver to perform the
manoeuvre. A very high torque indicates low possibility to control the vehicle. First steering
wheel torque peak is taken.
Driver intention following means how closely the vehicle responds (in terms of yaw motion) to
driver’s intention (commanded by the steering wheel).
time
SW
A
time
yaw
R
time
ay
time
Bra
ke
pre
ss.
time
SW
A
time
yaw
R
time
ay
time
Bra
ke p
ress.
Response gain = Response peak / SWA peak
delay
SWA peak
Response peak
time
SW
A
time
ya
wR
time
ay
time
sl R
EF
time
Bra
ke
pre
ss
time
yaw
R d
iff
time
ay d
iff
time
vx
Response
control
(ay)
Response
control
Response
stability
expected responseactual responseexpected response
actual response
expected
expected - realDIF
0 expected yaw response
> 0 over responsive yaw motion
< 0 under responsive yaw motion
Wheel lift is used to describe roll-over stability. Tip-up criteria is directly carried over from
NHTSA fishhook test. Tip-up condition is met when both inner wheels of the vehicle are
simultaneously lifted more than 50 mm (respect the ground) and for more than 20 ms.
B.8.4.9 Uncertainty
Uncertainty in measurement data shall be stated.
B.8.4.10 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
Deliverable D3.2
eVALUE 134 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.9 Highway exit, open loop test
B.9.1 Scope
This testing protocol describes the test procedure for testing the safety performance of the
subject vehicle when exiting a highway at too high speed. The vehicle has to follow a closing
radius trajectory. The present test procedure addresses the safety of different types of
subject vehicles (car or different types of commercial vehicles). The test is an open loop test.
vs
R
Figure B-22 Highway exit scenario
B.9.2 References
References are listed in Chapter B.1.
B.9.3 Definitions
Definitions are listed in Chapter B.1.
B.9.4 Test procedure - Highway exit
B.9.4.1 Principle
The test is based on the observation of the subject vehicle behaviour when executing the
manoeuvres specified in scenario C3-3: highway exit as stated in Chapter 2.2.
The test aims at determining the performance for the subject vehicle with respect to stability.
B.9.4.2 Test objectives
This testing protocol defines tests for open loop testing where a steering robot is used.
The objective of an open loop test is to test the technical performance of the vehicle, without
considering natural response and feedback from the driver. A steering robot is used to trigger
an actuation from the vehicle.
Deliverable D3.2
eVALUE 135 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Timing sequences of open loop tests in the highway exit scenario is presented in Figure B-
23.
Start of test
Velocity
stabilized
Steering manoeuvre
started
Stabilization function active
End of test
t0 t2 t1 t3
Figure B-23 Timing sequence of open loop test
B.9.4.3 Drivers
A steering robot shall be used in the subject vehicle.
B.9.4.4 Equipment
The following equipment is necessary during test or test preparations:
- track surface temperature sensor
B.9.4.5 Testing environment
Test conditions and environment are described in Chapter B.1.
B.9.4.6 Information required for the test
The resulting reports from the following inspections are used.
- Definition of subject vehicle including the targeted safety functions
- Environmental conditions
These reports are necessary to understand if e.g. normal conditions can be used during the
physical tests. Also these inspections help to get an understanding of the subject vehicle and
its safety functions.
B.9.4.7 Subject vehicle preparation
Subject vehicle preparation and conditioning are described in Chapter B.1.
Deliverable D3.2
eVALUE 136 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.9.4.7.1 Characterization of the lateral dynamics
The Slowly Increasing Steer (SIS) manoeuvre is used to characterize the lateral dynamics of
the subject vehicle. The manoeuvre is used to provide the data necessary for determining
the steering wheel angle (δ0.3g) capable of producing a lateral acceleration of 0.3 g. This
steering wheel angle is then used to determine the magnitude of steering required during the
manoeuvre.
Speed 80 km/h
Ramp steer 13.5 º/s
Figure B-24 Vehicle steer characterisation
B.9.4.8 Test procedure and data processing
B.9.4.8.1 Tests
The test procedure addresses highway exit according to the following test:
Test 1: highway exit (Figure B-22)
During the tests, the parameters in Table B-27 shall be measured.
Deliverable D3.2
eVALUE 137 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Table B-27 Measurements
Measure Typical range Recommended maximal
error
Distance 0 to 250 m ±1% (≤ 50 m) ±0.50 m (> 50m)
Speed 0 km/h to 110 km/h ±0.5 km/h
Position 0 m to 250 m X,Y: ± 0.50m
Lateral acceleration -15 m/s2 to 15 m/s2 ±0.15 m/s2
Steering wheel angle -360º to 360º ±1 º (<50 º) ±2 º (>50 º and <180 º) ±4 º (>180 º)
Steering wheel torque -30 Nm to 30 Nm ±0.1 Nm (< 10 Nm) ±0.3 Nm (> 10 Nm)
Yaw rate -50º/s to 50º/s ±0.3 º/s (<20 º/s) ±1 º/s (>20 º/s)
centre of gravity sideslip angle
-20° to 20° ±0.3 º/s
The test can be run either as a left turn or a right turn.
Technical requirements and safety performance indicators
The technical requirements and safety performance indicators for open loop tests are
presented in Table B-28.
Table B-28 Technical requirements and safety performance indicators
Technical requirements Safety performance indicator
Vehicle understeer shall be prevented in an highway
exit scenario
relative radius
Vehicle oversteer shall be prevented in an highway
exit scenario
slip angle
Vehicle roll-over shall be prevented in an highway
exit scenario
wheel lift
The test itself aims at determining the following safety performance indicators for the subject
vehicle: relative radius, slip and wheel lift. The relative radius (Rrel) is the difference between
the trajectory radius in the test run (Ri) and the trajectory radius in the initial test run (R1).
Deliverable D3.2
eVALUE 138 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
In all cases, measurement of radius is made at the end of the steering wheel ramp.
The slip angle at the centre of gravity of the vehicle (SlipCOG) is used as an oversteer
indicator.
The wheel lift is used to assess roll stability.
B.9.4.8.2 Test 1 Initial run
An initial test is performed to determine the initial radius.
Speed: 80 km/h.
Drive: highest manual gear or automatic drive in position D.
Curve manoeuvre
The test is performed without throttle (coasting)
Steering rate: 0.3 g/s
Steering angle: 0 to 6.5 * δ0.3g
B.9.4.8.3 Test 2 Successive runs
Successive tests are performed at increasing vehicle speed and steering wheel rate.
Speed: increased by 5 km/h steps from 80 km/h until a final speed of 110 km/h is reached.
Each test case should be performed once.
Drive: highest manual gear or automatic drive in position D.
Curve manoeuvre
The test is performed without throttle (coasting)
Steering rate: increased proportionally to vehicle speed increase (compared to initial run)
Steering angle: 0 to 6.5 * δ0.3g
B.9.4.9 Uncertainty
Uncertainty in measurement data shall be stated.
Deliverable D3.2
eVALUE 139 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
B.9.4.10 Result
The measurements and the safety performance indicators associated to the subject vehicle
for the current test procedure shall be documented in the test result. The values shall be
listed together with their respective uncertainty.
The execution of the test procedure shall be briefly described.
The test vehicle shall be identified with reference to the corresponding inspection.
Deliverable D3.2
eVALUE 140 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Annex C Resolved open issues
This annex presents how the open issues collected in deliverable D3.1 have been treated
and resolved.
Since the closed loop versions of testing protocols in cluster 1 and 2 have been removed to
this version of the testing protocols, the open issues related to those testing protocols are not
discussed in this annex.
Issue: TP:
What is meant with colour range for camera sensors? B2,B3
Resolution:
The colour range of a camera sensor defines the colour spectrum in which light can be detected. It is of course important that the target has a colour within that range.
Issue: TP:
Are all three radii (125, 250, and 500 m) necessary? B2
Resolution:
The different curve radii were inspired by the system classification in ISO 15623. However, in order to reduce the number of trials, 250 m was chosen as the only radius in the test cases of testing protocol B2. (see e.g. B.2.4.8.2)
Issue: TP:
Is maximum 40 ºC test track surface temperature a correctly chosen limit?
B2,B3,B4,B5,B6,B7,B9
Resolution:
The upper bound of track surface temperature was increased to 50 ºC in order to accommodate testing during sunny summer days in southern Europe. (see B.1.4.2)
Issue: TP:
Is 100 Hz sample frequency necessary for video recordings? B2,B3,B4,B5
Resolution:
100 Hz is preferable, but not mandatory. If 100 Hz is used, video data will as frequent as data from high-precision GPS-based measurement systems. (see B.1.5.1)
Deliverable D3.2
eVALUE 141 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Issue: TP:
Tire specification needs to be more specific. B2,B3,B4
Resolution:
A statement ―recommended by the OEM‖ has been added to the tire specification. (see B.1.5.2)
Issue: TP:
Is longitudinal vehicle alignment of ± 0.25 m too tough? B2
Resolution:
The open issue is probably incorrectly formulated since it is the lateral vehicle alignment that shall be smaller than ± 0.25. The issue was raised at a project meeting and it was decided to keep the requirement as it is. (see B.2.4.8.1)
Issue: TP:
The difference between technical indicators and safety indicators needs to be clarified.
B2,B3,B4,B5
Resolution:
Technical indicators have been removed to avoid confusion. (see e.g.Table B-2)
(Technical indicators are related to the technical performance of the system without explicitly telling the implications on safety)
Issue: TP:
The timing sequence of the test puts demands on time gap measurement in real-time, is that feasible? Maybe use clearance instead?
B2
Resolution:
Clearances are used instead of time gaps in the test cases. (see e.g. Table B-3)
Deliverable D3.2
eVALUE 142 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Issue: TP:
Test 1 and 2 in Fig. 3 does not correspond to Test 1 and 2 of the testing protocol.
B2,B3
Resolution:
The figure has been updated to reflect the tests. (see e.g. Figure B-3)
Issue: TP:
―The lateral acceleration of the subject vehicle shall not exceed 2.3 m/s2 hence the maximal speed shall be adapted to the curve radius.‖ Is this a valid requirement? Where does the requirement come from?
B2
Resolution:
The requirement has been removed from the testing protocol. (The origin is probably the ISO 15623 standard which stipulates a maximum lateral acceleration of 2.3 m/s2 for class II and III systems during curve testing)
Issue: TP:
How many trials should there be for each test case? B2,B3,B4,B5,B6,B7,B8,B9
Resolution:
The number of trials for each test case has not been decided. More testing and evaluation of the testing protocols are needed before these numbers can be determined. There is a discussion in D4.2 regarding this issue. (See also footnote in e.g. B.2.4.8.2)
Issue: TP:
Tyre, brakes and system conditioning missing. B2
Resolution:
Tyres and brakes conditioning are described (or rather there is a reference to the conditioning procedure of ISO 21994) in B1.5 in the general part. No proper procedure has yet been defined for system conditioning (the driving style prior to the activation of the system could affect its settings).
Issue: TP:
Which type of lane marking shall be used, worst case? What lane markings represent the worst case?
B4
Resolution:
The eVALUE project has not reached a decision on this issue. Worst case could mean well-painted lines but with low white coverage (dashed lines with long spacing) or it could mean dashed lines with short spacing but with worn lines (little paint left). Lane markings are in this version briefly discussed in the general part, i.e. B.1.4.1 instead of in TP B.4.
Deliverable D3.2
eVALUE 143 ICT-2007-215607 eVALUE-101031-D32-V20-FINAL.doc www.evalue-project.eu
Issue: TP:
―Subject vehicle moves in a straight course at the specified speed, along the centre line of the lane (±5cm) during 100m‖ Is this requirement realistic?
B4
Resolution:
It might be achievable with a steering robot. However the requirement has been removed.
Issue: TP:
Lateral speeds of ≤ 0.3 and ≥ 0.5 are too vague to be used as a test parameter in the test cases.
B4
Resolution:
More precise lateral speed values have been defined in B.4.4.8.2.
Issue: TP:
Is the radius 125 m too small for lane departure tests? B4
Resolution:
Yes. The requirement has been increased to 300-600 m. (See B.4.4.8.3)
Issue: TP:
New directions and targets have been added in the truck specific test cases, is that intentional?
B5
Resolution:
It was intentional since these test cases were based on truck-specific accident data. However it was decided to remove truck-specific test cases from the testing protocol to avoid confusion.