report on the results of adaptation, in-vehicle implementations … · 0.07 2010-08-12 adding to...

European Large-Scale Field Operational Tests on In-Vehicle Systems

7th Framework programme INFORMATION AND COMMUNICATION

TECHNOLOGIES ICT-2-6.2 ICT for Cooperative Systems

Large-scale integrating project Grant agreement no.: 223945

Deliverable 2.2

Report on the results of adaptation, in-vehicle implementations and

piloting

Version number Version 3.2Dissemination level PULead contractor FFADue date 20.06.2011Date of preparation 16.06.2011

euroFOT 16.06.2011

Deliverable 2.2 Version 3.2 ii Report on the results of adaptation, in-vehicle implementations and piloting

Author

Guillaume Saint Pierre (IFSTTAR)

Christoph Kessler (FFA)

Lucas Malta (VOLVO)

Contributors

John-Fredrik Grönvall (VCC)

Karsten Heinig (VOLVO)

Lucas Malta (VOLVO)

Barbara Metz (IZVW)

Leandro Guidotti (Poli)

Clement Val (CEESAR)

Christoph Kessler (FFA)

Sabine Kadlubek (FFA)

Walter Schwertberger (MAN)

Marian-Andrzej Obojski (VW)

Mariana Rakic (BMW)

Friedeman Kuhn (Daimler)

Project Co-ordinator

Aria Etemad

Telematics & Navigation Research

Ford Research & Advanced Engineering Europe

Phone: +49 241 9421 246

Fax: +49 241 9421 301

Email: [email protected]

Ford Forschungszentrum Aachen GmbH

Suesterfeldstr. 200

D-52072 Aachen

Germany

Copyright: euroFOT Consortium 2011

euroFOT 16.06.2011

Deliverable 2.2 Version 3.2 iii Report on the results of adaptation, in-vehicle implementations and piloting

Revision and history chart

Version Date Reason

0.01 2010-04-26 created by GSP (INRETS)

0.02 2010-06-01 Added inputs from POLI (Italian VMC) and German 1 VMC

0.03 2010-06-01 Adding an introduction from GSP

0.04 2010-06-23 Adding revisions from POLI and results from German 2 VMC

0.05 2010-06-28 Adding inputs from GSP

0.06 2010-07-22 Adding French and Swedish piloting results and various corrections by GSP

0.07 2010-08-12 Adding to VMC1 from CK

0.08 2010-08-17 Adding to VMC1 from VW (MAO)

0.09 2010-09-01 Adding to Italian VMC from LD and GSP

0.10 2010-09-27 Adding German VMC2 and French VMC conclusions.

1.0 2010-10-05 Finalization for internal review by GSP

2.0 2010-10-28 Addressing reviewer’s comments by GSP

3.0 2011-06-08 Deliverable revised according to the remarks from the EC

3.1 2011-06-16 Language check, formatting by FFA (jki,ck)

3.2 2011-06-16 Layout check, version control, accepted

euroFOT 16.06.2011

Deliverable 2.2 Version 3.2 iv Report on the results of adaptation, in-vehicle implementations and piloting

Table of contents

1 Executive summary......................................................................................................................... 1 2 Introduction ..................................................................................................................................... 2

2.1 Technical tests conducted during the project ...................................................................... 2 2.2 General considerations for the pilot phase .......................................................................... 3

3 Methodology ................................................................................................................................... 6 3.1 Identification of the most representative use-cases for each VMC ..................................... 6 3.2 Identification of representative scenarios ............................................................................ 7 3.3 Verification procedures ........................................................................................................ 7 3.4 Incidents .............................................................................................................................. 9 3.5 Conclusion on methodology ................................................................................................ 9

4 Technical functioning of the data acquisition systems in real driving situations ........................... 10 4.1 Verification procedures for logger unit (CO,CV,CVE) ....................................................... 11 4.2 Functionality verification procedures ................................................................................. 13 4.3 Post-processing related aspects ....................................................................................... 15 4.4 Piloting subjective data ...................................................................................................... 16

5 French VMC .................................................................................................................................. 17 5.1 Recall of the VMC .............................................................................................................. 17 5.2 Brief description of the data logging system ...................................................................... 18 5.3 Description of the upload centre ........................................................................................ 19 5.4 Description of the most representative use cases ............................................................. 20 5.5 Results of in-vehicle adaptation and piloting procedures .................................................. 21 5.6 Tested scenarios and incidents ......................................................................................... 26

6 German VMC1 (Ford, MAN, VW) ................................................................................................. 29 6.1 Recall of the VMC .............................................................................................................. 29 6.2 Description of the data logging system ............................................................................. 31 6.3 Description of the upload center ........................................................................................ 32 6.4 Description of the most representative use cases ............................................................. 33 6.5 Results of in-vehicle adaptation and piloting procedures .................................................. 35 6.6 Tested scenarios and incidents ......................................................................................... 40

7 German 2 VMC ............................................................................................................................. 43 7.1 Recall of the VMC .............................................................................................................. 43 7.2 Description of the Data Logging system ............................................................................ 44 7.3 Description of the upload centre ........................................................................................ 46 7.4 Description of the most representative use cases ............................................................. 47 7.5 Results of in-vehicle adaptation and piloting procedures .................................................. 47 7.6 Tested scenarios and incidents ......................................................................................... 52

8 Swedish VMC (Volvo, VCC) ......................................................................................................... 54 8.1 Recall on the VMC ............................................................................................................. 54 8.2 Brief description of the data logging system ...................................................................... 54 8.3 Description of the upload centre ........................................................................................ 55 8.4 Description of the most representative use cases ............................................................. 56 8.5 Results of the in-vehicle adaptation and piloting procedures ............................................ 59 8.6 Tested scenarios and incidents ......................................................................................... 61

9 Piloting subjective data, Italian VMC (CRF) ................................................................................. 65 9.1 Brief recall on the VMC ...................................................................................................... 65 9.2 Description of the upload centre technical characteristics ................................................ 65 9.3 Description of the most representative use cases ............................................................. 65 9.4 Procedures to be tested for questionnaires ....................................................................... 66 9.5 Development of piloting questionnaires ............................................................................ 66 9.6 Verification procedures used (See Annex 6 for details) .................................................... 67 9.7 Feedbacks from questionnaires ........................................................................................ 68

10 Conclusions .............................................................................................................................. 69 10.1 Technical problems encountered and solutions ................................................................ 71 10.2 Organisational problems .................................................................................................... 73 10.3 Lessons learned ................................................................................................................ 75

Annexes

euroFOT 16.06.2011

Deliverable 2.2 Version 3.2 v Report on the results of adaptation, in-vehicle implementations and piloting

List of Figures

Figure 1: Overview of the OEM, VMC, and associated technical details. ................................ 4

Figure 2: Examples of scenarios and verification procedures relationship .............................. 7

Figure 3: regulation systems schematic overview. ................................................................. 17

Figure 4: The two types of cars studied by the French VMC: The Clio III and the Laguna III. ............................................................................................................................................... 17

Figure 5. Low level instrumentation at French VMC .............................................................. 18

Figure 6. High level instrumentation at French VMC .............................................................. 18

Figure 7. Trip diary, completed by pilot drivers for each power on / power off cycle. ............. 21

Figure 8. Visible elements in French VMC instrumentation ................................................... 23

Figure 9 Overview of the ACC function (Ford, MAN, VW) with distance regulation which maintains a selected max. speed and/or constant time gap to the lead vehicle. ................... 29

Figure 11 Overview of the FCW function (Ford, MAN) with constant surveillance of time-to-collision to the lead vehicle and braking support. ............................................................... 30

Figure 12 Overview of the LDW, LA and IW function (MAN, VW) with surveillance of proper trajectory and optional assistance. .............................................................................. 31

Figure 13: SafeHMI schematic overview ................................................................................ 43

Figure 14: Communication of the data logger components .................................................... 44

Figure 15: ZGW connections .................................................................................................. 45

Figure 16: Architecture of DAS used by Daimler. ................................................................... 45

Figure 17: Trigger box used in the Daimler FOT vehicles ...................................................... 46

Figure 18: Schematic explanations for the systems studied at the Swedish VMC ................. 54

Figure 19: The steps that typically have to be considered when conducting an FOT, as explained in the FESTA methodology. The large arrows indicate the time line, and the blue ellipse indicates the steps validated by the pilot tests. ........................................................... 70

euroFOT 16.06.2011

Deliverable 2.2 Version 3.2 1 Report of the results of adaptation, in-vehicle implementations and piloting

1 Executive summary This deliverable provides an overview on the process and the outcomes of developing both, in-vehicle adaptation verification procedures and piloting tests to be used in the euroFOT project. It describes the work done during WP2500 to pilot testing the data acquisition chain in real driving conditions. Some parts of the document are related to and refer to WP2400 that has the goal the technical installation and adaptation of the data logging systems. The main results of WP2400 are in the form of installation manuals. These are not included in this report. They are, however, available in the document centre. This deliverable addresses organisational issues and describes the methodological process adopted during the euroFOT preparation phase of the experiment. It tries to provide guidelines on how to conduct harmonized small scale tests for heterogeneous vehicles and equipment. The different elements to be tested during an FOT pilot test are exhaustively described in a first part of the document together with templates suitable for reporting the results of the test. Due to technical and methodological differences between the large numbers of partners, the proposed methodology has been adapted and modified to suit the needs of the VMC’s. The results of the pilot test are provided by the means of VMC specific sections as each partner involved in euroFOT has to describe his own approach and results, and to explain the adaptations made.

For budget and time reasons, and due to the difficulties in the recruitment process, it was not possible to closely follow the FESTA approach; particularly for briefing and training the participants. The pilot tests were focused from the beginning to a rather technical approach. Indeed, the main difficulties encountered during the pilot tests were of technical nature and therefore closely related to vehicle’s embedded systems and level of equipment. All the technical issues were solved during the pilot test as this is a prerequisite for the large scale FOT to begin. Some organisational issues have been identified and adapted to the needs of the experiment. The whole process of piloting all the data collection process took more time than initially scheduled. This was because of the intrinsic complexity of the task.

The final conclusion summarizes the main results obtained by each VMC and gives feedback to the reader about lessons learned for future large scale FOT. The main issue with the euroFOT piloting tests was the difficulty to share a common methodology with so many different OEM using different sensors and equipment for a different purpose. Despite that, all the conducted pilot tests were successful in finding solutions and ensuring reliability of the data acquisition chain in real driving conditions. For many reasons, from confidentiality to practical feasibility, it is very difficult to report in details the tests conducted during the pilot tests. The piloting tests needed more time than scheduled by the FESTA methodology, and we recommend continuing piloting during the ramping up of the FOT in order to save time. This can be done by involving the same people in testing the data logging chain and technically following the experiments, and could ensure high-efficiency, smoother integration and decreasing overhead.

euroFOT 16.06.2011


2 Introduction This document is dedicated to the piloting phase (WP2500), while WP2400 work is covered by the installation manuals available on the document centre. The goal of the present deliverable is to report the results of piloting tests and procedures that have been conducted before starting the large scale experimentation.

Before the start of the pilots, preliminary tests have been performed by subproject SP3 regarding the performance, security, and reliability of systems. This was done to ensure that all technologies to be deployed would operate in accordance with the functional requirements and performance specifications defined in previous activities. These preliminary field tests have specific objectives and deal with different levels of analysis. These tests are not included in piloting and, therefore, will not be discussed here.

According to the FESTA handbook, the pilot test task should be performed early in the evaluation process. It represents an important step for the mobilisation and the dialogue between the various teams involved in an FOT.

In the subsequent phase, the following steps are considered:

The first set of tests aims to check the technical functioning of the data collection systems in real driving situations (quantitative data collection). They should enable to identify potential problems of sensor calibration or drift and thus to establish the periodicity of maintenance procedures during the FOT. They should also permit to validate the data collection procedure from data acquisition, data transmission to data storage.

The second level of preliminary field test deals mainly with the issue of assessing the usability and usage of the systems under study and of identifying the main critical issues associated with their use in real driving situations.

The third level consists of testing the feasibility of the overall evaluation process from the selection of the participants through the data collection. It was planned that the euroFOT pilot testing activity had to address the participant recruitment and briefing, and also the questionnaires issues (qualitative data collection). For practical reasons, some professional drivers (or colleagues from the institute/enterprise) have been asked to test the experimental protocol (briefing, explanations and testing of the vehicles) using the first equipped vehicles from the fleet. Such small drift from the FESTA methodology has been tolerated, but it is important to stress the fact that drivers involved during the pilot were not aware of euroFOT project.

The final aspect addressed during the pilots concerns the post-processing related aspects of the experiment. These aspects cannot be considered purely as a part of the data acquisition chain but more as the treatment phase, which is of course the main task of the analysis. These parts of the process include database management, data analysis software, performance indicators or derived measures computation, and answers to research questions.

This last part of the pilot test needed a finalised database structure and also sufficiently advanced data analysis tools. It was impossible to reach this level of precision as most of the algorithms and tools were not finalised before the end of the pilot. In this context, piloting served more as a tool to aid software development rather than a test bed. It has therefore been decided to minimize the importance of the pilot test for such aspects of the experiment and, consequently, this last part is not described in great details on the VMC results.

2.1 Technical tests conducted during the project

During the process of designing a large scale FOT experiment, many steps are needed to assess reliability of the whole data acquisition chain from a technical point of view but also

euroFOT 16.06.2011


from a purely organisational aspect. Different tests have therefore been scheduled through the whole preparation process.

• Small scale testing and validation (WP3600)

o The “small scale test” can be regarded as something like a “factory acceptance test”, a laboratory test of the whole data management chain covered in SP3. It has, from a purely technical point of view, tested the data chain from data acquisition in the data acquisition system (DAS), through the data upload and quality assurance procedures, to the database, to the analysis tool. It has validated the technical requirements, and the aim was not to test the systems’ suitability for study of the functions in “real life” as this is a matter for the formal pilot tests.

• Adaptation and in-vehicle implementation (WP2400)

o This work package concerns the technical installation and adaptation of the chosen DAS. The goal was to supervise the installation of the data loggers in vehicle prototypes with the help of the OEM and suppliers. Suppliers and OEM gave information about the CAN-bus and other function-specific needs. WP2400 also helped solving some problems encountered during the pilot test experimentation.

o These tests covered in-vehicle technical implementation of the material needed to collect data, and also the correct functioning of the system once integrated in the vehicles.

o Each partner was responsible for creating a detailed installation guide for equipping their fleet of vehicles with specified logging and sensing equipment. The guide describes in a structured way what and how to install so that a person responsible for the installation (e.g. a mechanic) can manage to install all the equipment with little or no help at all. Installation guides are available on the Document Centre in the “WP2400 Vehicle Implementation” folder.

• Piloting (WP2500)

o These tests have addressed real situations using the specific equipment. Based on technical specifications/documentation given by the DAS provider (SP3) and also on the SP4 inputs on the final experimental design, various verification procedures has been identified and tested in a small scale experiment.

It is important to point out again that only the pilot testing is covered by this deliverable.

2.2 General considerations for the pilot phase

The piloting activity is of much importance in the process of designing a large scale FOT since it is the final step before the real experiment. A wide range of objectives are covered by such a task:

• Test the entire data acquisition chain installed in a vehicle prototype driven in real conditions.

• Use verification procedures to identify the problems and solve them with the help of WP2400.

• Test subjective data collection, participant briefing and experimental design. More generally, test every aspects not directly related to objective data collection.

• Give feedback to the partners involved in the methodological aspects of the FOT in order to modify a potentially unfeasible method (research questions not precise

euroFOT 16.06.2011


enough, performance indicators (PI) not computable, lack of data or needed frequency not available, etc.)

• Identify and disseminate the best practices for piloting.

• Authorize the start of the large scale experiment after every necessary condition has been fulfilled.

Although various OEM’s with different systems and data acquisition equipments are involved in the euroFOT project, a common approach is needed to ensure the same quality level of the collected data and therefore the analysis to be conducted. This was one the major challenges of the pilot test activity. As one can see in Figure 1, the diversity of the partners involved made this goal very difficult to reach for both cultural and economical reasons.

Figure 1: Overview of the OEM, VMC, and associated technical details.

Each involved VMC was responsible for ensuring a good technical adaptation of the systems and a good piloting. Each VMC responsible person has therefore been in charge of collecting the verification procedures needed in his own VMC.

OEM Pilot test responsible person

Volvo Cars John-Fredrik Grönvall

MAN Walter Schwertberger

BMW Mariana Rakic

DAG Friedemann Kuhn

AB VOLVO Karsten Heinig, Marco Dozza

CEESAR Clément Val

Ford Christoph Kessler

euroFOT 16.06.2011


VW Marian Andrzej Obojski

CRF Gianfranco Burzio

Despite the complexity of the task, the piloting phase is a critical part in the whole FOT process, and therefore needs specific attention to ensure the good quality of the collected data. In practice, four main objectives have been addressed:

a) Develop a harmonised approach for the piloting methodology using common tools for pilot testing FOT equipment, methods, procedures and materials.

b) Deploy a small sample of FOT vehicles under a representative range of driving conditions that will be experienced in the FOT, as per the pilot testing protocol.

c) Fine tune FOT vehicles and technologies, systems, procedures and protocols, as required, on the basis of the pilot data yielded.

d) Signing-off the vehicles for pilot testing.

Part a) is covered by section 2 which proposes templates and a common language to assess reliability of the entire equipment.

Parts b) is related to methodology (identify use cases for example) but is mainly related to the identification of all the parts of the data acquisition chain that need to be tested during the pilot, with the insurance that nothing is forgotten. This is covered by section 3 which list and describe all the parts of the data collection process that may needed to be tested.

Part c) is not described in that document as it is a continuous process impossible to document precisely.

Part d) is addressed in the conclusion section by listing all lessons learned and some of the major encountered problems.

Sections 4 to 9 are VMC specific. They address the following points:

• Purpose of the VMC

• Brief description of the data logging system

• Description of upload center

• Description of the most representative use cases

• Results of in-vehicle adaptation and piloting procedures

o Suitability of installation procedures and quality control of installation

o Proper data acquisition, transmission and conversion

o Absence of driver annoyance in his/her own vehicle

o Recruitment, briefing and subjective data collection

• Tested scenarios and incidents

euroFOT 16.06.2011


3 Methodology Many discussions were needed to define the final methodology of the pilot test. This section presents the results of the approach developed and gives all the elements needed to describe correctly the conducted pilot tests. Some efforts have been made to share common vocabulary (see Annex 1 for a Glossary) and common templates (Annexes 2, 3, 4 and 5). The following elements represent the basis of the euroFOT approach to piloting and are described in this section:

• Use cases o Most representative use cases evaluated during the pilot.

• Scenarios o A tested driving situation defined by a set of verification procedures.

• Verification procedures: o Step by step description of the tests.

• Incident o Description of the circumstances and part of the equipment concerned by an

incident or a non-conform operation.

3.1 Identification of the most representative use-cases for each VMC

Each VMC needs to identify what are the most representative use-cases for each system tested. Identified use-cases should be listed by each VMC using a unique ID. The proposed use cases template can be found in annex 2. Below a real use case in the proposed template is presented.

Example (German 2 VMC):

ID Description General description of the use case

USE_SafeHMI_02 Driving on route with congestions

System and vehicle requirements

System status Possible states of activation of the system

System activity Possible activity of the system

Vehicle characteristics Test vehicle

Interaction between systems Possible interaction with other in-vehicle systems

Activity occurrence Define how often the system intervenes

ON-OFF ACTING (giving route information ) NOT ACTING CONSIDERING TRAFFIC INFO (dynamic routing) NOT CONSIDERING TRAFFIC INFO

Depends on route

Environmental requirements

Traffic conditions Required traffic condition

Environmental situation Required lightning, weather, visibility conditions

Road characteristics Required road characteristics

Geographical characteristics Required geographical characteristics relevant for testing the systems

High traffic density No specifications Different ways to No specifications

euroFOT 16.06.2011


destination are possible Comments

Familiarity is assed via button press for Daimler and via trip diary for BMW

3.2 Identification of representative scenarios

In addition to the use cases, it has been suggested to describe some scenarios that are representative of the activities scheduled in the VMC. Writing down some representative scenarios may be useful to assess if an activity is totally checked from the beginning to the end. For example, if the data analyst wants to use one of the performance indicators (PI) to answer one research hypothesis, it is necessary to check the possibility of computing the PI from the variables, then to check if those variables are present in the database, in the correct format and precision, and if the values are accurate. Doing this, it may be easier to discover some gaps in the complete process of trying to perform a single action. Each scenario will therefore be associated to a flow of necessary verification procedures (VP). In order to check if the scenario is a success, each VP needs to be checked separately. The proposed template to describe briefly the scenarios is given in Annex 3. The following graph gives an idea of what is meant.

Figure 2: Examples of scenarios and verification procedures relationship

3.3 Verification procedures

The verification procedures (VP) describe a single test, for a single element of the data acquisition chain. There is a need for these tests to be described step by step. Conditions of acceptation and use cases also need to be detailed precisely.

euroFOT 16.06.2011


In order to facilitate the work and to ensure harmonised presentation a common template has been proposed which can be found below and in Annex 4.

Verification procedure template:

Unique id Creation date Creator Summary (name)

[VP_x] [yyyy-mm-dd] [Creator of the VP.] [Short name of the verification procedure]

Importance level Version Stakeholder Type of material concerned

[1 (high), or 2 (low)] [0.1] [VMC1/VMC2/VMC3] Indicate which part of the data acquisition process is concerned

Description of the use case

Describe the situation in which the material is supposed to do something.

Description of the functionality to be tested

Describe which aspect of the material will be tested. What the material should be able to do in the situation identified below (use-case). This part will need some technical specifications for the material concerned which may not be available at this time. It is the responsibility of the VMC/OEM to get this specific information.

Description of the verification procedure

Describe step by step the verification procedure.

Importance level: 1: Procedure of such an importance that leads to a GO/NOGO condition.

2: Procedure that can be modified in the following way: delete, simplify or improve.

From the detailed list of procedure, it has been proposed to summarize them in the form of a validation plan (excel template provided in Annex 5). This validation plan is in the form of a checklist that should include the main information of the procedures, and can be used for reporting the results of the testing procedures and the envisaged solutions. Example:


VP_01 2008-03-22 G. Saint Pierre File creation when the vehicle is started


1 0.1 French VMC Data logger

Description of the use case

The vehicle engine is started as usual, and the driver starts to drive.


The Data logger should create a file containing all the measures in the correct format (this particular aspect will be the subject of another verification procedure). The file should be created if the engine is on during more than 30 seconds, or if the car is going further than 500 meters.


- Start the engine less than 30 sec. - Shut down the engine - Check if the file exist on the system - Start the engine more than 30 sec. - Shut down the engine - Check if the file exists and if the collected parameters are correct. - Start the engine and drive more than 500 meters and less than 30 sec. - Shut down the engine

euroFOT 16.06.2011


- Check if the file exists and if the collected parameters are correct. - Start the engine and drive less than 500 meters and more than 30 sec. - Shut down the engine - Check if the file exists and if the collected parameters are correct.

3.4 Incidents

All detected incidents or non-conform events need to be saved in a traceability tool to track the solutions that took place until the problem is solved. Two types of documents are necessary for that:

• Incident definition

• Failure report and tracking

The incident definition template is given at Annex 5 while an example of what can be an xls document for tracking is given in the next Figure:

3.5 Conclusion on methodology

This common framework has been constructed by the partners in order to satisfy all VMC’s needs. However, the work has been organised so that each of the VMC could maintain its own specifics, either due to the functions tested, the materials used, or the database structure chosen. This proposed framework was used by each of the VMCs as a basis to develop their own verification and testing procedures and templates1.

1 For confidential and technical reasons, in-depth verification procedures are excluded from this report.

euroFOT 16.06.2011


4 Technical functioning of the data acquisition systems in real driving situations

Some of the equipment installed in the vehicles is common among the partners. That includes the central logging unit (data acquisition system), video recording system and positioning system. In addition to this each partner have their own set of sensing systems, both sensors already available in the vehicles and additionally installed sensors. Below is a list with short descriptions of the common equipment setup.

• Data acquisition system (DAS): Logging unit that will collect and synchronize data delivered both by factory and additionally installed sensing systems. DAS shall also have wireless communication capabilities for uploading data to a central server as well as having the possibility to remotely control the unit. DAS shall be specified for automotive usage to manage any expected environmental impact.

• Video logging system: Several video cameras will be installed covering both internal and external views.

• Positioning system: A GPS system will log the position of the vehicle whenever it is in use.

• Extra special sensors: Some of the vehicle fleets will be equipped with special sensors.

o Eye/head tracking system (for monitoring the state of the driver’s head and eyes concerning e.g. position, gaze direction and eyelid opening)

o Lane tracking system (lane position, lane width etc.)

o Radar/lidar sensors (for monitoring the area around the vehicle)

In this section all the identified main categories of verification procedures are described. Some of them will be common, and some will be VMC specific. This section’s aim is to list all the possible categories of data acquisition chain parts that may eventually be relevant for the VMC’s. Verification procedures will be grouped so that there may be clear division between CAN only (CO) loggers, CAN+Video (CV) loggers, and CAN+Video+Extra sensors (CVE) loggers. In addition, to the processes and procedures identified in this report, there might be other procedures, requirements or methods that are VMC dependent. On the other hand, for different reasons, some of the processes and procedures listed here might not be needed for some VMC’s. Below are the identified categories that will be detailed in the next sections.

a) Verification procedures for logger unit (CO,CV,CVE) a. Mechanical b. Electrical c. Operational

i. Start up of operational verification application ii. Vehicle data iii. Audio logging (CVE) iv. Positioning systems (CO,CV,CVE) v. Driver/vehicle metadata (CO,CV,CVE) vi. Video logging (CV,CVE) vii. Driver monitoring (CVE) viii. Head/eye tracking equipment ix. Feet proximity sensors x. Non-video environment data sources (CVE)

euroFOT 16.06.2011


xi. Extra sensors (CVE) d. Driver interaction with the logger e. Acquisition of vehicle bus data

b) Verification procedures for logger sensing a. Functionality verification

i. Functionality verification - Standing still ii. Functionality verification - Driving iii. Specification of common needs for data logging

b. Data transfer i. Wireless ii. Manual

c. Upload centre i. Formatting ii. Synchronisation and timing iii. Data quality iv. Data storage

c) Post processing related aspects a. Database management b. Data analysis software c. Performance indicators and derived measures computation d. Answers to research questions (function specific)

The following verification procedures will make use of tracking tools and traceability tools. For example, to know what type of problem is encountered for certain vehicle, to know the frequency of those problems, to know the associated software or system configuration, etc. (See incidents related sections).

The installation handbook has to be tested. That means one of the first verification procedure will be to install the Data acquisition chain following the handbook in order to check if everything is understandable by the final user.

The next sections give a short summary of each identified category of verification procedures together with some reasons to understand why they are needed.

4.1 Verification procedures for logger unit (CO,CV,CVE)

This part concerns all the possible verification procedures related to technical aspects of the data logger.

Mechanical

When all units have been installed and correctly mounted according to installation guide the installation responsible (mechanic) shall verify that no units are loose and that they are securely mounted with enough space from other units to prevent physical damaged during driving. This shall be done by inspecting the mounting equipment (straps, screws etc.), both for correct connection of the mounting equipment and secure fitting by trying to move the units by hand.

Electrical

For electrical verification the mechanic shall make a first verification to check that all units are correctly connected and the power is according to specification in the installation guide. Any units equipped with a LED for power status shall be checked.

euroFOT 16.06.2011


Operational

The operational verification procedure will be semi-automatic. The DAS unit will have an operational verification application preinstalled which will verify the communication from units connected to the DAS. This verification can be done indoors in the workshop with the vehicle standing still and the engine running. The application will guide the mechanic through the steps and check the basic communication for all sensing units and report any detected fault. There shall also be a list of suggested actions to take in order to repair the fault.

Start up of operational verification application

A monitor, keyboard and mouse shall be connected to the DAS unit. The unit shall then be started (e.g. by starting the engine). Once start up is finished the operational verification application shall be started by the mechanic and the verification steps will start automatically. When the application has tested the communication with every connected sensing unit, any detected fault shall be reported to the mechanic. The information about the fault shall be as detailed as possible in order to make it easier to locate and repair.

Vehicle data

The operational verification application shall verify that all vehicle bus signals (e.g. CAN, J1587, LIN) are reaching the DAS unit in a correct way.

Audio logging (CVE)

The operational verification application shall test the audio connection and check that the sound level is correct.

Positioning systems (CO,CV,CVE)

The operational verification application shall verify that the connection with the GPS unit is working properly. All signals such as global position might not be available due to no satellite connection but other signals which are active shall be verified (e.g. time or counter).

Driver/vehicle metadata (CO,CV,CVE)

The operational verification application shall verify that driver ID and vehicle metadata are reaching the DAS unit.

Video logging (CV,CVE)

The operational verification application shall verify that video data from all cameras are reaching the DAS unit in a correct way. The application shall also present all videos live to the mechanic. This is due to even though the video signals are detected, the video image could be black because of other than communication related issues (e.g. lens hood still present) and the mechanic should still be able to detect this. The mechanic shall also verify that the video images are displaying the correct field of view according to specification. Any detected communication related fault shall be reported to the mechanic. In case of faulty camera field of view the mechanic shall adjust the specific camera until correct filed of view is achieved.

Driver monitoring (CVE)

The operational verification application shall verify that driver monitoring related data is reaching the DAS unit in a correct way.

euroFOT 16.06.2011


Head/eye tracking equipment

The operational verification application shall verify that eye tracking data is reaching the logging computer in a correct way. Not all eye tracking data will be available since no person is present in the driver’s seat but other signals which are active shall be verified (e.g. time or counters).

Feet detection sensors

The operational verification application shall verify that feet proximity data is reaching the DAS unit in a correct way.

Non-video environment data sources (CVE)

The operational verification application shall verify that data from non video environment sensors (radar, lidar etc) are reaching the DAS unit in a correct way.

Extra sensors (CVE)

The operational verification application shall verify that data from any additionally installed sensors are reaching the DAS unit in a correct way.

• Start-up and shutdown procedures

• Malfunction management, self diagnostics

• System status and health reporting (wireless, automatic)

• Remote administration/software upgrades and maintenance

• Periodic automatic transfers of subsets of collected data for test fleet supervision and data quality assurance

Driver interaction with the logger

In the case that such possibilities are given to the driver, it is necessary to ensure a good understanding of what to do.

Acquisition of vehicle bus data

The goal is to check simply the first level of vehicle bus data acquisition, without any processing of the raw signal.

4.2 Functionality verification procedures

Functionality verification

Once the basic operation of the system has been verified, a deeper verification of the system’s functional performance will be done. This verification is divided into two sections. One part while the vehicle is standing still and the other part while driving along a test track with known characteristics. The two parts include scenarios with several steps that shall be performed according to a specified procedure. Every step will be logged and later-on analysed where the result will be compared with what is expected. The analysis will give an answer on how well the systems are working and if there is any need for adjustments among the sensing equipment. Which steps to be performed are dependent on the type of vehicle and on the type of equipment the vehicle has installed.

euroFOT 16.06.2011


Functionality verification – Standing still

The first section of the functional verification is done with the vehicle standing still. The user (e.g. mechanic) shall have a printed copy of the correct procedure and document any deviation from the procedure directly in the comment field. The standing still procedure shall be performed at the same position for all vehicles within the same fleet in order to have equal environmental conditions (and also have the possibility to trigger the start by GPS position).

Functionality verification – Driving

Close to the work shop where the installation is done there shall be a test track where the driving part of the functional verification shall be performed. This shall be done directly after finishing the stand-still functional verification. The test track can be an ordinary public road with a shape that suites the specified verification procedure. For instance if one step shall be to drive through both right and left curves with a specific curvature, obviously, there need to exist such curves. Minor changes in the functional verification procedure are acceptable to adapt the procedure to existing and accessible roads. Though, it is essential that these changes do not risk the complete functional verification of the logger and sensing systems. Any deviation from specified procedure shall be documented by the user and later on considered when analysing the log data.

Specification of common needs for data logging

The exact procedure will vary depending on the type of vehicle and installed logger and sensing equipment. Some functionality is common between the different vehicles and shall be included in each functional verification, both for stand-still and driving scenarios. The table below specifies those functionalities.

No. Functionality Comment

1 At least two curves, both for left and right, with a curvature of at minimum XX.

Verify active steering wheel angle, lateral acceleration.

2 Good satellite connection for GPS positioning. There shall be at least five known GPS positions evenly spread out along the test track.

GPS positions

3 A section of straight road (>250m). Longitudinal acceleration

4 Lane markings with good visibility on both left and right hand side.

Activate lane tracking system, if present

5 Other vehicles present in same lane within 30m, which can be followed for > 5 seconds

Activate radar sensors, if present

6 Test track shall be as long as a change in fuel level at start and at the end can be separated.

Econometer

Data transfer

Data transfer is crucial and has to be verified by testing various scenarios. This holds especially for wireless transfer because of some possible drawbacks (tunnels, weakness of the signal.

euroFOT 16.06.2011


Upload centre

Once data have been uploaded to the VMC server, and after checking the presence of the data, necessary details for the needed quality of the database will be tested.

Formatting

Is the variable available in the correct format, size and type (Boolean, integer, real, how many digits etc.)?

Synchronisation and timing

Different sources of data will need to be time-stamped. This can be checked by searching for incoherent relationships among data. Acceleration and speed are a good example of this kind of relationship, when the value of the acceleration is positive the signal of speed should increase, otherwise there is an incoherent relationship between those signals that may be due to a bad synchronisation.

Data quality

A first level of data quality assessment will be done during the transfer, or just after. This step is a part of SP3 work on the data acquisition chain and has to be verified. According to the different data types, verification procedures should be provided for the following items:

• CAN Data

• Video Data

• Extra sensors data (eye tracker, radar/lidar, Feet proximity)

Each measure collected should have its own verification procedure or at least verified separately. Some procedures may apply to various measures depending on the type (for example, Boolean, categorical, ordinal or real).

Data storage

This aspect is mainly related to the size of the stored data. Will the server hard disks space be sufficient for one year of experiment? Are regular backups considered?

4.3 Post-processing related aspects

This last part of the pilot tests is related to the post processing of the acquired data. This ranges from the storage in the database to the research questions. This part is the most critical of the process, as many aspects of these tasks were not clear when pilot starts. It is a long and complex task to define and build such high level tools. The pilot test helped a lot in building such tools or verifies the PI computation and event detection, but this process is continuous in time during the whole experiment. Most of these tools needed the data acquisition chain to be tested and to operate in a correct way before starting their construction. This is the main reason why the VMC did not have the possibilities to entirely test these tools. Considering subjective data, it was possible to validate their acquisition during the pilot, at least for the Italian VMC who did not had a data logger unit, but only questionnaires. The other VMC provide less detailed verification procedures for this aspect.

euroFOT 16.06.2011


Data analysis software

A number of tools are necessary for the high-level data analysis. They are being developed by the euroFOT partners (mainly in SP3) before and during the experiment. Some others may be provided by external partners and may not need to be verified as they are already in use. Three kinds of tools have been identified:

• Tools from SP3: Maintenance checks, data reception quality, visualization.

• External tools: e.g. enrich data with road classes from digital map.

• Tools from SP6: Hypotheses testing, impact assessment etc.

The tools developed in SP3 should have been tested extensively in various conditions of use. Other tools are known to be working but it is necessary to verify that they give the expected results. For the global assessment tools (e.g. impact assessment relying on results from traffic models), the only thing to do is to check if all variables necessary for the assessment are collected. Most of the tools presented here were not ready when the pilot started, except the commercial ones (statistical analysis tools for example). It is not realistic to test all the tools related to data post processing during the pilot.

4.4 Piloting subjective data

This section is devoted to all the technical aspects related to subjective data collection. Even a questionnaire has to be tested on a small scale before submitting it to the final set of subjects. Many aspects are specific to this data type.

The piloting phase contributes to define many of the test details and of each specific test sub-phase. Questionnaire administering modalities (e.g. paper- web- or mixed- ways) have to be checked. These will influence the exact time span in which questionnaires have to be filled (short for interviews, long for paper) and could affect drivers’ availability, and such. All such aspects need to be analyzed and have to be administered according to piloting findings.

The following main procedures can be tested during piloting tests:

• driver recruitment and contact;

• questionnaire administration and submission;

• driver liaison centre and operational procedures defined in SP5;

• gathering of paper-based questionnaire;

• questionnaire digital version and web-based tool (i.e. Limesurvey);

• data entry in Limesurvey;

• data storage;

• data export and processing for data analysis.

euroFOT 16.06.2011


5 French VMC

5.1 Recall of the VMC

The French VMC, CEESAR is in charge of all operational aspects related to the execution of the FOT. CEESAR works closely with Renault, which contributes by providing the necessary technical information and supporting the recruitment process. 35 drivers are recruited to study the use of the Speed Regulation System (which is a combined system including the Speed Limiter-SL and a standard Cruise Control-CC, see Figure below).

Figure 3: regulation systems schematic overview.

Drivers are selected according to their car possession and the area of driving. They are asked to drive around the west of Paris, and must use one of the two following cars: Clio III or Laguna III (see Figure below).

Figure 4: The two types of cars studied by the French VMC: The Clio III and the Laguna III.

The subjects’ own cars are instrumented with a CAN, GPS and Radar data acquisition system. They will also drive during some periods an identical vehicle owned by CEESAR.

euroFOT 16.06.2011


Those vehicles have a more complete instrumentation, which adds video logging, lane tracking and eye-tracking on top of the standard equipment (please see the next section for details).

Piloting tests of the instrumented cars have been made by more than 15 people driving the west region of Paris and collected more than five thousand kilometres of data on various conditions.

The pilot tests at the French VMC have started in December 2009 and were finished at the end of May 2010 (6 month).

5.2 Brief description of the data logging system

The data acquisition system used at the French VMC is a modular system. It consists of a datalogger built by CTAG and several optional equipments, including a CEESAR designed videologger, which are described in report D3.1. Two instrumentation levels are used, the low level for the subject’s own cars, and the high level for CEESAR cars (see Figure 3 and 4, and Table 1).

Figure 5. Low level instrumentation at French VMC

Figure 6. High level instrumentation at French VMC

euroFOT 16.06.2011


Low Level High Level

Vehicles used 35 drivers’ owned vehicles.

5 vehicles owned by CEESAR and lent to participants

CTAG datalogger 2 Max 4 CAN Channels GPS GPRS data transfer

● 2 channels used

● ●

● 4 channels used

● (not used : manual

transfer)

TRW AC20 radar (not part of standard vehicle equipment)

● ●

VideoLogger (custom made, H.264 encoding)

●

Cameras (B&W, SuperHAD Exview)

4

Mobileye AWS (added, with special firmware)

●

Smarteye Eyetracker ● Table 1. Details of the low and high levels of instrumentation.

5.3 Description of the upload centre

Data is transferred by different means to a central storage, hosted on an application server called CMS (which stands for Central Management System). This server is a Windows Server. It is a high performance machine with a RAID storage array, backed-up on a regular basis.

Video data is transferred to the server by a network connection, when the highly instrumented vehicles are at CEESAR.

Digital data (CAN and extra sensors) are preferably transferred by GPRS, using the Upload Processor and Fleet Manager developed by IKA. However, depending on the instrumentation level, the travel patterns (the mix of trips length) and, most crucially, on the GPRS reception levels on the site where the vehicle is most often parked, remote transmission of data will not always be possible. Therefore, when on-line monitoring shows that an SD card is getting constantly filled, an on-site retrieving of data (SD exchange) will be programmed, and data from the SD will be directly transferred to the CMS server.

Once the data is transferred to the server, it is converted and enriched using a software with a similar architecture as the German 1 VMC. The main difference is that data processing is not passed to other machines in a computer array, but all done locally on the CMS server. This is possible because of the smaller dataset (5 times less data in the French VMC than in German 1 VMC), and the machine’s configuration (2 Hexacore Xeon processors and 24Gb of RAM).

Once converted and enriched (with additional derived metrics, detected events, computed performance indicators), data is stored in a structured way into to a relational database hosted on a dedicated server, the CDS (Central Data Storage).

When data from a specific trip is required to do new observation or calculations, it is loaded back as a file from the CDS to the CMS.

euroFOT 16.06.2011


5.4 Description of the most representative use cases

The use cases to be covered in the pilot mainly consisted of the following: baseline driving in any kind of context: this means using the Cruise Control on motorways and using the Speed Limiter in urban areas. Each use case varied according to certain situational variables such as traffic conditions or weather. The main use cases for the French VMC are described in Table 2 and 3.

ID Description

USE_Urban Urban driving


System status System activity Vehicle characteristics

Interaction between systems

Activity occurrence

SL: ON-OFF Or CC: ON-OFF

ACTING (i.e. limiting or maintaining throttle) NOT ACTING

manual / automatic gearbox

Continuously


Traffic conditions Environmental situation

Road characteristics Geographical characteristics

Any traffic conditions Any weather

Urban area (roads with frequents variations of maximum legal speed limit or that need frequent adaptation of the vehicle speed)

No specifications

Comments

This use case is more likely to be associated with the SL usage Table 2: Urban driving use case.

ID Description

USE_Motorway Motorway driving


System status System activity Vehicle characteristics

Interaction between systems

Activity occurrence

SL: ON-OFF Or CC: ON-OFF

ACTING (i.e. limiting or maintaining throttle) NOT ACTING

manual / automatic gearbox

Continuously


Traffic conditions Environmental situation

Road characteristics Geographical characteristics

Free flow Any weather Motorway (Roads without variations of maximum legal speed limit or that do not need frequent adaptation of the vehicle speed )

No specifications

Comments

This use case is more likely to be associated with the CC usage. Another use case is congestion on motorway which is not likely to be in favour of the system usage.

Table 3: Motorway driving use case. Since these two use cases cover pretty much any kind of situation, it has been decided that instead of searching to create specific driving situations, pilot drivers would be free to drive in any kind of context, and use the studied systems as they wish.

euroFOT 16.06.2011


This is a very stochastic approach, made possible by the length of the pilot (6 months, 5 vehicles, 15 different drivers) and the quantity of data extracted from it and analyzed (more than 300 individual trips, 150 hours of driving / 5 000 km). Moreover, driving conditions (traffic, weather, road type) have been controlled by using a trip diary that pilot drivers had to complete systematically when using the pilot cars (see Figure 5).

Figure 7. Trip diary, completed by pilot drivers for each power on / power off cycle.

Using data from those trip diaries, systematic verifications could be carried out on a large number of trips. Also, using data from trip diaries, some specific situations corresponding to potential issues with specific equipments were identified and more thoroughly analyzed. These included, for instance, the following: long trips, parking in bad reception area, bad weather conditions...

Along with this stochastic approach (including systematic and targeted verifications), some simulated situations were created on purpose in a normal driving environment to perform more specific tests. The aim of those created situations was to observe the behaviour of the data acquisition system in cases that were not likely to happen in a few thousands kilometres of pilot, but could still happen during the FOT. These included stress tests, unusual driving manoeuvres and unusual power supply cycles (successive engine stalls for instance).

Finally, other use cases covered more organizational aspects of the FOT, such as initial installation of DAS in vehicle and interaction with participants (recruitment, briefing, digital questionnaires...).

5.5 Results of in-vehicle adaptation and piloting procedures

The aim of the pilot test is, complementary to short scale and in-the-lab tests, to validate the suitability of the whole «FOT system» to carry out the experiment. The biggest part of this «FOT system» is of course the DAS and data processing infrastructure. However, many other aspects of the experiment could be serious pitfalls if not taken seriously. That’s why,

euroFOT 16.06.2011


even if technical suitability of the equipment has been the main concern, those other aspects (organization for instance) were piloted as well.

All the items which were tested using verification procedures during pilot help build a general checklist. The answer to each line of this checklist could be yes, no, or acceptable. In the latter case, an acceptable result meant that problems related with the item still existed at the end of the pilot, but that an acceptable workaround could be found. This checklist is too long and complex to be reproduced here, but the most important items, how they were assessed, and the main results of the testing are detailed in the next sections.

Suitability of installation procedures and quality control of installation

Installation procedures regarding low level equipments have been thoroughly defined then tested and refined during pilot. They are described in “Vehicle Installation Guidelines for French VMC” document. They include thorough testing of the equipment to be installed before and after installation in the vehicle.

They are implemented in such a way that a team of two technicians from CEESAR can completely equip and validate the installation within half a day, which is compatible with both logistics and resources constraints.

No such precise installation guide is defined for the 5 higher level instrumentation vehicles, as they are rather handled as “prototypes”. However, the same installation guide applies for the common devices in both instrumentation types. For the pilot tests the additional elements have been totally integrated in 2 vehicles (1 Clio and 1 Laguna), and partially integrated in the other 3 vehicles. Those 3 vehicles were completed later by the same technicians as the first 2, using their initial experience as a basis to quickly replicate the installations.

Installation of DAS in all vehicles will be performed by CEESAR technicians in our own workshop. Piloting consisted of instrumenting the last three vehicles following the operator’s procedure presented before, in real condition. A third person measured the duration of each task and took note of every little glitch during the operation, while the two operators remained focused on the installation. Modifications were done to the installation guide using the outcomes of this part of the pilot. It is now ensured that vehicles can be instrumented in the time planned, and with the expected level of quality.

Great care has been taken to ensure a very inconspicuous integration of the equipment. Figure 6 presents the elements which are visible. Note that in participants’ own cars with “light” level of instrumentation, only the radar is visible.

The safety of the DAS for the vehicle has been assessed in two different ways:

• During the pilot, after thousands kilometres, no battery drain was observed, no service warning was displayed in vehicle or recorded in their diagnostic memory.

• The CAN activity of instrumented vehicles has been recorded by RENAULT to assess its conformity to their specifications and the absence of influence from the datalogger.

euroFOT 16.06.2011


AC 20 radar

Cockpit Camera(in roof light)

Front Camera

Mobileye

Feet Camera

Eyetracker

Eyetracker

Face Camera

Figure 8. Visible elements in French VMC instrumentation

Proper data acquisition, transmission and conversion

This technical part, although it complements systematic tests carried out in the lab (WP3600), has been the biggest part of the pilot. Three different ways of assessing the equipment and software efficiency, corresponding to three different kind of use cases described above, have been used:

Observing DAS behaviour in rare events purposely simulated

The Intentional creation of strange patterns in ignition cycles allowed to identify several problems in the reliability of the data acquisition system. For instance, this is how a lack of reliability was found in the embedded PC used to log video data. This PC is supposed to throw an event to the application (i.e. give a defined error signal) when its ignition entry goes from on to off, in order to allow the software to shut down its activity before requesting the operating system to properly stop. This feature didn’t work in some patterns, which were later found in «normal» driving as well. Hence, the video logging software was modified accordingly to regularly read the ignition input, and generate its own «shutdown» event instead of using the hardware generated one.

Searching for issues likely to happen in specific situations found using the driver’s diary in the pilot database

Some situation can more likely result in issues regarding different items in the logging system. Using the drivers’ diaries, those situations were searched for in the pilot database and the behaviour of the potential impact to the system was accordingly observed. Examples are listed in the table below:

euroFOT 16.06.2011


Situations Potential issues in...

Long trips Data transfer

Post-processing

Parking in bad reception area Data transfer

Bad weather condition GPS reception

Radar measure

Video image quality

Lane tracking measure

Harsh illumination (snow, low sun...) Video image quality

Curvy road, or road with important slope Radar measure

Lane tracking measure

High buildings, tunnels GPS reception

Table 4: Driving situations and their linked potential issues on data logging. This approach allowed the identification and correction of a few problems and, more important, confirmed the validity of potentially risky technical choices in adverse conditions. For instance, the fact that the GPS antenna his hidden under a layer of plastic instead of being exposed on top of the roof doesn’t seem to affect the GPS reception significantly (comparisons were actually made, with two separate dataloggers in the same car, one having the antenna on the roof and the other in the chosen, hidden configuration). The choice of rather inexpensive general purpose cameras was also validated in high contrast situations.

Systematically, for each trip, verifying the reliability of the DAS

More than three hundred trips (including several hundred kilometres trips and a few seconds long parking manoeuvres) were thoroughly analyzed using the verification procedures listed below:

• VP_1: A trip folder containing CAN and GPS acquisitions files has been created on the SD card.

• VP_2: The datalogger connected to the CMS server and transferred data from trip start time (as noted by the driver in the diary) to trip end time + transfer timeout.

• VP_3: The trip folder is completely transferred to the CMS server after a reasonable amount of time.

• VP_4: The trip can be converted into a single Matlab standardized Trip file.

• VP_5: There has been a continuous acquisition of data for the duration corresponding to trip start time and trip end time (as noted by the driver in the diary).

• VP_6: GPS acquisition is continuous and starts a reasonable amount of time after departure.

• VP_7: GPS signal fits correctly over a route on digital map.

euroFOT 16.06.2011


• VP_8: Last GPS position corresponds to parking place identified by driver in diary.

• VP_9: GPS speed signal is correctly synchronized with CAN speed signal.

• VP_10: All CAN channels and signals are correctly acquired.

• VP_11: Unusual values on CAN signals can be explained.

• VP_12: Special events or attributes of the trip noted as remarks in the diary can be observed on signals.

• if car has video:

o VP_Vid1: A video trip folder is created.

o VP_Vid2: A list of video files is created for each channel.

o VP_Vid3: Each file in the list except the last one has the specified maximum acquisition length.

o VP_Vid4: Playing sequentially the list of file for each channel results in a continuous video stream of the trip duration.

o VP_Vid5: Files sizes per minute are inferior to maximum mean rate.

o VP_Vid6: For each channel, for the duration of the trip, video quality is sufficient (focus, luminosity, contrast).

o VP_Vid7: Some events which can both be observed on the signal and the video (e.g. braking) show good (within a frame) synchronization between the two.

The table below present an example of how the verification procedures are described using the proposed templates.


VP_08 December 2009 C. Val Last GPS position


1 (high) 1 French VMC GPS, Data logger, Data transmission

Can be automated Tools used during verification

no Visualisation tool software or manual check

Description of related scenarios

All scenarios except the events related ones


Last GPS position corresponds to parking place identified by driver in diary.


- Manually check the driver diary to identify the parking place location. - Identify the GPS coordinates of the parking place location. - Identify the last GPS coordinates transmitted by the logger (they should be repeated many times before

ignition is off). - Compute manually or using a simple script the distance between the two GPS positions. - If the computed distance is less than 25m, then VP is validated for that trip.

Table 5: Example of a verification procedure used in the French VMC. This approach allowed the identification of the largest number of problems. It allowed to iteratively improve the whole system to a point where its performance and the workarounds to the issues were acceptable.

euroFOT 16.06.2011


Absence of driver annoyance in his/her own vehicle

It is crucial for an FOT that the driver accepts his/her own vehicle to be instrumented, and that he/she totally forgets about it once it is done. Hence, no visual or auditive annoyance should result from the instrumentation. For the low level instrumentation, as the equipments are totally invisible inside the car, no visual disturbance is to be deplored. A few noises and rattles were heard in one of the pilot cars, but it was possible to go down to their origin and eliminate them.

It has been also experimented that standard maintenance operations of the vehicle in a shop didn’t cause any trouble: the instrumentation remained invisible to the previously uninformed technicians who serviced our pilot cars (although the datalogger is plugged on the standard OBD plug, this connection is hidden beneath the panel and a derivation is made to present an identical unoccupied plug in the usual location).

Finally, pilot cars were analyzed by experts for perceived quality at RENAULT, who confirmed that, after instrumentation, the car remained within tolerance concerning their requirements.

Since the instrumentation in high level instrumented vehicles is more intrusive than in participants’ personal cars, a minimal annoyance could be accepted. Subjective ratings of the pilot vehicles showed that the small visual masking’s caused by the face camera and the eyetracker unit were perfectly acceptable. However, some drivers complained about dry eye, resulting from the eyetracker’s IR illumination, although they apparently got used to it over time.

Recruitment, briefing and subjective data collection

Briefing and recruitment was piloted by role playing in front of uninformed colleagues. This resulted in training for our staff and many suggestions from the attendees, which could be used to improve the presentation and guidelines which are prepared for these phases.

The piloting of the questionnaires has been done in two steps: first the content, then the electronic implementation.

First, using English paper-based questionnaires, feasibility and acceptance were tested with uninformed colleagues speaking English with a good level (5 persons). This allowed several modifications and optimizations to be done collectively with all the partners on the content of the questionnaires.

Then, the final questionnaires have been translated by ERTICO. The translation has been assessed and a few corrections made by our team’s psychologist.

Finally, although all questionnaires will be completed during meetings with participants at CEESAR, an electronic version has been prepared using a web-based system (LimeSurvey), so that participants can directly enter their answer in electronic form.

The final part of the pilot was to ask several persons (5 persons different than the others) to complete first the paper based questionnaire, and then carefully enter the same answers in the electronic version. After that the coherence of the initial paper questionnaires with the Excel file exported from the electronic survey could be confirmed. Questionnaires took between 5 and 15 minutes to be filled by the subjects.

5.6 Tested scenarios and incidents

As explained in the methodology part of the deliverable, a quite different verification has to be done for a single trip depending on the road conditions. Different verification procedures need to be checked sequentially to ensure a proper validation of the data acquisition. Along

euroFOT 16.06.2011


the verification procedures, some incidents may be detected and need to be treated separately to solve the corresponding issue.

Here is the scenario corresponding to a usual recording of the trip data under normal conditions (urban driving or motorway driving use cases)


[SC_1] December 2009 C. Val Systematic DAS verifications

Version Stakeholder Type

1 French VMC Real driving

Environment

Data is recorded for an entire trip, on any type of roads, without adverse weather condition and under free flow traffic.

Preconditions

Scenario may vary according to the trip duration

Description of the main scenario

For every normal trip (e.g. urban driving and motorway driving use cases), systematic verification procedures needs to be checked.

- VP_1: A trip folder containing CAN and GPS acquisitions files has been created on the SD card. - VP_2: The datalogger connected to the CMS server and transferred data from trip start time (as noted by

the driver in the diary) to trip end time + transfer timeout. - VP_3: The trip folder is completely transferred to the CMS server after a reasonable amount of time. - VP_4: The trip can be converted into a single Matlab standardized Trip file. - VP_5: There has been a continuous acquisition of data for the duration corresponding to trip start time

and trip end time (as noted by the driver in the diary). - VP_6: GPS acquisition is continuous and starts a reasonable amount of time after departure. - VP_7: GPS signal fits correctly over a route on digital map. - VP_8: Last GPS position corresponds to parking place identified by driver in diary. - VP_9: GPS speed signal is correctly synchronized with CAN speed signal. - VP_10: All CAN channels and signals are correctly acquired.

Alternative scenarios

VP_11 and VP_12 are optional if unusual CAN values are detected. For vehicles with video logging, the following verification procedures are necessary in addition to the previous ones: VP_Vid1 to VP_Vid7.

Each incident (e.g. unusual values detected, wrong operation of the sensors etc.) has been documented at the French VMC in order to follow the process of solving them. Here is an example of an incident that occurred during high level instrumented cars pilot tests.

euroFOT 16.06.2011


1 INCIDENT / NON CONFORMITY

Reference INC_003

Status Solved

Issuer Date February 2010

Device Embedded PC Video logger

Supplier

Versions (hardware & software)

05-01-2010 release

Serial numbers N/A

Incident title Video logger shut down Circumstances (if they exist: scenario and verification procedure)

No specific circumstances could be linked to occurrence of incident.

Incident description The embedded PC for video logging is supposed to throw

an event to the application when ignition signal (KL15) goes from on to off, to allow the software to shut down its activity before requesting the operating system to properly stop. This feature didn’t always work.

Frequency (systematic/random)

Random

gravity (minor/serious/critical)

critical

2 TRACKING

Date Action Result February 2010

Incident reported on specific situations (after remote desktop session on videologger)

The embedded PC did not stop

March 2010

Incident also reported on normal driving situations

The embedded PC did not stop

April 2010

Incident solved: the video logging software was modified accordingly, to regularly read the ignition input, and generate its own «shutdown» event instead of using the hardware generated one.

Incident closed

euroFOT 16.06.2011


6 German VMC1 (Ford, MAN, VW)

6.1 Recall of the VMC

The German VMC, Operation Center 1 (German VMC1) is responsible for the partners Ford, MAN and VW. Operation Sites of German The VMC1 locations are Aachen (Ford), Munich (MAN) and Wolfsburg (VW). Functions to be analyzed are ACC (all partners), LDW (MAN, VW), FCW (Ford, MAN) and CSW (Ford). Data transmission, checking and storage are located at partner ika in Aachen. An overview of the four functions is given in the figures below.

Figure 9 Overview of the ACC function (Ford, MAN, VW) with distance regulation which maintains a selected max. speed and/or constant time gap to the lead vehicle.

euroFOT 16.06.2011


Figure 10 Overview of the CSW function (Ford) with warning to the driver if a comfortable or safe speed is exceeded while approaching curves.

Figure 11 Overview of the FCW function (Ford, MAN) with constant surveillance of time-to-collision to the lead vehicle and braking support.

euroFOT 16.06.2011


Figure 12 Overview of the LDW, LA and IW function (MAN, VW) with surveillance of proper trajectory and optional assistance. In this VMC the driver-owned or leased vehicles are equipped with a CAN, GPS and Radar data acquisition system (DAS). No extra instrumentation has been installed. Ordinary drivers (Ford and VW) as well as professional drivers (MAN) are participating.

6.2 Description of the data logging system

As described in Deliverable D3.1 the CTAG DAS proved to be the most cost-efficient solution for the first level of instrumentation (CO or CAN Only). One CTAG data logger is installed in each vehicle together with a combined GPS/GPRS-antenna. It can record up to four CAN-data channels The service provider for GPRS transmission in Germany is Vodafone. An SD-card with 2 Gigabyte storage in the DAS is used to store intermediate data. The large memory SD-card allows the storage of data during international drives without having to transmit them. This is important since the GPRS-connection is only valid in Germany and thus avoids high roaming costs. Up to 10 Megabyte per hour driving are recorded. The amount of information to be stored varies for each partner and was carefully studied during the selection process. To record all data at the given interval (some signals are measured at a rate of 100 Hertz) results in several 100 Megabytes per hour and is not feasible at the GPRS transmission speed. The chosen upper limit of 10 Megabytes, however, allows to store around 200 hours of driving outside of Germany without the need for data transmission. Data will be transmitted during recording and during a one hour interval after vehicle engine OFF. After turning off the vehicle and the end of the one hour interval, the logger switches off. The wake-up of the DAS is done via “clamp 15”, i.e. after ignition is detected. A continuous power connection, "clamp 30" is necessary for the DAS to transfer remaining data to the server after engine shut-off. Power drain on the clamp 30 is negligible after switch-off.

In the German VMC1 besides CAN (C) no video (V) or other extra sensors (E) are installed in the vehicles.

euroFOT 16.06.2011


6.3 Description of the upload center

The data transmission chain goes from the vehicle via GPRS to a high performance server located at the computing center of RWTH, Aachen. ika is connected to this server via high-speed intranet. At ika data is automatically converted to a standardized file format and afterwards checked with regard to data quality, integrity and plausibility. The data is passed through an extensive pre-processing network and stored in a datalogger-trip structure. Each trip is stored with date&time of the recording start of the logger in this structure. The timebase for the filename-time is UTC. Datasets are grouped according to the ID of the DAS.

The software tool chain framework designed for the German1-VMC by ika is presented in the figure below.

The architecture is based on a modular approach. The main functions cover data retrieval, processing and storage on an SQL based database as well as management and monitoring. The main data processing algorithms are not part of the framework by itself, but are embedded as extensions. They can be dynamically defined and selected at any time.

The software consists of several software components. Data is passed between these software elements each time one process step is accomplished. The coordination of the interaction between the software components is performed by the central manage

report on the results of adaptation, in-vehicle implementations … · 0.07 2010-08-12 adding to...

Documents