mobility services plugtest - etsidocbox.etsi.org/workshop/2015/201503_itsworkshop/session03_… ·...

RESULT SUMMARY OF 4TH ITS COOPERATIVE MOBILITY SERVICES PLUGTEST

© ETSI 2011. All rights reserved

Sebastian Müller 27.3.2015 ETSI Centre for Testing and Interoperability ITS Workshop, Helmond (NL)

Plugtest Organization

© ETSI 2011. All rights reserved 2

Organizer Test Tools

ITS modem for ETSI Test System Supporter

ETSI TC ITS Test Activities

© ETSI 2014. All rights reserved

Conformance Validation

Framework

ITS Test Framework ETSI EG 202 798

Test Architecture ETSI TR 103 099

CAM Tests ETSI TS 102 868

DENM Tests ETSI TS 102 869

GeoNetworking Tests ETSI TS 102 871

Open Source Platform

Standardized test system interfaces for

test execution, logging, and adaptation

Free tests, codec and adaptation

Tests implemented in the international

standardized testing language TTCN-3

ITS Plugtest 2015

Interop Tests

Conformance Tests

C2C CC Validation

Real Time Channel Simulation/Replay Tests

ETSI Plugtest Live Demo

65 person month of effort invested since 2010

ETSI Conformance Test System

CAM/DENM/GN and Security Tests with TESTINGTECH runtime environment ITS modem support from COHDA and COMMSIGNIA


Interop Test System


Attenuator matrix is controlled by the QOSMOTEC QPER software Message flow between the DUTs is supervised by a monitor device and displayed in the VECTOR CANoe software

Summary of test infrastructure

More than 20 vendors ! • 3 Open Source Implementations (see Annex)

ETSI provided Wireshark dissector for all ITS protocols including Security to assist the debugging Security PKI framework provided for Conformance and Interop tests via ETSI Event WIKI • Release of ETSI Security Conformance Tests via ETSI Forge available

since Feb 2015

4 ETSI Conformance Test Systems 3 Interop areas for face-2-face 2 Forwarding setups for 2 x 3 devices Each vendor scheduled for all tests 5 test session chairs


Interop Issues GN and DENM

GeoNetworking • Sender Position Vector vs Source Position Vector

(SePV deleted from packets but still used in algorithms)

• GBC Advanced Fwd Algorithm not often supported

DENM • Keep Alive not often supported


Interop Issues Security

No Profile 3 tests were executed due to the open base spec issue on ITS-AID usage for messages other than CAM and DENM Pseudonym Change needs to be defined better Some ITS stations did not send long packets (694 byte signed CAM including certificate chain) • TS 102 687 (DCC Access Layer) restricts maxPacketDuration

to 0,6 msec at 6MBit/sec Sender CAM Certificate inclusion • Reaction to request and unknown digest • Timing

Exceptional behavior tests very useful for vendors


ETSI Plugtest Live Demo


Demo system • 2 RSUs and 4

OBUs • HMI on Tablet • Set up under 2

days V2I use-cases: • Road Hazard

Warning • Traffic Jam

Ahead Warning V2V use-case: • Approaching

Emergency Vehicle

Vehicle 1: Vialis Vehicle 2: ITRI Vehicle 3-4: Commsignia

RSU 1: Commsignia RSU 2: Vialis

Results

All vendors had ITS modem with radio interface (no more SW prototypes) Most of the vendors have implemented Security Most of the vendors have implemented ETSI Upper Tester Interface 20% of all vendors passed all conformance tests (security included) More than 2600 tests executed Only 22 potential base spec issues identified (see Annex) Devices ready for deployment! © ETSI 2011. All rights reserved 10

PHY Performance Evaluation by HHI


Statistical Channel Emulation • Based on ETSI ITS channel models • Covers main characteristics of the

doubly-selective vehicular radio channel, e.g. multipath and Doppler

Replay Channel Emulation • Based on channel measurements

conducted by HHI • Covers exact channel properties of

selected measurement runs Interface to receive packet count

Channel Emulator

Statistical Model

Device B

Replay Data

Device A

Next step: Feedback to ETSI ITS WG4 on channel models (ETSI TS 103 257)

Example Result from HHI


Motion Emulation by octoScope/TASS/SAVARI


Emulate motion of vehicles and attenuation Monitoring BER performance Next Step: Demonstrate potential hardware in the loop test capabilities including application performance and behavior

Prescan Scenario

generator

Octoscope testbed

attenuator ITS G5 RF

signals position updates

octoBox

octoBox

DUT

DUT

Distances between DUTs

Mobility Emulation Scenarios by Qosmotec and TASS


Scope of next Plugtest

Test only features relevant for Day 1, e.g. no GeoUnicast, no TSB , no KeepAlive , i.e. like C2C CC Profile Develop security tests for exceptional behavior of DENM and conformance of certificates Traces / PathHistory . Test that data is interpreted correctly Include road operator in test infrastructure Robustness tests Extend demonstrated channel simulation scenarios of 3 DUTs to • dynamic channel simulations with up to 6 DUTs • emulation of real traffic scenarios

DCC tests (Sensitivity threshold correction, energy threshold test, header decoding test, DCC stability) Out of band emission + CAM Coexistence feature Drive Tests


Concerns

No PKI deployment plan/protocols available (for infrastructure) Pseudonym Changes / Anti finger printing No Profile 3 tests were executed due to the open base spec issue on ITS-AID SPAT/MAP needs to be made available soon DCC needs to be made available soon Make C2C CC profile available to all vendors of next plugtest (use only 1 profile in Europe for C2C Communication) Create an European wide compatible C2X profile © ETSI 2011. All rights reserved 16

On behalf of the Plugtest Team


Thank you!

ITS Vendors


Autotalks Cohda/Nxp Commsignia Ctag Denso *I-GAME (TNO + Viktoria)

Hitachi/Renesas Imtech Itri Kapsch

Marben Neavia Nec Nordsys Novero Qmic Siemens *Trialog Unex *Yogoko Vialis

* Partly Open Source

Issues GN EN 302 636-4-1 v1.2.1

Typo in 9.3.12.2 Source operations – Issue6982 ambigiuous description in 9.3.8.3 Forwarder operations – Issue6983 error in 9.3.12.2 Source operations – Issue6984 DPD in 9.3.11.3 forwarder and receiver operations – Issue6985 GN protocol constant itsGnGeoAreaLineForwarding is set to TRUE – Issue6986 clock drifting – Issue6987 Restructure of FWD algorithm – Issue6988 Duplicate Packet Detection – Issue6989 MAC and GN ID – Issue6990


http://forge.etsi.org/mantis/view.php?id=6982









Issues Security TS 103 097 V1.1.20

Uncomplete description of creating hash for signature of secured messages – Issue6950 Wrong description of considering the public key algorithm in the signature of CAMs – Issue6949 Wrong description of considering the signer info field in the signature of CAMs - Issue6948 ITS-S should accept request of unrecognized AA certificate from untrusted sender – Issue6972






New feature for next revision of Security TS 103 097

Request of unrecognized AA certificate create high channel load when all receivers reply with a certificate chain – Issue6969 Security link between CertID and relevant IDs (Pseudonym change) – Issue6979 ITS-S should stop requesting an unrecognized AA certificate if the issuer of the AA certificate is untrusted – Issue6973 Request of unrecognized AA certificate create high channel load when all receivers reply with a certificate chain – Issue6969 Do we need certificates in CAM each second – Issue6976 add example how to define time – Issue6978 Security link between CertID and relevant IDs – Issue6979









Issues DENM EN 302 637-3 v1.2.2

pseudonym change for active DENMs – Issue6980 StationID collisions - Issue6981




Useful Links

ETSI Certificate Toolkit (Generator, SignatureVerifier) http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/tools/itscertgen/ FOKUS SecureMessage & Certificates Validator https://werkzeug.dcaiti.tu-berlin.de/etsi/ts103097 Link to Wireshark dissectors http://forge.etsi.org/websvn/WIRESHARK_ITS_PLUGINS/releases/wireshark-1.12.x/ Link to ETSI conformance framework http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/ ETSI GPSD Server http://forge.etsi.org/websvn/ITS_GPSD_SERVER/ Blog http://www.etsi.org/news-events/past-events/846-plugtests-2015-itscms4?tab=3


http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/tools/itscertgen/

http://forge.etsi.org/websvn/ITS/branches/STF484_VALIDATION/tools/itscertgen/

https://werkzeug.dcaiti.tu-berlin.de/etsi/ts103097

https://forge.etsi.org/svn/WIRESHARK_ITS_PLUGINS/releases/wireshark-1.12.x/

https://forge.etsi.org/svn/WIRESHARK_ITS_PLUGINS/releases/wireshark-1.12.x/

https://forge.etsi.org/svn/ITS/branches/STF484_VALIDATION/

http://forge.etsi.org/websvn/ITS_GPSD_SERVER/

http://www.etsi.org/news-events/past-events/846-plugtests-2015-itscms4?tab=3

http://www.etsi.org/news-events/past-events/846-plugtests-2015-itscms4?tab=3

Contact


Sebastian Müller

Project Manager ETSI - CTI

T: +33 4 92 94 43 13 | M: +33 6 74 40 83 66 |

E: [email protected]

mobility services plugtest - etsidocbox.etsi.org/workshop/2015/201503_itsworkshop/session03_… ·...

Documents