Frederic RoullandXerox Research Centre Europe

Xerox Vehicle Passenger Detection System

Intelligent road traffic system trends

• From infrastructure to vehicle centric

• Ticket less, barrier less, free flow roads/parking lots

• On-board units, connected, autonomous vehicles

• Aim is to ease traffic

• From vehicle to traveler centric

• Mobility as a Service vs vehicle ownership

• Shared occupancy vs single occupancy

• Aim is to reduce traffic footprint

3

IdentifyVehicle and types

PeopleUser Privacy

TrackVehicle point to

point vs continuous

People behavior

ExecuteOnline tracking

Fast and lightweight models

LearnDomain adaptation

Training data capture/generation

Why do we need computer vision?

• Detect, track and identify people and vehicles

• For safety

• Speed

• Incident detection

• Dangerous behaviors detection

• Demand management

• Vehicle occupancy incentive

• Dynamic Road Tolling enforcement

• Dynamic Parking enforcement

• Internal security

4

• How can research help?

Xerox transport innovations using computer vision

5

Open road tolling

Vehicle occupancy enforcement

Car Park management

On street parking enforcement

Public transport crowd management

Vehicle Occupancy enforcement

6

CLOUD

LOCAL

PROCESSING

TICKET

POLICE OFFICER

IN BACK OFFICE

Applications

• Enforce Car pooling incentives – Dedicated (HOV) lanes

• Caltrans I5: January 2015 (ITS America Award for Best Partnership Project in Infrastructure of Things category)

– Dedicated gates• French border – June 2015

(CEREMA award for best partnership project)

– Specific toll fares

• Automate Passenger counting – Border crossing inspection

• pilot – Ongoing

– Analytics

Pilot at France’s border

3: Prediction: 0, 1, 2 or more passengers

8

1: Windshield detection 2: Side window detection

System accuracy

• Correctly predicts the number of passenger in 93.5% of the cars in overall

conditions (95.94% in Caltrans)

• Detects Single Occupancy Vehicle (SOV) with an accuracy of 95%.

Occupancy rate

Date / time Number of passengers

None 1 1 or more 2 or more

Global 1.36 June 3-6 95,1 % 91,8 % 92,5 % 95,2 %

Night 1,19 June 3-4

4h30 AM- 5h15 AM97,0 % 95,4 % 95,2 % 98,2 %

High Traffic 1,17 June 3-5

4h30 AM - 8h00 AM96,8 % 96,9 % 97,1 % 98,6 %

Low Traffic 1,53 June 3-5

8h00 AM - 5h00 PM93,8 % 87,4 % 88,5 % 92,3 %

Highoccupancy

1,88 Saturday June 6 1h00

PM - 4h00 PM91,6 % 80,4 % 81,2 % 85,4 %

At the core of the technology:

XRCE’s Generic Visual Toolbox (GVT)

• Towards holistic visual understanding

• Detecting and recognizing scene elements (objects, text, etc.)

• Tracking scene elements and interactions

10

Genericity id achieved by learning from the data

GVT facts

Approx. 120 patents and patent applications protecting

‒ the core technology itself: signatures, classification, …

‒ different applications and businesses: transportation, retail, BPO, …

‒ 18 international awards over the past 10 years

Developed in collaboration with some of the leaders in the field

‒ in the US: MIT, Harvard, UIUC

‒ in Europe: Oxford, EPFL, INRIA

Stable C/C++ code developed, enriched, fine-tuned over past 12 years

→ robust software components, difficult to replicate

11

Conclusion

• Xerox Vehicle Passenger Detection System

– An interesting example of new generation of visual sensors going

beyond car tracking

– Successfully piloted across the world

– Relying on powerful proprietary computer vision technology

– A wide potential of applications for car pooling incentives and people

counting