inrets, villeneuve d’ascq, december 15 th -16 th 2005 etiseo annotation rules data structure...
TRANSCRIPT
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
ETISEO Annotation ETISEO Annotation rulesrules
•Data structure•Annotation tool and format•Ground truth creation rules•Reference data generation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Data structure - Data structure - contextcontext
• Id• Type (ex: zone)• Name (ex: exit_zone)• Polygon
Context
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Data structure - objectData structure - object
• Id• Start time• Type (ex: person,
vehicle…)• Subtype (ex: car,
van…)• Info2D• Info3D
• Links• Speed• GT Nature• GT Status• GT Occluded (partially or
completely)• GT Connected with
Physical object
GT : specific to annotation purpose
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Data structure - eventData structure - event
• Id• Name• Start and end time• Status• Involved objects (physical object of interest
and contextual object)
Event
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
ETISEO Annotation ETISEO Annotation rulesrules
•Data structure•Annotation tool and format
•Ground truth creation rules•Reference data generation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Annotation tool and Annotation tool and formatformat
• Database - Excel file• Context annotation - Viper-GT tool and format
Annotation delivered with video:
• Ground truth - Viper-GT tool and format• Reference data (computed from GT) – Specific
tool and Viper-GT format
Annotation delivered at the end of Etiseo:
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Annotation tool and Annotation tool and formatformat
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Annotation tool and Annotation tool and formatformat
Viper-GT format example<?xml version="1.0" encoding="UTF-8"?><viper xmlns="http://lamp.cfar.umd.edu/viper#" xmlns:data="http://lamp.cfar.umd.edu/viperdata#">
<config> <…></config><data>
<sourcefile filename="file:ETI-VS1-CR-4-C1.avi"><object framespan="1:100" id="4" name="physical_object">
<attribute name="nature"><data:svalue value="physical_object_of_interest"/>
</attribute><attribute name="type">
<data:svalue value="person"/></attribute><attribute name="info2D">
<data:bbox framespan="1:1" height="182" width="86" x="138" y="59"/><data:bbox framespan="2:2" height="183" width="86" x="139" y="59"/>
<data:bbox framespan="3:3" height="185" width="86" x="141" y="60"/> <data:bbox framespan="4:17" height="187" width="87" x="143" y="60"/> <…>
<data:bbox framespan="40:100" height="212" width="10" x="3" y="10"/></attribute><…>
</sourcefile></data>
</viper>
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
ETISEO Annotation ETISEO Annotation rulesrules
•Data structure•Annotation tool and format•Ground truth creation rules
•Reference data generation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Contextual object
• Annotated during its period of movement
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Static physical object of interest
• Any physical object of interest is annotated on the whole video clip (ex: a parked vehicle)
First frame and last frame of a video clip
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Occlusions – general rules
• Apparition of an object: the BBox increase with object apparition including only its visible part,
• All BBox are limited by the image boundary,• Partial and complete occlusion will be
differentiate in the evaluation (attribute “occluded”).
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Static occlusions (by a contextual object)
• Partial occlusion: full BBox is drawn including hidden part,
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Static occlusions (by a contextual object)
• Complete occlusion - two cases: all “escapes” of the contextual object are
visible: full BBox is drawn all the time,
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
all “escapes” of the contextual object are not visible: the object is no more annotated when completely occluded
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
• Partial or complete occlusion: in both cases, the full BBox is drawn including hidden part.
Dynamic occlusion (by another object)
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Split object due to occlusion
• A unique BBox including all sub part is drawn.
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Artefacts• Only the part constituting the objects forms the
BBox, artefacts (shadow, reflection…) are ignored for instant.
• Annotated in video data set 2
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Sudden illumination changes (tagged frames)• Static object
• Object in movement: position interpolation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Held object
Example: bag-person, person-bike, door-vehicle
• The holder and the held object are both annotated (if visible),
• The carried object is connected to the holder.
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Specific case
• Person getting in/out a vehicle: annotated only when visible
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Group - Crowd• Group: multiple bounding boxes (persons +
group)
• Crowd : unique Bbox when individual annotation is impossible.
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Crowded video clip• Only object involved in event of interest are
annotated. No false detections evaluation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for objectsRules for objects
Multi-view annotation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation Ground truth creation rulesrules
• Rules for objects: Contextual object Static physical object of interest Occlusion Artefacts Sudden illumination changes Held object Group - Crowd Crowded video clip Multi-view annotation
• Rules for events
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for eventsRules for events
• Id• Name• Start and end time• Status• Involved objects:
Id of the physical object of interest in GT files
Id of the contextual object in context information files
Event – data structure
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for eventsRules for events
• Choice of event for each video clip in the event ontology => defined in the database xls file (provided with video)
Sequence Name ETI-VS1-RD-15
… … …
Content
Physical objects type Person, vehicle
Events of interest gets_in, gets_out, parked
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for eventsRules for events
• Automatic annotation of events with quantitative constraints, example:
Related to kinematics properties: stopped, running…
Related to spatial properties: close_to, inside_zone…
• Manual annotation of other events: holding, pushing, waiting, gets_in/out,…
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Ground truth creation – Ground truth creation – Rules for eventsRules for events
Example
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
ETISEO Annotation ETISEO Annotation rulesrules
•Data structure•Annotation tool and format•Ground truth creation rules•Reference data generation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
Reference data Reference data generationgeneration
• Filtering on object size• Object speed computation• Group generation• Multi-view and 3D reconstruction• Events with quantitative constraints
generation
INRETS, Villeneuve d’Ascq, December 15th-16th 2005
ETISEO Annotation ETISEO Annotation rulesrules
Documentation:• ETI-Data structure and output format• ETI-Video Annotation Rules