user interfaces for direct and remote hri for a swarm of robots assisting fire-fighters
DESCRIPTION
TRANSCRIPT
![Page 1: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/1.jpg)
User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters
Affligem, October 19, 2009
Elvina Motard Space Applications Services (Belgium)
Jeremi Gancet Space Applications Services (Belgium)
Amir M. Naghsh Sheffield Hallam University (UK)
Chris R. Roast Sheffield Hallam University (UK)
![Page 2: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/2.jpg)
Outline
1. Introduction and background– the EU FP6 Guardians project
2. Fire-fighting context, setting and operations
3. The Guardians baseline setup
4. Direct Human Swarm Interactions (HSI)
5. Remote Human Swarm Interactions
6. Conclusion and future work
![Page 3: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/3.jpg)
1. Introduction and background – the EU FP6 Guardians project
Guardians: EU-FP6 project, 2006-2010 (38 months)
=> Group of unmanned assistant robots deployed in aggregative navigation supported by scent detection
=> Swarm robotics
=> Human-Robot Interactions / Human-Swarm Interactions
=> Application domain: search & rescue, support to fire-fighting
=> Main scenario: large warehouse fire
![Page 4: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/4.jpg)
2. Fire-fighting context, setting and operations
=> Smoke – means heavy breathing apparatus, little to no vision, guidelines
=> Noise – means limited audio communication, establish protocols
=> Heat – physical stress
=> Touch – limited dexterity
=> 20 minutes maximum, risks to lives – mental stress
![Page 5: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/5.jpg)
2. Fire-fighting context, setting and operations
=> Fire event: little information available a priori...
=> 1st task: to assess the incident and the primary risks
=> Fire-fighters safety: high(est) priority!
=>Organized into teams, then groups of 2 – 3 fire-fighters
Incident Commander
Operations Commander
Sector Commander
Fire-fighter
Fire-fighter
Fire-fighter
Safety Officer
ECO
Zone 3
Zone 2Zone 1
Sector 3Sector 2
Sector 1
![Page 6: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/6.jpg)
3. The Guardians baseline setup
Incident Commander
Operations Commander
Fire-fighter
Fire-fighter
Fire-fighter
Safety Officer
ECO
Zone 3
Zone 2Zone 1
Sector 3
Sector 2
Sector 1
R
RRR
Guardians’ Base Station
Coordinator
Operator(s)
Sensor Data Specialist(s)
Sector Commander
![Page 7: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/7.jpg)
Incident Commander
Operations Commander
Fire-fighter
Fire-fighter
Fire-fighter
Safety Officer
ECO
Zone 3
Zone 2Zone 1
Sector 3
Sector 2
Sector 1
R
RR
Sector Commander
R
Guardians’ Base Station
Coordinator
Direct Human-Swarm Interactions
Operator(s)
Sensor Data Specialist(s)
4. Direct Human Swarm Interactions
![Page 8: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/8.jpg)
4. Direct Human Swarm Interactions
Purpose: supporting local fire-fighters operations
=> Guidance / navigation (e.g. toward exit in case of increasing risk)
=> Informing fire-fighters about current ambient conditions in the vicinity (e.g. heat level, chemicals)
=> Small distance forward scouting
![Page 9: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/9.jpg)
4. Direct Human Swarm Interactions
Light Array Visor: Baseline setup
![Page 10: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/10.jpg)
4. Direct Human Swarm Interactions
Light Array Visor (LAV): version 1
=> Phidgets toolkit
=> Iterative development process:
• Implementing the prototype
• Followed by exploration sessions with Fire-fighters
=> Analogical visual information
![Page 11: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/11.jpg)
Light Array Visor (LAV): version 2
=> Real operational fire-fighting helmet
=> Experimenting with a group of two fire-fighters
=> Logical commands encoding and display
=> Integration of internal measurement unit (IMU)
=> More stress (reporting, counting light flashes and memorising the mental map)
4. Direct Human Swarm Interactions
![Page 12: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/12.jpg)
5. Remote Human Swarm Interactions
Incident Commander
Operations Commander
Fire-fighter
Fire-fighter
Fire-fighter
Safety Officer
ECO
Zone 3
Zone 2Zone 1
Sector 3
Sector 2
Sector 1
R
RR
Sector Commander
R
Guardians’ Base Station
Coordinator
Operator(s)
Sensor Data Specialist(s)
Remote Human-Swarm Interactions
![Page 13: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/13.jpg)
5. Remote Human Swarm Interactions
Purpose: remotely supervising robots and fire-fighters operations
=> Integrated data for the global supervision of the situation
=> Provides means to control the robots of the swarm:
• High level commands / policy level control of groups of robots in the swarm
• Individual robot control, in case of specific need
=> Interface with decision makers in the fire-fighters hierarchy, thus allows to take informed decision
![Page 14: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/14.jpg)
5. Remote Human Swarm Interactions
Base station baseline setup
Khepera III
HNI Bebotx80Erratic
Robudem
Robotnik Guardian
Atrv-JR
![Page 15: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/15.jpg)
5. Remote Human Swarm Interactions
1
2
5
3
4
1. Main visualization zone
2. Viewport navigation
3. Commands formulation area
4. Overall map view
5. Mission log
Remote interaction: main user interface
![Page 16: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/16.jpg)
5. Remote Human Swarm Interactions
1
Fire-fighter and robots 3D representation
Robots sensors + video - joint visualization (ecological display paradigm)
High level command generation wizard
![Page 17: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/17.jpg)
6. Conclusion and future work
Direct HSI
- LAV2 more expressive for fire-fighters, although more stressful
- Further experiments planned with a number of swarm support scenarios
Remote HSI
- Operator role usability evaluation so far => further work to focus on Sensor Data Specialist and Coordinator roles
- Multi-user, joint operations to be experimented and evaluated
Interfaces design for fire-fighters support
- Complex but valuable approach to integrate user inputs into the design and development process
- Incremental usability evaluation process (formative feedback)
![Page 18: User interfaces for direct and remote HRI for a swarm of robots assisting fire-fighters](https://reader034.vdocuments.site/reader034/viewer/2022051818/54b5b4614a7959097e8b45e1/html5/thumbnails/18.jpg)
Thanks!
http://www.guardians-project.eu/