rescue robot project
DESCRIPTION
Rescue Robot Project. Project overview. Modular components to be re-used in future years Tasks and Team Robot graphical user interface ( Zwivhuya Tshitovha ) Robot control interface ( Jaco Colyn ) Robot visual system (Jonathan Dowling) - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/1.jpg)
Rescue Robot Project
![Page 2: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/2.jpg)
2
Project overview
Modular components to be re-used in future years
Tasks and Team1. Robot graphical user interface (Zwivhuya Tshitovha)2. Robot control interface (Jaco Colyn)3. Robot visual system (Jonathan Dowling)
8 Masters students working in the Robotics and Agents lab
![Page 3: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/3.jpg)
3
Importance of Research
Reduce risk for rescue workers Map generation Rescue tasks Detection and localisation of victims Detection of hazardous materials
Disasters that have used robots: Japan nuclear disaster Three Mile Island nuclear disaster World Trade Centre terrorist attack
![Page 4: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/4.jpg)
4
Rescue robots in disasters
Usage in Japan nuclear disaster
And in the World Trade Centre
![Page 5: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/5.jpg)
5
Rescue robots in disasters World trade centre (continued)
![Page 6: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/6.jpg)
6
The UCT Rescue Robot
![Page 7: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/7.jpg)
7
Research Questions
How can an effective system be built to automatically detect simulated disaster zone objects? human victims hazmat signs rolling E’s
Using the normal IR cameras available on the robot and
Applying existing computer vision techniques to extract the features
![Page 8: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/8.jpg)
8
Research Questions
How can a usable and efficient GUI be built by hiding information and preventing sensory overload?
![Page 9: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/9.jpg)
9
Research Questions
How can an intuitive robot control system be created by mapping the robots functions onto a human interface device?
![Page 10: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/10.jpg)
10
RoboCup Rescue Robot overview RoboCup – an international
organisation based in Sweden Urban search and rescue tests
![Page 11: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/11.jpg)
11
Tests and Competitions
Simulate human subjects and disaster environments
Robot vision tests Rolling E’s Hazmat signs Victim detection
Remote operation of robot through competition arena to carry out tasks such as retrieving an object
![Page 12: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/12.jpg)
12
Rolling E’s
![Page 13: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/13.jpg)
13
Hazardous Materials (Hazmat)
![Page 14: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/14.jpg)
14
Victim detection
![Page 15: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/15.jpg)
15
Deliverables Client for software engineering project: Stephen
Marais from Robotics and Agents Lab
The deliverables required for this project are: A visual system for the robot to be able to pass the
“rolling E’s” test Hazardous material signs detection Human victim detection Easy to use interface for controlling the robot via a
human interface device Intelligent graphical user interface that highlights
only important information to the operator
![Page 16: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/16.jpg)
Robot Control InterfaceGraphical User Interface
Goals and challenges:
![Page 17: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/17.jpg)
17
Fundamental Robot Control Goals Driving the
robot
Controlling the robotic arm
Adjusting the robots tracks
![Page 18: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/18.jpg)
18
Prevent Operator Mistakes No direct line of sight
to the robot
Robot is “on its own” Has to return by itself
Prevent Falling over Driving out of range
![Page 19: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/19.jpg)
19
Robot Control Challenges Efficient control
Single Teleoperator Reduce fatigue Interpret and perform commands in real time
Reduce input delay “lag” Efficient mapping of commands to the control
interface
Micire, M., Mccann, E., Desai, M., Tsui, K.M., Norton, A., and Yanco, H.A. Hand and Finger Registration for Multi-Touch Joysticks on Software-Based Operator Control Units. (2011), 88-93.
![Page 20: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/20.jpg)
20
Graphical User Interface challenges Designing a “clean” Graphical User
Interface (GUI) Reducing sensory overload – which
leads to fatigue Relay locations of important objects
that cannot be directly seen Only seen through the control interface
and GUI Interface must provide situational
awareness
![Page 21: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/21.jpg)
21
Robot Control Interface RC & GUI Iterative and Incremental software
engineering.
Human Computer Interaction (HCI) Human Robot Interaction (HRI)
Use case scenarios taken from Urban Rescue Scenarios
![Page 22: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/22.jpg)
Robot Vision(RV)
Goals and challenges:
![Page 23: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/23.jpg)
23
Fundamental RV goals
Detect Human body parts (primarily limbs)
Detect and identify Hazmat signs
Detect the “rolling-E”
![Page 24: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/24.jpg)
24
Robot vision system
Input stream of video The general procedure
Transform the image’s data into a simpler representation
Apply a machine learning algorithm onto the transformed data to detect features.
![Page 25: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/25.jpg)
Testing
![Page 26: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/26.jpg)
26
Testing Robot control interface and GUI
User centred design approach. Qualitative and Quantitative results Feedback can be generated from ▪ Test subjects ▪ Asked to complete specific tasks using the interface.
Performance measurements and quantitative results from simulation testing. Hazmat signs Human body parts Rolling “E”
![Page 27: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/27.jpg)
27
Testing
Requirements RoboCup Rescue Competition (RRC) Urban Search and Rescue
Initial testing on each sub-system
Final testing on the system as a whole Based on expected challenges in the
RRC
![Page 28: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/28.jpg)
28
Ethical, Professional and Legal Issues Demonstrate its capabilities on
dummies
No external users because robot quite expensive
Ethical clearance
![Page 29: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/29.jpg)
29
Related Work
Robot vision Mikolajczyk et al [1]
Robot control and Interface Richer et al [2] Adams [3]
![Page 30: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/30.jpg)
30
Related Work cont...
Team Description Papers 2003 Team RoBrno [4]
![Page 31: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/31.jpg)
31
Resources
Hardware 19 inch wide screen Robot Human interface device
Software OpenCV Bosch SDK ROS
![Page 32: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/32.jpg)
32
Risks
Accessing the robot
Robot hardware damage
Missing project milestones
![Page 33: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/33.jpg)
33
Anticipated Outcomes
System used for a rescue robot that is developed by the Robotics and Agents Research Laboratory.
Key success factors Robot vision -- Accurate detection in smallest
time Robot interface and control -- Usable and
efficient given by measure of time and effort
![Page 34: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/34.jpg)
Timeline
![Page 35: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/35.jpg)
35
Timeline
![Page 36: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/36.jpg)
36
Timeline
![Page 37: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/37.jpg)
37
Timeline
![Page 38: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/38.jpg)
38
References
[1] Mikolajczyk, K,Schmid, C & Zisserman ,A 2004, „Human detection based on a probabilistic assembly of robust part detectors‟ . European Conference on Computer Vision, Oxford, United Kingdom.
[2] Adams, JA 2002,‟Critical Considerations for Human-Robot Interface Developmen‟, Proceedings of 2002 AAAI Fall Symposium, 2002 - aaai.org, Rochester Institute of Technology Rochester, New York.
[3] Richer, J & Drury, JL 2006 „A Video Game-Based Framework for Analyzing Human-Robot Interaction ‟, Characterizing Interface Design in Real-Time Interactive Multimedia Applications. MITRE Corporation, United states of America
[4] Uschin, K, Inteam, S, Sae-eaw,N, Trakultongchai, A, Klairith, P, Jitviriya, W & Jenkittiyon , P,‟ RoboCup Rescue 2009 - Robot League Team iRAP_Pro (Thailand)‟,King Mongkut‟s University of Technology North Bangkok (KMUTNB) , Bangkok Thailand.
![Page 39: Rescue Robot Project](https://reader030.vdocuments.site/reader030/viewer/2022033021/56816697550346895dda7f7e/html5/thumbnails/39.jpg)
39
Questions
?