tools for programming in the real world self introduction for msra fellowship 2012 jun kato, the...
TRANSCRIPT
- Slide 1
- Slide 2
- Tools for Programming in the Real World Self Introduction for MSRA Fellowship 2012 Jun Kato, The University of Tokyo 1
- Slide 3
- Jun Kato 2 nd - year PhD candidate Igarashi Laboratory, The University of Tokyo Concentration: Human-Computer Interaction Especially, Tools for Programming in the Real World 2
- Slide 4
- Agenda My motivation for the research Introduction of the projects Ideas for new projects 3
- Slide 5
- MY MOTIVATION FOR THE RESEARCH Tools for Programming in the Real World 4
- Slide 6
- CRISTAL [SIGGRAPH09 E-Tech] Tools for Programming in the Real World Multi-touch [CHI09 SRC] Roboko [ CHI13 (to be submitted)] Sharedo [Ro-MAN12 (submitted)] Phybots [DIS12] Surfboard [UIST10 Demo] IntelliViewer [UIST10 SIC] Pressing [UIST09 SIC] DejaVu [UIST12 (to appear)] 5
- Slide 7
- Sharedo [Ro-MAN12 (submitted)] Roboko [ CHI13 (to be submitted)] DejaVu [UIST12 (to appear)] Phybots [DIS12] CRISTAL [SIGGRAPH09 E-Tech] Multi-touch [CHI09 SRC] Phybots [DIS12] Surfboard [UIST10 Demo] IntelliViewer [UIST10 SIC] Pressing [UIST09 SIC] CRISTAL [SIGGRAPH09 E-Tech] Multi-touch [CHI09 SRC] In the Real World From Robot to Human 6
- Slide 8
- Sharedo [Ro-MAN12 (submitted)] Roboko [ CHI13 (to be submitted)] Phybots [DIS12] DejaVu [UIST12 (to appear)] CRISTAL [SIGGRAPH09 E-Tech] Multi-touch [CHI09 SRC] Roboko [ CHI13 (to be submitted)] Surfboard [UIST10 Demo] IntelliViewer [UIST10 SIC] Pressing [UIST09 SIC] DejaVu [UIST12 (to appear)] In the Real World From Robot to Human 7
- Slide 9
- Surfboard [UIST10 Demo] IntelliViewer [UIST10 SIC] Pressing [UIST09 SIC] Roboko [ CHI13 (to be submitted)] Phybots [DIS12] DejaVu [UIST12 (to appear)] Multi-touch [CHI09 SRC] Roboko [ CHI13 (to be submitted)] Phybots [DIS12] DejaVu [UIST12 (to appear)] CRISTAL [SIGGRAPH09 E-Tech] Multi-touch [CHI09 SRC] Sharedo [Ro-MAN12 (submitted)] Tools for Programming From Keyboard to IDE 8
- Slide 10
- Surfboard [UIST10 Demo] IntelliViewer [UIST10 SIC] Pressing [UIST09 SIC] Roboko [ CHI13 (to be submitted)] Phybots [DIS12] DejaVu [UIST12 (to appear)] Multi-touch [CHI09 SRC] Surfboard [UIST10 Demo] IntelliViewer [UIST10 SIC] Pressing [UIST09 SIC] CRISTAL [SIGGRAPH09 E-Tech] Sharedo [Ro-MAN12 (submitted)] Tools for Programming From Keyboard to IDE 9
- Slide 11
- INTRODUCTION OF THE (SELECTED) PROJECTS Tools for Programming in the Real World 10
- Slide 12
- Projects Phybots DejaVu Roboko 11
- Slide 13
- Motivation Add mobility to our daily objects 12
- Slide 14
- Gulf of HCI and Robotics 13 Physical UIs Toolkit for: Robots HCI researchers & Interaction designers Target users: Robotics people Prototyping Focus: Reliability Low-level&Static Software API: High-level&Extensible Small&Cheap Hardware: Phidget Kit, $200 Medium-Large&Expensive K-Junior, $938
- Slide 15
- Goal of Phybots Add mobility to physical objects In a cheap and easy way Through high-level and extendible API With support for the whole prototyping process 14
- Slide 16
- Prototyping with Phybots 15 Hardware construction Programming with built-in API Test with runtime debug tool (Extend API when needed)
- Slide 17
- Localization and Locomotion API Hardware setup: easy and cheap Camera PC or Mac Robotic things 16 From our user study: Miniature drive recorder
- Slide 18
- Localization and Locomotion API Navigation by global coordinates Move Push TracePath 17 Task move = new Move( mouseX, mouseY); move.assign(robot); move.start(); From our user study: Beach flags with obstacles Y-axis X-axis
- Slide 19
- Localization and Locomotion API Vector field navigation Easy design of new behavior ex) Follow another robot Combination of existing fields ex) Move + collision avoidance 18
- Slide 20
- Runtime Debug Tool 19 Entity MonitorService MonitorWorkflow Monitor
- Slide 21
- User Studies Alpha version deployment To graduate students Current version deployment To HCI students To robotics students 20
- Slide 22
- Lessons learned: Mere mobility was not enough. Most apps only used a single API call. Parameter configuration was painful. Alpha version deployment 21 Lessons learned: Extendibility is important. Higher-level task management is desired. Support for testing phase is needed.
- Slide 23
- Current version deployment 22
- Slide 24
- Phybots A toolkit for prototyping robotic things Localization and Locomotion API Extensible Architecture Runtime Debug Tool Open-source software available at http://phybots.comhttp://phybots.com 23
- Slide 25
- Projects Phybots DejaVu Roboko 24
- Slide 26
- Motivation Input by the users hands, body, or physical objects Captured by cameras Processed in real-time Support for computer-vision- based programming is needed Wang, 2006 Wilson, 2005 Cao, 2003 25
- Slide 27
- Existing approaches Hardware, Software API Becomes more and more accessible Webcam, Kinect, OpenCV, etc. Development Environments Do not provide explicit support Ex) Breakpoint-based variable monitoring OpenCV Visual Studio Eclipse 26
- Slide 28
- DejaVu 27
- Slide 29
- DejaVu Canvas Frame data visualization Freehand sketch 28
- Slide 30
- DejaVu Canvas Capable of showing variable values at the different positions in the code 29
- Slide 31
- DejaVu Timeline 30
- Slide 32
- Conclusion DejaVu provides integrated support for the development of interactive camera-based programs that tightly matches their nature: Visual/Continuous/Non-reproducible the workflow: Record input/Change code/Replay CanvasWorkflow 31
- Slide 33
- Projects Phybots DejaVu Roboko 32
- Slide 34
- Programming robot actions is difficult. 33 DanceClearing dishesCooking Motivation:
- Slide 35
- Text-based Programming joint[0].setAngle(0.6); joint[1].setAngle(0.7); joint[2].setAngle(-0.24); arm.setPosition(4.3, 2.4); robot1.setPose(pose32); 34 Problem: We cannot imagine the actual pose of the robot Typical approach:
- Slide 36
- Visual Editing 35 Problem: We cannot design flexible robot behavior such as repeat this for 3 times when Typical approach:
- Slide 37
- Overview of the IDE 36
- Slide 38
- Workflow of the Programmer 37 1. Capture2. Drag and Drop Camera RobotPC
- Slide 39
- Getting and setting pose 38 Supported APIs:
- Slide 40
- Execute a sequential action 39 Supported APIs:
- Slide 41
- Not only a robot but a human (Kinect) 40 Project outcome: new application This work is an outcome of collaboration with a MSR researcher, Xiang Cao
- Slide 42
- Integrating visual representations into text-based programming environment 41
- Slide 43
- IDEAS FOR NEW PROJECTS Tools for Programming in the Real World 42
- Slide 44
- Ideas for New Projects (CENSORED) 43
- Slide 45
- Thank you for listening Ive designed various tools for programming in the real world. I am looking forward to further collaboration with MSRA researchers. 44