toolstone: effective use of the physical manipulation vocabularies of input devices

Copyright (C) 2000 SonyCSL

ToolStone:ToolStone: Effective Use of the Physical Manipulation Effective Use of the Physical Manipulation

Vocabularies of Input DevicesVocabularies of Input Devices

Jun Rekimoto and Eduardo SciammarellaJun Rekimoto and Eduardo Sciammarella

Interaction LaboratoryInteraction Laboratory

Sony Computer Science Laboratories, Inc.Sony Computer Science Laboratories, Inc.

2Copyright (C) 2000 SonyCSL

3Copyright (C) 2000 SonyCSL

Introduction Introduction Increasing complexity of software applications:Increasing complexity of software applications:

Difficult & time-consuming to select an appropriate Difficult & time-consuming to select an appropriate tooltool

Many toolpalettes obscure the screen (application Many toolpalettes obscure the screen (application area)area)

Visual attention to the application is often interruptVisual attention to the application is often interrupted by tool selection tasksed by tool selection tasks

Lack of a simple MDOF input methodLack of a simple MDOF input method 3D rotation, color selection, etc.3D rotation, color selection, etc.

4Copyright (C) 2000 SonyCSL

PadMouse PadMouse [[Balakrishnan et al. CHI98]Balakrishnan et al. CHI98] StrategicCommander StrategicCommander [Microsoft][Microsoft]

5Copyright (C) 2000 SonyCSL

6Copyright (C) 2000 SonyCSL

ToolStoneToolStone

7Copyright (C) 2000 SonyCSL

ToolStoneToolStone Semi-6DOF ManipulationSemi-6DOF Manipulation

Position (X, Y) & Orientation (0°- 360°)Position (X, Y) & Orientation (0°- 360°) Detection of contacting face (with the tablet)Detection of contacting face (with the tablet)

Simple, cord/battery-less deviceSimple, cord/battery-less device 2.5 × 4 × 5 cm, 222.5 × 4 × 5 cm, 22ｇｇ Easy to rotate in the handEasy to rotate in the hand Tactile impression tells its statusTactile impression tells its status

Can be used forCan be used for Tool selection (bimanual interfaces)Tool selection (bimanual interfaces) Multi-DOF control (e.g., virtual camera in VR)Multi-DOF control (e.g., virtual camera in VR) Physical license key (for software applications, etc.)Physical license key (for software applications, etc.)

8Copyright (C) 2000 SonyCSL

Several different ways of Several different ways of holdingholding

a b c

d e f

9Copyright (C) 2000 SonyCSL

ToolStone for bimanual ToolStone for bimanual interactioninteraction

NDHNDH (non dominant hand) (non dominant hand) for tool selectionfor tool selection

DHDH (dominant hand) (dominant hand) for tool manipulationfor tool manipulation

10Copyright (C) 2000 SonyCSL

11Copyright (C) 2000 SonyCSL

Assigning Assigning multiple tools for multiple tools for each angle and each angle and faceface

Control of parametersControl of parameters by moving a stoneby moving a stone

Pick&Drop by Pick&Drop by moving a stone moving a stone from one place from one place to anotherto another

12Copyright (C) 2000 SonyCSL

Example 1: Toolpalette SelectionExample 1: Toolpalette Selection

8 directions × 6 faces = 48 different 8 directions × 6 faces = 48 different toolstools

= 8 directions

= 6 faces

13Copyright (C) 2000 SonyCSL

Visual FeedbacksVisual Feedbacks

selected tool

indicating ToolStone direction

14Copyright (C) 2000 SonyCSL

Mapping of command space Mapping of command space and physical manipulation and physical manipulation spacespace

Required physical Required physical manipulationmanipulation

15Copyright (C) 2000 SonyCSL

Example 2: Color Example 2: Color SelectionSelection

Color selection is a 3D positioning task (HSV, RGColor selection is a 3D positioning task (HSV, RGB, …)B, …) Unnatural mapping is required with 2D devicesUnnatural mapping is required with 2D devices

Our solution: 2D Our solution: 2D DHDH pointing device pointing device + 1D + 1D NDHNDH toolstone movement toolstone movement

Other Apps:Other Apps:3D rotation,3D rotation,ZoomingZooming&panning&panning

16Copyright (C) 2000 SonyCSL

Example 3: Virtual Camera Example 3: Virtual Camera ControlControl

Selecting the “camera” face Selecting the “camera” face automatically activates the 3D automatically activates the 3D walkthrough modewalkthrough mode

NDHNDH camera position camera position + orientation + orientation

DHDH view angle, view angle, elevation angleelevation angle

17Copyright (C) 2000 SonyCSL

Sensing ArchitectureSensing Architecture

Uses an existing pen tablet (WACOM) Uses an existing pen tablet (WACOM) to recognize to recognize position, orientation, contacting face of position, orientation, contacting face of

the ToolStonethe ToolStone WACOM tablet can distinguish sensor coil WACOM tablet can distinguish sensor coil

types, position, and angletypes, position, and angle

Our ApproachOur Approach Embed 3 coils in the ToolStone deviceEmbed 3 coils in the ToolStone device each has a unique resonance frequencyeach has a unique resonance frequency

angles between tablet and coil (α≠β)

α

βα

β

coils

19Copyright (C) 2000 SonyCSL

ααφφ

20Copyright (C) 2000 SonyCSL

21Copyright (C) 2000 SonyCSL

22Copyright (C) 2000 SonyCSL

Trial UseTrial Use

Frequently used physical Frequently used physical operations sequence became a operations sequence became a chunkchunk

Sound (noise) caused by flipping Sound (noise) caused by flipping the device was noticeablethe device was noticeable

Visual guide is still necessary (for Visual guide is still necessary (for learning phase)learning phase)

23Copyright (C) 2000 SonyCSL

Summary & Future Summary & Future DirectionsDirections TooStoneTooStone

a semi 6DOF input devicea semi 6DOF input device novel 3coil architecture enables position+orientnovel 3coil architecture enables position+orient

ation+face recognitionation+face recognition Future WorkFuture Work

Self contained device (gravity sensor + optical Self contained device (gravity sensor + optical mouse sensor)mouse sensor)

Shapes other than cubesShapes other than cubes

24Copyright (C) 2000 SonyCSL

25Copyright (C) 2000 SonyCSL