toolstone: effective use of the physical manipulation vocabularies of input devices

Copyright (C) 2000 SonyCSL

ToolStone:ToolStone: Effective Use of the Physical Manipulation Effective Use of the Physical Manipulation

Vocabularies of Input DevicesVocabularies of Input Devices

Jun Rekimoto and Eduardo SciammarellaJun Rekimoto and Eduardo Sciammarella

Interaction LaboratoryInteraction Laboratory

Sony Computer Science Laboratories, Inc.Sony Computer Science Laboratories, Inc.


Introduction Introduction Increasing complexity of software applications:Increasing complexity of software applications:

Difficult & time-consuming to select an appropriate Difficult & time-consuming to select an appropriate tooltool

Many toolpalettes obscure the screen (application Many toolpalettes obscure the screen (application area)area)

Visual attention to the application is often interruptVisual attention to the application is often interrupted by tool selection tasksed by tool selection tasks

Lack of a simple MDOF input methodLack of a simple MDOF input method 3D rotation, color selection, etc.3D rotation, color selection, etc.


PadMouse PadMouse [[Balakrishnan et al. CHI98]Balakrishnan et al. CHI98] StrategicCommander StrategicCommander [Microsoft][Microsoft]


ToolStoneToolStone


ToolStoneToolStone Semi-6DOF ManipulationSemi-6DOF Manipulation

Position (X, Y) & Orientation (0°- 360°)Position (X, Y) & Orientation (0°- 360°) Detection of contacting face (with the tablet)Detection of contacting face (with the tablet)

Simple, cord/battery-less deviceSimple, cord/battery-less device 2.5 × 4 × 5 cm, 222.5 × 4 × 5 cm, 22ｇｇ Easy to rotate in the handEasy to rotate in the hand Tactile impression tells its statusTactile impression tells its status

Can be used forCan be used for Tool selection (bimanual interfaces)Tool selection (bimanual interfaces) Multi-DOF control (e.g., virtual camera in VR)Multi-DOF control (e.g., virtual camera in VR) Physical license key (for software applications, etc.)Physical license key (for software applications, etc.)


Several different ways of Several different ways of holdingholding

a b c

d e f


ToolStone for bimanual ToolStone for bimanual interactioninteraction

NDHNDH (non dominant hand) (non dominant hand) for tool selectionfor tool selection

DHDH (dominant hand) (dominant hand) for tool manipulationfor tool manipulation


Assigning Assigning multiple tools for multiple tools for each angle and each angle and faceface

Control of parametersControl of parameters by moving a stoneby moving a stone

Pick&Drop by Pick&Drop by moving a stone moving a stone from one place from one place to anotherto another


Example 1: Toolpalette SelectionExample 1: Toolpalette Selection

8 directions × 6 faces = 48 different 8 directions × 6 faces = 48 different toolstools

= 8 directions

= 6 faces


Visual FeedbacksVisual Feedbacks

selected tool

indicating ToolStone direction


Mapping of command space Mapping of command space and physical manipulation and physical manipulation spacespace

Required physical Required physical manipulationmanipulation


Example 2: Color Example 2: Color SelectionSelection

Color selection is a 3D positioning task (HSV, RGColor selection is a 3D positioning task (HSV, RGB, …)B, …) Unnatural mapping is required with 2D devicesUnnatural mapping is required with 2D devices

Our solution: 2D Our solution: 2D DHDH pointing device pointing device + 1D + 1D NDHNDH toolstone movement toolstone movement

Other Apps:Other Apps:3D rotation,3D rotation,ZoomingZooming&panning&panning


Example 3: Virtual Camera Example 3: Virtual Camera ControlControl

Selecting the “camera” face Selecting the “camera” face automatically activates the 3D automatically activates the 3D walkthrough modewalkthrough mode

NDHNDH camera position camera position + orientation + orientation

DHDH view angle, view angle, elevation angleelevation angle


Sensing ArchitectureSensing Architecture

Uses an existing pen tablet (WACOM) Uses an existing pen tablet (WACOM) to recognize to recognize position, orientation, contacting face of position, orientation, contacting face of

the ToolStonethe ToolStone WACOM tablet can distinguish sensor coil WACOM tablet can distinguish sensor coil

types, position, and angletypes, position, and angle

Our ApproachOur Approach Embed 3 coils in the ToolStone deviceEmbed 3 coils in the ToolStone device each has a unique resonance frequencyeach has a unique resonance frequency

angles between tablet and coil (α≠β)

α

βα

β

coils


ααφφ


Trial UseTrial Use

Frequently used physical Frequently used physical operations sequence became a operations sequence became a chunkchunk

Sound (noise) caused by flipping Sound (noise) caused by flipping the device was noticeablethe device was noticeable

Visual guide is still necessary (for Visual guide is still necessary (for learning phase)learning phase)


Summary & Future Summary & Future DirectionsDirections TooStoneTooStone

a semi 6DOF input devicea semi 6DOF input device novel 3coil architecture enables position+orientnovel 3coil architecture enables position+orient

ation+face recognitionation+face recognition Future WorkFuture Work

Self contained device (gravity sensor + optical Self contained device (gravity sensor + optical mouse sensor)mouse sensor)

Shapes other than cubesShapes other than cubes

toolstone: effective use of the physical manipulation vocabularies of input devices

Business

toolstone device

camera face

toolstone wacom tablet

d devices

d rotation

toolpalette selection

toolstone semi

toolstone direction