Daniel A. TaylorUniversity of Pitt-Bradford

IntroductionIntroduction

Touch sensitivity is fairly common in electronics today…

…but only for a single point of contact at any given time.

Touch-sensitive devices that allow for multiple points of contact have thus far been expensive or difficult to produce.

Why multi-touch?

Allows a user to interact with a system with more than one finger at onceTouch KeyboardsOperations using both hands

Also allows multiple users to interact on the same touch-sensitive platformInteractive wallsInteractive tabletops

Prior approaches to the problem Matrix of smaller sensors

Requires many connections, which severely limits resolution

Visually translucent - no touch-screen

Video camerasMeasuring of brightnessIdentifying tracking markers in gel

A new approach…

FRUSTRATED TOTAL INTERNAL REFLECTION

Total Internal Reflection

When light encounters a medium with a lower index of refraction (e.g., going from glass to air), its refraction depends on the angle at which it hits the border.

Beyond a certain critical angle, light is not refracted, but instead reflects entirely within the material.This is the basis for fiber optics and other

optical wave guides.

Frustrated Total Internal Reflection If another material touches that within

which the light is reflecting, the reflection is frustrated, causing the light to escape.

This has been used in the past:fingerprint imagingearly touch sensors (1970s!)tactile sensors for robotic grips

A Schematic of FTIR

(Kasday, 1984)

Using FTIR for touch sensitivity A clear acrylic sheet is used as the

touch surface.28 inches x 24 inches in prototype

Edges of surface lit by infra-red LEDs to produce total internal reflection.

A video camera is mounted under the surface and facing it.

When the surface is touched, the light escapes and registers on the camera.

Using FTIR for touch sensitivity Basic image-processing techniques are

performed on the camera output to identify the points of contact.

Computer-vision techniques are used to interpret the motion of contact points as discrete touches or strokes.

Processing easily handled in real-time by a 2 GHz Pentium IV processor

Advantages to this approach High capture rate and resolution

30 frames per second640x480

True zero-force touch sensitivity Inexpensive to construct Scalable to much larger (even wall-

sized!) surfaces Transparent: can be combined with rear-

projection display

How is projection achieved?

Disadvantages

Requires significant space behind touch surface for camera

Gloves, certain types of styluses, and even dry skin may not registera function of refractive index

Residues on surface (e.g., sweat) also produce FTIR effect that may build up

Any Questions?