Touch & Talk:Contextualizing Remote Touch for

Affective Interaction

Rongrong Wang (wangr06@vt.edu)Francis Quek (quek@vt.edu)

Computer Science Department &Center for Human Computer Interaction

Virginia TechJan 25, 2010

► video

Current Communication Milieu

► People communicate not only for info. but also for emotion conveyance.

► Three modalities dominate:

Audio Text Video

► Touch is largely ignored

Why Touch is Important?

► Touch is essential for human development and communication

► Touch is unique► Touch is immediate (i.e., conveys

affect directly without symbolic interpretation)

Touch: An essential channel to convey affect

Touch is important to developmentThe wire-mother-fed monkeys appeared to be pathological violent and aggressive, while the terry-cloth-fed monkeys (though not completely normal) did not express such characteristics.

Harlow, H.F.: The Nature of Love. American Psychologist 13 (1958) 673-685

Affective Touch in Humans

Touch is an important component of intimacy

Register et.al.: non-verbal communication is critical in the experience of intimacy, in ways that words alone cannot convey › They write: “It seems that there is an element of

intimacy that is more accurately expressed via other sensory modalities, such as sight and touch, …”

* Register, L.M., Henley, T.B.: The Phenomenology of Intimacy J. Soc. & Pers. Relationships 9 (1992) 467-481

Touch is Unique

In a discussion of the physiological perspectives of touch, Collier* concludes that “Touch should not be broken down into

separate features which are then used independently, or divorced from the total context in which it occurs.”

Touch is a rich and unique form of human interaction that may not be easily expressed in its logical constituent parts.

Collier, G.: Touch. Emotional Expression. Lawrence Erlbaum Associates (1985)

Touch is Immediate

Immediate vs “Symbolically” Mediated

The difference is not trivial: › If touch is immediate, it holds a unique

place that is not easily subsumed by some other modality.

› If it is mediated through symbol, it could be otherwise mediated.

› We argue that it is this immediacy that makes the exploration of digitally conveyed touch important.

Touch: a Contextualized Medium

The meaning of touch is significantly shaped by context.

Jones and Yarborough (1985): There is no one-to-one correspondence between meanings and touch behaviors. › The same touch may have different

meanings› Different touches may share similar

meaningsJones, S.E., Yarbrough, A.E.: A naturalistic study of the meanings of touch. Comm. monographs 52 (1985)

Previous Work in Remote Touch1986 White & Back Telephonic arm wrestling

1996 Strong & Gaver Shaker

1997 Dodge The Bed

1997 Brave & Dahley inTouch

2001 Nilsen & Vrana Vibrobod

2001 Chang et al LumiTouch

2001 Hansson & Skog LoveBomb

2002 Oakley & O’Modhrain Contact IM

2007 Motamedi Fabric screen

2005 Mueller & DiSalvo The Hug

2006 Bonanni et al TapTap2002 Chang et al ComTouch

2007 Brown & Williamson Shake2Talk

2008 Chan et al Tactile mouse for turn-taking

2007 Adcock et al Tug n’ Talk

2001 Alhalabi etal Tele-Handshake

1998 Fogg et al HandJive

2004 Rovers & Essen HIM

2006 Rovers & Essen FootIO

Making remote touch immediate

► Either through explicit design or user choice, previous systems were used to convey symbolic messages

► We argue that haptic touch applied without context reduces to low-bit-rate signaling channels

► Our study explores how context empowers remote touch to retain its privileged position of immediate affective conveyance

Our Haptic Device► Armband

Shape Memory Alloy

► Squeeze Object Flexiforce Force Sensor

►Microcontroller Board BasicAtom Pro 24

Experiment►Subjects listen to an emotive story recording

which is told by a professional actress

► In pre-study, another pool of subjects ranked key points in the story for affective meaning and emphasis

0

5

10

15

20

25

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Experiment (cont.)

►Measure subjects’ emotional state before and after the story session under 2 conditions With-Touch Without-Touch

►Goal: To find out whether remote touch channel can make any difference to subjects in emotional state

Subjects

► 16 subjects (6 females, 10 males) 14 are native English-speakers 2 have spoken English for more than 8 years i.e., subjects have no language problems

► Recruited via SONA system maintained by Psychology Department and CS graduate student mail-list. › Study balanced participants from both sources

The average age is 23.6

Measurements Pre/Post PANAS (Positive Affect Negative

Affect Schedules)› consists of 60 words of feelings and emotion› participants are asked to mark each word,

using a scale, according to how they are feeling right now.

Cheerfully (Not At All) 1 7

(Extremely)

Results► Compare emotion change cross conditions

(b) Mean Gain in Positive Emotion

T(13)=-1.4, p<0.1

(c) Mean Reduction in Sad Emotion

(a) Mean Reduction in Negative Emotion

(d) Mean Gain in Joviality Emotion

Our results show

remote touch reduces sadness significantly. It suggests that the touch amplified the effect of the story over the non-touch condition.

a trend to reduce general negative mood and to reinforce joviality

positive affect measures have more to do with excitement, alertness, attentiveness etc. – all of which relate to ‘positive energy’ rather than the emotions in our story.

Thanks for listening!

Questions ?