Dag Svanæs

Interaction Design for and with the Lived Body –

Some Implications of Merleau-Ponty’s Phenomenology

Dag Svanæs NTNU, Trondheim, Norway ITU, Copenhagen, Denmark

Presentation at CHI 2013, Paris

Dag Svanæs

Overview

•  Motivations for a theory of the body? •  The body in early HCI theory. •  Merleau-Ponty: The lived body. •  The interactive user experience. •  Reframing interaction as active embodied

perception. •  Examples of active perception. •  Designing with the intelligent body. •  Implications for design. •  Future work.

Dag Svanæs

Motivations for a theory of the body

•  Current digital products are designed for active bodies. They are often held, worn or carried.

•  Their success depends on an effortless integration with the living body of the user.

•  The success of a product like Google Glass is about more than ergonomics and usability.

Dag Svanæs

The body in early HCI theory

•  Card, Moran, and Newell: “The Psychology of Human-Computer Interaction”, 1983.

•  Cognitive science: The body is external to cognition. The body makes possible input and output, just like the peripherals of a computer.

•  Mind “has” a body. •  Blind spots: The way “body”

shapes “mind” and vise versa.

Dag Svanæs

Maurice Merleau-Ponty (1908-61)

•  Merleau-Ponty: “The Phenomenology of Perception”, 1945.

•  Heidegger refers to the distinction between Körper and Leib in German:

–  Körper is the body as a material object. (3rd person perspective).

–  Leib is the body as that through which I live my life (1st person perspective).

•  Merleau-Ponty: The lived body = Leib. (We are living bodies).

•  Merleau-Ponty starts out with a phenomenological analysis of perception from a 1st person perspective.

Dag Svanæs

Perception is active and directed

Eye Movement Studies (Yarbus, 1967): –  We actively construct our inner

image of the world. –  Perception is always directed

towards something. –  Millisecond level (“pre-cognitive”,

faster than cognition).

Perception! (passive)!

Action! (active)!

Early cognitive psychology: –  Perception is passive. –  Information paradigm. –  Action is different from

perception. –  Second and minute level.

Dag Svanæs

Task/intention colours perception

•  Same image, three different tasks. •  Different viewing patterns. •  Perception is not separate from cognition. •  We live 1/12 second in the past.

Dag Svanæs

Skills colour perception

Eye tracking of layperson (left) vs. artist (right) seeing the same work of art. Training as an artist changes the way people see pictures. Artists “see” all areas of the picture, while most of us focus on faces and objects in the scene.

Layperson Artist

Dag Svanæs

Perception involves the whole body

•  We explore objects with many senses.

•  The exploration is an active process.

•  We move, rotate, touch, smell, taste, squeeze the object and change our viewpoint.

Dag Svanæs

Instrument

•  Merleau-Ponty uses “instrument” to denote artefacts that enable the body to both act on the world with them and to sense the world through them.

•  A blind man's stick (white cane). •  Perception through the cane

requires active perception. •  The cane becomes an extension

of the body (from the 1st person perspective).

Dag Svanæs

Body schema

•  The body constantly maintains knowledge about its position (the limbs etc.), its structure, and its potential for action.

•  This is the body schema. •  The structure includes

the instruments/tools that have been integrated into its lived body.

Dag Svanæs

The body in everyday coping

•  Moving around and dealing with objects in our environment does not require “thinking”.

•  The body learns new skills and easily integrates new tools/instruments.

•  The body is intelligent! •  Challenge: Interaction

design for the intelligent body.

Dag Svanæs

Exploring Interactivity

”13 Rectangles” Kandinsky

Form + Color Form + Color + Interaction

Abstract interactive squares • What stories do people tell? • What metaphors emerge? • What dimensions emerge?

Dag Svanæs

Experiment ”Square world”!

15 high school students (age 16-17)."Explore the gadgets"Think aloud"Implicit metaphors (Lakoff & Johnson)"

“Understanding Interactivity”, Svanæs, 2000.

Dag Svanæs

Interaction patterns – very rapid

Millisecond scale

Dag Svanæs

Interaction = Active perception

•  When we interact with an interactive artefact, this is a kind of active perception.

•  This involves active perception at two levels.

1.  Eyeballs: The visual image is actively constructed.

2.  Eye-hand-artefact: The interaction is a sequence of action-reaction pairs.

Interaction gestalt

Dag Svanæs

Interaction as Perception: Scrolling

•  The scrolling wheel allows for active perception.

•  Using the scrolling wheel is very different from dragging the scrollbar: Active perception vs. Action.

•  Active perception is automated. Little cognitive workload.

Dag Svanæs

Interaction as Perception: Reading

•  Turning pages in a book while reading is part of active embodied perception.

•  eBook readers: Do they support active perception?

Dag Svanæs

Electronic Medical Records

•  Turning pages in a paper-based medical record is automated.

•  It is part of the embodied active reading process.

•  Gives good eye contact with patient.

•  Both smartphones and laptops-on- wheels required “actions”.

•  Gives little eye contact with patient.

Paper

Laptop on wheels

Smartphone

Dag Svanæs

Gaze-controlled scrolling

•  Automatic vertical scrolling of text, controlled by eye tracker during reading.

•  Feels like reading an infinite text!

•  The scrolling mechanism becomes an extension of the sensory apparatus (an instrument).

EyeScroll. Kumar & Winograd, 2007.

Dag Svanæs

Designing with the body

•  Merleau-Ponty: Abstract vs. concrete movements.

•  Concrete movements: Done as part of an activity in a context, no focus on the movement as such.

•  Abstract movements: Movements done “out of context”, to try out, to illustrate, to communicate.

Dag Svanæs

Nintendo Wii in physical rehabilitation

Dag Svanæs

Participatory design workshop

•  Lessons learned: •  The body is an important

resource in the design process.

•  Acting out different design alternatives opens up for kinaesthetic creativity.

•  Designing for the intelligent body should be done with the intelligent body, not through design representations like drawings and text.

Physiotherapists inventing a Nintendo Wii game for their patients through acting out.

Dag Svanæs

Implications for design

•  Reframe interaction: For many applications it makes sense to think of interaction as active perception.

•  Speed: Consider interaction techniques that allow for very rapid coupling between user actions and system feedback. Design at the millisecond level: faster than cognition.

•  Mapping: Fluid integration with the intelligent body requires action-reaction mappings that are easily “understood” by the body.

•  Running prototypes: Technology for the intelligent body requires high-fidelity prototypes .

•  Use the body in design: Interaction design for the intelligent body should primarily be done with the intelligent body – not through design representations.

Dag Svanæs

Future work: Designing for the body

•  Research-through-design project: –  Exploring the design

space of artificial human tails.

–  Applying the design principles from Merleau-Ponty.

–  What does it take to make an artificial tail that is fully incorporated into the human body schema?

Shippo by Neurowear

Monkey with long tail

Dag Svanæs

Future work: Designing for the body

•  Research-through-design project: –  Exploring the design

space of artificial human tails.

–  Applying the design principles from Merleau-Ponty.

–  What does it take to make an artificial tail that is fully incorporated into the human body schema?

Shippo by Neurowear

Monkey with long tail

Questions?