20140520 technology mediated reminiscence (phd oral exam)

Wenn-Chieh Tsai, PhD Candidate, Graduate Institute Of Networking And Multimedia, National Taiwan University

Advisors: Jane Yung-jen Hsu PhD, Rung-Huei Liang PhD 2014.05.20

Technology Mediated ReminiscenceDesigning Interactions with Digital Mementos

What is reminiscence?

2

3


4


Physical vs. Digital Mementos 5

Traditional/Tangible Memory Aids 6

7

What if digital media are more than just a reflection?

Initial Questions

✤ How do we design a reminiscence aid with digital materials?

✤ How do we mediate reminiscence through digital artifacts?

✤ How do we extract significant mementos from personal collections?

✤ What is a preferred state of technology-mediated reminiscence?

8

Figure 1. An illustration of the pathways and deliverables between and among Interaction Design Researchers and

other HCI Researchers. The model emphasizes the production of artifacts as vehicles for embodying what “ought to

be” and that influence both the research and practice communities.

Second, interaction design researchers create artifacts that

provide concrete embodiments of theory and technical opportunities. These design exemplars then become an

appropriate conduit for the transfer of HCI research to the

practice community. Design artifacts are the currency of

design communication. In education they are the content

that teachers use to help design students understand what

design is and how the activity can be done [7]. In research,

they describe a vision of a preferred state, increasing the

chance for knowledge transfer to the research, practice, and

education communities. Through exposure to the ideas in

the artifacts, the practice community can more easily

observe the value of different theories, models, and technology, and this can motivate them to follow the

threads back to the original research that might most impact

their work.

Third, use of this model results in a holistic research

contribution that reveals the framing of the problem and the

balance the researchers have made between the intersecting

and conflicting perspectives. The idea of contributing a

whole closely resembles the work of systems engineers in

HCI who focus on building whole systems. The Aware

Home constructed at Georgia Tech provides a good

example [11]. In this case the novelty was not in the

construction of the individual elements, but in the integration of many technical research contributions from a

variety of disciplines, into a single working system. The

difference between this type of contribution and the design

research contributions we propose involve both the intent and the process of the research. In making a technical

contribution of a whole, engineers first develop a

specification of what they need to make to meet a specific

need. Next, they take a research focus asking questions

such as can this be built? Is there a better way to build this?

In proposing a model of design research with a focus on the

production of artifacts, we build on Nigel Cross’s idea that

design knowledge resides in the product [7]. The artifact

reflects a specific framing of the problem, and situates itself

in a constellation of other research artifacts that take on

similar framings or use radically different framings to address the same problem. These research artifacts provide

the catalyst and subject matter for discourse in the

community, with each new artifact continuing the

conversation. When several related research artifacts have

been created, then researchers can use more traditional

design research methods to analysis the artifacts and search

for similar approaches designers have taken in addressing

common problems. The artifacts made through design

research have the potential to become pre-patterns [6] from

which design patterns [1] can begin to emerge.

Our model departs from the roles of the design researcher

discovered in our literature review and interviews: (i) design researcher as member of design practice team doing

CHI 2007 Proceedings • Design Theory April 28-May 3, 2007 • San Jose, CA, USA

498

“how”

“true”

“real”

Zimmerman, J., Forlizzi, J., and Evenson, S. Research through design as a method for interaction design research in HCI. Proc. CHI 2007, 493–502.

Research for Design Research through Design

9

Our Approach

Expected Contributions

✤ Knowledge answering the question: “How would you design a [reminiscence aid]?”

✤ A series of artifacts

✤ A concrete problem framing

✤ Articulation of the preferred state

✤ Documentation of the design process

Cross, N. Designerly Ways of Knowing. Springer, 2006.10

Digital Memory Aids

✤ Utility-oriented Design

✤ “Memory Prosthesis”

✤ “Memory Augmentation”

Sparrow, B., Liu, J., and Wegner, D.M. Google effects on memory: cognitive consequences of having information at our fingertips. Science 333, 6043 (2011), 776–8.11

Living Memory Box

Stevens, M.M. et al. Getting into the Living Memory Box: family archives & holistic design. Personal and Ubiquitous Computing 7, 3-4 (2003), 210–216.

authentic

12

Lifelogging

Sellen, A.J., Fogg, A., Aitken, M., Hodges, S., Rother, C., and Wood, K. Do life-logging technologies support memory for the past? Proc. CHI 2007, 81–90.

recollecting

13

CIRCA

Alm, N., Dye, R., Gowans, G., Campbell, J., Astell, A., and Ellis, M. A communication support system for older people with dementia. Computer 40, 5 (2007), 35–41.

objective facts

14

CaraClock

Uriu, D., Shiratori, N., Hashimoto, S., Ishibashi, S., and Okude, N. CaraClock. Proc. CHI EA 2009, 3205–3210.

indexical

15

FM Radio

Petrelli, D., Villar, N., Dib, L., and St, P. FM Radio  : family interplay with sonic mementos. Proc. CHI 2010, 2371–2380.

evocative

16

Audiophotography

Frohlich, D. and Fennell, J. Sound, paper and memorabilia: resources for a simpler digital photography. Personal and Ubiquitous Computing 11, 2 (2006), 107–116.

reconstructive

17

Pensieve

Peesapati, S.T., Schwanda, V., Schultz, J., Lepage, M., Jeong, S., and Cosley, D. Pensieve: supporting everyday reminiscence. Proc. CHI 2010, 2027–2036.

dialogical

18

Definition

Memory Aid Reminiscence Aid

Interaction Attribute

Indexical Evocative

Interaction Attribute Monological DialogicalInteraction Attribute

Authentic Reconstructive

Output/Product Facts (Objective) Meaning (Subjective)

Human Experience Recollecting Interpreting/Reflecting

Research Approach Cognitive Psychology Ethnomethodology

19

A Preferred State

20

The Reflexive Printer

21

It prints and deletes one digital photo from a personal archive everyday.

Users can have the thermal printout participating in their live,

22

23

Restore its original back to the digital archive,

!

24

Or, only keep the tangible embodiment in their lifeworld.

3 Framing Artifacts, 3 Lessons

✤ How to initiate a reminiscence dialog? (The Retro Jukebox)

✤ Experiential resonance for being evocative

✤ How to intervene this dialog? (The Oblivescence Board)

✤ Self-presentation for being dialogical

✤ What to expect from this dialog? (The SoundTag)

✤ Alternative perspective for being reconstructive

25Ylirisku, S., Halttunen, V., Nuojua, J., and Juustila, A. Framing design in the third paradigm. Proc. CHI 2009, 1131–1140.

26

Reminiscence Mode interpersonal interpersonal intrapersonal

Digital Media music photo soundContext hospital dorm room in the wild

Participant14 inpatients (> 65 y/o),

1 research nurse

two 22 y/o female close friends 10 young adults

Method unstructured weekly meeting

experimental research

semi-structured interview

Evaluation usability testing,user feedback memory retention event records,

user experienceHCI Paradigm pragmatic cognitive situated

Focus evocative dialogical reconstructive

The Retro Jukebox The Oblivescence Board The SoundTag

NSC 99-2218-E-002-002 Research Report 2008-2011

The Retro JukeboxMusic Listening in a Ward

27

How might we evoke a reminiscence dialog through digital media?

28

The Retro Jukebox is a conversation aid used in bedside activity for elder inpatients and their care givers. In addition to a music playing function, three cognitive (one categorization and two memorization) games are integrated in the interaction design to maintain inpatients’ mental fitness.

Usability Testing

29

Findings & Reflection

✤ Empathic resonance

✤ “I’m not feeling well!” (Patient)

✤ “I feel like a saleswoman every time I step into the ward with the device.” (Research Nurse)

✤ Situated usage in wards

✤ impromptu, appropriated

✤ Music-mediated experiential resonance

✤ embodied rapport31

CHI 2013

The Oblivescence BoardPhoto Sharing in a Close Relationship

32

How might we shift responsibility from digital to personal memory in a reminiscence dialog?

33

The Oblivescence Board is a digital memory board for people to remember more. The photos shared on it will fade over time according to a forgetting curve as human memory. The fading photos can be revived through users’ selective viewing behavior. Our hypothesis is that a “forgetful” and self-presentative artifact would result in more users’ cognitive participation and retention of their valuable memories.

Transactive Memory System

Processor 1 Processor 2

Dir 1

Memory 1

Dir 2 Dir 2

Memory 2

Dir 1

Wegner, D.M. A computer network model of human transactive memory. Social Cognition 13, 3 (1995), 319–339.

Our Model

35

Ebbinghaus Forgetting Curve

(Min.)

(%)

self-presentation of forgetting

being reminded

Ebbinghaus, H. Memory: a contribution to experimental psychology. Teachers College, Columbia University, 1913.

General Hypothesis

• If the limitation of memory of a system is embodied in a reminiscence dialogue, transactive memory would be formed and encourage users to remember more

37

Field Experiment

38

Findings

• Keeping memories on fading photos no significant memory decay (Z=-1.890, p=.059>.05)

• Losing memories on constant photossignificant memory decay (Z=-2.271, p=.023<.05)

• Tapping more on fading photost=-12.248, p=.000<.01

• Viewing correlates with event richnessr=.468, p=.037<.05

39

Reflection

✤ Limitations in our experimental approach

✤ ownership of the system

✤ fixed dialogical pattern

✤ Require a situated approach to unfold the phenomenon and felt experiences in one’s life

40

DIS 2012, IASDR 2013

The SoundTagSoundscape Association in Everyday Life

41

How might we provoke self-reflection and reinterpretation on personal memories?

42

The SoundTag, a self-associative artifact, is used to explore the design space of using sounds as tags to annotate and associate event records. It intends to provide alternative perspectives for personal reminiscence in everyday life. We utilize timbre as the metric to determine the similarity between soundscapes of events and implement a proof-of-concept on smartphones.

What if we tag the world with sound?43

Human Artifact

Event Cluster

Reminiscence Dialog

44Brown, N.R. and Schopflocher, D. Event clusters: an organization of personal events in autobiographical memory. Psychological Science 9, 6 (1998), 470–475.

Human Artifact

Event ClusterAuditory-Association

Reminiscence Dialog

44Brown, N.R. and Schopflocher, D. Event clusters: an organization of personal events in autobiographical memory. Psychological Science 9, 6 (1998), 470–475.

Human Artifact

Event Cluster

Reminiscence Dialog

45

The SoundTaga self-associative artifact 46

47

user-subjectiveDeterministic

system-subjectiveRandom

intersubjectiveSelf-Associative

Findings & Reflection

✤ Encoded personal meaning in sound tags

✤ System limitation increase sensibility in situ

✤ Internalization of self-association

✤ Personal value on digital mementos

48

Technological Imagination?"…cultivating and shaping the technological imagination is a cultural imperative of the highest order" (Balsamo, 2011)

More Than Physical Form

50Vallgårda, A. Giving form to computational things: developing a practice of interaction design. Personal and Ubiquitous Computing 18, 3 (2013), 577–592.

The Design Space

Interaction Attribute Evocative Dialogical Reconstructive

IntendedOutcome Self-initiation Responsibility-

Shifting Reinterpretation

HumanValue Empathy Subjectivity Reflexivity

TechnologicalMediation

Experiential Resonance Self-presentation Alternative

Perspective

DesignQuality

Familiarity, Serendipity

Extemporaneity, Negotiability

Ambiguity, Perceived

Drawbacks

51

DIS 2014

The Reflexive PrinterPerceived Drawbacks in Technology-Mediated Reminiscence

52

3 Themes (Norms)

✤ The normality of digital photo consumption:

✤ Powerful Artifact and Feeble User

✤ Fast Consumption and Slow Rumination

✤ Simple Materiality and Monological Performance

54

3 Designed Drawbacks

✤ One-bit Halftone

✤ memory recollection

✤ Slow Rumination

✤ memory storage

✤ Material Decay

✤ memory retention

55

Prototype for Probing

✤ 4 Users

✤ 1 Week

✤ Initial & Final Interviews

✤ Diary Method

56

Lessons Learned

60

✤ Just Enough Information

✤ Feeling of Being Accompanied

✤ Biomorphic Form

62

Field Study

✤ Two (one male and one female) young adults

✤ Digital photo natives

✤ Retrospective interviews (elicited by diaries)

Participant YT

2013.12.10

installation in YT’s room

12.12 12.20

1st week interview& app reinstall

3rd week interview

12.25

double prints

2014.1.3(print at 10:00pm) (print at 9:00am)

!

63

initial interview

Participant YJ

initial interview(pilot study)

2013.11.14

installation in YJ’s room

12.25

2nd week interview

2014.1.6

pilot final interview

11.26(print at 8:00am)

64

Findings

Humble form and function initiate intersubjective dialog

!65

✤ “The photo was printed when I woke up, and a series of characters were printed: ‘UUUUUU.’ It feels like it was talking to me by saying ‘you you you you you’, I was like, are you talking to me?” (YT)

✤ “It is not a robot, and it is for sure not a printer. Oh, yes, it should have a ‘wanting to communicate with me about something’ kind of setting.” (YJ)

66

Findings

Minimal information encourages active and personal interpretation

67

✤ “Although it is in black and white, it enables me to recall the colors and sound from the memory. [...] It is easier for me to recall the feeling of that moment when the photo is blurry. [...] That feeling is like, there is no word to explain or be written, maybe, it is just different from the real photo. Maybe it is the medium.” (YT)

68

Findings

Materiality enriches the context for reminiscence

69

✤ “I’ll bring it with me. I was thinking that maybe when I am free I could just conveniently take it out [of my pocket], because if I put it on the shelf, it could too easy get blown away by the wind.” (YJ)

✤ “I could simply take it from my pocket, but then I find out that it is not in these pants, so I have to go back and look for it” (YJ).

70

Findings

Embodiment anchors expectations in the routine grounds of everyday activities

!71

✤ “I imagine it as something that is always there. It gives me a sort of expectation and curiosity to find out which photo will be printed. And it could become something that I can interact with when I am free, or put aside when I am busy.” (YJ)

✤ “In the morning, I thought I heard the printing sound from the machine. I felt excited, but I continued my sleep. When I woke up, I found no printed photos—it was like I was dreaming” (YT)

72

Contributions

✤ 3 framing artifacts and 1 design example

✤ The design space of technology-mediated reminiscence

✤ 1 experiential design quality

✤ Design implications for reminiscence aids

73

Demo VideoThe Preferred State

https://docs.google.com/file/d/0B-rOc4S9n-4yeFVVR0RZZVRiaVE/edit





The Design Space

Evocative Dialogical Reconstructive

IntendedOutcome Self-initiation Responsibility-

Shifting Reinterpretation

HumanValue Empathy Subjectivity Reflexivity

TechnologicalMediation

Experiential Resonance Self-presentation Alternative

Perspective

DesignQuality

Familiarity, Serendipity

Extemporaneity, Negotiability

Ambiguity, Perceived

Drawbacks

76

Related Publications

77

✤ Tsai, W.-C. (2014). Technology-mediated reminiscence: designing interactions with digital mementos (Doctoral dissertation, National Taiwan University, Taiwan).

✤ Tsai, W.-C., Wang, P.-H., Lee, H.-C., Liang, R.-H., and Hsu, J.Y. (2014, accepted). The Reflexive Printer: toward making sense of perceived drawbacks in technology-mediated reminiscence. In proceedings of the 10th ACM Conference on Designing Interactive Systems (DIS 2014). (Full paper oral presentation)

✤ Tsai, W.-C., Hsiao, J.C.-Y., Lee, H.-C., Huang, C.-H., Hu, J.-C., Liang, R.-H., and Hsu, J.Y. (2013). Designing a reminiscence aid in personal soundscape. In proceedings of the 5th International Congress of International Association of Societies of Design Research (IASDR 2013), 5150–5160. (Full paper oral presentation)

✤ Tsai, W.-C., Lee, H.-C., Hsiao, J.C.-Y., Liang, R.-H., and Hsu, J.Y. (2013). Framing design of reminiscence aids with transactive memory theory. In extended abstracts of the 2013 ACM Conference on Human Factors in Computing Systems (CHI 2013), 331–336. (Work-in-Progress extended abstract and poster presentation)

✤ Tsai, W.-C., Hsiao, C.-Y., Lee, H.-C., Huang, C.-H., and Hsu, J.Y. (2012). In search of lost sounds: designing a reminiscence aid in everyday soundscape. In poster abstracts of the 9th ACM Conference on Designing Interactive Systems (DIS 2012). (Poster abstract and poster presentation)

Q & A

78