amazon s3 · the thesis of greg t. elliott is approved and is acceptable in quality and form for...
TRANSCRIPT
U C,I
Exploring the Role of Efficiency Through Device Design
T
submitted in partial satisfaction of the requirementsfor the degree of
M S
in Information & Computer Science
by
Greg T. Elliott
Thesis Committee:Professor Simon Penny, Chair
Professor Paul DourishAssistant Professor Bill Tomlinson
2007
c© 2007 Greg T. Elliott
The thesis of Greg T. Elliottis approved and is acceptable in qualityand form for publication on microfilm:
Committee Chair
University of California, Irvine2007
ii
T C
L F
A
A T
1 D A A E R Efficiency L 11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Personalizing Efficiency . . . . . . . . . . . . . . . . . . . . . . . . 31.3 Related Design Genre: “Slow Technology” . . . . . . . . . . . . 51.4 How Reflective Design, Ludic Design, and Critical Design can
benefit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.5 Exploring Efficiency in the tradition of CTP . . . . . . . . . . . . 91.6 Abstract Discussions of Efficiency . . . . . . . . . . . . . . . . . . 101.7 The Drift Table: A good start . . . . . . . . . . . . . . . . . . . . 141.8 Early Cinema as an Exploration of Efficiency . . . . . . . . . . . . 161.9 Another proposed method: Interruption . . . . . . . . . . . . . . 161.10 Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 H C T R , R , P I E 192.1 Software and Efficiency . . . . . . . . . . . . . . . . . . . . . . . . 192.2 A Brief History of the Effects of Technology on Efficiency . . . . . 292.3 Objectivity allows for the generalization of Efficiency . . . . . . . 31
3 E O S: T P C PS- 343.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.2 Summary of PersonalSoundtrack . . . . . . . . . . . . . . . . . . 373.3 Origins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.4 Machine Adaption and Symbiosis . . . . . . . . . . . . . . . . . 423.5 Mind, Body, Ears . . . . . . . . . . . . . . . . . . . . . . . . . . . 463.6 The Curse of ‘Context-Aware’ . . . . . . . . . . . . . . . . . . . . 483.7 Final Thoughts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
4 PS: E Efficiency I 544.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.2 What is PublicSoundtrack? . . . . . . . . . . . . . . . . . . . . . . 544.3 Private and Public Space . . . . . . . . . . . . . . . . . . . . . . . 574.4 Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 654.5 Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
iii
5 T D 745.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 745.2 Design Achievements . . . . . . . . . . . . . . . . . . . . . . . . . 745.3 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755.4 Software and Networking . . . . . . . . . . . . . . . . . . . . . . 825.5 Control Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 955.6 Final Notes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6 C 98
A PS 107A.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107A.2 Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108A.3 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108A.4 Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109A.5 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111A.6 Tempo and Context-Aware Music . . . . . . . . . . . . . . . . . . 112A.7 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113A.8 Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113A.9 User Evaluation in Progress . . . . . . . . . . . . . . . . . . . . . 115A.10 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116A.11 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117A.12 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . 117
iv
L F
3.1 PersonalSoundtrack, 3rd Iteration . . . . . . . . . . . . . . . . . . 39
5.1 Schematic of Step-Detection Circuit . . . . . . . . . . . . . . . . . 785.2 Step-Detection Circuit . . . . . . . . . . . . . . . . . . . . . . . . 805.3 Abstract Hardware Component Breakdown . . . . . . . . . . . . 97
A.1 PersonalSoundtrack system diagram showing inputs, actions,and flow of control . . . . . . . . . . . . . . . . . . . . . . . . . . 109
v
Acknowledgments
Simon Penny - Your philosophical and spiritual guidance made this workpossible. Thank you for not setting artificial limits on my ideas and forallowing me to explore them completely. Thank you for wonderful andenlightening discussions over coffee. Thank you for reminding me that I amand always will be an artist, first and foremost.
Paul Dourish - Your ability to crystallize my jumbled thoughts made thisdocument legible. Thank you for sending me a million and one papers,making me wonder how you found such obscure yet amazingly relevantwork. Thank you for teaching me how to write about complex andtroublesome issues.
Bill Tomlinson - Your dedication and insight helped me ground my work.Thank you for helping me see my own work in ways I hadn’t imagined.Thank you for being a lifeline when I got stuck in the mud. I will misswandering around Montreal around 2:00 AM.
David Kirsh - Thank you for everything you’ve done for me. I’ll leave it at thatsince there is too much for which to thank you.
Simon, Beatriz, Bill, Robert, Paul - Thank you for the ACE program. I willcherish this experience for the rest of my life.
Thank you to the Department of Information and Computer Science and theCalifornia Institute for Telecommunications and Information Technology forfunding my research.
Hanna - Thank you for being you. Thank you for talking with me as I workedthrough what must have felt like endless issues. Thank you for beinginterested in and helping me with the dregs of my ideas. Thank you for yourlove, support, and for making me laugh.
Aaron - Thanks for being my academic other half.
Dan - Thanks for bringing levity to the grim days. Actually, thanks for nothing.
Julie, Nyta & Pops, and the rest of the family - Thank you for always being therefor me.
Dad - I don’t know what I’d do without you. Thanks for being my best friend.
Mom - You are with me always. Words cannot express my gratitude.
vi
A T
Exploring the Role of Efficiency Through Device Design
By
Greg T. Elliott
Masters of Science in Information & Computer Science
University of California, Irvine, 2007
Professor Simon Penny, Chair
Despite the wealth of discussions about the ideology of efficiency (Efficiency),
individuals rarely reflect on how they personally define, evaluate, and act on
the ideology of efficiency. We have few ways to conceptualize Efficiency
without resorting to global economics, social theory, or user studies, and as
such it can be difficult to grasp how Efficiency impacts our daily tasks, thought
processes, and even moments of pleasure. The way I experience Efficiency is
potentially unique to me, not only because it impacts my life at the cultural
and individual level, but also because instantiations of Efficiency are subjective
and personal. Abstractly, we are all affected by this ideology similarly; in
practice, however, the ways it nuances my life are different than the ways it
nuances yours. Artistic practice allows us to ground the abstract
conceptualization of Efficiency in individual experience unique to each person.
I propose a genre of artifact design that allows the individual to explore the
role of Efficiency within the context of daily life, moving us away from abstract,
objective views of the ideology of efficiency.
vii
C 1
D A A E R Efficiency L
1.1 I
When we discuss the term efficiency, it is important to first separate efficiency
as ideology from efficiency as practice. The ideology of efficiency, from here
on referred to as Efficiency, is the grouping of a series of ideas that gives us the
impression of what ‘efficiency’ means. It is this abstract representation that
encompasses the many ways that Efficiency is instantiated in life. Often, when
we speak about Efficiency, we are referring to particular facets of the ideology.
In practice, Efficiency often takes shape as an influence in a production,
time-discipline, optimization, and cultural value. Production is made efficient
through the minimization of waste and the maximization of output.
Time-discipline is heavily dependent on Efficiency since it refers to the ways we
use, manage and construct expectations based on the measurement of time.
Optimization is a direct instantiation of Efficiency as the general notion that
any process can be analyzed and adjusted to improve the ratio of production
to waste. Cultural values that stem from Efficiency are expectations about the
overall benefit of Efficiency, the desire for it in processes and objects and
ourselves, and the ways inefficiency is not desirable.
While the role of Efficiency as both ideology and instantiation has been
well-traversed [78, 30, 5, 7, 47, 77, 69, 1], the rise of ubiquitous technology
1
provides opportunities for a revived discussion of Efficiency. By embedding
computational technologies into the environment, we also embed the value
systems those technologies contain [79]. In the case of computational systems,
the primary value system is one of Efficiency. Our perceptions and
conceptualizations of the world are further influenced and shaped by these
value systems. This reveals the necessity for a close examination of what these
technological systems represent, as well as how we experience them on a
personal level. How is Efficiency felt by each of us, how do we perceive it, and
where do we find it in the context of daily life? Does Efficiency live only in
stereotypically work-based situations, or has it bled into other, more personal
aspects of life as well?
Despite the wealth of discussions about the ideology of efficiency,
individuals rarely reflect on how they personally define, evaluate, and act on
the ideology of efficiency. We have few ways to conceptualize Efficiency
without resorting to global economics, social theory, or user studies, and as
such it can be difficult to grasp how Efficiency impacts our daily tasks, thought
processes, and even moments of pleasure. The way I experience Efficiency is
potentially unique to me, not only because it impacts my life at the cultural
and individual level, but also because instantiations of Efficiency are subjective
and personal. While at an abstract level, we all are impacted by Efficiency in
roughly the same way, the ways it nuances my life are different than the ways it
nuances yours. Efficiency is easy to generalize away from personal experience,
as I will discuss, even when written about through personal narrative.
Artistic practice, however, allows us to ground Efficiency in individual
experience unique to each person. I propose a genre of artifact design that
allows the individual to explore the role of Efficiency within the context of daily
life, moving us away from the abstract, objective views of Efficiency.
2
1.2 P Efficiency
While there are many ways Efficiency is talked and written about, the design
genre I propose here aims to help us define and contextualize Efficiency for
ourselves. Efficiency is felt in our lives not in abstract, objective ways, but in
concrete, subjective ways. In practice, instantiations of Efficiency are defined
and evaluated by the person(s) determining what is efficient through the
exclusion and inclusion of specific data [73]. It is a personal term by nature,
and yet our treatment of that term exists primarily in text that is abstracted
away from our personal experiences. I suggest designing objects that help us
examine the role of Efficiency through an exploration of its subjectivity and the
multiple definitions of Efficiency. That is, we can create objects that do not seek
(and aggressively refuse) a singular definition, and help us experience
Efficiency as the fuzzy, intimate, personal, and messy experience that it is.
It is important to note that the design genre I propose does not seek to pass
judgement on Efficiency. Instead, we can think of the proposed design genre as
an attempt to reveal how and when Efficiency influences us.
In addition, it is easy to assume that tasks indicate the presence of Efficiency.
For example, it’s doubtful that scenarios like ‘waiting at a bus stop’ or
‘listening to music’ make us think of Efficiency, but writing a thesis chapter
might. We may assume we know when we are being efficient, and that we
know when we are not. These are assumptions about the nature of efficiency
as an ideology that need to be deconstructed. What does waiting for a bus
imply? Does it imply boredom, wasting time, or filling time with trivial tasks?
These assumptions do not consider the reach of the ideology of efficiency. A
person waiting at a bus stop could be mentally reorganizing their schedule for
the next day or working out their finances. In that case, the person may feel
Efficiency as anxiety about doing ‘nothing,’ or as a need to do something with
3
their idle time. We can’t assume that Efficiency is absent because the scenarios
are not stereotypically work-like. The nature of the ideology is that it worms
its way into everything, affecting each of us so intimately that perhaps we do
not recognize its influence anymore. I have heard stories of people that dance
in the shower to get their daily exercise when they have back-to-back meetings
all day. The obsession with Efficiency can shape what we do, how we think,
and how we perceive our world. By revealing these moments, each of which is
personal and likely unique to the individual, we begin to uncover the shape of
Efficiency and where it lives.
Efficiency is so pervasive that even the refusal to be efficient has meaning:
someone decides to sleep in late and finds joy in refusing to get up ‘on-time.’
The fact that one could react pleasurably indicates the ubiquity of Efficiency
where a resistance to the dominant ideology, in this case time-discipline, feels
good. Again, the presence of an efficiency ideology is not surprising or novel,
but it has not been explored well at such an intimate level.
Efficiency is embedded in culture through many channels, one of which is
increasing use of ubiquitous computational technologies. As I will discuss in
detail in the next chapter, the domain of computational technologies is born of
an efficiency ideology and perpetuates that value system through both how it
is constructed and what it produces. Efficiency is embedded in the values of
computation, and when we populate the world with software and hardware
objects we also populate the world with the value system [79], in this case with
Efficiency. We begin to perceive and construct our notion of the world through
the objects we interact with, where the values implicit in objects like laptops
shape the way I understand my world [79]. Many computational devices are
personal items that we interact with on a daily basis; each time we interact
with an efficient machine, we interact with its value system. The design of
4
computational technologies to explore Efficiency re-appropriates this popular
channel for Efficiency dissemination, providing us with a means to question
and reflect on it.
1.3 R D G: “S T”
A design style called “Slow Technology” is a genre related to the one I
propose, but instead of exploring Efficiency through design seeks to avoid
Efficiency in design. Inherent in its rejection of efficient tool design are broad
assumptions about the ideology of efficiency. Because Slow Technology
proposes an alternative to the efficient computational devices that surround us
daily, it could benefit from a better understanding of how the efficiency of
those devices impacts us personally.
Slow Technology is the creation of computational designs for moments of
reflection as a reaction to the dominant paradigm of efficient tool design [35].
While in general this is not problematic, Hallnas and Redstrom’s justification
for its existence reveals outdated assumptions about the nature of Efficiency.
Slow Technology was developed because computer technology is no longer
limited to office workers and scientists, who primarily rely on it for the
efficient execution of tasks, and thus we need to reconsider how such
technology is designed [35]. They propose that outside of offices and labs we
should design for non-efficient experiences. Unfortunately, this assumes that
life outside the office is markedly different than life inside the office. For the
most part, that may be true (despite the two quickly converging); however,
Efficiency doesn’t live in the top drawer of your work desk - it follows you
home and influences every aspect of your life. Slow Technology is an attempt
to design for something other than efficiency, but perhaps before we tackle
5
that challenge, we can benefit significantly from a consideration of how each
of us conceptualizes and puts into practice the ideology of efficiency, and the
ways in which we differ in that process. Slow Technology is a response to an
assumed perception of Efficiency, and designs that are alternative to that
assumption may not be alternative to a well-informed perception of Efficiency.
For example, Hallnas and Redstrom refer to Efficiency only in terms of
computational tools that make us more efficient at completing tasks. They
suggest Slow Technology as a means to help us stretch time, where efficient
tools help us shrink time (tasks take less time to complete and tools are quickly
learned and usable). As a design genre, Slow Technology invites us to reflect
on time itself, allowing us to elongate and play with it. And yet ‘efficient’ and
‘quick’ are related but not identical. Efficiency may be evaluated along the time
dimension, but it need not include that dimension. Sometimes efficiency is the
‘best’ solution given a set of resources (e.g. circuit design). Time is only one
way to slice Efficiency, and to pigeonhole efficient tools as those that reduce
time not only underestimates the nature of Efficiency, but also limits our
perception of how these tools influence us. Some tools are efficient because
they present data in a way that hopes to maximize our understanding and
ability to manipulate that data (there are too many software tools to list as
examples). Some tools cross into unusual areas, helping us achieve the most
efficient nap possible [67]. How are these notions of efficiency influencing us?
Designs that help us reflect on time do not relate to these kinds of efficient
computational tools. Hallnas and Redstrom clearly did not intend to explore
the realm of Efficiency with their designs, but instead hoped to inspire us to
slow down and help us reflect on both technology itself as well as time. Yet
their assumptions highlight our tendency to generalize and speak about
Efficiency in broad strokes, such as along the dimension of time, often ignoring
6
the capacity for this ideology to structure our world. That kind of abstract
definition is created by looking at Efficiency through a keyhole. It is difficult to
generalize instantiations of Efficiency without creating an incomplete
representation because of its subjective and context-sensitive nature.
The point here is that in order to see where we’re going, we need to
understand where we’ve been. How can we propose alternatives to efficient
design when we know so little about how efficient design impacts our lives? If
the goal is to create artifacts that impact us in ways that are different than
efficient tools, we have to explore how efficient tools impacts us within the
context of everyday life. Reworking everyday artifacts to inspire intimate
reflection on Efficiency can help each of us uncover how the ideology of
efficiency acts as a scaffolding on which we construct our representations of
the world.
1.4 H R D, L D,
C D
Sengers’ Reflective Design, founded on other design genres [17, 18, 28],
focuses on exposing the value systems inherent in technological design. The
design genre I propose is not only critical for moving beyond the dominant
‘technology as a tool’ paradigm as Hallnas and Redstrom has attempted to do,
but also can benefit the work of Sengers, Dunne, Gaver, and others.
Instead of looking at reflection as a counter to Efficiency as in Slow
Technology, we might instead think of Efficiency as a structuring agent that
limits our ability to improve and implement alternative design genres.
Computational technologies are primarily designed and perceived as efficient,
functional tools [35]. Devices that promote reflection often cannot be
7
categorized as a tool, and do not fit that hegemonic perception. As a result,
they can also be misunderstood or dismissed outright, similar to Sengers’
point that the ironic or subtle commentary in Critical Design can be easily
missed. The ideology of efficiency severely limits the acceptance and
understanding of technologies that do not function as tools, undermining
attempts to move beyond tool-based computational designs. We could
shoehorn reflection into tool-like objects, or, as Sengers suggests, employ
user-centered design; however, to successfully break the perception of the
computer as an efficient tool, we should expose the value system of
computational technologies that perpetuates Efficiency through everything
from circuit boards to designers. As I will touch on in the next section,
Efficiency is no longer confined to mass-production factories and offices, but is
embedded in our culture at an intimate level. In order to explore this value
system, we need to move beyond the abstracted visions of Efficiency and focus
on how it influences the personal, daily lives of people. By engaging the
public in a discussion, we may learn how Efficiency guides our lives in
tangible, contextualized ways. Using this foundation we can more effectively
loosen the grip of ‘technology as tool’ and clear a path for design genres like
Critical, Ludic, and Reflective Design, as well as Slow Technology.
It is worth mentioning that examining Efficiency follows Sengers’ advice,
where ‘... reflection on unconscious values embedded in computing and the
practices that it supports can and should be a core principle of technology
design.’ [70]. We can use reflective design to reveal the role of Efficiency,
opening it for further critique while simultaneously improving the foundation
and motivation for reflective design.
8
1.5 E Efficiency CTP
My proposal is at the heart of critical technical practice [2, 3] 1, where we must
actively question our own design decisions and any shifts those decisions
take. In addition to choosing an alternate route, e.g. reflection instead of
Efficiency as we see in Slow Technology, we should question that alternate
route as it appears. That is, not only should we pursue ideas like Reflective
Design and its neighbors, but also we should interrogate why this type of
design has become a focus of the HCI community and others at this point in
time. Explicit in Reflective Design is a call for computer scientists and
computational designers to adopt and put to work the criticality of artistic
practice, further clarified in Leahu, et al.’s work with Sengers [49]. Along with
this push for criticality comes the responsibility to, in essence, question our need
for the adoption of criticality. Designs that allow us to reflect on Efficiency can
help us understand the desire for reflective design. This inevitably brings up
questions about the role of Efficiency, given its authoritative position in
computational design. Perhaps the emerging consumer markets for
ubiquitous computation have inspired a focus on alternative design genres.
Because computational technology has become increasingly individualized
2, bringing the value system of Efficiency into everyday artifacts, we can no
longer rely on generalized notions of Efficiency. We should analyze Efficiency
from the point of view of the public in everyday situations, where this
ideology lives. Ideas like ‘Slow Technology’ and Reflective Design provide a
starting framework for understanding what we might do instead of efficient
tool design, but there is a largely undiscussed issue of what we have done
1Though I agree with Sengers’ modification of CTP that it should occur throughout thedesign, instead of limiting it to solving technical impasse
2By this I mean the rapid transition from huge, corporate or governmental servers that noindividual could afford to the pocket-sized, customizable devices we have now
9
with efficient tools. I mentioned earlier that Efficiency has been
well-considered in terms of economics, labor, capital, and even the term itself
and its roots in theology and spirituality [41], but the role of Efficiency on an
individual level is not adequately discussed. It is at that level that we may find
the most interesting effects of the ideology of efficiency, and from which may
come the largest catalyst for reflective design. The way this criticality is
reveled in our designs must be open for discussion not only between
designers, but also between users [70]. Technological devices provide the
perfect means to begin such an interrogation given their ubiquitous and
personal nature. We can piggyback on the pathway laid by efficient tools by
repurposing laptop software, iPods, cell phones, alarm clocks, televisions,
video game systems, traffic light cameras, etc. Since these devices are the
champions of efficient design that consistently propagate that value system,
reappropriating them may create significant cognitive dissonance in their
users. That dissonance is our opportunity to question the role of Efficiency.
While one might argue that we should wait for reflective design to mature
before that kind of analysis, I suggest that questioning a potential
paradigmatic shift is critical in its infancy. If we wait for it to embed itself in
design and establish a solid foundation, we may give up the ability to
successfully shape it and become aware of its assumptions.
1.6 A D Efficiency
The following section reviews a few ways Efficiency is discussed by other
authors, and how they can be improved by the design genre I propose.
Banta’s “Taylored Lives” [5] comes the closest to placing Efficiency in the
context of everyday activities, following the spread of the ideology of
10
efficiency across the US. And while her narratives often seem to relate to
current trends, the book does not try to reveal the efficient practices that you,
the reader, engage in unconsciously. Instead, we as readers follow trends
throughout previous time periods from which we may or may not find
resonance in our own lives.
In the period Banta writes about, Efficiency was not concealed but laid bare
for all to see. Today, the move to optimize and streamline life is tucked away
in products and consumer trends. Always-on wireless technology helps us be
productive when away from the office, portable computers allow us to work
anywhere at any time, and even music is marketed in 3-minute sound bytes
that may test well in user studies. While we might feel free from the reigns of
Efficiency, empowered to work on our own schedules at our own times, we
have instead ingrained this ideology so deeply into our personal lives that we
are no longer aware of its presence in all but the most typical scenarios. To
truly generate awareness about our commitment to this ideology, we should
intervene in the processes and tasks that make up our lives, continuously
presenting opportunities for reflection on this hegemonic ideal. The “one best
way” has slipped underground, into the technology we buy and the trends we
follow.
In “Work, Time-Discipline, and Industrial Capitalism” [78], Thompson uses
narratives about people to ground his abstract discussion of time-based
efficiency, but as we will see with Kellheler, these narratives are limited by the
media. We learn about the influences of time-structured work and how time
became currency, and while we as readers can appreciate this evolution, we do
not learn to uncover the concrete moments in our own lives that reinforce that
value system. We are often unaware of how and when often Efficiency
influences us, because we have no tools for becoming aware in real-time.
11
Thompson’s paper is even more abstracted away from the public because it
exists as an academic paper that requires a strong academic background to
unravel. It is, in a sense, impenetrable by the public and thus ineffective at
helping us explore this issue.
A more direct and personal exploration of Efficiency can come through
artifact design. Computational artifacts (mobile phones, laptops, alarm clocks,
software, etc.) do not reveal their commitment to Efficiency and how their use
guides our behavior, and Thompson’s abstractions do not provide us with the
tools to uncover them on our own. If we were to imbue our technical artifacts
with a reflection on Efficiency, with a questioning stance such that common
interactions with these devices raise the ambiguity and pervasiveness of the
ideology of efficiency, we might be able to realize and act more consciously on
moments of efficiency. Because computational devices are also pervasive (by
no small coincidence), they provide the perfect platform for reaching the
public at a direct level. Not to say that we want to refuse Efficiency, but instead
to become aware of its presence in everyday situations and thereby open it up
to questioning and discussion.
Even when authors discusses a highly detailed point of view of an
individual, we still do not achieve a personal exploration of Efficiency. Kelleher
discusses how Efficiency influences two factory workers: “To work work meant
to be disciplined ... Seamus’s glassblowing team exemplified it. They kept
their heads down at the factory and worked very quickly, almost in a unified
motion. They watched the clock constantly and measured, for themselves,
how long it took to make particular items of glassware. They always tried to
produce more, to prove themselves and what they called ‘our inventions,’ the
procedures they introduced to speed up production.” [42]. We can perhaps
understand how these workers felt, but it is nonetheless abstracted away from
12
our own lives via the the era, the specific lives of the workers, the text on the
page, the language of the author, the specification of efficiency as quantity per
unit time, etc. Even if the person reading this text was a factory worker that
could relate fully to the narrative, the reader experiences this discussion out of
context. The author is defining and exploring Efficiency as she sees it unfold,
but we cannot assume the factory workers would explore it in a similar
manner. Furthermore, the scenario is stereotypically efficient, and does little to
reveal the unique moments in the reader’s (factory worker or otherwise) life
where Efficiency may live. By providing people with an object that allows them
to explore Efficiency in the moment, as that ideology is enacted and lived, we
effectively remove that layer of abstraction.
McCullough refers to communication as “low-bandwidth,” appealing
generally to the language of the sciences, specifically computer science and
engineering: “I do not argue that low-bandwidth communication is useful, but
instead point out that its use reshapes the dynamics between people.” [54].
That kind of phrasing reveals a way of framing the world under the value
system of computation, and makes sense only because we are used to likening
our world to computation and technology. A primary component in the values
of computation is Efficiency, and we should not ignore how much it influences
mediums like e-mail, text messaging, MySpace / Facebook-type messages, or
even cell phone calls. We should not generalize the impact of Efficiency on
people in these cases. For a given person Facebook may serve as a means of
inefficiency: a distraction from work, or a break between bouts of productivity.
For others it may be used to maintain networking between colleagues for the
benefit of jobs and opportunities, requiring little effort and time to sustain
friendships. In fact, the same system may be both a source of efficiency and
inefficiency for the same person depending on the context the tool is used in and
13
the state of mind of the individual, and may differ entirely for a different person.
This is at the heart of examining the role of Efficiency at an intimate level. We
know very little about how this ideology works into mundane, trivial
moments, as well as non-trivial moments. One strategy for design might be to
piggy back on these forms of efficient communication and allow for moments
of reflection about Efficiency; that is, we could rework or modify these tools to
allow people to discover their own lived experience of Efficiency.
1.7 T D T: A
The Drift Table [28, 9] is perhaps the only example that almost fits an
exploration of Efficiency. This table provides a small hole in a table through
which a small part of a map is shown. Users can move the map in any
direction, but the map moves excruciatingly slowly. It becomes impossible to
use the map in the way one would conventionally use a map, instead
appealing to Ludic Design, the notion that humans are playful by nature [28].
It raises an interesting question about the role of Efficiency by simply being a
non-functional, inefficient map. This lack of functionality is both what makes
it useful as an example, but also limits how one might interrogate the role of
Efficiency. First, it succeeds by being so incredibly bad at accomplishing a task
while simultaneously appearing as if it has typical functionality. The primary
question of course is ‘If it doesn’t do anything, why should I use it?’ The
obsession with functionality in computational devices and the assumption
that they will help us ‘get something done’ reveals a lot about the influence of
Efficiency. The Drift Table’s packaging is important because coffee tables are
typically found in the home - an intimate and personal environment. As a
representation of Efficiency, computational technology changes the manner in
14
which the home is perceived and personalize the ideology of efficiency. We’ve
seen this already with smart homes, smart kitchens, etc. [8]. Why is it that
adding an LCD screen to a coffee table instantly changes it from a coffee table
into a tool? Not surprisingly, as Hallnas and Redstrom say, it’s most likely the
dominance of the “computer as tool” paradigm. Typically, computers help us
get things done. The Drift Table allows us to reveal our commitment to
Efficiency by presenting itself as a representative of Efficiency and yet
aggressively breaking with that tradition. When asking questions of the Drift
Table like “What does it do?” and “How do I work it?” we find no answers.
Instead, the Drift Table allows us to interrogate the relevance of those
questions. The Drift Table presents no obvious improvements to life at home,
in the way that a more intelligent microwave might. It doesn’t help you do
anything faster, or better, or more accurately. Its presence helps us rethink how
we perceive computational technology, and is persuasive because it endows
typically non-technical objects with computation. It shows us how easy it is to
override previous perceptions of objects with notions of Efficiency. Coffee
tables are typically social centers, holding magazines, drinks, food, etc. And
while the coffee table is efficient at holding things (as are all tables), its usage
does not make us more efficient on the same scale as computational devices,
and it doesn’t radiate an ideology of efficiency in the same way as a cell phone.
All of these perceptions are immediately brushed aside when viewing the Drift
Table, as the LCD screen captures our attention and changes the entire table
into a computer. This transformation reveals the ideology of efficiency. Much
of the technology designed for smart homes overrides culture and tradition
[27, 8]. The Drift Table counteracts this tendency and opens it for questioning.
The Drift Table, while primarily a Critical Design, falls under the category
of Slow Technology in that it helps elongate time. Both the table and Slow
15
Technology are instantiations of ways to examine the role of Efficiency, but
developing an inefficient device is only one method for questioning the role,
albeit the most obvious.
1.8 E C E Efficiency
O’Malley discusses the impact of the cinema on the cult of efficiency [60]. In
their own way, original movie theaters allowed the interrogation of the role of
Efficiency. Movie-goers were allowed to enter and leave the theater as they
wished, and movies ran all-day without obvious organization in time. The ‘...
early silent films threatened middle-class values, especially concerning time
and its productive use.’ [60] While it is interesting that early theaters
accidentally de-structured the reliance on Efficiency, it is more interesting that
the did so without appealing to inefficiency. In fact, this cinema-style probably
made people more efficient in seeing movies because they could come and go
as needed or allowed by their jobs, family constraints, etc. Movie-goers could
make optimal use of their free time given the lack of constraints in the cinema.
In this sense, the cinema questioned the role of Efficiency by making people
more efficient 3.
1.9 A : I
One method I propose for bringing to the surface the role of Efficiency is the
careful application of interruption. HCI and productively-oriented disciplines
3Efficiency, like statistics, can be skewed and evaluated along many dimensions, the selectionof which implies the bias of the one evaluating the efficiency of a system and the overall plasticityand subjectivity of efficiency. I describe this notion in further detail in Chapter 2. One mightfind a way to perceive of the Cinema as being inefficient (say from the point of view of thetheater itself), but my point here is that the interrogation of Efficiency through design need notrely on the promotion of inefficiency.
16
tend to treat interruptions as a negative [55, 25, 61], causing breaks in work
and productivity. Even ‘Calm Technology’ and ‘Ambient Technology’ seek to
minimize interruptions by backgrounding information [82]. While today there
is clear demarcation between work and life, this was not always so, as I will
discuss in Chapter 2 [78]. What is considered an interruption today rests
heavily on the notion that work can be separated from the rest of one’s life, and
that an interruption in one’s work for the purpose of managing life is negative.
Furthermore, even interruptions regarding work that disrupt from work are
negative. The notion of interruptions as a negative in this context depend on
the idea that one seeks to achieve maximal productivity with minimal waste,
where an interruption is a ‘waste of time’ that reduces productivity.
By strategically using interruptions, I suggest that we can reveal an
underlying commitment to Efficiency not only in work, but also in everyday
situations. Being interrupted on the walk to class to play a game may be
highly annoying, but might also expose just how efficient we have become at
using our time. Or, even better, such an interruption might reveal how much
time we ‘waste’ (despite the cultural belief that Efficiency is positive).
1.10 F T
By designing to reveal Efficiency, not only do we bring the discussion of the
term to an individual level, but also we draw it from the abstract into the
concrete. We have seen how Efficiency is often discussed in texts and how that
technique, while critical to the knowledge about Efficiency, cannot provide the
grounding necessary to explore the ideology in day-to-day tasks. This chapter
has offered several examples of how one might intervene and question
Efficiency in different ways, and suggested how interruption may be used to
17
penetrate the ideology as it guides our actions and thoughts. These
techniques, especially when instantiated in computational artifacts, can open
up the elusive and invisible ideology to further critique, analysis, and public
discussion.
18
C 2
H C T R , R , P I E
“As the seventeenth century moves on the image of clock-work
extends, until, with Newton, it has engrossed the universe.”
- E.P. Thompson
In this chapter, I argue that our long tradition of an ideology of efficiency
has resulted in and is reinforced by computational technologies. This includes
the physical construction of hardware, the ability for these technologies to be
produced by and produce efficiency (embody and produce), the theoretical
frameworks within which software and hardware operate, and the tactical use
of non-technical language to propagate a devotion to Efficiency.
2.1 S Efficiency
Computational technologies, both hardware and software, are a natural result
of and strategically reinforce the tradition of Efficiency on several levels:
physical structure, theoretical framework, means of production, use of
language, and through the ways they are perceived and used. Unlike previous
systems, however, computational technologies both embody and produce
efficiency, reinforcing it both by construction and use. Chun argues that
software itself is an analog to ideology [11], but I argue here that
19
computational technologies actually implement an ideology of efficiency.
Furthermore, “... software sustains and depoliticizes the notions of ideology
and ideology critique ... [people] attribute to software, metaphorically, greater
powers than have been attributed to ideology” [11]. I should clarify that one
can build a computer that does not rely on or promote Efficiency, but these
computers are typically not in use today (or are extremely marginalized). I
speak only of the dominant computing machines that have been in use since
the mid-20th century. That is, those that use circuit boards and software and
hardware algorithms.
2.1.1 P
We can grossly divide the physical constituents of a computer into software
and hardware, both of which carry forward the ideology of efficiency. While
software may not seem like a physical structure, it is stored physically and its
primary function is to manipulate voltages. The hardware provides the
foundation on which software is able to function. Furthermore, the storage of
code includes both the physical structure onto which it is stored (e.g.
hard-drive), as well as the format in which it is stored (e.g. compressed,
compiled, text, etc.). I will attempt to discuss each in its own terms, but the
discussion will bleed into both realms at times.
H
Computer hardware relies on extreme precision: first, in the construction of
parts (e.g. microprocessors), second, in the strict management of power, and
third, in the actual layout of the parts as they compose a circuit.
The mass-production of computer chips, memory, hard-drives, displays,
etc. relies on machine production rooted in the Industrial Revolution and
20
inherits the ideology of efficiency from its roots. At this point, it is physically
impossible for a human to create a .65nm microprocessor without help from a
computer. We rely on other machines and their representation of Efficiency to
generate new computer parts, who in turn inherit the efficiency ideology.
Mass-production is dependent upon the ideals of Efficiency [5], and thus give,
in perhaps a small sense, its productions a tradition of Efficiency through
creation; however, often computer hardware is then used as a means to
improve the efficiency of mass-production by replacing inconsistent laborers,
achieving inhuman levels of precision, increasing hours of operation,
maintaining plant conditions, etc. In this way, computer hardware becomes a
producer of Efficiency, where it carries forward the tradition not only through
its own production, but also through that which it produces. Furthermore,
hardware is used in personal productivity applications, such as the writing of
this document. These concepts apply to software as well, as I will address
shortly.
At the micro level, the motherboard in my laptop depends entirely on
efficient power management designs and infrastructures. Voltage
management is of extreme importance to our volatile computational
technologies. They would not operate unless electricity was tightly controlled
and managed, as most systems require highly specific voltages ranges in order
to operate. From the lithium-ion battery, to the battery charger, to the wall
socket, to the generator, my laptop relies on a vast infrastructure of power
management. At each point, we find commitments to Efficiency. The batteries
size and heat have been reduced as much as possible (without inflating cost, of
course), and their power output and life-span have been maximized. These
design goals are important to both a company in that it wishes to minimize
cost while maximizing appeal to customers, and to the customer in that he/she
21
desires lighter, smaller, longer-living devices that can operate faster processors
without burning their skin. Next, the battery charger is responsible for taking
as input a wildly powerful electrical signal and transforming into a very
precise output to my laptop’s battery. While size is an issue here as well, more
important is the precision of the chargers operation. If a charger wasted half
the energy it took as input in converting the signal to a more manageable form
for digital machines, it would be essentially a ‘broken’ charger. It is assumed
that the battery charger will efficiently manage and convert power, in that it
will waste little energy and provide optimal output and charge rate (the faster,
the better). Simply transmitting electricity along a grid is dependent upon the
reduction of waste - the signal must be insulated to ensure minimal loss of
power along its route to my home. Computer power management relies on a
vast infrastructure that is, by definition, invisible and thus difficult to reveal
[72]. Each of these steps has been critically influenced and shaped by a
devotion to, and the hegemonic position of, the efficiency ideology. Again, it is
not my intent to claim that these methods are unwelcome, but instead to
interrogate the roles Efficiency plays.
Finally, we find traces of Efficiency embedded directly into the design of a
circuit board. Setting aside the aesthetic and craft-like nature of trace-routing
and board layout, a primary concern is that of the length of traces and their
ability to transmit voltages with minimal loss. That is, the primary reason a
circuit board layout looks the way it does is a result of the designer’s attempts
to minimize wasted signal, striving for perfect noise-free signal transmission
(bringing to mind Shannon’s unfortunately titled “Theory of
Communication”). Boards that do not achieve near perfect signal efficiency
often do not work, or fail within a short period of time.
22
A S, C, T
An interesting question arises: do inefficient, functional circuit boards exist?
This leads us to an important point about the ideology of efficiency,
specifically about objectivity and subjectivity: instantiations of Efficiency are
relational. The term is meaningless when talking about only one object. If I
build a circuit board by hand to the best of my ability, I could claim it efficient
and I would be right if no other circuit boards exist. The only way my board
becomes inefficient is if I or someone else develops a more efficient board to
which I can compare my previous design. The determination of efficiency
requires comparison, and in vacuum becomes useless. If we have only one
version of an item, it is and will always will be the most efficient version of
that item. What arises from this is a curious duality in nature of the term: on
the surface it is binary where a design can be efficient or inefficient; if it is not
the most efficient, it is inefficient since efficiency in its most general form refers
to maximal productivity and minimal wasted effort or expense. But in
groupings, one might speak in shades of efficiency: “My board is more
efficient than this one, but less efficient than that one.” The term’s subjectivity
becomes visible, and the contextual-nature of it is revealed. Thus, for a board
to function it must be somewhat efficient, but need not be the most efficient
design. The plasticity and context-sensitive nature of the word is troubling.
We can speak of a functioning board as being ‘just efficient enough’ to work. On
the surface, this statement seems strange given that Efficiency tends to imply
optimality, but such optimality is in fact a holdover from the objectivity
inappropriately granted to Efficiency. In fact, the terms ‘less efficient’ and
‘more efficient’ are more honest about the inherent subjectivity of the term.
23
S
Efficiency as optimality is so embedded in the design of software that even a
programmer’s skill typically refers to his ability to create efficient code, where
software is a “means both of quality control and of disciplining programmers,
methods of cost accounting and estimation, methods of verification and
validation, techniques of quality assurance” [52]. Here, Efficiency takes subtle
forms such as the subjective speed of the program, the use of resources, the
compactness of written code, the overall size of the program, the readability of
code, etc. Inefficient programs are typically referred to as bad programs. If a
coder writes a program that uses 90% of a machine’s processor cycles while
producing minimal functionality, he has written bad code1. If a program uses
100% of a machine’s RAM regardless of the program, he has written bad code.
If a program ‘feels’ slow, regardless of functionality, or takes up 5 gigabytes of
space on the hard-drive while simply writing ‘Hello, world.’ to the screen, he
has written bad code. There are very few methods for evaluating software
development that do not rely on Efficiency. Even those evaluations that focus
on the usability of the software subscribe to a desire for Efficiency: HCI is a
field devoted primarily to fast, ‘natural’ interaction with machines. Here, the
less time it takes one to learn how to operate a piece of software, the better that
piece of software is. Of course there are exceptions to this rule, such as
emotional satisfaction, enjoyment factor, and other ‘non-technical’ evaluations,
but efficiency remains a primary measurement.
Often, software is advertised as a means to increase the productivity of the
user. Not only does the software itself conform to and reinforce Efficiency, but
also its function is often to help people become more efficient producers. In
1I exclude endeavors that may be pursued in a more artistic, critical technical practice [2]methodology, as my discussion of software focuses on its general use and conceptualization.
24
the way that the use of efficient machines produces other, more efficient
machines, hardware/software systems designed for user productivity also
produce efficient producers. From Baudrillard [7], this effect extends not only
to work but also to leisure, as we’ve seen through video games [64] where
users compete for high scores, most wins, most kills, fastest runs, etc.
Hardware and software systems are often advertised as more precise for
gamers, providing less noise and higher scores. Typically, gaming skill is,
when stripped of all its dressing, the ability to produce the maximal output
(score). Software embodies the ideals of Efficiency through its value system,
and also is designed to elicit increased productivity from its users with
minimal waste (e.g. effort, time, etc.).
Additionally, there is a general rule of thumb in the programming
community: ‘Don’t reinvent the wheel.’ This rule applies to both hardware
and software. The idea is obvious, but beneath it lies a deeper commitment:
one should reuse code when possible as ‘tried-and-true’ code is often faster,
better tested, more reliable, more readable. That is, it uses fewer resources and
feels quicker, it has been debugged such that it has the fewest errors, and
should other programmers look at your code they will waste less effort
analyzing it. The premise is that in each case, a programmer should strive to
waste as little as possible, and achieve maximal productivity via reuse. Even
from the standpoint of development time, code reuse minimizes coding time.
Efficiency is embedded in the creation and usage of software. It is nearly
inescapable at this point, crossing work and leisure boundaries at will. The
effect of Efficiency on software and hardware is so profound that when one
designs for inefficiency, the product essentially becomes non-functional, like
the Drift Table [9].
25
2.1.2 T
Algorithm design is evaluated in terms of Efficiency. Computer Science is
founded on the desire to produce algorithms that are both faster and reduce
resource use. The analysis of algorithms uses ‘Big-O’ notation, where the
optimality of an algorithm is judged based on how long it takes to complete in
a worst case scenario. Furthermore, an additional layer of how likely that
worst case is to occur is taken into account. For example, if an algorithm takes
O(x2), then in the worst case it will take exponential time to complete;
however, if in general it runs in O(1), or constant time, it may be acceptable to
use the algorithm if it is easier to implement. The primary means for
determining the usefulness of an algorithm is a mathematical determination of
its speed and resource use. Clearly, efficient algorithms are desirable and
inefficient algorithms are not. In general, ‘good’ programmers effectively use
efficient algorithms in their code.
2.1.3 T L
Efficiency has been embedded into unusual facets of software and hardware
design, particularly through the use of language. We’ve seen an example of
this in Day’s discussion of the ‘conduit metaphor,’ where the sheer abstraction
and vague sense of certain language allows for broad application of
mathematical and computational theory to domains far beyond their intended
target [12]. There is a unique use of the term ‘elegant’ when applied to this
technical domain, which actually works to identify the programmer’s style.
Elegant solutions are somewhat amorphous, but in general imply a clever
method for achieving efficiency. For example, an elegant piece of code may
solve a problem using only one array when one would typically uses several
26
variables and arrays. In this case, the more judicious use of program memory
is smiled upon. The term extends to any solution that uses less of a resource
than would normally be required, be it processor cycles, threads, memory,
space, code lines, etc. The phrase elegance speaks to the style of the
programmer. This is fascinating because not only are programmers applauded
for efficient coding, but thy are also assigned an individual style based on the
ways they achieve that efficiency. Far beyond an objective measurement, the
term now stretches to include the programmer’s insight, creativity, and ability
to mold efficiency in unique ways. Elegant programmers are masters of
efficiency and optimization.
In the hardware world, elegance is applied in almost the same way where it
acts as a measure of style through efficient use of resources. One example
might be the ability to use only one layer on a PCB board using tactical traces
and clever layout, which reduces the cost of the board, the length of the traces,
and size. Another elegant choice might be to use the second layer only for
ground, allowing all ground traces to simply connect to the second layer
through vias and reduce the overall use of copper on the board, allowing for
better placement of pin outs, an increase of signal transmission integrity and
greater readability. In all cases, elegance refers to the reduction in use of
resources while providing maximal productivity.
By embedding itself in language, Efficiency is buried deeper into the
structure of computational technologies. While colleagues use terms like
elegant or clever, they are primarily referring to the design’s efficiency. This
ideology of efficiency is so entrenched in our notions of technology, it is
assumed that software and hardware systems should be efficient.
27
2.1.4 E
It is important to discuss how our computational technologies are valued and
perceived because these technologies are embedded throughout our world
[79]. They can be embedded directly through physical devices and software,
theoretically through mathematical communication theories such as
Shannon’s [71], and metaphorically through an idea like the ‘conduit
metaphor.’ The effects of these technologies on our conceptualization of the
world are profound. Day argues that information theory was “... used to
extend the range of these notions across social and political space during the
period of the Cold War” [12]. Chun reveals how the female clerks that
operated the mechanics of computational machines, also known as ‘wrens,’
were subject to evaluation in terms of the machines they operated on: “This
man-machine synergy, or interactive real-time (rather than batch) processing,
treated Wrens and machines indistinguishably, while simultaneously relying
on the Wrens’ ability to respond to the mathematician’s orders” [11]. By
conflating computational technology and our environment, we conflate the
value systems as well. As the value systems of software and hardware merge
with other value systems, we see a reinforcement of the ideology of efficiency.
In one sense, computers are simply one more step in the maintenance of an
efficient ideology, but in another way actively promote Efficiency by acting as
both an embodiment of and producer of efficient behavior. It is vitally
important to bring to the surface computational value systems in order to
better understand how they impact us.
28
2.2 A B H E T
Efficiency
It is important to highlight the progression of Efficiency through technology,
not just the ways it is embodied today. This brief history will track how
technological advances in time-discipline embodied and reinforced Efficiency.
Additionally, the roots of term itself have been carefully dissected [41] and as
such I will only delve into a discussion of its interpretation when necessary.
2.2.1 T Efficiency
The introduction of locally standardized time radically reshaped work
processes, shifting priorities and replacing evaluation methods. Their
existence helped reorient employers and laborers away from task-oriented
work, where task requirements were the structuring element, towards
time-based work. Thompson makes three points regarding this change:
task-oriented work places control and management in the hands of the laborer,
blurs the line between ‘work and life,’ and appears ‘wasteful and lacking
urgency’ to time-based laborers [78]. The introduction of clocks brought along
a means for abstracting labor away from the individual in a pseudo-Marxist
manner, where the “... employed experience a distinction between their
employer’s time and their ‘own time”’ [78]. Technical production, while not
the sole force behind this change [30], was a primary factor in its occurrence.
2.2.2 T
It is difficult to assess ‘waste’ in terms of production without a clear
demarcation between work and non-work, because waste implies an ability to
29
judge when one is being unproductive. If life and work are intermixed in such
a way that it is difficult to discern one from the other, as in task-oriented work,
waste is highly difficult to assess. Efficiency is built upon the ideas of the
production without waste. Time-based labor allowed for the measurement of
waste, and as a result paved the way for an abstracted measurement of
efficiency. I will return to this concept later.
2.2.3 R-
Perhaps the most paradigmatic shift, however, was the re-conceptualization of
time itself [78]:
“And the employer must use the time of his labour, and see it is
not wasted: not the task but the value of time when reduced to
money is dominant. Time is now currency: it is not passed but
spent.”
Here we see the beginnings of ideals that hold true today. Time becomes a
basis for judging labor, simplifying evaluation of work by reducing it to
quantifiable measurements. Where before employers used subjective
measurements to compute timing of work, time-based evaluation provided
easily computable abstraction. Previously, subjective judgments of corn
density allowed employers to evaluate their laborers: “...if it be good thick and
fair standing corn, then [the laborer] may mow two acres, or two acres and a
half in a day; but if the corn be short and thin, then he may mow three ...” [78].
In contrast, even imprecise (as often they were) watches and clocks provide
the illusion of objective measurement through mechanical abstraction, where
production can be evaluated in terms of hourly rates.
30
2.2.4 A Efficiency
In the spirit of craft, watchmakers refined their skills at producing exceedingly
accurate clocks, sometimes claiming variation of less than a second per two
years [78]. Such technological advances, perhaps unconsciously, strengthened
the role of Efficiency by improving the precision of abstraction. That is, as
watches became more precise, ‘objective’ measurement of performance
became more precise. Where before an imprecise clock left shades of gray for
interpretation, early 19th century watches removed what ambiguity remained
through sheer mechanical precision.
To put it another way, the objective representation of the world replaced the
subjective interpretation. A watch was accurate and not influenced by human
imprecision, and as such the onus of evaluation was focused on the abstracted
technology rather than the situated person.
The trend of technology towards greater precision, I contend, still exists
today and continues to reinforce the abstraction of efficiency and the
promotion of it as objective measurement. The most obvious example, as I
have noted, is that of computer software and hardware.
2.3 O
Efficiency
The illusion of objectivity of time, made available through technological
artifacts like clocks, paved the way for the evaluation of people and things in
terms of efficiency.
Jollands notes that “... in the context of the rationalist spirit of the
Enlightenment and the commercial activity of 18th century Europe, was
31
applied more widely to the transient world. In doing so, the core meaning of
efficiency shifted from a theological, spiritual basis to a Western-scientific,
‘logical-positivist’ realm” [41]. I propose to add to this the false notion of
objectivity that is a part of the conceptualization of efficiency. That is,
instantiations of Efficiency are subjective interpretations, but are often
conceptualized as objective measurement (appealing to logical-positivism and
rationality). This idea alludes to Von Foerster’s notion that objectivity itself is
a delusion [64], but adds to that idea as the false sense of objectivity is what
allows Efficiency to exist as it does.
This ‘logical-positivist’ version of efficiency appears rational only under the
guise of objectivity and logical-positivism - that one can stand back from the
element being measured and judge waste and productivity. A personal
assessment of one’s own efficiency is irrational in that it allows no external
measurement for verification. Instantiations of Efficiency are bound to the
notion of objectivity by their construction through abstraction and a reliance
on external comparisons. I used the phrase ‘the guise of objectivity’ because
all interpretations of efficiency are subjective and context-dependent [73],
drawn from the observer and imposed on the participants.
For example, in a shoe factory, one might evaluate the factory as efficient if
the shoe production to material wasted ratio is desirable; however, one could
easily chose other criteria for determining efficiency, such as the number of
employees to shoe production ratio.
The criteria chosen in determining efficiency reflect the interests of those evaluating
a system’s efficiency, which is a subjective interpretation of the system.
Yet it is the illusion of objectivity that allows Efficiency to propose solutions
that are, as Taylor characterized it, “the one best way” [5].
Without presentation as objective measurement, the values of Efficiency
32
change; we see that the concept itself is highly dependent on its presentation
as objective (despite its inherent subjectivity).
In the supposed objectivity presented by clocks, also we find that
reasonable measurements of efficiency filter the world in a way that hides the
complex structure of its elements. For example, an employer with a clock may
measure worker efficiency as the amount harvested per hour. While this
perspective is not wrong, it is clearly subjective. A laborer may be inefficient
in terms of harvest per hour ratios, but may be exceptionally efficient in terms
of long-term yield by paying attention to conditions and only harvesting in
areas that will not damage the soil. Efficiency operates within the false realm of
objectivity through which measurements are verifiable and repeatable,
ignoring the subjective exclusion of ‘inconvenient’ properties of the system.
In order to combat the treatment of Efficiency as abstract and objective, we
can rework and repurpose technological artifacts into objects that question the
existence of Efficiency. Where before technology helped reinforce and embody
that ideology, designing to reveal Efficiency in our lives re-appropriates
technology in an effort to demystify and contextualize this cloud of ideas as
real-world, tangible moments in each of our lives.
33
C 3
E O S: TP CPS
Every object has a story. Objects of design are the result of a complex history
of iterations, design choices, motivations, assumptions, reworkings, impasses,
etc. In Reflective Design [70], we are encouraged to fuse reflection into design
allowing it to shape, motivate, and inform our work; as a result, the stories
behind reflectively-designed objects contain an even richer texture.
Regrettably, the story is seldom visible by inspection of an object alone, and
often is lost as the process of creation fades from the designer’s memory.
Critical Design tells its story by embedding alternate perspectives in the
actual design [[70][17][18]]; however, because this story is abstractly packaged
into the design itself, it is likely legible only to its creators. Ludic Design
promotes reflection through usage, where a story is told through interaction
with an object [28]. This story, however, is not the story of creation but a story
of interaction. Neither Critical Design nor Ludic Design provide users or
designers with a background story. Objects are presented as is, perhaps
accompanied by a user study or a few implications for design. I propose that
the simple act of telling an object’s story via autobiography greatly enhances
the practice of Reflective Design. Process reveals assumptions, philosophical
commitments, the significant role of intuition, and other nuances that may be
hidden from the creator, other designers and users. Additionally,
documentation of process may help prevent a misunderstanding or
34
misappropriating of its Critical Design elements [70].
I’ll begin with a bit of 2nd order reflection on writing this chapter in the
next section. The third section moves into the origins of PersonalSoundtrack,
while the remainder of the chapter is broken into three main philosophical
motivations which serve to provide grounding and structure amidst the
disorganization inherent in documenting process.
3.1 I
Telling an object’s story is not so easy. This document is focused on the process
of creating my device, PersonalSoundtrack. I describe the origin of the idea,
my reflection throughout its evolution over time, my reluctance to
acknowledge its political stance, the accidental discovery of philosophical
underpinnings, and reasons behind choices for which my device has been
labeled ‘insufficient’ [[68][14]]. In developing this paper, not only did I
reacquaint myself with many of the project’s intricacies, but also I came to
terms with how difficult it is to describe the process of creation and reflection,
in part because the process is non-linear. For example, initial design decisions
are later contextualized by my updated perspective of the project, sometimes
reinforcing those initial decisions through unrelated justifications. Though I
conceptualize the device easily in my head, verbalizing that characterization
requires unpacking dense, ill-defined structures.
3.1.1 L W
Agre discusses how his technical training made it nearly impossible to
decipher non-technical works [3]. At the beginning of his journey, he
attempted to read philosophical works through a technical lens, which
35
obfuscated the meaning. He was forced to develop new lenses in order to read
material outside his domain. Those in interdisciplinary work are no doubt
familiar with this experience, yet there was a new moment I had not
experienced during my interdisciplinary work until now. In writing this
chapter, I have found that (in keeping with appropriate metaphors) I have only
a few pencils with which I can write. My technical training has made me adept
at discussing technical internals, presenting my opinions as ‘objective,’ etc.
This chapter, however, is autobiographical, and as such frightens me. I find
myself constantly thrust against my technical training, deleting ‘I,’ ‘me,’ ‘my’
and replacing them with the ‘unbiased’ ‘We,’ ‘us,’ and ‘our’ (a curious
technique used regardless of the number of authors). I do so explicitly to
accept my individuality, and I take responsibility for my bias. When speaking
of the development of my ideas, I feel awkward writing about such personal
experiences. It is my writing ‘lens’ that leads me to devalue my personal
opinion and the story of my device. Nevertheless, I intentionally forgrounded
the bias of my work, prioritizing my opinion and personal motivations
regardless of their appropriateness in scientific texts. While the extent to
which I personalize this material may seem excessive to some readers, it is
done purposefully to question where such language is appropriate. This
writing style may seem excessively personal under the sciences, but somewhat
impersonal under the arts. Thus, a secondary goal of the remaining chapters is
to probe which styles of writing are acceptable when one is writing an
interdisciplinary thesis.
3.1.2 P B
This document needs a short preface, best stated by Agre [2]:
“In writing a personal narrative, I am assuming some risks. Few
36
narratives of emergence from a technical worldview have been
written; perhaps the best is Mike Hales’ (1980) remarkable book
Living Thinkwork about his time as a manufacturing engineer
using operations research to design work processes for chemical
production workers. A sociological inquiry is normally expected to
have an explicit methodology. The very notion of methodology,
however, supposes that the investigator started out with a clear
critical consciousness and purpose, and the whole point of this
chapter is that my own consciousness and purpose took form
through a slow, painful, institutionally located, and historically
specific process.”
In accepting my bias, I’m heartened by the fact that Agre has already laid a
foundation for this process of autobiography.
3.2 S PS
Before this chapter can continue, it is important to know a bit about the device
I created. PersonalSoundtrack is a tiny music player - the third iteration is
approximately the same size an iPod Nano (Figure 1). It is a:
“...mobile music player that makes real-time choices of music
based on user pace. Standard playlists are non-interactive streams
of previously chosen music, insensitive to user context and
requiring explicit user input to find suitable songs. The
context-aware mobile music player described here works with its
owner’s library to select music in real-time... ” [24]
The device uses an accelerometer to detect pace and chooses songs by
comparing steps-per-minute (SPM) to beats-per-minute (BPM). To use the
37
device, one simply places the device in his/her pocket, puts on earphones, and
begins moving. “Music is seamlessly matched to the user’s speed, putting the
user ‘in tune’ with the music. By continuously adapting to user pace, the
device remains ‘in tune’ with the user without explicit control...” [24]
In my previous paper, I discussed how the device is useful, laid out user
scenarios, compared it to related devices, revealed the hardware and software
components, and presented implications for future work. An early section of
the paper is devoted to justifying the device:
“Many users attempt to plan for mobile activity by pre-defining
playlists that correspond to specific activities or moods. From
Suchman [1], plans alone do not dictate actions, but instead
provide scaffolding that individuals can use to organize action.
Thus, users attempt to follow previous plans while continuously
adapting their actions to the environment [2]. Pre-defined playlists
cannot adapt to such ever-changing situations without explicit user
input. Manually selecting music requires both user attention and
memory when mobile navigation inherently demands the majority
of user resources [2]. Context-aware playlists that automatically
choose music in real-time and in response to user movement, can
better match the unpredictability of mobile activity.” [24]
Not included in this previous paper, however, was a discussion of the
iteration process. This was done not because the iteration process was not
interesting, but because it was not suitable for a technical or scientific
discussion. There was no place for what is the content of this paper. As a
result, the motivations and socio-political implications of the device were not
represented. In my focus on the end result, on justifying its existence, and
presenting a valid user study, the reflective process and philosophical nature
38
Figure 3.1: PersonalSoundtrack, 3rd Iteration
39
of the device was marginalized. In terms of Ludic Design the device creates
reflection via usage, but again the process of creation and design is
ill-represented. Furthermore, to discuss the process of development only in
terms of the final product can be disadvantageous as the device may or may
not successfully represent that which was pursued. Without a story of process,
critical discussions of the device are not privy to the assumptions or
motivations in a meaningful way. Many assumptions of the designer may not
be explicitly visible through the object or its use alone. To go a step further,
many assumptions of the designer may not be visible to the designer
him/herself. Instead, by unravelling and verbalizing the process, it is my hope
that the motivations become as important as the success or failure of the
device if only to invite deeper critical conversations between designers and
users. Even if in creating this document I don’t reveal additional assumptions
to myself, some may become obvious to you.
3.3 O
Several years ago, my original idea for PersonalSoundtrack centered around
driving, not walking. I loved to drive. It was a passion of mine, and I often
spent time slaving over music CD’s for my car. When the right song came on
at the right time, the adrenaline rush was addictive. Sadly, the discs I created
seldom had the ‘right’ kind of music. At the time, I wasn’t sure why they
weren’t what I wanted and attempted to improve the situation by creating my
own music.
I wrote and recorded a set of songs that might be appropriate for driving
(aptly titled “Driving Music”). I sat at my computer and attempted to imagine
driving scenarios such as waiting at a red light, speeding on the freeway, local
40
commuting, accelerating, etc. For each scenario I thought of, I wrote a piece
that I felt fit that scenario. For example, waiting at a red light was represented
as a single hi-hat with minimal drums and quiet atmospheric effects, where
high-speed freeway driving was characterized by upbeat, intense, semi-frantic
tracks. Once the music was written and burned to disc, I memorized the
locations of each track type. When driving, if pulled up to a red light, I quickly
switched to the track titled “Waiting at a red light.” I actually continued
manually changing tracks in an attempt to create semi-contextualized music
and test my idea. Obviously, this method was temporary and was instantly
irritating.
A few years later, I attempted to convert this idea into a semi-automated
form. From a technical standpoint, it was difficult to automate my previous
manual selection of songs. I thought I needed to sense speed and movement in
a way that correlated to the music. I didn’t know how to measure the speed of
the car in a general way, and didn’t want to rip my car apart to get a reading.
Image-processing of the speedometer was not elegant, and a mechanism to
measure pedal distance was beyond my skills. Surprisingly, this technical
limitation was hiding a deeper flawed assumption.
3.3.1 P I S B
As I go through my process and re-encounter what I did, I see issues that were
invisible to me before. I was having trouble imagining how to correlate the
car’s speed and movement to song choice. Surely, there was some algorithm
that would play the ‘correct’ song, I just wasn’t smart enough to figure it out,
right? The idea of a ‘correct’ song was a fundamentally flawed aspiration.
What I was looking for was a way to interpret and put to work an ill-defined
space of context, to divine from data a sense of appropriateness. I didn’t want
41
to have song speed correlate to car speed, I wanted to choose an ‘appropriate’
song. As I’ve said, I wrote this off as technical limitation, but I see it now as
primarily a philosophical limitation. To assume that a bodiless computer
could interpret and predict personalized context via a series of data
dimensions was untenable [16]. You might be thinking, doesn’t
PersonalSoundtrack attempt to do the same thing? After all, it simples
correlates song speed to step speed. The important difference is that
PersonalSoundtrack does not claim to choose ‘appropriate’ songs, instead it
offers music that is directly connected to bodily movement. It’s a subtle
distinction that I will discuss further in the final section.
3.3.2 T V, P U
After deciding that a car version was out of the question, I chose to work with
walking and running because it seemed more technically viable. I assumed I
would return to the driving version as my technical skills increased; however,
as the project progressed technically and theoretically, I remained committed
to a body-based version for reasons far beyond technical limitations (only in
part because of the context issue above). Though a driving version became
technically available at later stages, it became philosophically unworkable.
3.4 M A S
One of the primary goals of the project evolved into the desire to create a
visceral interface based on a user’s unique ways of navigating the world. This
happened in part because my lack of technical skill forced me to create an
interface based on user pace, and in part because of my background in HCI
(resulting in a general desire to create interesting interfaces).
42
I saw the potential to embrace a user’s unique gait. At the time, this was
desirable because users would easily be able to use the device if they knew
how to walk; the device would be as easy to learn as a pedometer. Children,
young adults, adults and seniors have had no trouble using the device with
nothing more than a one-sentence description of what it does. This basic
principle is still valuable in terms of HCI, but has also flowered and found
support in theoretical canon. Specifically, the machine was primed to do more
adaptation than the human. Let’s unpack that idea.
3.4.1 A D
Typically, people are required to adapt to computer interfaces. The most
obvious example of this is the computer keyboard, where user adaption to a
static interface is represented on several levels. First, the physical shape of the
keyboard is generalized; it is ‘one-size-fits-all.’ None of the unique attributes
of my physical body are taken into account, such as the size of my hands, or
the reach or strength of my fingers. I am forced to wrap myself around the
keyboard, often contorting my hands into painful positions. Second, the
QWERTY layout was specifically designed to impede the ability of the user,
again forcing users to employ bizarre combinations of movements in order to
type common words (despite the advancement of DVORAK and other key
mappings, my laptop still uses a QWERTY layout) [31].
3.4.2 A H N-AM
Even worse, the concept of the keyboard-screen-pointer (e.g. the Graphical
User Interface, or GUI) was initially developed in 1958 with the intent to use
humans as militaristic band-aids for the S.A.G.E. air defense system [19]. The
43
role of the human was to adapt to the machine and solve problems it could
not. Humans were hardware components in a larger circuit. Adaption fell on
the shoulders of humans, while machines simply provided an unchanging
interface. The machine didn’t have to know anything about the soldiers - the
soliders were required to learn everything about the machine. Interestingly
enough, these two paradigms used by the military in the 50’s are still in place
today. My laptop’s interface is based directly off the S.A.G.E. system’s
interface, and my computer does not adapt to me (I learn it).
3.4.3 A S
PersonalSoundtrack offered, instead, a symbiotic relationship with a machine.
The burden of adaption was placed primarily on the machine, and adaption
by the human was voluntary. When both adapt to each other, the system
becomes homeostatic where human and machine settle on points of stability.
Furthermore, the system does not exist without machine adaption - it is a a
core commitment of the device.
If PersonalSoundtrack had been created in the spirit of S.A.G.E. and the
keyboard, it would require its users to walk in a very specific way in order to
function. This, of course, sounds as ridiculous as it should. Imagine, for a
moment, a city of people all walking in exactly the same manner. We each
have a unique gait, and we should not be forced to modify this gait in order to
use a piece of technology. Settling into this philosophy, it became increasingly
clear how often I am forced to change my behavior in order to interface with
technology. We often modify our behaviors and ways of working in order to use
computational devices. PersonalSoundtrack represents, then, an attempt to
break from the hegemonic paradigm of non-adaptive machines. At a
fundamental level, the project began to take on a deeper philosophical
44
structure, where the technology existed primarily as a theoretical scaffolding.
Returning to the original, car-based version of PersonalSoundtrack, it
becomes clear why that idea is given this new theoretical commitment. In
order to adapt directly to the personal walking style of each user, the machine
needed to be coupled directly to that user. By adapting to a user’s footsteps
instead of a car’s general speed or movement, the device becomes a direct
extension of the body. That is, the computer becomes a meaningfully
embodied technology, where the body is fundamentally and directly
connected to the machine. A device that tracks a car’s speed and/or movement
is not breaking the paradigm of adaption because said device is adapting to
the user’s interface with the car. That is, the user has already learned the
mechanisms of operating the car and is limited by them. The car’s interface
acts as an insulating layer in which we lose the unique traits of different
individuals. PersonalSoundtrack exploits and welcomes unique physical
interactions that are not embraced by the car. For example, to drive a car one
learns how to press pedals and turn the wheel. A steering wheel and pedals is
non-adaptive, and requires users to adjust themselves to the its controls
(adaptive controls in a car are often rudimentary, such as seat adjustment). If
PersonalSoundtrack adapted to the output of the car’s movement, it would be
adapting to the user’s adaptation to the car.
A driving version excises a primary theoretical motivation for the
development of the device. It is interesting to note that this motivation was
not present in the device’s infancy. Though the technical limitation initiated
me turn toward a body-based interface, it was a deeper drive for embodiment
that caused me to remain with the the current version.
45
3.5 M, B, E
I’m not clear as to when this next philosophical issue arose. I’m sure it
occurred during testing, most likely during the completion of the first
prototype in 2005. In any case, during usage I began comparing
PersonalSoundtrack to typical MP3 players. Non-adaptive portable music
players (e.g. iPod) ignore the body 1 The experience is cerebral; one listens
with their ears only. As acceptable as that idea may appear, it was quite
unusual for me the first time I experienced it. Trained primarily as a musician,
experiencing music without tapping my feet or hands, closing my eyes, or
moving my head in time with the beat, was foreign and extremely frustrating.
I could have tapped my hand or moved my head to the beat of the song while
walking but, since the tempos hardly matched, it was a bit difficult to do.
Over time, I adjusted and learned to listen to music without my body.
When I began using PersonalSoundtrack in my everyday life, I stumbled onto
an additional perspective: the device relied on both mind and body. This
perspective was revealed not by designing, building, or analyzing the device,
but by using it. As in Ludic Design, the usage caused reflection; however, the
reflective effects impacted me, the designer, during development.
During that time, I was involved in Simon Penny’s interdisciplinary theory
seminar [63]. In it, we discussed the commitment to Cartesian philosophy by
traditional A.I. and computer systems. That is, the mind/body split that had
irritated me grew out of a long tradition of prioritizing the mind, both
phenomenologically [[16][46][80]] and through situated cognition
[[75][48][38]].
It was no coincidence that the body was relegated to the background, as
1Apple recently introduced the Nike+iPod system. This interesting opportunity for em-bodiment is ignored. The Nike+iPod system in no way detracts from the commitment todisembodied experience. It is simply a digitally-enhanced, complicated pedometer.
46
that ideology was deeply rooted in the discipline of computer science and
engineering.
3.5.1 A P M D
Suddenly, I found myself entrenched in a heated political and philosophical
debate during attempts to simply debug my device. I only wished to see if it
was working, and now I was forced to deal with much larger issues.
Did I want to argue that computational devices should embrace the body as
well as the mind? Did I want to take a strong political stance against the
tradition of Western culture? My intent had never been to construct a talking
point for that theory, or any specific theory for that matter. It felt misleading to
simply present the device as a potential advancement in HCI (which I did
[24]), and ignore the complicated social implications. Given my involvement
in an interdisciplinary program, I was encouraged to take on challenges like
this. I accepted the politics inherit in my project.
While the device had become theoretically interesting in terms of adaption,
it wasn’t until this point that I began to retro-actively re-characterize the
project. This was a critical moment for the project and myself. During
development, my decisions had primarily been based on technical reasoning
and intuition; however, if I was to critique a fundamental assumption of
Western culture and its impact on technology, all previous design decisions
must now be evaluated using this new non-technical lens.
My original design was significantly more ‘context-aware,’ and with this
new lens I became increasingly uncomfortable with the choices I had made.
47
3.6 T C ‘C-A’
The first prototype was unnecessarily more complex than the current version.
The laptop-based prototype used Java and had plenty of processing speed and
memory to accommodate complex programs. My first program only used
location to determine which music to play. Using the PlaceLab [39] libraries for
Java, an alternative to GPS that uses wireless routers to triangulate position, I
provided an interface for users to manually define their location. For example,
if I was in the offices at A.C.E., the program would pick up the routers nearby
and ask me to classify my location. I would type in something meaningful to
myself, such as ‘my office.’ On my way home, as the device found new routers
and lost known ones, it would again ask me to classify my location. Once my
typical locations were manually classified, I could go through my library and
choose which songs I felt fit each context. So perhaps I like downbeat and jazz
at my office. I would simply label all those songs ‘office’ using this program.
Then, as I walked around and switched from location to location, I would get
different types of music based on the area I was in.
I then decided that since my location was used to choose music, what about
my footsteps? After adding that into the equation, I picked up a bluetooth
accelerometer and wrote some simple routines to pick up my steps.
Everything worked ok, but it needed something. It need more context. I set
out to enhance this music device by giving it access to other information. I
wanted to add in weather detection, so that I could classify different types of
music based on the current weather. I was excited that I could have Funk-Jazz
pop up if the forecast was bright and sunny. I developed a plan to have it
detect other devices and alter music based on those data points. I was context
crazed. And after a few critiques and discussions about the device, I took a
step back. For some reason, I didn’t feel right about all this context. What does
48
my location really have to do with the type of music I want to listen to? Do I
really want to codify all these classifications? What does it mean if I carve up
my environment into digital areas? The computer scientist in me was at war
with the musician in me. The scientist was enthralled with digitally
augmenting the world around me, but the musician was wary of looking at
the world like that. Music was about experience, about emotion, about getting
into a groove. None of the context I used (or imposed) had anything to do
with why I loved music. It had the effect of hollowing out the experience.
About this time, the A.I. issues discussed earlier began to arise. I
questioned the assumption that more context was better, that the context I was
deriving was meaningful, that any of it had anything to do with music. Lastly,
I started thinking about making this device a non-laptop version. All the
power, battery life, memory, and interfaces would be gone. What would I use?
How would I even go about creating a small version of this monstrosity with
all its wireless tentacles and data mining?
When my projects get out of control, I have a tendency to reduce. Perhaps
that’s my HCI training, or just a style of work, but I immediately started
removing everything that was expendable. At this point, I hated the location
classification. It required Java libraries and large amounts of data storage.
Philosophically, I didn’t agree with what it said about the world. I dumped the
weather idea and the detection of other devices. I was left with step detection.
There was something fascinating about that experience that I couldn’t explain.
With all this talk of mind/body splits, it seemed like it might be worthwhile.
After all, it was the only feature left.
What bothered many of my classmates was the way my device interpreted
steps as context. What if I like running to slow songs? I heard many variants
on that question, and perhaps out of desperation settled on my answer: this
49
device wasn’t about finding the music you want to listen to (as is often seen as
a negative in my field [49]), it was about presenting an alternative way to
interface with a computer and your music. For a while I had been attempting
to play appropriate songs, but maybe that wasn’t an interesting pursuit. How
could I possibly determine which songs you want to hear, anyway? My
statement now, is, ‘I don’t feel that a machine can determine what a person
wants to hear at any arbitrary moment, nor would I want one to if it could.’
My perspective on the project shifted significantly. No longer was I concerned
with choosing music, but instead was concerned with helping people
experience their music in a different way. My musician side was happy with
the idea that music would be felt physically and embodied the way it was for
me. My HCI side was happy that the interface was easy to learn, and my
Computer Science side was happy with inverting the metaphors of adaption
[3]. My intuition about context was supported through a seminal paper by
Dourish, which I read some time later [15].
3.6.1 SM: T D C D
Two citations have already taken PersonalSoundtrack as an attempt to choose
appropriate songs [68][14], not only missing the subtlety of the Critical Design
elements of PersonalSoundtrack, but also using it as “... evidence of support
for the very values on which I hope to cause critical reflection” [70]. This is
perhaps my fault, as I mentioned the word ‘inappropriate’ during the
discussion of a simple learning mechanism. Unfortunately, I used the phrase
‘context-aware’ in the title of my paper only as a means to make the paper
easily understandable by the CHI community. My CHI paper focuses on the
idea of being ‘in-tune’ with the music via one’s body, not having music be
‘in-tune’ with one’s life (the designers of XPod go so far as to say the XPod
50
picks ‘correct’ music) [14]. PersonalSoundtrack is a direct refutation of both of
these works.
Both the XPod and LifeTrak attempt to choose the ‘correct’ music to play
given a user’s context. Unlike these two, PersonalSoundtrack does not play
‘correct’ music given one’s pace, but instead plays songs that have a physical
connection to the user. These songs may or may not be what the user wants to
listen to, but that isn’t the point. The connection between the user’s speed and
the song speed is a way to bridge a fundamental attribute of music to a
fundamental attribute of human movement. It is a project about embodiment,
not about a smarter MP3-player. The project actively refuses the idea that a
machine could choose the ‘right’ song at the ‘right’ time, and lays bare its
simple connection between user and music. Instead of hiding behind complex
algorithms and mysterious choices of music, it is obvious how songs are
chosen in PersonalSoundtrack and that I, as a designer, take responsibility for
that connection. XPod and LifeTrak still carry the bias of the designer, despite
the efforts to hide that bias behind additional AI, but whisk it away beneath
the computational complexity. PersonalSoundtrack is explicit about the bias of
its designer, revealing it through usage: once you understand that song speed
matches foot speed, it’s quite easy to see that such a connection was my choice
as a designer to program, not the handiwork of a a smart MP3-player.
Both the XPod and LifeTrak attempt to choose the ‘correct’ music to play
given a user’s context using several complicated methods for determining
what the user wants to hear. Unlike these two, PersonalSoundtrack does not
play ‘correct’ music given one’s pace, but instead plays songs that have a
physical connection to the user. Furthermore, the premise of my previous
paper was that the interface is embodied and refuses to impose context on its
user. There is no ‘correct’ song, only one that matches what your body is doing
51
at that moment - nothing more. The connection between the user’s speed and
the song speed is a way to bridge a fundamental attribute of music to a
fundamental attribute of human movement. It chooses songs that match one’s
pace, creating a blindingly obvious causality (whether desirable or not) and
purposeful ambiguity around the type of song, its emotional meaning, etc.
It was left for the user to create his or her own context from the songs chosen, where
they felt physically connected enough to draw mental connections. I never meant to
imply that PersonalSoundtrack picks songs one wants to hear; instead, I wanted to
state that the device presents music in a way that is meaningful to the body, and thus
to the mind.
In writing this document, one goal was to present my work in a way that
reveals the motivations for its construction. The CHI paper was written using
CHI terminology, formatted with learning mechanisms and user studies, and
as a result presents the work in a way that is conducive to CHI but easily
misconstrued.
3.7 F T
The story behind PersonalSoundtrack goes back two years, revolves around
my introduction and settling into interdisciplinary culture, contains three
design iterations informed by several philosophical and political
commitments.
Until now, this story did not exist.
Only a month ago I couldn’t remember how events had transpired to result
in my device. Often, I had to go back and contextualize previous sections as
more memories came to me. The process of documenting a process is painful
and slow.
52
Staring at the PersonalSoundtrack on my desk, I see only a glimpse into an
entire world of thought and meaning. The device is a representation of
process, not an articulation of it. Now, however, I am able to reread this
document, and reflect on it in ways I could not reflect on the device itself.
During my writing I’ve attempted to determine if this chapter could have been
written during design. I am inclined to say no. Perhaps a better method
would be to keep a notebook of design choices and the motivations for them,
epiphanies, theories and their context, etc. Not only would this provide a
useful (though limited) history of reflective design, it might also aid the
process of reflective design itself. That is, having to write down the
motivations for a design choice forces one to consider that choice in a reflective
manner.
If we are to embed reflection into design, we cannot rely on an end-product
to convey our reflection. That is not to say one should not build something -
reflection via use is also good practice. However, I suggest that one of the
greatest tools of reflective designers is a commitment to describing process as
well as developing product.
Every object has a story, and it is the responsibility of the designer to tell it.
Otherwise, deep meaning and reflection may fade from memory, or, worse,
may be retold by someone else.
53
C 4
PS: EEfficiency I
4.1 I
PublicSoundtrack is my first attempt to create a device that helps the
individual explore the role of Efficiency within the context of everyday events,
the theory proposed in Chapters 1 and 2. The device and theory were
developed simultaneously, where developments in the theory would lead to
developments in the device and vice versa. Similar to Chapter 3, this chapter
will follow the process and motivations behind PublicSoundtrack, but will
also expand on the ways in which the project supports or contradicts an
exploration of Efficiency, and future directions for the project.
4.2 W PS?
PublicSoundtrack is a collection of custom portable music players (e.g. iPod)
that interrupt you and invite you to ‘waste time.’ I use scare quotes around the
phrase ‘waste time’ because the notion of wasting time is one this project seeks
to unpack. The idea is to take advantage of the ubiquity of portable music
players and re-appropriate their daily usage to help us explore Efficiency in
daily activities.
Publicsoundtrack starts with an ordinary MP3-player and equips it with
Bluetooth and the ability to detect your footsteps. In action, when you pass by
someone wearing a PublicSoundtrack device, the music player uses Bluetooth
54
to detect that person and sounds an alarm that lets you know you’re near
another listener. Once the alarm sounds, whatever you were listening to
previously is paused. In order to turn you music back on, you need to tap or
dance the same tempo as the listener you are near. It acts like an unscheduled,
semi-voluntary moment to interact with someone. By semi-voluntary I mean
that the device decides when the dance is available, but the users decide if
they want to engage it or not. The interaction is not completely voluntary, nor
is it completely involuntary. Tempo is determined by beats-per-minute, so if
you tap your foot once every second, your tempo is 60 beats-per-minute. If the
other listener is tapping faster or slower than you, nothing happens; however,
once you two decide on and collectively tap one tempo, say 60
beats-per-minute, each of your music players chooses and plays a song at 60
beats-per-minute. The catch is that each of you hears a song from your own
library. Neither of you can hear what the other person hears; instead you
listen to your own music as it is synched up to another’s music. You can end
this simple interaction by walking away. These ad-hoc social interactions can
be as large as six people or as small as two, but all group members must
cooperate and tap the same tempo. The system is forgiving enough to allow
individual expression as long as all individuals remain close to the group’s
beat, so that those that feel inspired can dance as they like.
4.2.1 S A: B S
Imagine a person, let’s call her Julie, waiting at a bus stop listening to her
music player. A second person, Bob, also wearing a music player, is on his
way to the same bus stop. Bob takes a seat at the bus stop, when an alarm
sounds and his music is paused. The same happens to Julie. At this point, they
can either ignore each other and continue whatever they were doing
55
previously, or they can take a moment to tap a collective beat to turn their
music back on. Once the music has resumed, they can again go back to
ignoring each other, or engage the system further by interacting longer or by
promoting the interaction to dancing. After a few minutes of interaction, a bus
pulls up and Julie gets on. Bob waits for the next bus. As Julie leaves, the short
interaction is over and they each go back to what they were doing.
4.2.2 S B: L
Three students are working independently in a library, and one is listening to
her music player. A second student decides to listen to music, pulls out her
music player and puts it on. At that moment, an alarm sounds in both
students headphones. The students look around, trying to figure out whose
music player is causing the interruption. Once they find each other, they can
either ignore each other and sit in silence or tap a collective beat. They can tap
from their seats, subtlety, without disturbing others. If additional students
notice these two, they may decide to put on their music players and join the
interaction. They can do this from across the room without getting up because
the range of the device is approximately a 20 feet radius. Other students may
enter the library already wearing their music player and will be instantly
thrust into the ad-hoc social interaction. Each time a new person joins, the
entire group must resynchronize and agree on a collective tempo. Student can
leave without affecting the group, and eventually the group session dies out as
the students return to their work.
56
4.2.3 S C: S
While the previous two scenarios are interesting, there are somewhat
idealistic. The most common use of PublicSoundtrack would be when it is
ignored. Two people pass by each other on the stairs, each listening to their
music player. One is on the way home, one is on the way out. The alarm
sounds. They pause for a moment, consider stopping and continue on with
their lives. Once out of range, their music will resume and the interruption
will be over. In this case, the alarm serves as a reminder that each has better
things to do than spend a moment interacting with a stranger.
4.3 P P S
One might wonder why I am writing about Efficiency and not about private
and public space, social interaction, etc. PublicSoundtrack evolved to have
two layers: the interrogation of Efficiency and the juxtaposition of private space
within public space. This thesis primarily covers the first, though I will touch
on the second in the section on future directions. It’s useful to dissect the
project and reveal how the layers interact. The interruption and opportunity
to ‘waste time’ support an exploration of Efficiency, while the collective beat
tapping and ad-hoc dance circle supports an exploration of private and public
spaces. That is, interruptions into everyday events provide us with a moment
to reflect on those events, and dancing to a group beat while hearing
individual music re-sensitizes us to the way our private life is often put on
display for public viewing.
In the next section, I will discuss how PublicSoundtrack both supports and
betrays an exploration of Efficiency at an embodied and personal level.
57
4.3.1 A I E Efficiency
In Chapter 2, I touched on how we might use interruption to highlight
moments of efficiency. Through the careful application of interruption, we
might be able to puncture one’s thought process and allow them to mentally
poke around that moment. Because we can’t be sure how or when we are
involved in the ideology of efficiency, one way to start discovery is to allow for
spontaneous interruption that does not rely on pre-conceptions about
Efficiency. If we as participants in the ideology of efficiency try to identify
when we are being efficient, we’re victims of the assumptions and bias toward
Efficiency we are trying to uncover. So, you and I might assume the ideology of
efficiency is wrangled into obvious areas of life, such as at the office or while
doing dishes. Unfortunately, this assumption fails to account for the
pervasiveness of Efficiency, where a walk on the beach can be more used
productively or a vacation includes an optimal amount of sight-seeing.
PublicSoundtrack does not attempt to categorize situations as efficient, seeking
to identify and interrupt certain tasks, but instead interrupts regardless of
context. That means that any task, be it mental or physical can be interrupted.
If this system sought to pinpoint only efficient situations, it would reflect my
perception of what Efficiency is and betray the idea that each of us defines and
experiences Efficiency differently. How could I know when you are are being
influenced by Efficiency? I might guess accurately if I confined the detection of
efficient behavior to offices and laptops, but then the project is boring and
useless - we know plenty about the influence of Efficiency in those spaces.
With PublicSoundtrack, if you happen to walk by someone with the device
an interruption occurs. Interaction also varies depending on signal strength
and interference, meaning it’s even less likely to predict when the alarm may
sound. While it’s true that we may predictably run into certain people along
58
our daily routes, when and where these interactions occur are subject to the
individual lives of each person.
I’m not purporting that PublicSoundtrack is free of bias, but rather that it
does not propose or act on pre-defined situations and rules for determining
Efficiency situations. This task is left to the users, so that they may interpret the
interruption as they wish. Furthermore, because PublicSoundtrack is housed
in a pocket-sized package it is difficult or impossible to detect visually. A pair
of headphones signals the possibility of an interaction but does not guarantee
it, for the reasons just listed: signal strength, the lives of other listeners, that
they are wearing PublicSoundtrack and not an iPod, etc. This foundation of
indiscriminate interruption (excluding the obvious constraint that users must
be wearing one of my custom music players) allows us to uncover efficient
behavior because the system can foreground itself in any situation regardless
of how we conceptualize that situation, be it twiddling our thumbs or trying to
catch the subway to work.
Once an interruption has occurred, we have the opportunity to invoke
reflection on the task that was interrupted. This is the difficult bit and
unfortunately PublicSoundtrack is not as successful here as I had hoped. The
issue, of course, is how didactic to be. We could interrupt and simply ask
someone “How are you being efficient right now?” This is unhelpful for
several reasons. First, the question is unwieldy because we have few mental
tools for conceptualizing Efficiency at such a personal level (the development
of which is a tangential goal of this project). Second, as Sengers [70] notes, in
general, we want to avoid that kind of didactic preaching. PublicSoundtrack
employs an alternate method by exploiting the tension caused by the
interruption of potentially efficient behavior. It heightens the anxiety created
by the interruption by asking the user to stop their previous task in order to
59
engage a new one. That is, it brings to the surface the tension between
finishing what you were doing and engaging the interruptive interaction. By
interrupting you, and presenting an opportunity to do something else,
PublicSoundtrack foregrounds the tension created by the disruption of
Efficiency (if the behavior that was interrupted was efficient in some way).
Additionally, because the interaction that is offered is playful, quick and easy
to engage and the interruption caused stops previously playing music, one
must actively choose to not interact and remain in silence until nearby
listeners move out of range. It sets up a scenario that makes it difficult to
ignore anxiety and tension.
To further enhance agitation, PublicSoundtrack leaves it up to the users to
decide when to stop. This is important for an individualized experience of
Efficiency because it may take longer for some to feel anxious than others.
Open-ended interaction increases anxiety by forcing users into awkward
social situations. The interaction does not have winners and losers, nor does it
have a point-system or a clear-cut ending. Halfway through an interaction,
users might drift back to thinking about what they were doing previously,
wondering when the ‘game’ will end. This increases anxiety as well as leads
the user into a more reflective state in which they worry about how to handle
the awkward moment.
As I discussed in detail in Chapter 2 and earlier here, we can’t assume that
trivial scenarios like waiting at a bus stop are free of Efficiency. If Julie from the
bus stop had been mentally sorting and planning important events for
tomorrow, an interruption could be incredibly distracting and cause a
significant amount of anxiety. Julie might think to herself that she doesn’t
have a time for this distraction; here, PublicSoundtrack has successfully
highlighted what may be a moment of efficiency.
60
Julie may, instead, be shy and not want to engage a stranger. There are
dozens are reasons why someone may not engage the device, such as shyness,
social training to avoid strangers, fear for safety around strangers, feeling too
tired, feeling angry about a job interview, etc. While these issues are a problem
with PublicSoundtrack, I can only hope to cause some users to reflect on
Efficiency, not all.
The point is to draw to the surface how and when the ideology of efficiency
is influencing us, not at a later date but within the moment. Perhaps Julie was
not aware she was being efficient with respect to time at a bus stop. By
highlighting it we draw her attention to it. Creating a palpable sense of
Efficiency draws it out of the abstract, and out of economics, capitalism and
social politics. Putting the ideology on display and revealing it in the context
of one’s life, even in trivial situations, are the first steps toward defining what
Efficiency means at a personal level.
Surprisingly, PublicSoundtrack works best when people don’t have time to
interact with each other. The alarm can serve as a reminder that they have no
time to ‘waste,’ have other things to do, or schedules to keep. The interaction
exists only as potential that is not fulfilled, serving as a source of reflection that
may grow stronger as the reminders occur more frequently.
That’s all well and good if these moments cause the user to think about
Efficiency or productivity, but what if they don’t?
4.3.2 I
Perhaps it’s not necessary to say, but PublicSoundtrack is an attempt to
explore Efficiency and by no means a definitive example. It succeeds in some
ways, and flounders in others. First, there are no guarantees that interactions
will cause reflection on Efficiency. Second, the interaction offered may be
61
misinterpreted or may overshadow the larger point, a symptom of Critical
Design in general [70]. Third, users might game the system and use it only
when they wish to such that interruptions no longer cause tension. Lastly,
music listening is not typically an efficient activity and therefore not may be
worthwhile to interrupt if one is looking for the influence of Efficiency.
First, PublicSoundtrack offers no guarantee for reflection on Efficiency. This
is only an issue if we frame it in the world of computer science, where devices
are often evaluated by functionality; however, we need to be careful when
applying computer science evaluations to artistic computational devices like
PublicSoundtrack. Despite its appearance as a standard electronic gadget, it is
an artistic piece that follows the spirit of “Device Art” [45]. Furthermore, it
uses that framework to dig at the role of Efficiency. Electronic gadgets that look
like PublicSoundtrack are typically judged based on what they help us
accomplish, with the hope that they can help us complete tasks faster or easier
or make our lives better. PublicSoundtrack, while packaged as a consumer
gadget, does not offer a way to make one’s life better or easier. Instead of
helping one accomplish a task, it functions by interrupting other tasks. It is
antagonistic towards Efficiency, making its users less productive and derailing
optimization. PublicSoundtrack is positioned between Faux Technology and
Device Art. This allows it to penetrate daily life, where users can see
themselves using it. At the same time, in usage it calls into question the
existence of devices like itself and asks users to think about what these devices
do for us. Specifically, it offers reflection on how computational devices
reinforce Efficiency through increased productivity and optimization, and by
proxy the role of Efficiency in one’s lives.
Second, despite my best efforts (as I’ll discuss in the section on process), the
interaction or ‘game’ PublicSoundtrack offers tends to take precedence over
62
the interruption. This contributes not only to the way it is perceived by users,
but also to the way it is received during presentation. The interruption may
reveal efficient behavior and ignite tension, while the interaction is intended to
focuses that tension. The ‘game’ I first designed was far more complicated,
and I deliberately simplified the interaction in order to focus attention on
anxiety. I don’t feel the simplification was sufficient because the interaction
still focuses attention on tension as much as on social awkwardness.
The interaction had to be enticing and easy to engage in order for it to serve
as a source of tension. That is, if the experience PublicSoundtrack presented
was too annoying or demanding, it may never be used even if users were
willing to make time to use it. The argument can be made that the current
interaction is too annoying, and does detract from the reflection. I tend to
agree, but was unable to design a more fitting experience. The problem was to
design interactions that were engaging but not too engaging. Several
possibilities for fixing this issue came up during development, as I’ll discuss
shortly.
Third, users might game PublicSoundtrack, only using it when they wanted
to use it. This is actually less of a problem than it appears. Even if one could
section out areas where they would expect usage, they would have to turn off
the device whenever he/she didn’t want to be interrupted. That would require
users to not listen to music in the majority of scenarios they already do - public
places - or carry around two portable music players. If the alarm sounds, users
can simply move out of range of each other within a minute (at which point
their music resumes). Given that, significant changes in current behavior
seems unlikely. Second, if a user did decide to hunt for other players, there’s
no guarantee those users want to be interrupted. For the users being
interrupted by the gaming user, PublicSoundtrack functions as it should. It
63
only functions differently for the user hunting down others, and even then
he/she is essentially increasing the number of potential interactions, not
necessarily increasing the probability that others want to make time for an
interaction. For groups of friends that wish to meet at certain times to engage
in the synchronized dance, almost every computational device is susceptible
to that kind of aggressive re-appropriation. Even if a group of friends found a
spot to use the system as they wished, a stranger could easily walk by and be
added to the circle, interrupting the friends’ scheduled dance circle. So, while
it’s possible for users to re-appropriate the system, it’s by no means fool-proof
and attempting to design out that kind of repurposing is a losing battle in
general (not to mention that re-purposing is often interesting); however, I
would be interested in the ways users might accomplish this, and what that
says about either the the project or efficient behavior.
Lastly, we might assume that invading someone who is listening to music
will not interrupt efficient behavior, given that listening to music is typically a
non-optimizable, non-efficient behavior. This is understandable, but ignores
the context in which portable music players are used. These players are used
in multiple scenarios, amidst a myriad of tasks, as either background music,
traveling music, working music, etc. The act of listening to a portable music
device for the sake of listening to music rarely happens in isolation. iPods are
small and portable because people take these devices with them, to
accompany them during other events in their lives. The act of listening to
music does not constrain other tasks being performed by the user, and just
because someone is listening to music doesn’t mean we can assume they are
not multi-tasking. I sit writing this chapter at a coffee shop because I have a
half-hour to kill, and instead of twiddling my thumbs or staring at the sky, I’ve
decided to write. I happen to be listening to my iPod. If this music player I’m
64
listening to were a PublicSoundtrack device, an interruption right now would
definitely be interrupting my efficient behavior to optimize my time.
So, while PublicSoundtrack is by no means a ideal, the main issues it suffers
from are not unreasonable. Its existence can catalyze other projects or a
general interest in the exploration of Efficiency.
4.4 P
This section details the process and motivations for PublicSoundtrack,
including but not limited to the design process and why I sought to discuss
Efficiency.
4.4.1 M U B
PublicSoundtrack is based off the platform I used for PersonalSoundtrack. I
originally purchased the Gumstix motherboards with the Bluetooth chip so
that I could troubleshoot and debug my code wirelessly. The only other
method for directly interacting with the Gumstix was via a serial board that
was large and not usable in a final version. Once I had finished
PersonalSoundtrack, I began thinking about how I could take advantage of the
wireless functionality. Projects like tunA [6] used wireless technology for
transmitting actual music, but I wasn’t interested in spreading my music to
strangers. PersonalSoundtrack had been developed to be intensely isolating,
focusing the user’s attention inward to the body and on physical connections
to music. With PublicSoundtrack, I sought to make a project that included the
social aspects of walking in public, in ways that differed from SonicCity [53]
and tunA.
65
4.4.2 F D
My initial idea was to create a multi-player game based on the concepts from
PersonalSoundtrack. I thought this would be interesting because the device
could use only aural cues, allowing users to keep their eyes on the world
instead of burying their heads in 3” screens. I spent a significant amount of
time developing one such game, though ultimately it wasn’t finished because
the project wasn’t interesting enough for a thesis project at ACE. The game I
designed was an aurally-augmented version of tag, that used Bluetooth as a
means to detect and communicate between players. Similar to
PublicSoundtrack, an alarm would sound and interrupt listeners when other
PublicSoundtrack users were nearby, at which point a game of tag was offered
to the users. If users engaged the game, a seeker and runner were
automatically chosen by the system and announced to the users via their
headphones. As the game began, music was chosen and matched to the each
user’s pace. The music heard by each user would respond to different
elements of the chase. For instance, if the runner was running slower than the
chaser, the runner’s music might become more intense. If the runner was
faster than the chaser, the runner might hear more joyous music and the chaser
would hear more depressing music. Either way, the music would be matched
to pace and enhance the game. The system also had support for multiple
teammates. If several chasers were following a single runner, the runner’s
music might become frantic while the team of chasers would hear their
pre-chosen “team song.” If a runner hid behind a wall and stopped moving,
his music might get quiet and suspenseful, where the chaser’s music could
help him/her find the hiding runner. The music played was governed by a
simple set of rules based on pace and distance to other players, which was
determined via signal strength. As I developed this project, I was also
66
documenting and writing about the theories behind PersonalSoundtrack. It
was through that documentation process that I begin investigating Efficiency.
4.4.3 I O
I didn’t develop an obsession with Efficiency overnight - the process matured
over the course of a year and half. During the development of
PersonalSoundtrack, I noticed a few computer scientists developing
technology similar to PersonalSoundtrack. Their approaches were
technologically similar but subscribed to radically different philosophies.
Other researchers, whose work followed mine sought to use user-pace-based
music as means to optimize your workout, or help you achieve a ‘perfect’ run
[14, 68, 59]. That is, they had you set up playlists ahead of time with planned
heartrates and speeds, such that when you went out on a run your music
would follow tempos that helped you achieve maximally-beneficial runs. I
couldn’t put my finger on it, but the absolute last thing I wanted to create with
PersonalSoundtrack was a gym-optimizer. It was after this moment of contrast
that I realize how PersonalSoundtrack could in fact help someone be less
efficient, allowing them to become physically absorbed in the music as it
followed them, encouraging meandering and ‘wasting’ time. Instead of
developing playlists ahead of time to plan out a perfect run,
PersonalSoundtrack had no concept of a playlist, and simply followed
whatever physical movements you were inclined to make. In fact, one of the
primary reasons I developed PersonalSoundtrack was because pre-defined
playlists are created in a completely different context than the context in which
they are experience [24]. If anything, a pre-defined playlists were
philosophically refused by my project, where they were touted as an feature in
other projects. At this point, I became emotionally involved in distinguishing
67
my work from the perception of it as a gym-trainer.
If that wasn’t enough, just before my first publication on
PersonalSoundtrack, I had been in talks with Simon Penny about how difficult
the project was to analyze using typical Computer Science metaphors (this still
holds true). The closest either of us came was a description using 2nd-order
cybernetics metaphors such as Walter’s turtles, the Ashby’s Homeostat or Beer
and Pask’s U-Machine [81, 65], and so began exploring what seemed to be the
primary underpinnings of Computer Science metaphors. In addition to
Input/Output Theory and Control Theory, Efficiency seemed fused to
computation. I became increasingly intrigued by the general ideology of
efficiency: it seemed important not only to designers, but also deeply
integrated in the culture of the computer. The more I explored this realm, the
more I felt a strong desire to explore this idea through artistic design.
Before summer of 2006, I began looking into how Efficiency was integrated
into our culture and felt it was not confined to work only but influenced our
notions of play as well. Our notions of life in general were largely defined by
Efficiency [8, 78, 5], which meant our notions of play were also affected. Bell
and Kaye discussed how Efficiency has become a way to design life and how
computational devices were recruited as tools for and objects of the homes of
the future. Penny further explored how play is influenced by efficiency and
optimization [64].
Meanwhile, the multi-player tag game was taking shape and I noticed it
had the potential to mix work and play in ways that could help us rethink
those categorizations. Since it was an audio-based device, the kind that is
often used to accompany both work and play, I saw an opportunity to delve
further into this issue via the project. If the tag game was impromptu and not
initiated by the user, could it invade moments of work and inject a moment of
68
play? The goal would be to explore how work and play might be connected
via the ideology of efficiency, where efficiency was a way to evaluate and
conceptualize both subjects. I was eager to fit my project into the scope of
Efficiency, but I decided that the issue of work versus play was tangential,
though related, to Efficiency. I needed to hone my focus on Efficiency itself, and
exploring it. The next step was to figure out just what I meant when I spoke of
Efficiency.
4.4.4 W E?
I never had a satisfying answer to that question. I could not explain what I
meant in a way that made sense to everyone. If I resorted to personal anecdote
then my audience could relate, but I couldn’t formalize a definition. This
puzzled me. Often, it was clear that particular instantiations of Efficiency were
subjective and specific to each case, and yet I was trying to talk about it
generally. I couldn’t figure out how we could have a sense of Efficiency, but not
be able to talk precisely about it in the context our lives. Part of the problem
was my inability to explain myself, but additionally, economic, socio-political,
or mathematical facets of Efficiency were difficult to apply to my life in a way
that felt meaningful. So I broadened my scope until I was left with a nearly
useless definition: the maximum output with the least waste. The more
abstract I became, the less useful my definition became. What I didn’t realize
was that I should have been focusing more on personal anecdote and less on
abstraction. How can Efficiency impact us every day, our actions, thoughts, and
choices, and yet none of us really has a any sense of what Efficiency means in
that context? This question plagued me as I continued the development of my
project.
69
4.4.5 R
By now I had ditched the aural-enhanced tag game, and set out to interrogate
what could be done after an interruption that would start poking at Efficiency -
whatever I meant by that term. I had assumed that the interaction that
occurred after the interruption should be fun, but in several discussions it
seemed impossible to interrupt someone and then ask them to have fun. This
still holds true with the final version of PublicSoundtrack, as I think this
system of devices is mildly annoying in usage. I developed a less structured
game that sought to disrupt with other people that might be behaving
efficiently. PersonalSoundtrack acted as a passive system for inefficiency via
meandering, and PublicSoundtrack became a more aggressive attempt to
derail efficient behavior. The first idea along these lines was a game where
players could stomp the ground and send out virtual shockwaves to nearby
listeners and temporarily disrupt their music. So, if two listeners walked by
each other, the alarm would go off, and they could stomp the ground to
‘bump’ the song the other person was listening to. A ‘bump’ meant speeding
up or slowing down the other person’s music by a few beats-per-minute. The
idea here was that users could aurally disrupt other users as a representation
of the mental disruption from what they were doing previously. This behavior
was fully-functional. In theory, it seemed fun, but in practice it was a little
boring. I have not given up on it, but something about it did not feel right.
Eventually, I decided to add in a cooperative version, where users could tap
the same beat and listen to different songs synched up to that beat. Now I had
a disruptive mode and a cooperative mode. Each mode could be entered or
left at any time, such that if all users were cooperating any user could decide
to stop cooperating and disrupt the others and ‘bump’ their music. The
interactions of cooperation and disruption became a little complicated and I
70
preferred the cooperative mode. I ended up removing the disruption
completely.
4.4.6 I S T R D
I happened on a paper called “Slow Technology” by Hallnas and Redstrom
[35], which proposed designing for reflection instead of Efficiency. It was a
primarily an alternative design strategy instead of an unpacking of Efficiency.
It was clearly related, but didn’t give me a way to think about Efficiency on a
personal level. Sengers’ “Reflective Design” interested me because it brought
computational devices into the world of the individual and presented
scenarios for reflection. Sengers, Agre, and others made a general call for
technologists and designers to design for reflection and laid out how one
might do that.
After months of thinking about Reflective Design and Slow Technology, I
realized that I already had all the pieces in front of me and just had to put
them together. First of all, instantiations of Efficiency are a subjective in
evaluation. Second, it is difficult to explore the subjectivity of the term via the
writing of someone else, especially when that writing deals with global scale.
Third, I was busy building a personal computational device that could invade
its user’s personal life. Fourth, Reflective Design and Slow Technology talked
about using computational devices for reflection.
Once I put these pieces together, I realized that I had trouble explaining
what I meant by Efficiency because I was looking for a abstract characterization
of highly personal events. That is, I knew what it meant to me, in the context
of my life since I’d spent so long thinking about it, but I didn’t know what it
meant to anyone else. This situation felt odd because, as I’d read many times,
the ideology of efficiency clearly affects many of us if not all of us on a daily
71
basis. And yet it was difficult to talk about at that level. At that point I realized
that “What do you mean by Efficiency?” should be rephrased as “How does
each of us conceptualize Efficiency in our lives?” and that very question should
be the focus of the device. I thought of this right around the time I was
removing the disruption mode from the device. I felt that the interruption
alarm plus the cooperative tapping could cause people to feel tension between
their previous task and a trivial, goofy interaction with a stranger. The
cooperative tapping was easy to engage, silly, not too annoying, and trivial to
encourage participation.
4.5 F D
There are a few options that might help PublicSoundtrack better explore
Efficiency. First, I might look at making the system more didactic without
sacrificing artistic quality. I had debated long about adding an aural timer that
would tell the users how long they had spent dancing with a stranger. This
might heighten the anxiety and clarify the area of reflection, but may cross the
line of preaching to the audience. Another option is to, instead of offering an
annoying or mildly entertaining interaction, offer a highly engaging and
enticing interaction. The point of this would be to aggressively derail users
from their previous task with the hope that once they get back to what they
were doing, they feel guilty, depressed, or anxious that they had ‘wasted’ that
time. This strategy is more masochistic but potentially effective. The difficulty
would be separating it from other activities people use to ‘waste’ time.
The primary future of PublicSoundtrack lies in delving into the way it
combines the private and public spaces. This will likely include the removal of
the interruption alarm, and a modification of music synchronization. At the
72
time of writing this, I’m working on developing a more passive system that
would synchronize people on the street without large modifications to what
they were already listening to. For instance, imagine a busy street where a
quarter of the people walking are stepping in synch with each other, listening
to music at the same speed, all without their knowledge. Their headphones
might flash when they step such that at first, everyone’s headphones flash
spastically because no one is in synch. Slowly, as they are automatically,
passively synchronized to each other, the lights begin to flash in time with
each other, until all headphones on that section of street flash at the same
tempo. Because PublicSoundtrack currently includes in it the
PersonalSoundtrack technology, such that when users are not near other
listeners, the device becomes PersonalSoundtrack, users of PublicSoundtrack
version two would always have their music synchronized to their pace, and
now their pace would also be synchronized to everyone else near them
without the interruptions used in version one. This project would seek to
sensitize us to the way we live in private worlds while navigating public
spaces, inviting reflection and discussion about the ways we juxtapose and
navigate these spaces simultaneously everyday.
73
C 5
T D
5.1 I
Technically, PublicSoundtrack can be broken down into three major sections:
hardware, software and networking, control flow. What follows is a technical
description of how I developed PublicSoundtrack, including schemata,
diagrams, code, and sketches. Because PublicSoundtrack is based off
PersonalSoundtrack’s design, much of the discussion will span both projects.
Though kept to a minimum, I will sometimes delve into significant issues
encountered during production that instigated design decisions.
5.2 D A
In moving from PersonalSoundtrack to PublicSoundtrack, I achieved several
design goals:
1. Power solution that requires a single-cell Lithium-Ion battery
2. Pocket-sized
3. Externally-based OS that resists corruption, is easily replaced, and
plug-and-play
4. Reduction of computational load on Gumstix
5. Custom circuit for measuring steps that is cheap, light, requires little
power, and be robust, and fits 1”x1” x .15” space
74
6. Custom circuit must accurately detect steps in less rigid and stable
environments (pockets instead of belt)
7. Cost of all parts must be below $250
5.3 H
The hardware component of PublicSoundtrack includes the modification of
Gumstix motherboards and audioboards [33, 32], the development of a custom
step-detection circuit, and the miniaturization of PersonalSoundtrack.
5.3.1 C
When developing PersonalSoundtrack, I decided to use the Gumstix
motherboards because they are extremely small, low-cost, moderately
low-power boards that run linux. This meant I had greater choice in
developing the software, and left open the potential for cross-platform
development, as well as source code that was meaningful to other
programming communities. The Arduino board [4] was considered because of
its moderate size, but it bulky components not suited for this application.
The Gumstix motherboards use on-board flash memory to store the
operating system - an extremely light-weight modified Linux kernel similar to
Familiar [51]. A separate audio board, Audiostix2, was purchased to support
audio out. The motherboard includes an Infineon Bluetooth chip. The boards
require 3.5V - 5V and, with Bluetooth enabled, run for approximately 5-6
hours on a 900mA lithium-ion battery.
75
5.3.2 M
To develop the pocket-sized version, I had to cut the previous thickness down
to 1/4 the size of PersonalSoundtrack and reduce its height and width to the
size of the Gumstix audio board. I did this by removing the Compact Flash
board and replacing it with motherboard-based MMC storage, using li-ion
batteries, creating a custom step-detection circuit that is the size of a silver
dollar, and rewriting the code to compensate for the excessive vibrations as a
result of the device being placed in the pocket. PersonalSoundtrack was
securely fastened to the waist with a large belt, which helped minimize noise
on the accelerometer. While this works, and other developers rely on
armbands and such for accurate accelerometer data, I wasn’t satisfied with
being forced to wear a belt or armband. The pocket version only requires
vertical orientation, and works fine while being bounced off the leg during
walking. Unfortunately, the largest part of the device is the antenna used for
bluetooth communication. I have found a flat bluetooth antenna, and am
looking into replacing the current antenna. Horizontal orientation will be
compensated for in subsequent versions.
In the following sections I discuss how I was able to miniaturize the system.
5.3.3 P
The first modification necessary was a portable power source.
PersonalSoundtrack used four (4) AAA rechargeable nickel-cadmium
batteries. These batteries took up a significant amount of space, and thus for
PublicSoundtrack I used a 900mA 3.7V Lithium-Ion flat battery from
SparkFun [21]. This battery was wired to the main power terminal, using the
alternate ground terminal so that the Gumstix could use non-portable 5V
76
wall-wart power supplies, if necessary, without requiring one to de-solder the
Lithium-Ion battery. Though I could have used larger batteries (¿ 900mA) to
achieve longer life, I severely restricted myself with regard to the size of the
final product. Because I wanted PublicSoundtrack to be easy to imagine as a
standard electronic product, I needed to compete with the size of modern
portable audio players (e.g. iPod Nano).
5.3.4 C, -
Second, I developed a custom circuit onto which I could offload extremely
time-sensitive operations. In PersonalSoundtrack, none of these
high-frequency operations were offloaded, and resulted in occasional
computational “hiccups” and mediocre battery life. For PublicSoundtrack, a
primary goal was to reduce computation on the Gumstix boards significantly.
The custom circuit (Figure 2) primarily includes a 16F648A PIC [56] chip and a
2-axis accelerometer breakout board [22]. This particular PIC chip is capable of
UART transmission at TTL voltages, allowing it to speak easily with the
Gumstix, whose STUART interface also ran at TTL. The most computationally
expensive operation in the system was reading data from the accelerometer.
The accelerometer reports approximately 100 times per second, easily enough
to overload the Gumstix operating system given the overhead of running an
operating system. The PIC chip, on the other hand, running at only 20Mhz, is
able to read from the accelerometer at that high-rate without issue.
Once the PIC chip was set to read from the accelerometer, there was still the
issue of serial transmission. If the PIC chip simply sent across all of the data it
read from the accelerometer, we would not have improved the overall system
at all. Instead, the PIC chip needed to intelligently interpret the data from the
accelerometer and speak with the Gumstix motherboard as infrequently as
77
Figure 5.1: Schematic of Step-Detection Circuit
possible. This, clearly, became a software issue and I will cover that in the
software section. The PIC chip reads data from the accelerometer, determines
if a step has occurred, and if it has, sends time-independent data representing
that step. In this way, delays in the serial line are not important because the
data being sent is decoupled from the time it was sent. The complete circuit
required a few additional components, such as a 18.432Mhz crystal to be used
as a reference clock for serial transmission, capacitors on the crystal to stabilize
the clock, and a standard 10pF capacitor across power and ground to stabilize
the incoming power from the battery. I designed this circuit with significant
help from Tom Jennings, who helped me understand the basic needs for the
crystals and capacitors, as well as guiding me through the debugging of the
circuit design. Once the breadboard version of the circuit was functional, I set
out to design the final version.
An additional constraint on the final board was overall size. The Gumstix
78
boards leave little room for additional components, and the Lithium-Ion
battery was approximately 70% of the length of the motherboard. This left a 1”
by 1” square of space for the custom circuit, and only .15” inches of height.
Due to time constraints, I was unable to complete an SMT (surface-mount)
in-circuit programming design. I was forced to use non-SMT components and
hand-solder this 1” square component. I cut off the pins for the PIC chip, and
developed several layouts in an attempt to fit these size constraints.
I continually worked to improve the layout, as shown in Figure 1, but in the
end non-SMT boards of that size tend to look like a rat’s nest in spite of the
best layouts. Serial connections, transmission (TX) and ground were wired
from the PIC chip to the gumstix board’s STUART pins. The Gumstix pins
were microscopic, and required special fine wire. In my experience, that wire
lost transmission power if it was strung too long, and so I used the fine wire
only for the quarter-inch off of the Gumstix and used heavier threaded wire
for the remaining distance.
5.3.5 P R
In addition to powering the Gumstix boards, I also needed to power the
accelerometer and additional custom circuits. Not only did this mean
additional wiring to both components, which required creative threading and
wire distribution, but also it required a verification that the Lithium-Ion
battery would effectively power all components. The PIC chip datasheet states
that it runs at 20Mhz at 4V, and the serial and accelerometer reading was
carefully tuned to that clock rate on the order of microseconds. While not
guaranteed to run at 20Mhz at less than 4V, the custom circuit I developed
runs consistently at 20Mhz despite the Lithium-Ion battery providing slightly
less than 4V ( 3.75V). Additionally, the accelerometer required 3-6V. After
79
Figure 5.2: Step-Detection Circuit
80
testing the system with the 3.7V Lithium-Ion battery, it appeared to work
correctly. Fortunately, the system did not require any additional manipulation
or conversion of voltage in order to power all components.
5.3.6 R
The battery was soldered through the Gumstix board to a sliding on-off switch,
where the switch simply disconnects the positive rail; however, because the
battery was soldered in, it was difficult to recharge. While my SMT circuit
design included the Max1555 [13] chip for in-circuit recharging of the battery,
the non-SMT circuit barely fit in the space under the motherboard as it was -
there was no room for the Max1555 chip. To solve this issue, I wired ground
and power to two broken off DIP pins which I embedded in the wood of the
final package (Figure 2). I purchased a small pre-soldered Max1555 recharging
circuit [23] to which I soldered the male DIP pins. The purchased recharging
circuit allowed for 5V wall wart power source. The final recharging was quite
tidy and plugged directly into PublicSoundtrack for easy recharging.
5.3.7 U-P
Finally, a problem that plagued both PersonalSoundtrack and
PublicSoundtrack was the through-hole audio jack. The Gumstix boards use
components that are easily broken off after several months of usage. The
audio-in jack was the most susceptible to damage given that one often plugs
headphones into and out of portable music devices with varying degrees of
care. Even worse, if the audio jack was ripped off, the traces in the board were
nearly impossible to resolder at my skill level. Additionally, because the audio
jack is on the side of the motherboard, it would require the headphones to jack
81
into the long side of the final product, making it nearly impossible to place in
the pocket as I had planned. To solve this issue, I purchased a separate 1/8”
panel-mount audio jack and soldered it to the back of the on-board audio jack
using flexible wire. The panel-mount audio jack could be easily mounted to
the final packaging, and any trauma suffered by that jack was not translated to
the board itself, allowing me to repair or replace the panel-mount jack as
needed. This also allowed me to mount the audio jack in a more opportune
place on the final package, at the short end on the top end of
PublicSoundtrack, such that the headphone jack stuck up out of the pocket.
5.4 S N
The software and networking aspect of PublicSoundtrack was considerably
challenging primarily because of the immaturity of the Gumstix platform. The
Gumstix is a highly-customized product despite its underpinnings in Linux
and well-documented hardware parts. There were five distinct software
efforts: customization and compilation of the Linux kernel, running the main
operating system from an SD flash card instead of the on-board flash memory,
the main application, plugins and modules for the main application, and the
step-detection circuit software.
5.4.1 O
The Gumstix motherboards are shipped with a pre-installed Linux kernel that
contains many modules, but did not include many that I needed - namely,
Python, bluetooth and audio libraries, etc. In order to customize the kernel,
one checks out a particular revision from the Gumstix SVN and customizes the
kernel as you would do any Linux flavor. This process is split into kernel
82
modification, which is the inclusion and exclusion of kernel-level libraries and
drivers, and the modification of the library files, programs, and library files in
the operating system itself. The process was maddeningly slow given that
changes had to be recompiled and re-flashed onto the Gumstix motherboard.
That process averaged 30 minutes using a Compact Flash card, and even
longer using serial transfer. The Gumstix-flavor of linux is updated daily, and
often include desirable features but, unfortunately, also include numerous
bugs. For example, when I began PersonalSoundtrack the latest revision had
broken Bluetooth and Compact Flash support. I had to walk back through
each revision (of which there were well over 1000 at the time) until I found one
that was broken in an acceptable way. Furthermore, there was almost no
documentation available and all help was limited to the mailing-list only.
Direct contact with the developers is helpful for specific questions, but not
very helpful for general guidance.
Unfortunately, many of the audio drivers and library files were not
automatically included in the image despite their inclusion in both the kernel
and OS. There was a long process of tracking down each file that was
incorrectly omitted from the build. These files ranged from the blatantly
obvious (libao.so) to the rare(libreadline.so). Furthermore, the OS was flashed
to on-board memory in the form of a read-only compressed image making it
impossible to make even trivial tweaks. The process was so laborious that I
currently have stored in multiple locations the configuration files for five
different revisions based on what is broken in each, as well as the entire 2GB
build of each. In spite of my detailed “how-to” that I wrote during this
process, I’m afraid that if I revisit the process I may have to start over mentally
and programmatically in order to get everything just right.
83
5.4.2 I - -
Using on-board flash memory is ok to a degree, but it suffers from significant
drawbacks and thus I sought to work around it if possible. First, Replacing an
OS image is slow and requires archaic commands that are difficult to
remember and each has the potential to completely brick the Gumstix
motherboard. These commands are hand-typed into the Uboot terminal, and
each command should be double-checked to prevent devastating mistakes.
For example, if one accidentally overwrites the first sector of the
flash-memory, the entire motherboard needs to be factory re-flashed (unless
one happens to have a JTAG programmer handy). The command “protect on
1:0-1” tells Uboot to not overwrite the protected sector, but “protect on 1:1-0”
allows the critical section to be overwritten. This kind of risk is unnecessary.
Second, the image needs to be properly unmounted before a power off or it
can corrupt the entire OS, including all custom files. If this happens, one has
no choice but to reflash the OS using the 15-30 minute process. This is due to
the fact that the Gumstix uses the JFFS2 format [40] which requires complete
and consistent cleanup before powering down. This meant that if the batteries
died, the entire Gumstix could become corrupted. This happened often.
Again, this was unacceptable. Third, uncompressing and reading the JFFS2
image is an excruciatingly slow process - a process that is executed each time
the Gumstix is powered on. Because the Gumstix does not support any type of
sleep mode, only its automatic low-power idle mode, it is often powered
down completely to save battery. A full boot using the on-board flash memory
often took as long as a full minute. For comparison, most portable audio
players wake from sleep in under 5 seconds, and the iPod can awake within 1
second. Even slow mobile phones rarely take longer than 30 seconds to boot.
84
5.4.3 P OS
To resolve these issues, I decided to move the OS onto a more convenient
memory location. While PersonalSoundtrack used Compact Flash to store
audio files, PublicSoundtrack uses MMC instead. It turned out that a Gumstix
developer had hacked together a way to boot a development OS from the
MMC card. Using modified versions of his scripts, it became possible to boot a
fully functional, read-write OS from the MMC card. This process, however,
required a custom ramdisk image and custom startup scripts to bypass the
JFFS2 boot process. These tasks were terribly tedious and even easier to ruin
than the previous OS compilation. Furthermore, the only revision that
supported this special boot process did not support the Bluez bluetooth
libraries. I had to patch the old revision with updates from the newer versions
in order to reinstate it. Once this process was complete, however, all of
aforementioned issues went away. First, I no longer need to interface with
Uboot in order to replace the OS: I could pull out the MMC card from the
Gumstix, plug it into my laptop, and copy over a pre-compiled and
compressed version in less than 5 seconds. Second, the OS image stored on the
MMC card was not compressed using the JFFS2 format and thus could be
uncompressed and read completely in about 2 seconds, as compared to the
previous 20 seconds. Third, the new image format was not as susceptible to
corruption, but even if it does become corrupt, replacing the image is as easy
as copying a file to the MMC card. Additionally, there were other benefits of
using the MMC-based OS. For PublicSoundtrack, I needed multiple devices.
Instead of having to go through a complicated installation and initialization
process as is necessary with the JFFS2-based OS, I could simply copy over the
OS and program files to the MMC card, plug it in, and turn it on. The Gumstix
was up and running within 30 seconds. Also, this made it very easy for others
85
to install when I put instructions online. Users could simply download an
archive of files, uncompress it, and copy to their MMC card, stick it in the
Gumstix, and everything would work as it should. This was now a good
solution from both a hardware and software standpoint.
5.4.4 T
The main application was written in Python for two reasons: the Gumstix
boards have Python in the buildroot by default, I wanted to learn Python, and
Python supports modules written in almost any language. That last item was
particularly helpful in case I needed to do audio file decoding or other
real-time computationally intensive processes. While Python is very powerful,
it cannot compete with the pure speed of C at it’s current state. Furthermore,
Python had many modules that would be useful for my application, such as a
SQL database interface, a Bluez interface, an AO interface, and MP3 decoding
interface. The application uses the model-view-controller principles, though
obviously modified given that the device has no visual interface. There is a
central controller file that handles all communication through the application,
such that no modules speak directly to each other. That is, the audio player
portion knows nothing of the step detection portion, such that I need to
change only the Controller file if I want to modify interactions.
D
The application is broken into four main parts: controller, music playing, step
detection, and bluetooth networking. The controller is, as I said responsible
for communication, but acts as the main decision center, where it decides
when and by how much to time-stretch the music, when to change songs,
when to connect to other devices, etc. The music playing module handles the
86
MP3 decoding using PyMad [83], plays or pauses music, and performs the
actual time-stretching. The step-detection module was much larger in
PersonalSoundtrack than it is in PublicSoundtrack. This is because much of
the step detection code was offloaded to the external PIC chip in
PublicSoundtrack. Thus, the current step detection module on the Gumstix
board does basic averaging of the user’s pace based on the data sent by the
PIC chip, and exists primarily to act as a serial interface for the PIC chip. The
step detection module is in charge of alerting the ‘controller’ class when a step
has been detected.
The bluetooth module sets up and handles all bluetooth network
connections, including a server, multiple clients, and asynchronous
transmission and receiving of data. The bluez API for the gumstix limits a
server to 5 clients. To handle asynchronous data transfer, each time a client
connects to a device, a new thread is started for reading from that client.
Reading from a client is a blocking call, but because that block exists in a
separate thread, it does not impact the rest of the program. These read threads
are started and killed only when clients connect or disconnect, so they scale
well with demand. The bluetooth network is capable of killing off a client if
the client is powered off or if the client move too far away simply by checking
for read errors. On average, a disconnect takes around 10 seconds. Each
device is able to detect devices up to 20 feet away.
A -
Time-stretching is achieved using very inexpensive operations that produce
mediocre sound quality. To speed up a song, each group of decoded music is
artificially shortened slightly, and to slow down a song, each group of decoded
music is artificially lengthened. This method is acceptable for +/-3BPM
87
changes, but does shift the pitch and introduces musical artifacts that can be
noticed by a keen listener. I looked into using proprietary time-stretching
algorithms, but most required non-ARM chips and thus were not available to
me. I was resistant to time-stretching in general because it typically reduces
sound quality and generally interferes with what the artist had intended to
produce. Thus, I felt the limitation imposed by my amateur time-stretching
was acceptable.
M BPM
Instead of using automatic-beat detection algorithms which often fail to work
accurately, I decided to rely on my own ability to tap the beat of a song. I sat
down one night and tapped out the beat of 100 of my favorite songs. I stored
this data in the standard MP3 ID3 tag, so that the information would be
non-proprietary. Using an exporter I wrote for iTunes, a SQLite database is
constructed from these songs and the BPM tag is stored as a field in the
database. A user’s pace is calculated and then a query is performed on the
database. All songs within a few BPM of the user’s pace are selected and
played in random order. Additionally, songs that are multiples of the user’s
pace are also selected. If the user’s pace is 100 steps-per-minute, songs will be
chosen near 50 beat-per-minute, 100 beat-per-minute, and 200 beat-per-minute.
This admittedly low-tech solution to beat detection is intentional.
MusicBrainz.org is a site that houses meta-data for songs, and uses a digital
audio-fingerprint to match a song with the correct meta-data (artist, title, year,
etc.). They have a cross-platform software tool that will scan your music
library and update the tags on all songs and uses an audio-fingerprint instead
of the current meta-deta in your music’s tags to identify the tracks. I have
spoken with the developers of this site and they are planning to add a BPM tag
88
later this year. This would support a community-maintained BPM
information for many tracks, such that eventually the MusicBrainz software
tool could automatically update the BPM tag on all your music. I find this idea
much more promising than the hope that software will be able to analyze
music and correctly determine BPM data. MusicBrainz works because people
take the time to submit meta-data, and there’s no reason to think the BPM data
would be any different.
5.4.5 P
PublicSoundtrack relies on several open-source python libraries: pyMad,
pyAO, pySQL, pySerial, and pyBluez [83, 10, 36, 50, 37]. In order to use these
libraries, they had to be cross-compiled for the Gumstix’s ARM instruction set.
Some libraries were plug-and-play, such as pySerial. Any python libraries that
do not rely on C or other languages were fully compatible without
modification. pyAO was difficult to integrate because of previous issues with
missing library files. pySQL, pyMad, and pyBluez however required
cross-compilation. Unfortunately, the cross-compiler that is built with the
Gumstix Linux variant was unable to process the python scripts; however, by
unpacking the C files and toying with options I was able to manually compile
the files and pack them back into the package for use on the Gumstix. This
issue affected many users, and as such I have made my cross-compiled
versions of these libraries available for other developers via my website.
5.4.6 S-
The goal of this circuit was to read the pulse-wave modulation data from the
accelerometer, determine if a step had occurred, and send out data that was
89
time-independent over serial to the Gumstix. In addition to these goals, I was
able to program the PIC chip to dynamically sensitize to both the
accelerometer and the user. All of the code for the PIC chip was written in
assembly.
I used accelerometers that report a pulse-wave approximately 130 times per
second. This wave can be used to determine the acceleration applied using the
ratio between the ‘on’ and ‘off’ sections of the pulse. The first method was to
have the PIC check the accelerometer, and increment a counter for as long as
the accelerometer was sending a ‘1.’ When the accelerometer stopped sending
a ‘1’ and began sending a ‘0,’ the PIC chip would increment a separate counter.
That process would indicate a single pulse. Afterwards, the two counters
could be used to compute the ratio and thus the acceleration. The problem
with this method is that in assembly, variables are only allowed to be 0-256.
Given that the PIC chip has no overhead, it was able to check the accelerometer
approximately 20k times per second. In order to get meaningful data, the first
step was to slow down the PIC chip. I set a delay of 5 microseconds between
reads, which was still extremely precise but avoided overflowing my counters.
Unfortunately, division in assembly is unpleasant. Without access to a C
compiler for my PIC chip, I found a workaround. I threw away the ‘off’ half of
the PWM and set a static threshold for the ‘on’ portion. So, if the PWM was
‘on’ for more than 70 counts, the acceleration was strong enough to count as a
step. The danger here is that this method doesn’t take into account the
variability of the accelerometer as a hardware component that is greatly
affected by temperature and humidity. The accelerometers aren’t guaranteed
to have a standard pulse-length, but they are guaranteed to have a standard
ratio. That is, the total length of a pulse will vary between accelerometers, but
the ratio between ‘on’ and ‘off’ will not. I ended up solving this problem
90
without using the ratio, which I’ll come to shortly.
I hooked up an LED to test the circuit, and found that it often flashed
several times for each bounce, often so fast I could only see it after recording
the circuit with a camera. The issue was that a the data received from the
accelerometer during walking looks like data from a seismograph or a lie
detector. Each step is not distinct but instead has multiple peaks and troughs.
Thus, the PWM was exceeding the threshold multiple times for a single step.
I tried several complicated algorithms to chart the peaks and troughs, look
intelligently for the ‘real’ peak and failed. I resorted to the static threshold
with which I had started. The basic problem is that after I detect the step, I
need to stop paying attention to the accelerometer for a short time. My first
solution was to set a delay of 250ms after a step was detected. If one was to
step every 250ms, one would be running at 300 steps-per-minute which is an
extremely high-rate of running. The idea behind this solution was that most
people won’t be olympic sprinters, so a ceiling of 300 steps-per-minute seems
reasonable (for reference, some of the fastest music averages around 200
beats-per-minute). Unfortunately, this delay introduced a non-linear output,
where 60 samples read and 120 samples read were not related as they should
be. If a step is detected every 60 samples, that is, 60 pulses from the
accelerometer go by before we detect another step, then theoretically 120
samples should give us twice the BPM of 60 samples. In fact, 140 BPM allowed
60 samples to be read between steps, and 40BPM allowed 120 samples to be
read, where I was expecting 140 BPM / 60 samples and 70 BPM / 120 samples.
This non-linearity was caused by the delay of 250ms I had used to filter out
extra steps. I had assumed that the same number of samples would be read in
250ms regardless of the acceleration applied to the accelerometer. In reality,
the delay was not reliably timed. Instead, I decided to use simpler logic: if a
91
step was detected, I check the number of samples that have been read since the
last step detected, and if that number of samples is not humanly possible, I
ignore the step. That means instead of delaying, I let the PIC go back to
reading from the accelerometer, and though it may detect a single step five or
six times, it won’t do anything about the extra steps since too few samples
have been read since the last step.
The PIC chip is set to send serial data to the Gumstix only when a step is
detected. Since it filters out bogus steps, the maximum rate of transmission is
300 steps-per-minute, or five times per second. On average, we can expect
transmissions near two or three times per second. This is easily digestible for
the Gumstix. Furthermore, the Gumstix sends a single number that is not
dependent on transmission speed or time. When the PIC chip detects a step, it
sends across how many samples it read since the last step. The Gumstix then
converts the samples read into steps-per-minute, and calculates and averages
the user pace. This solution frees up the Gumstix to spend the majority of its
time playing music and handling network communications.
Lastly, the PIC chip’s sensitivity was based on a hard-coded number that
was insensitive to the user and the unique pulse length of the accelerometer. I
wanted the PIC chip to calibrate itself to its own hardware, and then being
able to sensitize and de-sensitize itself to its user. It may not need mentioning,
but each of us walks with a different amount of force depending on our
weight, the shoes we are wearing, the speed we are moving, and a whole set of
other items. Without dynamic sensitivity, the step-detector wasn’t doing an
acceptable job. Because I cannot divide on the PIC chip, I cannot average.
Instead of trying to take a global approach as one might do with significant
computation power available in most modern computers, I took a local
approach. We know the step-detector is too sensitive when it registers steps at
92
the maximum allowed steps-per-minute, 300. We know this because if we set
the accelerometer to be as sensitive as possible, a hard step will produce
devastating acceleration that may result in more than twenty detected steps.
Thus, even despite the filtering done on the PIC chip, a normal hard step on a
sensitive accelerometer may cause two or three steps at 300 BPM, the
maximum allowed by the code. If we define a 300BPM step detection as ‘too
sensitive,’ we now have a way to determine how sensitive the system is.
As a point of clarification, the accelerometer is always +/-1.5G, but I have
set a threshold for the PWM that defines what counts as a step. If I move that
threshold too high, none of the pulses will increment the counter higher than
the threshold, and the system will be completely insensitive. If I set the
threshold too low, every pulse will be above the threshold counter and thus
count as a step, so that the system is completely sensitive. What I wanted was
a way to adjust that threshold dynamically, based on the data from the
accelerometer.
To accomplish this, I set up a history of the last five steps. Each time a step
occurs, the earliest step is removed and the latest is added to the top of the
stack. And, each time a step occurs, I check the past five steps and look for
steps near the maximum allowable step-speed. If there are any steps near the
maximum, we can conclude that a bounce occurred. At this point, the history
of steps is clear, and the threshold is incremented by one. If another bounce is
detected, the threshold is incremented by one again, and so on until no more
bounces occur. Conversely, we need the system to re-sensitize quickly should
the user start walking softly, walk on a softer surface, slow down, etc. To
accomplish this, re-sensitization occurs only in large jumps. If the system has
not seen a step from a user in more than one second (as counted by the
samples read from the accelerometer), it will decrement the threshold by five.
93
It will continue in this fashion until it detects a step, at which point it ratchet
up the threshold until no additional bounces are detected. The system can
move from completely desensitized to completely sensitized in two seconds,
and can desensitize to the perfect sensitivity in only two to three steps. Finally,
if a bounce has been detected, the PIC chip will not send that bounce or the
next two steps. This means that while the PIC chip is adjusting, no data is sent
to the Gumstix and the music playing is not modified. This eliminates the
transmission of bogus data and further minimizes the computational load of
the Gumstix.
This same system is used to calibrate to the accelerometer, compensating
for temperature and humidity. When PublicSoundtrack is first powered on,
the threshold starts at zero and is ratcheted up almost immediately to the
point at which no more ‘bounces’ occur. At a threshold of zero, every pulse
from the accelerometer will cause a step, such that every step is at the
maximum allowable rate. Once the threshold moves high enough, idle
accelerometer activity will no longer cause a step, the system will set the
current threshold as its baseline, and will never sensitize past that point. The
calibration process takes less than one second, and is often too fast to see.
PublicSoundtrack was meant to detect both the left and the right footsteps,
but after months of usage I feel that method works best when the device is
mounted in the center of the body. In order for it to be placed in the pocket on
either the left or the right side, the system needs to automatically desensitize
itself only a few extra points after finding a stability point. That would allow it
to detect steps only from the foot of the side it is on. Because songs are chosen
that are multiples of the user pace, (100 steps-per-minute results in the same
set of songs as 50 steps-per-minute), none of the other code would have to
change. I would like to implement this in the future.
94
5.5 C F
The basic flow of control contains two states: in the first state, there are no
other devices nearby, so PublicSoundtrack functions identically to
PersonalSoundtrack. A step is detected, pace is calculated, a song is chosen,
and with each subsequent step the song speed is compared to the pace. If the
pace has changed radically for long enough, a new song is chosen. If the pace
is only slightly different, the song is time-stretched to match the pace, then
slowly moves back toward the original pace, continually drawing the user
back to the original song speed.
When other devices are detected, PublicSoundtrack pauses the music,
resets everything it knows about the user’s pace, and waits. As the user starts
to tap or move, each step is distributed as a single number to all other nearby
devices. The data sent is a single number, e.g. 100, where 100 means 100
steps-per-minute. This data is not time-dependent, such that if the 100 is sent
too slowly or more quickly than anticipated, the system is not impacted.
When a device receives step information, that pace is compared to the local
pace. If they are close enough the local system begins to countdown to playing
a song. If the users can step at a similar pace for approximately 4-6 steps music
will play. Each incoming step is compared to the local pace, and if any one of
them is not synchronized with the local pace, the countdown is reset. That
means that all nearby players must tap the same tempo, and that if one of
them fails or changes tempo everyone’s counter is reset. Since everyone has a
local counter, and all steps are received within 2ms or less, everyone’s local
counter stays more or less synchronized. They all are privy to the same data,
and thus will act on it in the same way in a reasonably short amount of time.
Once a tempo has been established and music is playing, the system will
attempt to time-shift each person’s music to match the group. Time-shifting is
95
done without any digital knowledge of the music playing other than its BPM.
In a two person example, if they have established a beat of 100BPM, both
songs that play are at 100BPM, but their beats will most likely not occur at the
same point in time. Without time-shifting, the participants would experience
“beating,” where they are always off from each other by the same amount. To
time-shift the music, the Controller class timestamps when steps are detected
and received from other devices. That is, if I step before you, my timestamp
will be earlier than your step, which has a later timestamp. Using this
information, we can adjust the music. We have to assume that most of the time
people can tap a beat, but the system is quite graceful when handling those
that can’t, don’t, or forget to for several moments. If your step landed before
mine, my device will speed me up very slightly since I stepped after you.
Likewise, your device will detect that you stepped before me and will slow
you down slightly. Each device time-stretches the song for only a quarter of a
second for 1/2 a BPM. This change is nearly imperceptible by the user, and is
similar to a DJ momentarily spinning a record faster or slowing it down.
The result is that users tap their foot to the song in their headphones, and
within a few seconds they will be aligned with the other person’s steps. Once
users feel synched up, they may dance or move to their own music for a bit
without worrying much about synchronization. Since people aren’t robots,
their steps will often vary a lot and they will likely end up desynchronized
over time. At this point, users would need to focus for a few seconds and let
the system re-synchronize their steps. The normal, expected pattern of usage
would be this back and forth between synchronization and
de-synchronization.
96
5.6 F N
Future versions may not require the Gumstix boards (Figure 3), which would
allow for more compact size and fewer hardware restrictions. I have drawn up
an abstract hardware component breakdown that would use several
microcontrollers in place of the Gumstix operating system. This kind of deep
specialization was too intensive to be completed during schooling, though I
look forward to attempting it in the future.
Figure 5.3: Abstract Hardware Component Breakdown
The PublicSoundtrack platform is one I plan to continue work on, both in a
continuation of the PublicSoundtrack philosophy as well as in other projects.
As a technical canvas, this platform is a robust, wireless, expandable,
rechargeable and re-creatable mobile device that runs Linux, allowing it to be
customized and adapted for many unique purposes.
97
C 6
C
The rise of ubiquitous technology provides opportunities for a revived
discussion of Efficiency. Specifically, by embedding computational technologies
into the environment, whose primary value system is one of Efficiency, we also
embed the value systems those technologies contain [79]. Despite the wealth
of discussions of Efficiency, the term is rarely explored directly by the
individual, where the individual reflects on how they define, evaluate, and act
on the ideology of efficiency in his/her own way. We may assume we know
when we are being efficient, and that we know when we are not; however,
these assumptions fail to account for the pervasiveness of Efficiency, where a
walk on the beach can be used productively or a vacation is expected to
include an optimum amount of sight-seeing. These assumptions about the
nature of efficiency as an ideology need to be deconstructed, yet we have few
ways to conceptualize Efficiency without resorting to global economics, social
theory, or user studies. Thus, it can be difficult to grasp how Efficiency impacts
our daily tasks, thought processes, and even moments of pleasure.
How is Efficiency felt by each of us, how do we perceive it, and where do we
find it? How does each of us conceptualize Efficiency in our lives? Artistic
practice allows us to ground these questions about Efficiency in individual
experience. I have proposed a genre of artifact design that allows the
individual to explore the role of Efficiency within the context of his/her daily
life, moving us away from the abstract, objective views of Efficiency.
This thesis contains several examples of how artists and technologists
might designs artifacts that question Efficiency, including the careful
98
application of interruption. These techniques, especially when instantiated in
computational artifacts, can open up the elusive and invisible ideology to
further critique, analysis, and public discussion.
I have discussed foundational design genres in this area, such as those by
Hallnas and Redstrom and Sengers. While designing for purposes other than
those that serve Efficiency is important, we should not position reflection as a
counter to Efficiency. Instead, we can think of Efficiency as a structuring agent
that limits our ability to improve and implement alternative design genres.
The design genre I suggest can help us loosen the grip the ‘technology as tool’
hegemony and clear a path for design genres like Critical, Ludic, and
Reflective Design, as well as Slow Technology.
I have attempted to put into practice these theories about Efficiency with
projects like PersonalSoundtrack and PublicSoundtrack. The first,
PersonalSoundtrack, is intensely isolating, focusing the user’s attention
inward to the body and on physical connections to music. The second,
PublicSoundtrack, aggressively forces the user’s attention outward onto
strangers. PersonalSoundtrack explored Efficiency through the passive refusal
of optimization and the support of meandering and wandering, though it was
not initially intended to support the design genre of this thesis. It served as a
catalyst for the second project, PublicSoundtrack, which aggressively brought
to the surface a reflection on Efficiency by interrupting its users and creating a
tension between a trivial, brief interaction and the user’s previous task.
PublicSoundtrack is an attempt to explore Efficiency and not a definitive
example. It is my hope that projects like the Drift Table, PersonalSoundtrack
and PublicSoundtrack can act as inspiration for other artists interested in
exploring Efficiency through computational artifacts.
Future work on Publicsoundtrack will involve an exploration of how it
99
explores the private and public spaces. Forthcoming versions seek to sensitize
us to the way we live in private worlds while navigating public spaces,
inviting reflection and discussion about the ways we juxtapose and navigate
these spaces simultaneously everyday.
We have seen how Efficiency is often discussed in texts and how that
technique, while critical to the knowledge about Efficiency, cannot provide the
grounding necessary to explore the ideology in day-to-day tasks.
Computational artifacts that are used in real-time can draw to the surface how
and when the ideology of efficiency is influencing us, not at a later date but
within the moment. Technological devices like the Drift Table,
PersonalSoundtrack and PublicSoundtrack provide the means to begin an
interrogation of Efficiency given their ubiquitous and personal nature. Since
computational devices are the champions of efficient design that propagate the
value system of Efficiency, re-appropriating them may create significant
cognitive dissonance in their users. That dissonance is our opportunity to
question the role of Efficiency. By designing to reveal Efficiency, we
contextualize the discussion of the ideology within our daily lives, giving us
the tools to critique it from an individual perspective.
100
B
[1] T. W. Adorno. The Culture Industry: Selected Essays of Mass Culture.London: Routledge, 1991.
[2] P. E. Agre. Bridging the Great Divide: Social Science, Technical Systems, andCooperative Work, chapter Toward a Critical Technical Practice: Towards aTechnical Critical Practice: Lessons Learned in Trying to Reform AI.Lawrence Erlbaum Associates Inc, 1997.
[3] P. E. Agre. Computation and Human Experience. Cambridge UniversityPress, 1997.
[4] Arduino. http://www.arduino.cc/.
[5] M. Banta. Taylored Lives: Narrative Productions in the Age of Taylor, Veblen,and Ford. University of Chicago Press, 1993.
[6] A. Bassoli, C. Cullinan, J. Moore, and S. Agamanolis. Tuna: A mobilemusic experience to foster local interactions. In Proceedings UBICOMP,2003.
[7] J. Baudrillard. Consumer. In M. Poster, editor, Jean Baudrillard: SelectedWritings. Ed. And Intro. Mark Poster. Stanford, California: StanfordUniversity Press, 1988.
[8] G. Bell and J. Kaye. Designing technology for domestic spaces: A kitchenmanifesto. Gastronomica, 2(2):46–62, Spring 2002.
[9] A. Boucher and W. Gaver. Developing the drift table. SPECIAL ISSUE:The art of prototyping, 13:24–27, 2006.
[10] A. Chatham. pyao. http://directory.fsf.org/pyao.html.
[11] W. H. K. Chun. On software, or the persistence of visual knowledge. GreyRoom, 18:26–51, 2004.
[12] R. E. Day. The “conduit metaphor ” and the nature and politics ofinformation studies. Journal of the American Society for Information Science,51:805–811, 2000.
[13] A. Devices. Max1555.http://www.maxim-ic.com/quick view2.cfm/qv pk/4002.
[14] S. Dornbush, J. English, T. Oates, Z. Segall, and A. Joshi. Xpod: A humanactivity aware learning mobile music player. In Proceedings of theWorkshop on Ambient Intelligence, 20th International Joint Conference onArtificial Intelligence (IJCAI-2007), January 2007.
101
[15] P. Dourish. Where the Action Is: The Foundations of Embodied Interaction.MIT Press, 2001.
[16] H. Dreyfus. What Computers Still Can’t Do: A Critique of Artificial Reason.MIT Press, 1992.
[17] A. Dunne. Hertzian Tales: Electronic Products, Aesthetic Experience andCritical Design. Art Books Intl., 2000.
[18] A. Dunne and F. Raby. Design Noir: The Secret Life of Electronic Objects.Birkhauswer, 2001.
[19] P. Edwards. The Closed World: Computers and Politics of Discourse in ColdWar America. MIT Press, 1996.
[20] P. Ekman and R. Davidson. The Nature of Emotion: Fundamental Questions.Oxford University Press, 1994.
[21] S. Electronics. http://www.sparkfun.com/commerce/categories.php.
[22] S. Electronics. Accelerometer breakout board. http://www.sparkfun.com/commerce/product info.php?products id=400.
[23] S. Electronics. Lipoly charger - single cell. http://www.sparkfun.com/commerce/product info.php?products id=726.
[24] G. Elliott and B. Tomlinson. Personalsoundtrack: Context-aware playliststhat adapt to user pace. In CHI ’06 extended abstracts on Human factors incomputing systems, April 22-27 2006.
[25] J. Franke, J. Daniels, and D. McFarlane. Recovering context afterinterruption. In Proceedings 24th Annual Meeting of the Cognitive Science,2002.
[26] D. Garlan, D. Siewiorek, A. Smailagic, and P. Steenkiste. Project aura:toward distraction-free pervasive computing. In IEEE PervasiveComputing, pages 22–31, 2002.
[27] W. Gaver. Designing for homo ludens. I3 Magazine, 12, June 2002.
[28] W. W. Gaver, J. Bowers, A. Boucher, H. Gellersen, S. Pennington,A. Schmidt, A. Steed, N. Villar, and B. Walker. The drift table: designingfor ludic engagement the drift table: designing for ludic engagement thedrift table: designing for ludic engagement. CHI ’04 extended abstracts onHuman factors in computing systems, 2004.
[29] L. Gaye and L. Holmquist. In duet with everyday urban settings: a userstudy of sonic city. In Proceedings of New Interfaces for Musical Expression,2004.
102
[30] P. Glennie and N. Thrift. Reworking e.p. thompson’s ’time,work-discipline, and industrial capitalism’. Time and Society, 5:275–299,1996.
[31] S. J. Gould. The Panda’s Thumb of Technology. Natural History, 1986.
[32] Gumstix. Audiostix 2 audio in/out. http://gumstix.com/store/catalog/product info.php?cPath=31&products id=158.
[33] Gumstix. Basix motherboard 400mhz with bluetooth. http://gumstix.com/store/catalog/product info.php?cPath=27&products id=155.
[34] Gumstix. Circuit boards. http://gumstix.com, 2006.
[35] L. Hallnas and J. Redstrom. Slow technology – designing for reflection.Personal Ubiquitous Comput., 5(3):201–212, 2001.
[36] G. Haring. pysqlite2. http://www.initd.org/tracker/pysqlite.
[37] A. Huang. pybluez. http://org.csail.mit.edu/pybluez/.
[38] E. Hutchins. Cognition in the Wild. MIT Press, 1996.
[39] Intel. Placelab, 2007.
[40] JFFS2. Journaling flash filesystem. http://sourceware.org/jffs2/.
[41] N. Jollands and M. Peterson. Wasting ‘efficiency’: Sisyphus and thepolicy analyst. Paper presented at Think Tank on Ecological Economics,November 2003.
[42] W. Kelleher. The Troubles in Ballybogoin: Memory and Identity in NorthernIreland, chapter Mapping Moves, pages 23–60. University of MichiganPress, 2003.
[43] S. Kristofferson and F. Ljunberg. ”making place” to make it work:Empirical exploration of hci for mobile cscw. In Proceedings of theinternational ACM SIGGROUP conference on Supporting group work, pages276–285, November 1999.
[44] F. Kuo, M. Chiang, M. Shan, and S. Lee. Emotion-based musicrecommendation by association discovery from film music. InInternational Multimedia Conference, pages 507–510, 2005.
[45] M. Kusahara. Device art: A new form of media art from a japaneseperspective. Intelligent agent, 6(2), 2006.
[46] G. Lakoff and M. Johnson. Philosophy in the Flesh: The Embodied Mind andIts Challenge to Wester Thought. HarperCollins Publishers, 1999.
103
[47] D. S. Landes. Revolution in Time: Clocks and the Making of the ModernWorld. Belknap Press, 2000.
[48] J. Lave. Cognition in Practice. Cambridge University Press, 1988.
[49] L. Leahu, C. Pederson, J. Thom-Santelli, P. Dmitriev, and P. Sengers.Uptake of situationism considered hamrful. In To Appear in CHI ’07Proceedings, 2007.
[50] C. Liechti. pyserial. http://pyserial.sourceforge.net/.
[51] F. Linux. http://familiar.handhelds.org/.
[52] M. S. Mahoney. Finding a history for software engineering. IEEE Annalsof the History of Computing, 27:8–19, 2004.
[53] R. Maze and M. Jacobs. Sonic city: Prototyping a wearable experience. InProceedings of the 7th IEEE International Symposium on Wearable Computers,volume October 21-23, page 160, 2003.
[54] M. McCullough. Digital Ground: Architecture, Pervasive Computing, andEnvironmental Knowing, pages 67–94. MIT Press, 2004.
[55] D. C. McFarlane. Interruption of people in human-computer interaction.Master’s thesis, The University of Oregon, 1990.
[56] Microchip. Pic 16f648a. http://www.microchip.com/stellent/idcplg?IdcService=SS GET PAGE&nodeId=1335&dDocName=en010212.
[57] T. Mitchell. Machine Learning. McGraw-Hill, 1997.
[58] D. Norman. Emotion and design: attractive things work better. ACM Press,2002.
[59] N. Oliver and F. Flores-Mangas. Mptrain: A mobile music and physiologybased personal trainer. In MobileHCI, Helsinki, Finland, September 2006.
[60] M. O’Malley. Keeping Watch: A History of American Time. New York:Viking/Penguin, 1990.
[61] A. Oulasvirta and P. Saariluoma. Long-term working memory andinterrupting messages in human-computer interaction, volume 23, pages53–64. Taylor and Francis, 2004.
[62] A. Oulasvirta, R. Tamminen, S., and J. V., Kuorelahti. Interaction in4-second bursts: the fragmented nature of attentional resources in mobilehci. In Proceedings of the SIGCHI conference on Human factors in computingsystems, April 2005.
104
[63] S. Penny. Reader for The Living and the Life-Like: Emergence, Complexity,Artificial Life and Generative Art. ACE Interdisciplinary Theory Seminar,Winter 2006.
[64] S. Penny. Art and instrumentality: Productivity, criticality and pleasure.Class Reader for Information, Representation, Cognition, Knowledge, 2007.
[65] A. Pickering. Beyond design: Cybernetics, biological computers andhylozoism. In International Conference on the Philosophy of Technology, 13-15Oct 2005.
[66] R. Plutchik. The Emotions. University Press of America, 1991.
[67] Pzizz. Pzizz - treat sleep problems and increase energy.http://www.pzizz.com, 2007.
[68] S. Reddy and J. Mascia. Lifetrak: music in tune with your lif. InProceedings of the 1st ACM international workshop on Human-centeredmultimedia, 2006.
[69] B. Russell. In praise of idleness. Online, 1932.
[70] P. Sengers, K. Boehner, D. Shay, and J. Kaye. Reflective design.Proceedings of the 4th decennial conference on Critical computer: between senseand sensibility, August 20-24 2005.
[71] C. Shannon. The Mathematical Theory of Communication. University ofIllinois Press, 1949.
[72] S. L. Star. The ethnography of infrastructure. American Behavioral Scientist,43:377–391, 1999.
[73] J. Stein. The Cult of Efficiency. House of Anansi Press Ltd, Toronto, 2001.
[74] S. Strachen, P. Eslambolchilar, R. Murray-Smith, S. Hughes, andS. O’Modhrain. Gpstunes: controlling navigation via audio feedback. InACM International Conference Proceedings Series, volume 111, pages275–278, 2005.
[75] L. Suchman. Plans and Situated Action: The Problem of Human-MachineCommunication. Cambridge University Press, 1987.
[76] S. Tamminen, A. Oulasvirta, K. Toiskallio, and A. Kankainen.Understanding mobile contexts. In Proceedings Mobile HCI, pages 17–31,2003.
[77] M. Taussig. Devil and Commodity Fetishism in South America. University ofN. Carolina Press, 1983.
105
[78] E. Thompson. Time, work-discipline, and industrial capitalism. Past andPresent, 38(56-97), 1967.
[79] N. Thrift and S. French. The automatic production of space. Transactionsof the Institute of British Geographers, 27:309–335, 2002.
[80] F. J. Varela, E. Thompson, and E. Rosch. The Embodied Mind: CognitiveScience and Human Experience. MIT Press, 1991.
[81] G. Walter. An imitation of life. Scientific American, 182(5), 1950.
[82] M. Weiser and J. S. Brown. Designing calm technology. PowerGrid Journal,January 1996.
[83] J. Wilkinson. Pymad. http://spacepants.org/src/pymad/.
106
A A
PS:C-
Previously Published: G. Elliott and B. Tomlinson. Personalsoundtrack:
Context-aware playlists that adapt to user pace. In CHI 06 extended abstracts on
Human factors in computing systems, April 22-27 2006.
A.1 A
This paper describes a mobile music player, PersonalSoundtrack, that makes
real-time choices of music based on user pace. Standard playlists are
non-interactive streams of previously chosen music, insensitive to user context
and requiring explicit user input to find suitable songs. The context-aware
mobile music player described here works with its owners library to select
music in real-time based on a taxonomy of attributes and contextual
information derived from an accelerometer connected wirelessly to a laptop
carried under the arm. We are in the process of evaluating this prototype with
25 users who will compare the systems context-sensitive playlist to random
shuffle. On the basis of user feedback and analysis, a hand-held device will be
implemented for testing in less constrained mobile scenarios.
PersonalSoundtrack allows users to experience their music with both mind
and body, providing a unique embodied experience of their personal music
library. In mobile environments where attention is a limited resource, users
can spend less time deciding what music to enjoy and more time enjoying it.
107
A.2 K
Interaction design, interactive music, mobility, wearable computing,
emotional state, inherent feedback ACM Classification Keywords H.5.2
Information interfaces and presentation (e.g., HCI): User Interfaces
A.3 I
Portable music players (e.g. the iPod) allow users to listen to music in multiple
mobile scenarios. They listen on the way to class, on the subway, during bike
rides, jogs, or workouts, etc. Many users attempt to plan for mobile activity by
pre-defining playlists that correspond to specific activities or moods. From
Suchman [75], plans alone do not dictate actions, but instead provide
scaffolding that individuals can use to organize action. Thus, users attempt to
follow previous plans while continuously adapting their actions to the
environment [76]. Pre-defined playlists cannot adapt to such ever-changing
situations without explicit user input. Manually selecting music requires both
user attention and memory when mobile navigation inherently demands the
majority of user resources [76]. Context-aware playlists that automatically
choose music in real-time and in response to user movement, can better match
the unpredictability of mobile activity. By monitoring user pace, the mobile
music player becomes a personal DJ that automatically slows, speeds, and
changes songs to match the user’s movement.
The system described here detects user pace and chooses songs by
comparing SPM (steps-per-minute) to BPM (beats per minute). To start the
music, a user holds the system, places earphones on, and begins walking.
Music is seamlessly matched to the users speed, putting the user in tune with
the music. By continuously adapting to user pace, the device remains in tune
108
with its user without explicit control; however, PersonalSoundtrack uses a
simple mechanism to learn inappropriate song choices should users decide to
explicitly skip songs.
Figure A.1: PersonalSoundtrack system diagram showing inputs, actions, andflow of control
A.4 S
PersonalSoundtrack can adjust to many scenarios, providing unique personal
experiences using familiar music. Feedback received from beta testers inspired
the following hypothetical scenarios:
109
A.4.1 M J
A woman grabs her headphones and heads out for an early morning jog. She
begins a warm-up walk and with each step music fades in to match her pace.
Each beat of the song occurs when she takes a step, synchronizing her with her
music. As she speeds to a jog, the music matches her pace by cross-fading into
a faster song. Suddenly she feels a surge of energy and decides to sprint to end
her jog. The device chooses an upbeat song that emotionally fuels her sprint.
She slows to a casual walk to cool down, and the music slows and adapts. Her
music has reflected her morning workout, providing a pleasing emotional and
physical experience beyond a simple jog.
A.4.2 IW
Bored on his 10-minute walk to the subway, a man has neither time nor desire
to stop walking and create a playlist; however, randomly selected music is less
than satisfying. He is not sure what he wants to hear, so he chooses the
context-aware playlist and a song begins to play. While the song is not his
favorite, he doesnt mind hearing it. As he walks, he notices he is walking to
the beat of the song, or rather, the beat is reflecting his steps. He begins to
enjoy the nuances as each step coincides with a bass pluck or a kick drum hit.
His attention drifts back to the world around him, allowing the music to adapt
to him as he walks.
A.4.3 L
Leaving in a hurry, a boy walks quickly toward the bus stop. Music fades in at
a heightened tempo, increasing his anxiety and speed. As he nears the bus
stop, he notices the bus has already arrived. Fearing he might miss it, he runs
110
to catch it. His quick steps cause the music to change to a maddeningly fast
song as he races to stop the bus. Reaching the doors just in time, he grabs a
seat and relaxes as his music cross-fades into a slower beat.
A.5 RW
Many researchers have created innovative, context-aware music devices. Maz
and Jacobs [53] developed SonicCity: a wearable jacket that sensed light,
noise, movement, and proximity to algorithmically generate music in response
to the environment. While the users movement was used to introduce
randomness or set initial tempo, music was primarily generated in response to
environmental input [53, 29], rather than selected from a users music library
based on user context as in this project. Strachan et al. [74] designed gpsTunes,
an mp3 player that uses mobile GPS to guide users to locations by panning
and changing music volume. Bassoli et al. [6] created the tunA project which
allows users to listen to what other nearby users are listening to using
hand-held devices.
Drawing inspiration from SonicCity, we applied a modified notion of
embodied music interaction to pop culture. By playing music from the users
library of songs, PersonalSoundtrack integrates with the existing and
successful iPod platform. This greatly extends the reach and accessibility of
the project, as it can easily be incorporated into everyday use. SonicCity
algorithmically generates music, acting as a digital instrument that relies on
the creativity of its user, making it less suited for general use.
Furthermore, SonicCity and tunA rely on external factors for context, so
that music primarily reflects the environment. In contrast,
PersonalSoundtrack prioritizes personal movement in determining context.
111
This is useful in two ways: public and private use is appropriate, and the
musical experience is intimately tied to the user. The system provides a novel
personal experience of ones music, where the user is always in tune with their
music without conscious effort. Neither attention nor visual interface is
required, making it ideal for mobile contexts. Control is implicit and highly
personal: music is chosen to match whatever pace the user finds comfortable.
A.6 T C-AM
Mobile devices often suffer from the limitations of user attention and poor
integration into the environment [43, 62]. At home, one can take time to
carefully select songs to play; in mobile contexts, attention is a limited resource
that results in short bursts of attention [62, 58]. Given this mobile constraint,
automatic music selection is possible by appealing to the affective system, that
can quickly assess valence without conscious thought [44]. The affective
system responds well to tempo and rhythm as they strongly influence human
emotion [26]. Based on the dimensional model of emotion [66],
PersonalSoundtrack follows a three-dimensional version (Arousal, Valence,
Stance) [20] where tempo (arousal) directly affects enjoyment and receptivity
(valence and stance). The device attempts to synchronize user arousal with
music tempo to promote positive valence and open stance. When users
explicitly skip songs, the device learns and modifies future selections. This
simple learning mechanism is described below.
112
A.7 H
The prototype has been implemented using a Mac laptop and a Bluetooth
accelerometer for rapid testing and development. Mac laptops are an ideal
environment for testing as they support wireless Ethernet, Bluetooth, and
multiple programming languages. Since many people are accustomed to
walking while carrying a laptop, it does not impose an undue burden on beta
testers that might skew reactions. We have plans to develop a hand-held
version based on the Gumstix [34] platform. We use a 3-axis Freescale
MMA7260Q accelerometer, reporting at a rate of 150hz with a sensitivity of
1.5+-g. A simple pedometer was tested, but was significantly less accurate
than the accelerometer.
A wireless accelerometer allowed the designers to test multiple placements
for accurate user pace detection. Popular iPod holding areas were examined,
such as hip, arm, and pocket, as well as novel holding areas such as wrist,
shoulder, and ankle. Hip placement of the accelerometer was the most reliable
and accurate. Wrist and ankle movements heavily depend on gait, while arm
and pocket locations decrease sensitivity. User steps are detected by
measuring subtle impact along the vertical axis of the body, working well with
high-impact (running) and low-impact (bicycling) movement. Software
controls are used to normalize noisy data.
A.8 S
Initial software was written in Java, using the Quicktime for Java API for
sound, and the PlaceLab API [39] for wireless access point detection. While
not discussed explicitly in this paper, location detection is implemented and its
potential use is being explored. An HSQL database imports and stores the
113
users iTunes library, WiFi access points, song probabilities, etc. In processing
output from the accelerometer, several sensitivity controls were needed. First,
two thresholds were required for changing a songs rate and for changing
songs. Second, a moving average of SPM was needed to find the perceived
user pace, versus the actual user pace. Precisely timed and typical user steps
lead to SPM variations near 5 BPM and 15BPM respectively, due to physical
gait and environmental obstacles. As a result, the perceived SPM is
determined by a system that is difficult to influence at first, but becomes easier
to change if the users pace change is consistent. Random fluctuations in SPM
have little influence on perceived SPM, while consistent changes (e.g.
changing from walking to running) strongly influence perceived SPM. Third,
after beta testing, it was determined that many users felt they were walking
in-time to the music, but technically were moving too fast or too slow. The
system should not punish users who, within reason, believe they are walking
to the beat but are not, nor should it assume the user is wrong. The device
determines if the user is unable to walk to the songs BPM, or if the user has
purposefully changed pace. This was solved by averaging the users previous
fifteen steps. If the average pace is stable and within the range required to
change songs, the music will not change because the system assumes the user
believes they are walking to the beat. Surprisingly, it is desirable that the
system is highly insensitive to most pace changes, while remaining sensitive to
deliberate pace changes.
This software is appropriate for walking, jogging, running, biking, jump
rope, etc. Certain activities such as biking may require a BPM that is slower
than actual pedal revolution (SPM), as pedal revolutions might be
disproportionately faster than walking or running. This issue will be
addressed in future work.
114
The devices used a simple machine learning approach to affect the
probability of songs being played based on the following equation
p = 100 − 25s + t (A.1)
where p is the likelihood of a song being played, s is the number of times
the song has been skipped within the first fifteen seconds of play, and t is the
total number of song choices made by the device that session. We plan to
explore more complex learning mechanisms [57] that use variables such as
location, terrain, time of day, weather, etc.
A simple graphical interface provides two controls: one to adjust the
devices sensitivity, and one to skip songs. In the final prototype, the Next Song
button will be implemented in hardware, and a hardware version of the
sensitivity control may be included depending on the user study.
A.9 U E P
To evaluate this project, the experimental hypothesis is that the selection
mechanism will produce results that are more appealing to people than
random shuffle. In order to test this hypothesis, we added an option to
PersonalSoundtrack that allows it to play randomly shuffled music instead of
pace-matched songs. Each user carries a laptop under his/her arm, while
listening to headphones plugged in to it. The accelerometer is worn on the
waist. This same interface can play both random shuffle and pace-matched
songs, so subjects cannot differentiate song selection based on interface.
We plan to observe the behavior of 25 college students in their everyday
environments. Subjects bring in their music library for use in the experiment,
and BPM is hand-calculated for a subset of songs. Subjects are asked to walk
115
to three specific points on the UCI campus while listening to music, each walk
lasting about 10 minutes. The experimenter rides ahead and waits for the
subject at each point. While each session is different, all are located in the same
general area as the others. The device randomly chooses to use random or
pace-matched playlist for the first session, and the non-chosen playlist for the
second session. For the final session, subjects are asked to choose to hear
music in the style of session one or session two. In each walk, users can skip
songs as often as they like.
After the final walk, subjects are interviewed and asked why they chose one
style over the other, how they felt about mid-song transitions, etc. During the
session, the number of songs skipped is recorded for comparison. This data
will be used to help understand the devices behavior given the users explicit
actions.
A.10 FW
Upon completion of the user study, the researchers will finish a second
prototype: a hand-held Gumstix device. Modifications will include interaction
revisions, such as how the device interprets pace, how it learns from implicit
and explicit interaction, and if it should change music only near the end of a
song.
With the hand-held prototype, the investigators will begin a second user
study that will compare the PersonalSoundtrack to random shuffle, radio, and
user-defined playlists for several activities such as walking, running, and
biking. This iterative design process will provide a more complete
understanding of how useful and effective the device is in complex scenarios.
116
A.11 C
This project seeks to create a context-aware device that can meaningfully
adapt to users without requiring their adaption to the device. It uses simple
contextual cues to provide a more personal experience, adapting to the user on
an intimate level. Because PersonalSoundtrack requires a trivial amount of
learning from its users, it is immediately viable for nearly all age groups and
user types.
Our digital and physical tools can significantly shape the way we work and
play. It is important to explore devices that adapt to our unique work practices
in order to improve HCI. PersonalSoundtrack does not require users to change
how they walk, but attempts to change itself based on the users unique
walking pattern. We hope this device provides a simple example of an
adaptive machine that successfully functions in the real world with minimal
user learning.
A.12 A
We thank Bonnie Nardi, David Kirsh and the ACE (Arts Computation
Engineering) Program at UCI.
117