Physical Computing: A Review of Past, Present and Future Application Areas

Yasmine El-Glaly, Daniel Shanahan, Cheryl Stinson Center for Human Computer Interaction

Virginia Tech {yasmineg, dshan88, cstinson}@vt.edu

INTRODUCTION Physical computing is an abstract research area rapidly gaining momentum in the human-computer interaction (HCI) realm. However when confronted with the question “What is physical computing?” few researchers can come up with a quick answer. Many of us have a conceptual notion of what constitutes physical computing, and if presented with a specific device or application can quickly classify whether it fits into our framework or not. It is also common to hear researchers attempting to describe physical computing in terms of other research areas such as: pervasive computing, invisible computing, ubiquitous computing, tangible interaction, and context-based computing. Yet at this point in time there is still no established guidelines defining physical computing. In order to begin the process of developing a theoretical framework for physical computing, we the authors are going to present our own view of physical computing. We are not arguing the validity of our vision, rather offering our vision to the HCI community to open the floor for further discussion.

We began our process by considering the vision presented by Weiser in his 1991 reflection titled ‘A Computer for the 21st Century’ [1].

‘There is more information available at our fingertips during a walk in the woods than in any computer system, yet people find a walk among trees relaxing and computer frustrating. Machines that fit the human environment instead of forcing humans to enter theirs will make using a computer as refreshing as taking a walk in the woods.’

Technology has forever changed how society interacts with our world. However we do not need to limit ourselves to the desktop workstation and cubicles. Weiser saw the potential for technology to weave itself into society seamlessly, and envisioned interactions that benefited human kind. We also drew motivation from the Stankovich’s vision of physical computing systems [2].

‘[Physical computing involves] seamlessly integrating computing with the physical world via sensors and actuators’

Using these perspectives, we define a physical computing as research involving programmable devices that unobtrusively enable mobility, communication, and control.

In this article, we present our view of physical computing by classifying existing applications into four categories: physical computing for quality of life, physical computing for health care, physical computing for transportation, and physical computing for public safety and security.

PHYSICAL COMPUTING FOR IMPROVED QUALITY OF LIFE Over the past few decades we have seen an incredible shift in society as physical computing has pervaded our daily existence. We are reaching a point where technology is no longer tethered to a single context, and digital information can be accessed from almost anywhere. This increased mobility has fundamentally changed the way we exist, and has incredible potential to positively improve our quality of life. In this section, we will review various applications which demonstrate how physical computing enhances quality of life. The three focuses of this section are physical computing in daily interactions, entertainment, and smart technologies.

Daily Interactions Over the past few decades we have seen an incredible shift in the typical daily interactions of industrialized societies. If you think back twenty years it was an anomaly if someone owned a cell phone, however today we shake our head in wonder when we hear of someone who does not. Mobile technologies are infiltrating every facet of our lives, and are providing a constant influx of digital information and social media. New paradigms of workplace and social interaction are constantly appearing, and are largely fueled by the continually decreasing cost and increasing availability of mobile devices. Laptops, cell phones, global positioning system (GPS) devices, personal digital assistants (PDAs), tablets, and wireless networking devices are just a few examples of physical computing technologies that are readily available to society today.

Mobile technologies have existed for several decades now, however it is only in the last decade that they have flooded the consumer market. Therefore it is not surprising that we have seen a huge shift in the research focus during this time. When researchers first started integrating mobile technologies over networks, most of the agendas were academic in nature. An early study from 2002 exemplifies this tendency [3]. In this study,

researchers were looking at how laptops could share their computational power over wireless networks to help solve large-scale computational problems. At this point computational power was still very expensive, and it was a natural tendency to look for ways to ease the cost of large computational problems. The heart of the research focused on dynamically dividing the computations, as well as handling the network communication. The researchers demonstrated the potential benefits of such shared-computation, however realized that the social element was a potential barrier to its success as their structure depended on users volunteering to share computational resources.

Looking just four years later we can see that the research mentality begins to move to a more user-centered viewpoint. Rather than looking at how physical computing can benefit from users, researchers begin to investigate how users can benefit from physical computing. This is highlighted in [4], where we see the smart phone (a cellular phone combining multiple sensors) being presented as the ideal ubiquitous device. No longer are mobile devices simply a tool to leverage for more complex problems, but rather represent an entirely new paradigm of use and interaction.

As we move towards present day, research is still user-centered; however we can see a strong surge towards context-aware capabilities. We are seeing an increased demand for physical computing devices to accurately sense where a user is in the world, and what they are doing in order for the device to respond appropriately. GPS devices and cellular networks are two common systems used to discern this location information; however both suffer reduced accuracy in different conditions. A recent study is attempting to use a hybrid approach to enable uninterrupted location-based service [5]. These researchers are investigating both centroid and nearest neighbor position-estimation algorithms. While centroid algorithms are easier to employ, nearest neighbor performs better in most cases and is generally the better option if users will traveling between both urban and suburban areas.

It is obvious from our research that physical computing is increasingly a factor in our daily interactions. It is also apparently that wireless capabilities are paramount to uninterrupted and seamless service. Society wants to access digital information anytime and anywhere while not being hindered by the technology. Looking to the future we see an exciting time for physical computing, since the shift of wireless solutions from being fixed in hardware to fixed in software opens up a world of possibility [6]. This shift to software allows wireless radios to be easily reconfigured, increasing the pace of innovation since we no longer need to wait for hardware production to implement a new protocol. This also means lower costs for custom wireless solutions, opening the

door for new and creative physical computing technologies.

Entertainment Compared to other application areas of physical computing, the field of entertainment is perhaps the most exciting. Whereas other areas are still dominated mostly by computer scientists and engineers, the creativity fueling physical computing in entertainment is fostering a fusion of a wide variety of disciplines.

The physical computing class at the University of Washington is a good example of this phenomenon [7]. The class is designed as an architectural design studio, and is offered to students in a wide variety of majors including: art, music, architecture, computer science, and engineering. The class features a 6-week long project completed in groups of 4-5 students, where students are encouraged to combine their shared knowledge to create a digital installment. An example of the work generated is the project titled ‘Alphabet Paint Space’. The project featured a motion-tracked area, and motions of the tracked users were used as ‘brushes’ to create a painting (displayed on a projection screen). Image processing was used to render abstract representations of the movements. On the display, the images faded over time and were constantly being overlaid with new ones – resulting in an ever-evolving, dynamic work of art.

Using technology in the generation of music is something that has been done for decades. In fact several music genres can be attributed to this process of generating music electronically. Therefore it is no surprise that physical computing is influencing this creative domain as well. A good example is in the recent use of real-time user-generated data from both performers and audience members to generate music [8]. These researchers are using kinematic sensors to detect motions, and physiological sensors to detect both somatic and autonomic activity in their BioMuse system. The data is used to quantify the emotional state of both the performer and audience, and to determine the musical output. These researchers are redefining musical performance- audience members are no longer simply witnesses to the creative process, rather they have become integral components to the process itself.

Another application area of physical computing in entertainment is in mobile applications. Youth culture is gravitating towards mobile applications for the entertainment they provide. This is especially exciting for researchers, since mobile applications can be developed to engender secondary benefits such as learning, research data, and physical activity. A good example of this concept is in NEAT-o games [9]. NEAT-o games are based around research that suggests that simple daily activities (such as walking) are incredibly important to physical health and can prevent conditions associated

with a sedentary lifestyle. In this specific study, users were equipped with a wearable accelerometer and a PDA that featured two mobile applications: NEAT-o-Race and NEAT-o-Sudoku. Users earned activity points based on their overall movements, which they could use to move their avatars in the NEAT-o-Race game, or to help them solve puzzles in the NEAT-o-Sudoku game. Several studies were conducted to assess the usefulness of the concept. It was found that the devices increased physical activity over both short- and long-term periods.

The future of physical computing in entertainment applications is very bright. Until recently interest in the area was limited to academia and industry, as the cost of technology was high. However the cost of entry is rapidly decreasing (both in price of materials and in time to learn the background necessary) and we are witnessing the formation of a do-it-yourself (DIY) culture [10]. Industry is responding to this culture, and the newest gaming interfaces (such as Nintendo Wii and Xbox Kinect) are providing open-sources libraries and toolkits to allow individual developers to customize the hardware to suit their own purposes [11] We can expect to see this culture continue to grow, and provide unique and creative innovation in the years to come.

Smart Technologies As defined in [12] a smart technology is one that is able to acquire and apply knowledge about the environment and its inhabitants in order to improve their experience in that environment. Smart technologies have been linked with physical computing since their advent, as they rely heavily on sensors and wireless equipment to track persons and events to make decisions about how to control the environment.  

The Aware Home is one of the earliest smart technologies to take advantage of physical computing [13]. The research began in 1999, and was focused on developing a home that was aware of its inhabitants and their activities. The initial objective was focused on providing a supportive living environment for the elderly, and included an embedded sensor network and wearable computers for the inhabitants. The Aware Home was designed to detect footsteps and determine who they belonged to using force-sensitive floor tiles. It also featured a system of small radio-frequency tags and audio/visual displays for keeping track of frequently lost objects like keys. The design also included specially support for the elderly including persistent communication channels with family, and crisis detecting technology.

A few years later researchers begin extending the smart home concept to reduce energy consumption [14]. In this study, researchers developed location-awareness probability models to predict the most-likely paths of inhabitants in order to efficiently manage power

consumption. Their prototype, the MavHome (Managing An intelligent Versatile Home) used hand-held devices and a wireless network to ensure only appliances near the occupants were activated.

A recent study highlights a potential future for smart technologies [15]. The research goal was to bring public awareness to issues of power consumption. The researchers installed sensing equipment in the engineering building at Eastern Washington University and developed online tools for accessing and visualizing the power consumption data. The research project allows students and the public to gain awareness about energy consumption, and to demonstrate how modern buildings can be outfitted with technology to track and reduce expenditures. This project exemplifies an important future of physical computing in smart technologies. Rather than solely focusing on making the environment better for the user, we are seeing a social transition towards energy efficiency and environmental protection. Technology played a major role in causing the environmental problems our world faces today; however technology could be an answer through physical computing. Sensor networks, ambient technologies, and context-aware interfaces can help control energy consumption, and help maintain the ecosystem around us.

PHYSICAL COMPUTING FOR HEALTH In this section, we will review various application of physical computing in health care, assistive technology, and physical activities. The applications in this review range from early 70s to 2011.

Health Care One of the early health care computing systems was proposed in [16] in 1971. A specialized facility, using automatic techniques was designed for comprehensive monitoring and treatment of critically ill patients. This unit, as shown in Figure 1, is centered on a digital computer and is operated by an interdisciplinary team of physicians, nurses, technicians, engineers and computer scientists.

Figure 1. Bed module in the shock ward [16].

The two-bed patient ward was designed to provide optimal placement of equipment while preserving optimal access to the patient. Beds were developed that provided flexibility and compatibility with mobile fluoroscopic equipment used to view catheter placement. Highly adjustable instrument supports of stainless steel and plexiglass were installed along both sides of the bed to accommodate pressure transducers, densitometers, pumps, urinometer, and flush system. Wiring from sensors on or near the patient was passed along support arms to the head of the bed and from there into an instrument island. This was located away from the wall to permit access to the patient from all sides. The wiring and tubing from the island to the control center and computer was shielded by multichannel flat metal ducting. An automated system for blood sampling and injection of fluid and medication was also developed for the bedside. Displays (both digital and analog) and trend plots served as sources of clinical data at the bedside. The miniaturization and optimal placement of the equipment preserved access to the patient and facilitated management at critical times during life-threatening illness.

In 2003, authors of [17] introduced the concept of the Hyper Hospital to offer patients a means of effective human communication during medical care. Hyper Hospital is a networked virtually reality interface that is dedicated to patient care. It features wearable computers that enable caregivers to operate various support devices connected to the network. This study found that hands-free operation of the wearable computer was important, as it enabled care workers to provide continuous physical care to the patients while accessing the computer. Their work featured an ocular interaction technique which used signal processing to take advantage of the dipolar potential of the human eye. The technique allowed care givers to interact with the graphical interface using eye movements and blinks.

In 2004, authors of [18] made a strong claim stating that “The ubiquitous computing for any time, any where, any network, any device and any service will become the core paradigm of the future information oriented society”. As a case of ubiquitous health care service, they introduced the Art Therapy service. Art Therapy uses sensors for recognizing contextual information such as body temperature, blood pressure, and humidity. Based on this information, appropriate music and paintings are provided to the patient. Art Therapy aims to provide a healing effect for both physical and mental diseases. The paper describes the system and technology, however does not provide any evidence to demonstrate its effectiveness.

Researchers have also expanded physical computing to the discipline of maternal and child health. Researchers in [19] developed an intelligent information system for maternal and child health care. The system features

wearable sensors for detecting physical signs, and uses cloud computing and data mining technologies to process the data and design medical models. With this system, new services can be designed to and improve the efficiency of maternal and child health care system.

As we look to the future, applications of physical computing for health care are going to continue to be a major priority. An exciting new direction that shows promise is the use of physical computing to aid in the monitoring of emotional health. Researchers in [20] designed a model of home health care system with an emotional recognition function. Their system integrates a wearable physiological signal acquisition module with communication modules to monitor the physiological and emotional state of a user.

Assistive Technology Physical computing is also a strong contributor to the field of assistive technology. Research work presented in [21] describes three interaction methods that were designed for reading six-dot Braille characters from the touchscreen of a mobile device. A prototype device with a piezoelectric actuator embedded under the touchscreen was used to create tactile feedback. The three interaction methods, scan, sweep, and rhythm, enabled users to read Braille characters one at a time either by exploring the characters dot by dot or by sensing a rhythmic pattern presented on the screen. The methods were tested with five blind Braille readers as a proof of concept. The results of the first experiment showed that all three methods can be used to convey information as the participants could accurately (91-97 percent) recognize individual characters. In the second experiment, the presentation rate of the most efficient and preferred method, the rhythm, was varied. A mean recognition accuracy of 70 percent was found when the speed of presenting a single character was nearly doubled from the first experiment. The results showed that temporal tactile feedback and Braille coding can be used to transmit single-character information.

Another assistive aid for the blind and visually impaired is an electronic travel aid, an assistive technology designed to increase their independent mobility while traveling in unknown environment. An example of such is described in [22]. The research examined a pedestrian navigation system for the blind which used a microcontroller with synthetic speech output. This aid is a portable, self-contained system that allows blind people to travel without the assistance of guides. It is designed for a battery-powered portable model. In addition, it is focused on low power consumption, small size, lightweight, and easy manipulation. The system provides information to the user about urban walking routes using spoken words to indicate what decisions to make.

A state of the art assistive technology, a dynamic haptic

matrix display is studied in [23]. It is used to provide physical tactile diagrams for visual 2-D graphics. It is composed of a Braille cell, which constitutes the tactile display, embedded in a computer mouse casing. Also embedded in the mouse casing is an RF transmitter that communicates the absolute position of the matrix to a graphics tablet. Custom electronics link the device with the software and enable the pins to be driven to one of four amplitude levels with a usable bandwidth of 125 Hz and an update rate of 1 kHz. In software, each pin is considered to have an infinitely small tip. The amplitude level of each pin is determined by locating its position in a (4 level) texture map. The coordinates of the device’s position on the graphics tablet are fed into the driving software, which looks up the eight elements of the texture map matrix corresponding to the levels of the eight pins; these levels are then output to the driving electronics. When using the device, the speed was limited to below 50 mm/sec because of hardware limitations. The study carried in [23] showed that there is no difference found between the performance of blind or visually impaired participants and sighted participants, so the application of the device may be extendable to areas outside of assistive technology.

Physical computing applications in the field of assistive aids are not limited to visual aids; they are extended to other disability aids such as mobility aids. For example, researchers in [24] developed a personal assistance device that provides autonomy and health monitoring capabilities for wheelchair users. This research is thought to be useful for patients with neuromuscular disorders, stroke victims, the elderly and others with limited mobility. The wheelchair described in the research monitors the user’s physiological characteristics using both passive and active sensors, and displays the data to a touch screen.

Researchers from CMU and other universities present a futuristic vision of assistive devices in [25]: “Tele- services and assistive devices will play an ever-increasing role in providing home, assisted living and hospital services, prevention of falls, injury mitigation, and a host of other services. In this regard, tele-presence through multiple entities such as tele-immersive displays, tele- operated, and haptic devices can be beneficial. Consider a tele-physical home-helper that would assist the elderly in cleaning up the home, preparing food, administering medicine and other tasks. Such a helper would be semi- autonomous, carrying out many operations in a fully autonomous mode, such as navigating through the home, scrubbing and vacuuming, recognizing voice commands and gestures from the patient (or client), maintaining the inventory of food and medicine, and ordering supplies over the Internet, while also accepting direct commands remotely, say, from a physician or family member. Achieving this vision requires, in part, trust of personal devices as well as trust in transmitted information.”

Physical Activities Physical computing is not only a relevant component of health care systems and assistive technology; it is also becoming prevalent in the area of physical activities. In 1999, cooperation between medical and engineering researchers led to a specialized device used to monitor physical activity [26]. Their device used accelerometers at different parts of the body to monitor the physical activity of the person. Their early prototype can predict whether the person is lying, sitting, standing, or moving with an accuracy of 90%. Ten years later, another group of researchers proposed a physical fitness condition measurement system based on radio frequency identification (RFID) technology in [27]. The system is composed of a RFID tag, a RFID reader, a main control board, and various peripherals including: blood pressure monitor, ear thermometer, height measure, weight scale, balance measurement, and a speedometer. The RFID tag and reader are used to read and store a user’s id and his physical fitness conditions. The main control board with processes the user’s id and subsequently triggers the peripheral device to perform measurements of the user’s physical fitness condition. The measured data is then transmitted and stored in the RFID tag and database. Testing results of this system proved its validity and accuracy. This system can be used as a reference to manage one’s health, or as a medical diagnosis to determine medical treatment, rehabilitation and training.

One of the most recent trends in physical computing for physical activity is the use of wireless sensor networks (WSN) and gaming to improve physical health. One such system is presented in [28], with the fundamental goal to improve a patient’s cardiovascular health. It features three components: the body area WSN, the game environment and the data acquisition manager. Using the WSN on the patient's body allows real time motion and medical data to be collected. The information is then filtered and used inside the gaming environment to control the patient's avatar. The data also provides an adjustment mechanism to change gaming parameters according to the medical status of the patient. A neck physiotherapy case study was used to illustrate the applicability of this approach.

PHYSICAL COMPUTING FOR TRANSPORTATION Transportation has been and will continue to be a huge application area for physical computing systems. From smart cars to global positioning systems (GPS) to traffic control systems, physical computing has become prevalent in transportation today.

Smart Cars For more than a decade researchers have been designing advanced car-driver interfaces. In 2001 and 2002 Carnegie Mellon students designed a multimodal car-driver interface, Companion [29].

Companion can read from data sources such as the vehicle’s data bus, GPS, and the driver’s PDA. It uses this information to alert the driver of important events via three types of reminders. Time-based reminders notify the driver of upcoming scheduled events, space-based reminders notify the driver of events that are located near the current location, and space-time based reminders notify the driver of scheduled events that they may miss due to the time required to get from the current location to the event location. The HUD is used for important, low-detail information since it is the least distracting display. The audio output is the primary means of communication with the driver since it can convey more information than the HUD, but is far less distracting than the LCD display, which is only used for detailed, low-importance information. The least distracting input modality is the hand gesture interface. A wave of the hand temporarily stops all system output in emergency scenarios where talking on the phone or listening to music may be distracting or dangerous. Due to its limited vocabulary and phrase matching the speech recognition rate of this system is very high. Speech recognition can be used to control the touch screen LCD to reduce driver distractions. More recently, in 2011, the authors of [30] used physical computing technology for non-contact monitoring of a car driver’s vital signs. A system of this nature could immediately contact medical help in case of an emergency. The vital measurements that the authors used were capacitive electrocardiogram (cECG) monitoring, mechanical movement analysis and inductive impedance monitoring. They founds that both cECG and inductive impedance monitoring are viable, but the mechanical movement analysis included far too much noise to use without further optimizations. The automotive industry continues to adopt smart car technology into high-end vehicles, ensuring funding and a strong future for researchers to pursue the perfect driving interface.

Figure 2. Fingerprint scanner for identification, FaceCam for facial recognition and eye tracking, GestureCam for gesture commands, microphone for speech recognition, Heads-Up Display (HUD) for important messages, and an LCD touch screen display for more details.

GPS GPS’s have been one of the most impactful advances in driving in recent history due to their ability to pinpoint vehicle location virtually anywhere using satellite communication. Among the many uses of GPS’s is giving directions to a target destination or locating a person in an emergency. The authors of [31] present an application that recommends events such as concerts to users and then provides them with start-to-finish directions to the event location. The recommender produces a list of suggestions based on user input and preferences. After the user selects a destination from the suggestions, the system calculates a route. Like most current GPS’s, driving and walking routes are supported, but this system also provides start-to-finish routing using a combination of public transportation methods and walking. The system will even alert the user by integrating live public transportation data with her current location if she has to hurry to catch a bus or train. The authors of [32] present a unique system to develop personalized, practical, fast driving routes by using both real-time and historical traffic conditions and driver behavior. To procure traffic condition data they used taxicabs with GPS units as mobile nodes. These taxi drivers are assumed to have developed a knowledge base of the traffic conditions at a given time and day and the shortest physical routes to a destination. This system automatically learns a particular user’s driving behaviors over time from the user’s recent mobile phone GPS data. The authors used data generated by 33,000 taxi drivers in Beijing over a three-month period and found that their system was able to provide accurate estimates of route time for a particular user. The authors of [33] introduced an application to help users of public transportation make better use of their transit time. Their system, HapticTransit, uses real-time location data to show the user’s position and haptic feedback to alert the user so they do not miss their stop. This application allows the user to partake in other activities such as reading without worrying about missing their stop. The authors of [34] aimed to reduce the time between a traffic accident and when emergency personnel are called to the scene because reducing accident response time by one minute corresponds with a 6% decrease in mortality rate. To achieve this goal they created an application that uses accelerometers and acoustic data collected by a smart phone to detect a collision. An emergency dispatch server is then automatically notified and the report data such as GPS location of the collision are automatically sent. Their system is similar to in-vehicle systems like OnStar, but is much cheaper and can be used in any vehicle.

Traffic Control Systems The modern air traffic control (ATC) system has not changed fundamentally since the 1960’s [35]. There are two main components of traditional ATC systems: radar and paper flight strips. Radar tracks the two-dimensional progress of aircraft, whereas the flight strips allow

controllers to track aircraft and modify their flight plans. Over the years air traffic controllers have generally accepted advances in radar systems, but have been very resistant to changes to paper flight strips. The authors of [35] are convinced after their study of air traffic controllers that not enough is known to get rid of the paper strips altogether and that replacing the physical interaction between controllers and papers flight strips will be extremely difficult. Their suggestion is to keep the physical strips, but to use them as the interface to the computer. In this way the physical interaction with the paper strips is preserved, but the efficiency benefits of computing can still be utilized. The authors of [36] take a different approach to optimizing ATC systems. They designed a multi-user tabletop surface to enable collaboration, communication, and mutual awareness of other controllers. In studies they found that their tabletop system was able to effectively support communication and coordination. Mutual awareness was achieved with certain teams of controllers and seemed to have to do with controller experience level. More recently researchers have been trying to apply the principles of ATC systems to car travel. This is a much more difficult problem than that of ATC since there is no central authority that knows the future path of each vehicle, or even how many vehicles are on the road. The authors of [37] describe a Vehicular Ad Hoc Network (VANET), WiSafeCar, which utilizes Vehicle-to-Vehicle (V2V) and Vehicle-to-Infrastructure (V2I) communication techniques. The WiSafeCar platform consists of vehicles, roadside units that act as base stations, and weather stations; future versions will also be able to connect to mobile users. VANETs have the capability to enhance safety by providing real-time collision and weather information to all other vehicles and infrastructure in the network. Vehicles in the VANET connect to the network via an ad hoc mechanism instead of a continuous connection. When a vehicle passes a roadside base station it downloads the up-to-date real-time service data from the base and uploads its own gathered data to add to the network knowledge base. In addition to gathering real-time location-based weather information from the weather station, the system can also get weather clues from windshield wiper, fog light, and ABS usage.

PHYSICAL COMPUTING FOR PUBLIC SAFETY AND SECURITY Physical computing can provide immense benefits to the safety and security of the general public.

Emergency Notification Systems One of the most important recent developments in public safety has been the advancing of emergency response systems. On April 16th, 2007 the Virginia Tech community was the victim of the deadliest shooting incident by a single gunman in U.S. history. The Clery Act of 2008 was passed shortly thereafter to require

higher education institutions to “immediately notify the campus community upon the confirmation of a significant emergency or dangerous situation involving an immediate threat to the health or safety of students or staff occurring on the campus” [38]. Prior to the massacre on campus the university’s emergency notification system (ENS) could update the university homepage, Virginia Tech news website, broadcast email alerts, broadcast voice messages, campus sirens and loudspeakers, a recorded emergency hotline and public media outlets. Since the attack on the VT community the ENS has added two new communication channels: Phone Alerts and Desktop Alerts. The university homepage can perform in a “light” mode in case of extreme network demand. Broadcast email is an internal system that can send more than 40,000 alert messages to university email accounts in less than five minutes. The university installed over 500 message boards on campus; alerts take less than 10 seconds to reach the message boards on average. VT Phone Alerts is a subscriber-only service that allows users to sign up to receive text messages, instant messages, emails, or phone calls in an emergency situation. Phone Alerts currently delivers to over 80,000 points of contact. Desktop Alerts provides a way for friends and family of Virginia Tech students and faculty to receive alerts in an emergency situation while not adding further delay to delivery of messages to the Blacksburg community. M-Urgency is a physical computing system for public safety that takes advantage of the widespread use of smart phones [39]. This system allows mobile users to stream live video, audio, and location information from their devices to a local public safety answering point such as an emergency dispatch office. M-Urgency also gives the dispatcher the ability to forward the information stream with a simple drag-and-drop action. In their survey of future goals and applications for physical computing the authors of [2] discuss future emergency response systems in the form of wireless sensor networks. These networks would be made up of thousands of cheap sensor nodes, each with limited resources. These nodes would collectively assess the emergency situation, detect survivors, notify rescue teams, and identify an exit path for those caught in the disaster. However, limited resources and environmental unpredictability of sensor nodes make ensuring real-time constraints extremely difficult. In practice it is more likely that wireless sensor networks will provide a probabilistic guarantee of reliability. Another issue that needs to be addressed before these systems are viable is the efficient energy consumption and timely delivery of information in wireless sensor networks.

Justice System The justice system has recently taken advantage of improvements in physical computing. Sometimes judges have to visit jails to conduct a trial. These jail trials can waste the judge’s time and taxpayer money. Jail trials also

provide an opportunity for attempted murder of prisoners, attempted escape, and potential smuggling of weapons, drugs, or other items into prison. One solution brought forth in [40] is conducting such jail trials through video conferencing. The judge, lawyers, witnesses and prisoner all have microphones and are shown over video conferencing while they proceed. [40] estimated it would cost 313% less to implement this video conferencing solution instead of jail trials, given a three-year lifetime for the technology. Using video conferencing instead of jail trials resulted in a 50% savings in time for the judge in their study. It also gives the benefit of convenient expert witness testimony via a live video link, and provides an opportunity for replays of key testimonies. Falavigna [41] presents another application of physical computing to the justice system – using automatic speech recognition for an automated transcription system. However, this method produces a word error rate of about 40%. Although these results are in line with other automated speech recognition systems, an error rate this high is clearly not acceptable for practical use without further improvements.

Military and Robotics In 1983 the Strategic Computing Program (SCP) funded $600 million worth of research projects incorporating artificial intelligence into military applications, including battle management systems [42]. An important battle management system was Igloo White, which sensed acoustic and seismic information to act as an electronic barrier between North and South Vietnam. This system led to indiscriminate killing since the sensors could not distinguish between a soldier and a civilian. It also presented people as “blips on a display screen – blips that must be stopped, rather like a video arcade game”, leading to important ethical questions regarding autonomous killing.

Autonomous vehicles and robots are an important asset for exploration of harsh environments such as other planets. In 1997 NASA sent the Sojourner rover to Mars, where it became the first spacecraft to drive on another planet autonomously [43]. The Sojourner was a purely reactive system since it did not store a permanent terrain map. Its autonomous capabilities were resource management, contingency response, and terrain navigation. The two Mars exploration rovers (MER), vast improvements on the Sojourner, were sent to Mars in 2004. They provide additional capabilities such as safe terrain navigation while avoiding geometric hazards and automatically detecting science events. The next generation rovers are the Mars Science Laboratory (MSL) rovers, which offer terrain prediction and autonomous science. Autonomous science describes the process of learning what measurements to expect so that the rover can automatically detect and record interesting measurements. Moving forward these increasingly

autonomous rovers will be crucial to our understanding of other planets and moons in our solar system.

CONCLUSION In this paper, we have presented our view on physical computing. We see physical computing as the research involving programmable devices that unobtrusively enable mobility, communication and control. We have surveyed the existing research that fits this definition, and classified it into four categories: physical computing for quality of life, physical computing for health care, physical computing for transportation, and physical computing for public safety and security.

The future of physical computing is bright, especially with the collaboration of more interdisciplinary research teams. The authors of [44] take a look back at the twentieth anniversary of Weiser’s 1991 seminal paper to see how far the field of physical computing has come. They conclude that although some of Weiser’s visions have been realized, we still have a long way to go before fully accomplishing his ideal. For example, moving content between one’s devices is still very often a difficult task. To achieve his goals we must make devices easily interoperable in a secure manner.

REFERENCES 1. Weiser, M., The computer for the 21st Century.

Pervasive Computing, IEEE, 2002. 99(1): p. 19-25.

2. Stankovic, J.A., et al., Opportunities and obligations for physical computing systems. Computer, 2005. 38(11): p. 23-31.

3. Phan, T., L. Huang, and C. Dulan, Challenge: integrating mobile wireless devices into the computational grid, in Proceedings of the 8th annual international conference on Mobile computing and networking2002, ACM: Atlanta, Georgia, USA. p. 271-278.

4. Ballagas, R., et al., The smart phone: a ubiquitous input device. Pervasive Computing, IEEE, 2006. 5(1): p. 70-77.

5. Akgul, F.O. and K. Pahlavan. Location awareness for everyday smart computing. in Telecommunications, 2009. ICT '09. International Conference on. 2009.

6. Partridge, C., Realizing the future of wireless data communications. Commun. ACM, 2011. 54(9): p. 62-68.

7. Gross, M.D., A Physical Computing Studio: Exploring Computational Artifacts and Environments. International journal of architectural computing, 2003. 1(2): p. 169-190.

8. Knapp, R.B., J. Jaimovich, and N. Coghlan. Measurement of motion and emotion during musical performance. in Affective Computing and Intelligent Interaction and Workshops, 2009.

ACII 2009. 3rd International Conference on. 2009.

9. Fujiki, Y., et al., NEAT-o-Games: blending physical activity and fun in the daily routine. Comput. Entertain., 2008. 6(2): p. 1-22.

10. Kuznetsov, S. and E. Paulos, Rise of the expert amateur: DIY projects, communities, and cultures, in Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries2010, ACM: Reykjavik, Iceland. p. 295-304.

11. Sung, K., Recent Videogame Console Technologies. Computer, 2011. 44(2): p. 91-93.

12. Cook, D.J. and S.K. Das, How smart are our environments? An updated look at the state of the art. Pervasive and Mobile Computing, 2007. 3(2): p. 53-73.

13. Kidd, C., et al., The Aware Home: A Living Laboratory for Ubiquitous Computing Research, in Cooperative Buildings. Integrating Information, Organizations, and Architecture, N. Streitz, et al., Editors. 1999, Springer Berlin / Heidelberg. p. 191-198.

14. Roy, A., et al. Location aware resource management in smart homes. in Pervasive Computing and Communications, 2003. (PerCom 2003). Proceedings of the First IEEE International Conference on. 2003.

15. Durfee, J.K. and M. Weiser. Improving access to public building energy use to promote better energy policy. in Technology Management in the Energy Smart World (PICMET), 2011 Proceedings of PICMET '11:. 2011.

16. Carrington, J.H., et al., Physical Arrangements at the Bedside in Support of Automated Systems for Patient Care. Biomedical Engineering, IEEE Transactions on, 1971. BME-18(2): p. 149-154.

17. Mizuno, F., et al. Development of hands-free operation interface for wearable computer-Hyper Hospital at home. in Engineering in Medicine and Biology Society, 2003. Proceedings of the 25th Annual International Conference of the IEEE. 2003.

18. Seong-Im, Y., et al. The implementation of art therapy service as an ubiquitous health-care service. in TENCON 2004. 2004 IEEE Region 10 Conference. 2004.

19. Lei, C., et al. An Intelligent Information System for Maternal and Child Health Care. in Computer Software and Applications Conference Workshops (COMPSACW), 2011 IEEE 35th Annual. 2011.

20. Jiang, Z., et al. Design of wearable home health care system with emotion recognition function. in Electrical and Control Engineering (ICECE), 2011 International Conference on. 2011.

21. Rantala, J., et al., Methods for Presenting Braille Characters on a Mobile Device with a Touchscreen and Tactile Feedback. Haptics, IEEE Transactions on, 2009. 2(1): p. 28-39.

22. Bousbia-Salah, M. and M. Fezari. The Development of a pedestrian navigation aid for the blind. in GCC Conference (GCC), 2006 IEEE. 2006.

23. Headley, P.C. and D.T.V. Pawluk. Roughness perception of textures on a haptic matrix display. in World Haptics Conference (WHC), 2011 IEEE. 2011.

24. Postolache, O., et al. Pervasive sensing and computing for wheelchairs users health assessment. in Bioengineering (ENBENG), 2011. ENBENG 2011. 1st Portuguese Meeting in. 2011.

25. Rajkumar, R., et al. Cyber-physical systems: The next computing revolution. in Design Automation Conference (DAC), 2010 47th ACM/IEEE. 2010.

26. Martinez, K., J.K. Hart, and R. Ong, Environmental sensor networks. Computer, 2004. 37(8): p. 50-56.

27. Sung-Tsun, S., C. Chian-Yi, and H. Chin-Ming. RFID based physical fitness condition measurement system. in Networked Computing and Advanced Information Management (NCM), 2011 7th International Conference on. 2011.

28. Fergus, P., et al. A framework for physical health improvement using Wireless Sensor Networks and gaming. in Pervasive Computing Technologies for Healthcare, 2009. PervasiveHealth 2009. 3rd International Conference on. 2009.

29. Siewiorek, D., A. Smailagic, and M. Hornyak, Multimodal Contextual Car-Driver Interface, in Proceedings of the 4th IEEE International Conference on Multimodal Interfaces2002, IEEE Computer Society. p. 367.

30. Walter, M., et al., The smart car seat: personalized monitoring of vital signs in automotive applications. Personal Ubiquitous Computing, 2011. 15(7): p. 707-715.

31. Zenker, B. and B. Ludwig, ROSE: assisting pedestrians to find preferred events and comfortable public transport connections, in Proceedings of the 6th International Conference on Mobile Technology, Application & Systems2009, ACM: Nice, France. p. 1-5.

32. Yuan, J., et al., Driving with knowledge from the physical world, in Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining2011, ACM: San Diego, California, USA. p. 316-324.

33. Jacob, R., et al., HapticTransit: Tactile feedback to notify public transport users. International

Journal of Digital Information and Wireless Communications, 2011. 1(1): p. 197-211.

34. White, J., et al., WreckWatch: Automatic Traffic Accident Detection and Notification with Smartphones. Journal of Mobile Networks and Applications, 2011. 16(3): p. 285-303.

35. MacKay, W.E., Is paper safer? The role of paper flight strips in air traffic control. ACM Transactions on Computer-Human Interaction, 1999. 6(4): p. 311-340.

36. Conversy, S., et al., Supporting air traffic control collaboration with a TableTop system, in Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work2011, ACM: Hangzhou, China. p. 425-434.

37. Sukuvaara, T., et al., Wireless traffic safety network for incident and weather information, in Proceedings of the first ACM international symposium on Design and analysis of intelligent vehicular networks and applications2011, ACM: Miami, Florida, USA. p. 9-14.

38. Mulhare, M. The Emergency Notification System and Protocol at Virginia Tech. in Suppression, Detection and Signaling Research and Applications (SUPDET). 2011. Orlando, Florida: National Fire Protection Association.

39. Krishnamoorthy, S. and A. Agrawala, M-Urgency: a next generation, context-aware public safety application, in Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services2011, ACM: Stockholm, Sweden. p. 647-652.

40. Nasir, A., Use of ICT: a step towards good governance in developing countries, in Proceedings of the 1st international conference on Theory and practice of electronic governance2007, ACM: Macao, China. p. 296-300.

41. Falavigna, D., et al., An automatic transcription system of hearings in Italian courtrooms, in Proceedings of the 2nd ACM workshop on Multimedia in forensics, security and intelligence2010, ACM: Firenze, Italy. p. 99-104.

42. Beusmans, J. and K. Wieckert, Computing, research, and war: If knowledge is power, where is responsibility? Communications of the ACM, 1989. 32(8): p. 939-951.

43. Bajracharya, M., M.W. Maimone, and D. Helmick, Autonomy for Mars Rovers: Past, Present, and Future. Computer, 2008. 41(12): p. 44-50.

44. Ebling, M. and M. Baker, Are we there yet? Pervasive Computing, IEEE, 2011. PP(99): p. 1-1.