Games as Neurofeedback Training for Kids with FASD Regan L. Mandryk1, Shane Dielschneider1, Michael R. Kalyn1, Christopher P. Bertram2, Michael

Gaetz2, Andre Doucette1, Brett A. Taylor1, Alison Pritchard Orr2, Kathy Keiver2 1University of Saskatchewan

Saskatoon, SK, Canada, S7N5C9 1-306-966-2327

{firstname.lastname}@usask.ca

2University of the Fraser Valley Abbotsford, BC, Canada, V2S 7M8

1-604-504-7441 {firstname.lastname}@ufv.ca

Figure 1. Columns show low, medium, and high levels of texture-based biofeedback. Rows show customizations of the same effect for two different games: top) Static Sprite (cracks) over Portal 2, bottom) Static Sprite (mud) over Nail’d.

ABSTRACT Biofeedback games help people maintain specific mental or physical states and are useful to help children with cognitive impairments learn to self-regulate their brain function. However, biofeedback games are expensive and difficult to create and are not sufficiently appealing to hold a child’s interest over the long term needed for effective biofeedback training. We present a system that turns off-the-shelf computer games into biofeedback games. Our approach uses texture-based graphical overlays that vary in their obfuscation of underlying screen elements based on the sensed physiological state of the child. The textures can be visually customized so that they appear to be integrated with the underlying game. Through a 12-week deployment, with 16 children with Fetal Alcohol Spectrum Disorder, we show that our solution can hold a child’s interest over a long term, and balances the competing needs of maintaining the fun of playing, while providing effective biofeedback training.

Categories and Subject Descriptors H5.2 [Information interfaces and presentation]: User Interfaces. - Graphical user interfaces.

Keywords Biofeedback, neurofeedback, games, FASD, ADHD.

1. INTRODUCTION Fetal alcohol exposure is the most prevalent cause of intellectual impairment in the western world [17]. An accurate account of the incidence of fetal alcohol spectrum disorder (FASD) is unknown but estimates range from 3 per 1000 live births to 10 per 1000 children being affected by prenatal alcohol exposure [10], which translates to thousands of affected infants born each year in Western Canada [3]. Children with FASD are often also diagnosed with Attention Deficit Hyperactivity Disorder (ADHD) [2]; using biofeedback (BF) to train brain function self-regulation has been effective at reducing the symptoms of ADHD, and at reducing differences of ADHD children from normative databases of elecroencephalography (EEG) [6,9].

Biofeedback training systems encourage a specific mental or physical state in a user through a closed biofeedback loop. These systems gather a child’s physiological state through sensing hardware, integrate this state into a computer-based interactive system, and present the feedback so that the child can work to adjust their state. Biofeedback training systems often use games for interaction because playing games is intrinsically motivating for most children. Biofeedback games work by altering the game mechanics (i.e., rules and procedures) based on the child’s physiology; however, traditionally, biofeedback games have not been engaging enough to hold a child’s interest over the repeated sessions needed for effective training [14].

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Interaction Design and Children’13, June 24–27, 2013, New York City, New York, United States. Copyright © 2013 ACM 978-1-4503-1918-8…$15.00.

Creating engaging biofeedback games remains difficult because the game’s mechanics must be altered to create the biofeedback loop. This means that each biofeedback game is a custom creation, which is both expensive and time consuming; choosing to play off-the-shelf games as biofeedback games is simply not possible. As a result, biofeedback games have two main problems. First, they tend to be toy applications that don’t hold a child’s interest in the long term, which is a problem because biofeedback training requires repeated exposure to yield successful results [19]. Second, a child who wants to play a biofeedback game has little choice over the game genre, and may not be motivated to play a game from a small selection with little appeal.

In this paper, we present a system that turns off-the-shelf computer games into biofeedback games. We propose to close the biofeedback loop by altering display graphics instead of game mechanics. We present a graphical overlay on top of a running game that obscures the underlying game when the child is not in the desired physiological state (see Figure 1). Our system works with off-the-shelf games, so children can choose games that they like. The amount of obfuscation varies parametrically in real time, and is driven by physiological sensors. Our graphical overlays can be chosen from an all-purpose set or be customized to be consistent with the visual style, theme, or genre of the game, so that they appear to be integrated with the underlying game. In addition, the graphical effects are consistent with current abstract in-game visualizations that players are already familiar with using (e.g., tunnel vision representing poor in-game health).

Because our solution is a novel approach to biofeedback, we investigate three main questions surrounding its efficacy. First, does altering display graphics instead of game mechanics work as a biofeedback mechanism? Second, do graphical overlays ruin the fun of playing games? Third, can our system remain motivating over the long term needed for successful biofeedback training? We answer these questions by deploying the system over 12 weeks with 16 children with FASD, a target demographic for brain-based biofeedback training. We conclude by discussing how our approach can be applied to interaction other than games, to open biofeedback training to those who are uninterested in games or who would prefer to integrate it into their day.

This research presents the first general solution for turning off-the-shelf software into biofeedback systems where the user chooses which physiological trait to train. We focus on games, leveraging the millions of dollars and years of development that go into triple-A titles, and ensuring an engaging play experience that will hold a child’s interest. Our low-cost system provides the opportunity for biofeedback training – previously only available in clinics – to children directly in their homes.

2. RELATED WORK Biofeedback training has been used to help patients with Asperger’s Syndrome [20], to reduce the frequency of seizures in patients with epilepsy [6], and to improve the behaviour of children with ADHD [9]. There is also evidence of successful biofeedback training for children with tic disorder, autism, schizophrenia, and learning disabilities (see [6]). In healthy individuals, biofeedback has been used to improve working memory and attention [21].

2.1 Biofeedback Games Instead of providing biofeedback through simple graphical feedback, games are used because they are intrinsically

motivating for many users and will encourage participation, potentially resulting in improved training compliance [14]; however, many biofeedback games (e.g., [20], [21]) might be better described as interactive systems because they lack the uncertain and quantifiable outcome of a game [15].

In biofeedback games, players must maintain a particular physiological state to make progress, generally accomplished by adjusting the game’s mechanics (i.e., the rules and procedures of play). For example, in a bowling game, the ball might roll toward the gutter as the player becomes stressed. The player responds by self-regulating their sensed physiological state in the desired direction (see Figure 2–left). Note that biofeedback games are not the same as ‘brain-training games’. Although both claim to help users improve cognition, biofeedback games require a closed loop of physiological sensing and real-time feedback; brain-training games are mental exercises made fun with game-like elements.

2.1.1 Commercial History Industry manufacturers have investigated biofeedback gaming since the 1970s, when Thought Technology created CalmPrix1, a racing game that came packaged with an Apple II mouse modified with galvanic skin response (GSR) electrodes. Other biofeedback gaming systems include the unreleased Atari Mindlink2 in 1983, The Journey to Wild Divine3 in 2001, and the Nintendo 64 biosensor4 included in the Japanese version of Tetris 64 in 1998. In 2005, Smart Brain Technologies5 released a home-based EEG system. Recent biofeedback game devices include two popular consumer-grade EEG devices – the NeuroSky Mindset6 and the Emotiv EPOC7. These devices measure EEG via electrodes that are held in place by a headset; both include a variety of games. In particular, the NeuroSky features Focus Pocus8 – a 2012 biofeedback game designed specifically for children with ADHD.

2.1.2 Academic History Early academic research in biofeedback games did not intend to create therapeutic systems, but to create compelling play experiences (see [12] for an overview). Sensed physiology can be indicative of a user’s emotional state, and if used to manipulate a play experience, could create engaging affective games [5]. Relax-to-Win (2001) [1] involved racing dragons whose speed was controlled by GSR; BalloonTrip (2003) [16] also used GSR as a game control; Brainball [8] is a game where a user’s EEG controls a physical ball rolling on a table. There are also examples of systems that use software development kits to integrate biofeedback with existing games. In AlphaWoW [12], players trigger their shapeshifting ability through EEG; in [4], some game mechanics in Half-Life 2 (e.g., enemy spawn points, screen shaking) are controlled with heart rate; and in AffQuake [13], the player’s avatar jumps when startled and grows with player excitement. Finally, there are many examples of heart-rate control in games for the purpose of improving physical fitness (see [18]).

These systems all use indirect physiological control [11], where players must work to change their physiological state through a 1 http://www.thoughttechnology.com/thewall2.htm 2 http://www.atarimuseum.com/videogames/consoles/2600/mindlink.html 3 http://www.wilddivine.com/ 4 http://nintendo.wikia.com/wiki/Bio_Sensor 5 http://www.smartbraintech.com/ 6 http://store.neurosky.com/products/mindset 7 http://www.emotiv.com/ 8 http://ballantinespr.com/News/NeuroSky/NeuroSky_Focus_Pocus.html

mediating mechanism. For example, to make the dragon race faster in Relax-to-Win, a user must reduce their GSR by relaxing. Indirect control is also the principle used in biofeedback games; however, the games presented here were not developed as therapy, but as novel systems that provide new forms of play.

2.1.3 Biofeedback Games in Practice Although biofeedback games used in clinical practice resemble the simple systems presented here, they have seen some clinical success. Biofeedback games have been used to help people manage stress, relax, focus, and manage anxiety see [6], and have seen particular success when with special populations – for example, to help children with ADHD and anxiety learn to manage their symptoms [6], or to help children with pelvic floor dysfunction improve voiding dysfunction [7].

In research closer to ours, Pope and Palsson [14] created a hardware solution5 that worked with Playstation games by altering the performance of a game controller based on a user’s EEG. In a study comparing their game-based system to a traditional biofeedback system with children with ADHD, the authors found that both approaches resulted in improvements, but that children and parents were happier with the game system [14]. Like our research, the authors were interested in leveraging the motivation provided by off-the-shelf games; however, our approach is not limited to games played with a controller or on a Playstation, uses a software solution that is agnostic to the underlying game (their solution affected performance in the game), and is not limited to a single physiological sensor (they train a specific band of EEG).

3. TEXTURE-BASED BIOFEEDBACK Biofeedback systems have two general requirements. First, they must sense a user’s physiological state; and second, they must provide this sensed state to the user through a feedback mechanism. Both of these requirements should occur without delay, as close to real time as possible. Our biofeedback system had two additional requirements. First, as we wanted to engage children over the long term, our system had to work with off-the-shelf games. Second, the computational resources needed to run the system should not affect the performance of the game.

Figure 2. Biofeedback loops: Left – Traditional loop; Right – Our revised texture-based loop.

Together, these four requirements inspired our use of texture-based overlays, rendered in real time in a transparent overlay on top of a user’s primary task of playing a game. Traditionally, biofeedback games work by not allowing the user to progress unless they are in the desired physiological state. In our case, the textures obscure the graphics related to the user’s primary task,

making it less enjoyable to play, and potentially impossible to progress if there is enough obfuscation. Similar to traditional approaches, we vary the feedback depending on the user’s state; the textures have different obscuring parameters (e.g., opacity, position, coverage) that vary continuously along a scale, providing varying levels of obfuscation of the game display. Players want to play with as little obfuscation as possible (see Figure 1), motivating them to maintain the desired physiological state.

Figure 2-left shows the traditional biofeedback loop, completed through modifying game mechanics. This approach requires that each game be custom designed specifically with the biofeedback loop in mind. With texture-based biofeedback (see Figure 2-right), texture overlays obscure the screen to complete the biofeedback loop. This approach is agnostic to the underlying game, requiring no access to source code, thus can be used with any off-the-shelf game. In addition, the user’s physiological state is integrated into our system rather than into the game itself, so any physiological system can be trained, regardless of game choice.

Our system has three main components; the biofeedback game interface accesses two libraries: one senses a player’s physiological state and the other renders the textures to the display. In the following sections, we describe these components.

3.1 Physiological-Sensing System Current biofeedback games (e.g., Focus Pocus) integrate the sensor with the game mechanics, leaving users no decision over game choice or what aspect of their physiology they wish to train. Our system separates the selections of sensor and game.

Our physiological sensing system is managed by a custom library called SensorLib, which is a multi-threaded library written in C# that provides an interface for external third-party sensors. It handles the connection to the sensors, the data-buffering, digital signal processing, and offers the data through a high-level .NET interface. SensorLib aggregates third-party software development kits (SDKs) into a single interface for ease of programming; new sensors are easily added if the signal can be accessed via an SDK.

3.2 Texture-Rendering System Our texture-rendering system (TextureLib) was built in C++ using Microsoft DirectX 10 graphics libraries, and displays an overlay window over any application. The overlay window is rendered over top of other windows even when it does not have focus, is transparent, and allows keyboard and mouse events to pass through it so interaction still occurs with the applications running “behind” the overlay. TextureLib renders visual representations in the transparent overlay by making use of DirectX resources (DDS files) and pixel shaders written in High Level Shader Language (HLSL). Our software requires Windows 7 and a video card that supports DirectX 10 (common in gaming computers).

3.2.1 Visual Appearance of Textures The visual representations that we use are made up of an effect (a pixel shader), and any number of resources (textures and colormaps) that can be edited using image editors. TextureLib contains five pre-packaged effects that can be customized.

Tunnel Vision creates a semi-transparent texture with a definable encroachment area on the screen. The location and size of this area, the fade-in threshold for the texture to become opaque, and the texture colour can be controlled. Note that the next four effects can be applied to Tunnel Vision, combining the effects.

Figure 3. Columns show progression of low to medium to high levels of texture-based biofeedback. Rows show customizations of effects: 1) Tunnel Vision (vines) over Up, 2) Tunnel Vision (veins) over Hulk, 3) Fractal Noise (mist) over Homecoming, 4) Fractal Noise (fire portal) over World of Warcraft.

Fractal Noise uses a noise texture to render semi-transparent textures. Multiple octaves of a noise texture (e.g., Perlin noise) are used for variation. The colour and the opacity can be controlled.

Waves fills the screen with drops that generate ripples. Generated with a 2D wave simulation, the resulting height field is rendered with specular lighting. The size, frequency, and coordinates of drops and the size and decay of the ripples can be controlled.

Static Sprite renders static 2D image sprites. The number, starting position (x, y coordinates), speed, acceleration, rotation speed, and size of the sprites can be controlled. In addition, particle system parameters for sprites can be specified to create visual representations such as explosions.

Animated Sprite renders animated 2D image sprites using a sprite sheet. The number, starting position, speed, acceleration, rotation, and size of the sprites can be controlled.

3.2.2 Game-related Texture Customizations Although our system can use any visual effect to provide biofeedback, we feel that the experience of biofeedback games might be improved if the graphical feedback is visually consistent

with the theme of the underlying game. For example, rain falling in a golfing game may contribute to a better gameplay experience than using that same effect in an ice hockey game, where the concept of rain inside an arena makes little sense. By visually customizing the effects, our system can appear to be integrated with the underlying game. However, the appearance of the effects not need be as literal as falling rain on a golf course to appear to be integrated with the game. Effects that relate to the theme, narrative, or world of the game may also contribute to a good experience. For example, a fiery portal that grows and shrinks to reveal the underlying display may be effective for use with a fantasy game, even though it has no literal meaning in the game.

Customizing the textures is a process can be done by a developer, but also could be done by an end user with no programming experience. Simply substituting a different image file or editing an image with standard image editing tools can customize existing effects. For example, this approach can be used to change an effect of mud splatters into slime splatters. We customized the five effects included in TextureLib to demonstrate a wide range of visual appearances (see Figures 3 and 4). In the following examples, we used each effect twice, with different graphical resources, to show how the appearance can be changed.

Figure 4. Columns show progression of low to medium to high levels of texture-based biofeedback. Rows show customizations of effects: 1) Waves (droplets) over Spearfishing, 2) Waves (frost) over NHL ‘11, 3) Animated Sprites (spiders) over Crysis, 4) Animated Sprites (particles) over NetRumble.

We used the Tunnel Vision effect to create vines, which grew over the jungle setting in an adventure game, in a fairly literal customization. We also used the effect to generate pulsing veins, which we deployed over a game based on the Incredible Hulk, in a less literal, but still thematically consistent visual representation. Fractal Noise was used to create a mist effect over a survival horror game, and a fiery portal over a fantasy roleplaying game. These effects are visually different, but require only a few changes to the graphical resources and parameters. We used the Waves effect to create water droplets over a spearfishing game in a literal customization of the effect. This effect was also used to create a frost effect, which we deployed over an ice hockey game. We used the Static Sprite effect to create cracks that appeared in the screen over a first-person shooter game with puzzle-solving aspects, where the world involves a lot of glass walls. This effect was also used to create mud splatter on the display in an outdoor racing game. Finally, we used the Animated Sprite effect to render spiders crawling over the screen during play of a first-person shooter game that takes place in a jungle setting, and particle explosions in a 2D space shooter game.

Some of the effects we created are fairly specific to certain games or environments (e.g., spiders on the screen), while others are

more generic (e.g., a flowing mist). Generic effects can be treated as a separate interface element, specific to the biofeedback system. In this case, the textures would not be customized to match the game being played; however, our system would still have the advantage of operating with off-the-shelf games. Casual games are designed to be quick to set up and fast to play, involving play times of only a few minutes. With a generic effect, a user could switch games a number of times within a single training session using our system; the visual effect would stay present between games, and the biofeedback experience would be seamless throughout the training session.

For more advanced users, parameter values of existing effects can be programmatically changed at run-time by a client application. For example, this can be used to change a mist effect into a smoke effect by changing the colour and opacity values. Also, developers can create new effects by implementing their own shaders.

3.3 Biofeedback Game Interface The biofeedback game interface gathers the user’s physiological state from the physiological sensing system (using SensorLib), and renders textures in the overlay (using TextureLib) corresponding to the user’s state. Any sensor in SensorLib can be

used to drive the biofeedback training, while any effect that can be created with TextureLib can be used as feedback. To use a sensor not currently available, SensorLib would need to integrate the sensor’s SDK, whereas creating a new visual effect does not require changing TextureLib, but rather using the built-in customization tools. To work with off-the-shelf games, our interface must be agnostic to the user’s underlying task, so the selection and launching of the game occur outside of our interface using the standard Windows interface.

We created a simple graphical user interface that allows users control over their biofeedback training. A commercial system designed for an end user would hide some features; whereas, one designed for a trained biofeedback clinician would include more control. Our interface was developed for use in a series of experiments, and resembles the features that would be presented to a clinician. Users can select which aspect of their physiology they wish to train (e.g., range of EEG), determine normalization values for the effect from a calibration procedure or accept values entered manually, preview and choose the effects and the obfuscation levels, and view the signal strength for the training hardware. In addition, we included an interface for saving log files for further processing throughout the evaluation of our system.

4. TEST DEPLOYMENT Because of the novelty of our approach, we were interested in answering three questions through a test deployment:

1. Does altering display graphics instead of game mechanics work as a biofeedback mechanism in games?

2. Do graphical overlays ruin the fun of playing games? 3. Can our system remain motivating over the long term needed

for successful biofeedback training?

We chose to evaluate the system with children with FASD, as fetal alcohol exposure is a prevalent cause of intellectual impairment in the western world [17], and children with FASD experience symptoms similar to those with ADHD [2]. Biofeedback training of brain function self-regulation (called neurofeedback (NF) training) using EEG has been effective at reducing the symptoms of ADHD [6,9]. Also, differences in EEG between children with ADHD and normative databases have been reduced with NF training. Specifically, those with ADHD exhibit higher power in the Theta band of EEG (related to decreased attention and less retention of material) and lower power in the low Beta band of EEG (related to increases in both hyperactivity and impulsivity) [19]. NF training helped children with ADHD lower the ratio of Theta/low Beta activity, by either lowering Theta activity or increasing low Beta activity [6,19].

We tested our system with 16 children (9 male) between the ages of 8 and 17 (median=11), who were all diagnosed with FASD, and played video games regularly. The children participated as part of a larger study on the effects of FASD interventions.

4.1 Biofeedback Training during Deployment Physiological Sensors. We used the Neurosky Mindset7, which is a single-electrode EEG device, as the input for our system. The Mindset was chosen for its simplicity of deployment, robustness of signal, SDK quality, and because it integrates headphones into the device. We modified the Mindset by moving the electrode from the forehead to EEG location Cz, on the top of the head, which produces a better signal for NF training [19]. The reference electrodes were positioned on the ear via the audio headphones.

Games. We provided a selection of games from which players could choose. Games were required to run in windowed mode, maximized to the screen (rather than full screen mode) to allow the overlays to display properly. In addition, games could not contain objectionable content, including violence, sexuality, or harsh language. Games were accessed via Steam, and initially included NBA 2K10, Osmos, World of Goo, Bejeweled Twist, and Blur. Part way through the deployment, we added additional games, including Capsized and Plants vs. Zombies.

Training. Users trained two or three times per week, in a research lab at the University of the Fraser Valley, with our system for 12 consecutive weeks between October 2011 and April 2012. Each session lasted about 60 minutes, with 30-45 minutes of game play. Users chose which games they wanted to play; however, all players used the mist effect at obfuscation levels set by the experimenters, based on a standard pre-play calibration procedure. Obfuscation levels were set at the beginning of each training session so that thresholds could adjust with the players over time.

Measures. We asked users to fill out a survey with questions related to the play experience after 12 weeks of training. All questions were answered using a 3-point scale (yes-maybe-no) suitable for the age of our users. In addition, we processed the log files for each training session. EEG data was logged every 500ms, and these data were aggregated over each session.

4.2 Results Does altering display graphics instead of game mechanics work as a biofeedback mechanism in games? Users generally agreed that they “wanted the mist to go away” (13-yes, 1-maybe, 1-no). Wanting to play without the obscuring textures is important to motivate the progression of NF training. Players also agreed that they were “able to control the mist to make it go away” (12-yes, 3-maybe). That players felt in control over the textures suggests that altering display graphics works as a biofeedback mechanism.

Users controlled the obfuscation of the mist with their Theta/low Beta ratio, because children with ADHD have been shown to have elevated ratios as compared to population norms [19]. Figure 5 presents average ratios for the beginning and end of the deployment. Because children participated at different times and for a different number of sessions, we performed a median split on the number of sessions and classified training as either beginning sessions or end sessions. Data for two children were removed due to the connection with the Neurosky mindset being below threshold (80%) for the majority of the sessions. Figure 5 shows that players were successful at lowering the ratio in the later training sessions as compared to the initial sessions. A paired-samples t-test supports that this difference in average theta/low beta ratio is significant (T13=2.16, p<0.05). We do not claim that our results demonstrate successful NF training; however, the lowered ratios do suggest that altering display graphics based on physiology has potential as a biofeedback mechanism.

Figure 5. Mean±SE Theta/low Beta ratios for the first half of the sessions as compared to the last half of sessions.

Do graphical overlays ruin the fun of playing games? Players agreed that “playing the biofeedback games was fun” (14-yes, 1-maybe). The NF textures did not fundamentally alter the gameplay experience so that it was no longer enjoyable, and fun games remained fun to play when used as part of our system. However, the kids also agreed that they “would have preferred to play the games without the headsets and the mist” (14-yes, 1-maybe). That kids wanted to play without NF is expected, and also fundamental to how the training system works; players need to prefer to play without obfuscation to help motivate training.

Players did not feel that “playing the games was challenging” (5-yes, 1-maybe, 9-no). It is good that the players did not feel excessively challenged by the games because the challenge of keeping the textures from obscuring the screens should be the focus of the NF training activity.

Can our system remain motivating over the long term needed for successful biofeedback training? Our test deployment shows that kids enjoyed playing the biofeedback games, even when asked at the end of the 12-week period. Thus our texture-based biofeedback system did not break the enjoyment of gameplay. In fact, after a number of training sessions, players became bored with the available selection of games and asked that we add more. Once we did, the kids were again happy to play. As long as a child is motivated to use the computer (i.e., through gameplay or other computer-based activities), our system will remain motivating.

5. DISCUSSION 5.1 Summary of Findings Our test deployment shows that kids enjoyed playing the biofeedback games, wanted the mist to go away, and felt able to control the mist to make it disappear. Our texture-based biofeedback system did not ruin the fun of games; rather it provided the opportunity for choice and variety so kids could maintain enjoyment. When players became bored with the available selection of games, we added more. By giving choices, we were able to retain the interest of the players over a long span of time (3 months). Had players tired of a traditional biofeedback game, with game mechanics that adjust to physiological state, there would have been no way to renew their interest in play, short of building a new game or new game levels. Our test deployment also showed that obfuscating overlays have potential as a feedback mechanism for biofeedback training; most users agreed that they were able to control the overlays. Also, the lowered ratios of Theta/low Beta activity in later play sessions suggest that our biofeedback training approach holds promise.

5.1.1 Interpretation of Results Our results indicate the potential of obscuring overlays as a biofeedback training system; we do not claim that our results show successful NF training for kids with FASD. The data gathered in our deployment are preliminary and representative of a small population. In addition, the EEG data used in our analyses was gathered during play from a consumer electronics device, not a multi-channel high-frequency EEG system with a standard electrode array. To determine whether our approach is successful in NF training, we are conducting a large-scale study involving pre- and post-testing using validated outcome measures.

5.1.2 Generalizing Beyond Casual Games We deployed our system alongside a selection of games from which players could choose. Limited primarily by technology

(must be played in windowed mode) and content (no objectionable content), we also were limited by the timing of the training sessions. Because users played every few days for half an hour, we offered casual games. Casual games are those that are easy to learn with limited instructions and simple rules and controls; they have short play times and allow users to put the game on hold, thus lending themselves well to our experiment protocol [22]. These types of games also lend themselves to our approach to biofeedback training, as the texture overlays do not present significant risk to players. For example, if a user lost a game of Bejeweled during training, she could simply start a new game. How our system would function with deep narrative-based games designed to immerse a player, which contain leveling tasks or boss battles (e.g., Mass Effect, Uncharted) remains to be seen. Whether our system could work with time-critical collaborative games (e.g., Team Fortress, World of Warcraft) also is unclear. That our textures do not break the ‘fun’ offered by the casual games tested in our deployment does not imply that the system will work for all games. Commercial games are engineered to provide a compelling and emotional experience for the player – our intention is to help users self-regulate. There is a tension between these goals that requires further investigation.

5.2 Presentation of the Biofeedback Textures Our textures were abstract representations of brain state. A user may understand that the opacity of flowing mist is indicative of their brain state; however, it is possible that, in some applications, concrete visual representations may prove superior. Numerical representations or progress bars are possible using TextureLib. Also, our textures were agnostic to the underlying interface of the user’s primary task. Although ours is a general solution that will operate with off-the-shelf software in a black-box manner, TextureLib can be aware of interface elements. For example, animated sprites could be directed toward a screen location, static sprites could reside over interface elements, or effects could follow the mouse cursor. Consider a dynamic icon that resides in a user’s system tray, or as part of a player’s interface in game. Connecting a texture-based visual representation to an on-screen or cursor location introduces new possibilities for biofeedback training without requiring access to underlying applications.

Some games allow limited access to the game state through SDKs (e.g., Valve Software’s (Portal, Team Fortress, Half-Life) Source Engine and SDK). With access to game state through an SDK, biofeedback games could be highly customized to gameplay, appearing to be an integrated solution presented by the developer. For example, different overlays could be presented at different locations (e.g., indoor / outdoor); overlays could be combined with in-game information (e.g., in-game health pack could reduce obfuscation); or overlays could follow in-game elements (e.g., obscuring text bubble of a non-player character conversation).

5.3 Interaction Beyond Games Although we focus on biofeedback games in this paper, our system is general enough to operate over most computer-based tasks. The premise of biofeedback games is that the desire to play motivates users to alter their physiological state in the preferred direction. For children who do not enjoy playing games, our system can also be used over web browsers, such as Firefox or Chrome, and thus function with any task that a user performs in a web browser (e.g., search, chat, social sites, media viewing). Alternatively, our system can be used with other off-the-shelf software (e.g., drawing application). Given that most consumer-

level biofeedback training systems are game-based, our approach creates possibilities for a new population of users by providing an opportunity for children uninterested in games to participate in biofeedback training. In addition, our system decouples the physiological sensing system from the activity being performed on the computer, so users can choose their training activity separately from their choice of which physiological trait to train. So a child can train, for example, their brain activity, or their anxiety, by playing a game or chatting online with friends.

There is still the opportunity for developers who wish to create integrated biofeedback training solutions – where the sensing is directly integrated with the application – to do so. Our system does not eliminate the prospect of integrated solutions; we simply provide a general-purpose solution that decouples sensing from activity, appealing to a broad range of tasks, domains, and users.

6. FUTURE WORK AND CONCLUSIONS Our research demonstrates a new approach to biofeedback training, where the physiological sensing is decoupled from the user’s primary task. We show that this approach has potential through a test deployment. To investigate whether our system can help children with FASD reduce their symptoms related to ADHD, we are conducting a large-scale study.

We used casual games as the player’s primary task and are presently exploring how the texture overlay system affects gameplay with more immersive games with narrative, time-critical tasks, and longer playtimes. As part of this exploration, we are focusing on how different presentations of the biofeedback display balance the competing (and perhaps mutually exclusive) desires for engaging gameplay and effective biofeedback training.

Biofeedback games have the potential to help children self-regulate their physiological function; neurofeedback games – those that help users self-regulate their brain function – hold promise for special populations, such as children with FASD. However, choice in biofeedback games is limited, they are difficult to create, and have little depth of play. We present a solution for turning off-the-shelf games into biofeedback games. Our texture-based overlay solution for decoupling the sensing from the game gives kids the choice of what aspect of their physiology to train and which game to play. Our approach provides biofeedback training to children previously uninterested or unable, and creates enough choice in interaction to support the long-term and repeated use that is necessary for success.

7. ACKNOWLEDGEMENTS Thanks to NSERC and the GRAND and NeuroDevNet NCEs for funding. Thanks to Bassam Khaleel and the UofS Interaction Lab.

8. REFERENCES [1] Bersak, D., Mcdarby, G., et al. Intelligent biofeedback using

an immersive competitive environment. Proc. of UBICOMP 2001 Workshop on Ubiquitous Gaming (2001).

[2] Bhatara V, Loudenberg R, & Ellis R. (2006). Association of attention deficit hyper-activity disorder and gestational alcohol exposure: An exploratory study. Journal of Attention Disorders, 9, 515–522.

[3] Clarren, S., & Lutke, J. (2008). Building clinical capacity for Fetal Alcohol Spectrum Disorder diagnoses in Western and Northern Canada. Canadian Journal of Clinical Pharmacology, 15(2), e223-e237.

[4] Dekker, A. & Champion, E. Please Biofeed the Zombies: Enhancing the Gameplay and Display of a Horror Game Using Biofeedback. Proc. of DiGRA'07, 550-558.

[5] Gilleade, K., Dix, A., & Allanson, J. Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me. Proc. DiGRA'05, 2005.

[6] Heinrich, H., Gevensleben, H., & Strehl, U. Annotation: Neurofeedback — Train your brain to train behaviour. J. Child Psychology and Psychiatry, 48, (2007), 3!16.

[7] Herndon, C., Decambre, M., & McKenna, P. Interactive computer games for treatment of pelvic floor dysfunction. The Journal of Urology, 166, (2001), 1893–1898.

[8] Hjelm, S. Research + design: the making of Brainball. interactions 10, 1 (2003), 26-34.

[9] Lubar, J.F., Swartwood, M.O., et al. Evaluation of the effectiveness of EEG neurofeedback training for ADHD in a clinical setting as measured by changes in T.O.V.A. scores, behavioral ratings, and WISC-R performance. Biofeedback and Self Regulation, 20, (1995), 83–99.

[10] May, P., & Gossage, P. (2001). Estimating the prevalence of Fetal Alcohol Syndrome: A summary. Alcohol Research & Health, 25(3), 159-167.

[11] Nacke, L.E., Kalyn, M., et al. Biofeedback game design: using direct and indirect physiological control to enhance game interaction. Proc. CHI '11, 103-112.

[12] Nijholt, A., Plass-Oude Bos, D., & Reuderink, B. Turning shortcomings into challenges: Brain-computer interfaces for games. Ent. Computing 1, 2 (2009), 85-9.

[13] Picard, R.W. Affective Computing, MIT Press, Cambridge, 1997.

[14] Pope, A.T. & Palsson, O.S. Helping video games ‘‘rewire our minds”, NASA TR (2001).

[15] Salen, K. & Zimmerman, E. Rules of play: Game design fundamentals. MIT Press, Cambridge, MA, 2003.

[16] Sakurazawa, S., Yoshida, N., Munekata, N. et al. A computer game using galvanic skin response, Proc. ICEC’03, Pittsburgh, USA, 2003.

[17] Spohr, H.L., Willms, J., and Steinhausen, H.C. Prenatal alcohol exposure and long- term developmental consequences. The Lancet, 341, (1993), 907-910.

[18] Stach, T., Graham, T.C.N., et al.. Heart rate control of exercise video games. Proc. GI '09, (2009), 125-132.

[19] Thompson, M. and Thompson L. The Neurofeedback Book: An Introduction to Basic Concepts in Applied Psychophysiology. The Association for Applied Psychophysiology and Biofeedback, Colorado, USA, 2003.

[20] Thompson, L., Thompson, M., & Reid, A. Neurofeedback outcomes in clients with Asperger's Syndrome, App. Psychophysiology Biofeedback, 35, (2010), 63-81.

[21] Vernon, D., Egner, T., Cooper, N., et al. The effect of training distinct neurofeedback protocols on aspects of cognitive performance. International Journal of Psychophysiology, 47, (2003), 75–85.

[22] Casual Game White Papers | IGDA. International Game Developers Association (2008).