mediated input

8/3/2019 Mediated Input

1/18

Mediated Input

Chris DeLeon Jan 2012 Expanded from a paper originally prepared for Dr. Ian Bogosts Philosophy of Sportclass

Simulation Test

To test the accuracy and realism of iRacing1 (Motorsport Simulations 2008), the

creators of the television show Top Gearcontacted the worlds best iRacing

driver, Greger Huttu, and put him behind the wheel of a real racecar in Atlanta,

Georgia [Read 2010]. Gregers lack of physical preparation to tolerate the G-

forces in real racing prematurely ended the event, leading him to give up 15 laps

after he first had to stop due to throwing up inside his helmet. Otherwise, hisperformance was quite good the racing simulation held up well under scrutiny.

As Top Gearreported in their online piece, The telemetry confirms it. His braking

points are spot on. He's firm and precise on the throttle. And in the fastest corner,

he's entering at 100 mph compared to an experienced driver's 110 - a sign of

absolute confidence and natural feel for grip.

Aside from the need to physically tolerate g-forces, driving a racecar mostly

consists of managing the cars input devices steering wheel, accelerator,

brakes, and shifter to determine and correct for motions of the car on the track.

Further complication arises through the need to account for the positions of othercars, if present, and on longer races managing pit stops by pacing engine heat,

fuel consumption, and tire wear. Switching between input devices from

videogame peripherals to the real mechanical interface, as Greger experienced

on Top Gear, requires the user to acclimate a bit, and to recalibrate the mapping

of their intentions to car movements. However the input provided in this case is

still the same fundamental kind and complexity, serving to translate and amplify

driver intention into changes in vehicle behavior. Moreover, both input and

feedback of that inputs effects are fluid and continuous, making it possible to

apply real-time corrections: if the driver means to apply a little more force right, or

a little more gas or brake, compared to what they witness their current level of

input accomplishing, that correction can occur subconsciously within a fraction of

a second [Swink 2009; Ericsson 1996].

1 As a point of clarification, despite the i at the start of the title, iRacingis

software developed for Windows PC, not for the Apple iPhone or other iOS

devices.


2/18

Why not use a similar approach to take the worlds best player of Madden NFL

12(EA 2011) and stick them into an American football game against NFL

athletes? To avoid the possibility of failing due to poor physical conditioning, or

wrongly matched body type, imagine for sake of argument that our extraordinary

Maddenplayer just happens to be a large, strong adult male in peak condition.Perhaps this player is a Special Forces soldier recently returning to civilian life, a

professional athlete from another rough sport, or (purely for illustration) someone

with Olympic gold medals in both weight lifting and running events. The critical

distinction in our case is that this otherwise athletic individual has no prior

practice playing American football. It seems very likely that even if the persons

heart is in great shape, the persons muscles are properly conditioned, and the

players Maddenskills are completely unmatched by others, he would

nevertheless have next to no ability in the context of professional football. He

would need to throw with precision, catch with ease, and manage rapid changes

to his bodys movement on the turf without hesitation.

In Gregers case with the racecar, he developed skills around feeding the right

input and reactions to his computer peripherals. Top Gearsimply asked him to

adapt from manipulating the controls he knew to manipulating the controls inside

an actual car. In both cases, the interface mediatesthe players input,

unambiguously and consistently transforming minimal expression of intention into

mechanical realization of that action. Much of the skill in driving a racecar well

comes down to being able, in real-time, to appropriately judge how much to

accelerate, brake, shift gears, and redirect the tires to follow ideal lines and pass

others, but at a mechanical level its the car (or simulated car) that sees to thelow-level execution of those intentions.

People that dont understand NASCAR, mocking it for example as hours of

turning left, harbor an incorrect comparison between stock car racing and their

casual experiences with foot races divided into lanes. This parallel mistakenly

imagines the difficulty in racing as the challenge of moving fast, which in

automotive competitions gets somewhat trivialized by powerful engines. Further,

regulations in stock car racing specifically require similarity between cars

[NASCAR 2004]. That trivialization and enforced consistency of mechanical

aspects, combined with the lack of clear lane divisions, elevates auto racing to a

more mental competition. Because every car on the track has comparable abilityto efficiently accelerate, brake, and steer, the advantage instead comes down to

superior timing, control, and decisions (fuel usage, when/how to pass, and so

on), rather than competing over simply trying to move faster.

By comparison, being successful in a sport requires evaluating opportunities at

eye level (Maddenprovides a strategic, elevated view), bracing for imminent risk

2


3/18

of serious impact, and throwing an accurate pass to a moving target over great

distance. These activities happen every play, and require a mastery of practiced,

split-second coordination to take action based on the innumerable variables in

personal and peer movement. Even sports with less contact still place strain on

the limits of perception and attention, requiring athletes to remain ready for

rapidly changing goals, for example how a baseball pitcher needs to watch forpossible steals [Gumbrecht 2006]. In a videogame the pitcher can instantly throw

straight to any base with a button press or two. A live athlete risks missing that

throw if its rushed faster than they can manage. Players spend decades drilling

these skills not just the timing of whento apply them, but also the subtle tacit

details involved in howto apply them to better perform certain mechanically

complex tasks under pressure, and to better estimate their unique abilities in

various circumstances. Control isnt mediated in real football or baseball, isolated

from the mechanical realization of intention; intention is inseparably tied up with

ever-changing personal and situational limitations on execution. The aspects of

auto racing handled by the car are demanded instead of an athletes body inconventional sports: translating intention to accelerate and maneuver into

realized action becomes different for every player, a function of subtle difference

in bodily preparation and tacit knowledge.

Transplanting a Maddenplayer onto a football field is moving them from a

domain with mediated input which boils down a long-range, accurate pass into

a button press to a situation without mediated input, in which that skilled action

needs to be honed by years of practice.

Thinking that a players ability in a sports videogamecould transfer to real bodily

athletics is no stranger than expecting anything done with videogame controller

to prepare someones body to execute a reverse group (diving), a triple axel

(figure skating), or a front tuck (gymnastics). These actions, no less than the

skills required for playing football or baseball, require practiced full body

coordination, handling disorientation, control over reflex, and the automatic,

subconscious orchestration of hundreds of muscles over fractions of a second.

Videogames utilize mediated input to simplify away nearly every complexity

involved in an actions execution except the timing, which the player is

responsible for managing via button presses and analog inputs.

In my high school years as a folkstyle wrestler, and my experiences with boxingand fencing in college, I gained an appreciation for a category of skills that I

never needed for Street Fighter II(Capcom 1991): drilling moves to suppress

counterproductive but natural reflexes weve evolved for being hit in the face,

violently rushed, or deliberately though briefly entering dangerous situations in an

attempt to achieve surprise advantage. Nothing I have ever experienced in a

videogame came close to that type of sensation; I could watch my virtual

3


4/18

character on-screen get punched, kicked, and thrown all day. Children playing

baseball have to learn to suppress the reflex to hide from an incoming ball.

Theres another side to this, too: gaining a feel for the level of physicality allowed

in a game like basketball or hockey before it might result in a penalty for being

excessive. Mediated input makes that line easier to not cross in a videogame.

Mastering when to press buttons in Fight Night Champion(EA 2011) is not

preparing anyone to throw or take a good punch. Carefully timed mouse press

and release in the classic Links 386 Pro(Access Software 1992)golf simulation

games cannot convey the countless details critical to a successful golf swing in

real life; at best the observation of on-screen animations may offer some basic

information about what the end result should look like, but its obvious enough

that seeing someone make a shot in basketball or hit a homerun in baseball can

hardly prepare someone to succeed at the task.

And yet, flight simulators work so well that theyre often integrated into thetraining of both civilian and military pilots [Aldrich 2005]. As is the case for

racecar driving, piloting a real aircraft also happens via mediated input. Thus,

going from simulation to real flight is a case of shifting from mediated input to

mediated input, rather than changing from mediated input to an activity where the

players body then has to perform different, complex, coordinated actions.

Thrown into an athletic activity after practicing only with a mediated one, the

player may have a clear idea of what to do and when, but that cannot help stir

the body to do the precise and difficult moves necessary to realize those

intentions. The players acquired sense for what to do and when could

unproductively feed those intentions into the next step of performance which, in

the unmediated case, replaces automatic execution with a need for tacit skill to

transform those intentions into action. Mediated input handles that last step for

players, which is a convenient way to bypass a decade of practicing

fundamentals in order to instead focus on the game at another level, but its not a

substitute for practice.

Separating Execution from Strategy

To further illustrate the effects of mediating input, well next explore the dart

game Tic-Tac-Toe [A1Darts.com 2011]. Players throw darts into different zones

on the board to earn Xs or Os in corresponding tic-tac-toe positions:

4


5/18

In Tic-Tac-Toe darts, players take turns throwing darts at the areas indicated

on the left. After earning 3 points for any one area of the dartboard, that player

can place their mark on the tic-tac-toe grid. Hitting multiplier rings scores 2 or

3 points at a time, earning positions faster. The center square, E, is scored by

hitting the bullseye. Once a grid position is claimed it is permanent.

Traditional tic-tac-toe, played on paper without any need for darts, is a solved

game. Solved, in this context, means that all possible moves have been

considered, leading to the discovery that optimal play by both sides will either

always end in a draw, favor whichever player moves first, or favor the player that

moves second. For example computer scientists solved the game of checkers

after 16 solid years of distributed calculations [Sreedhar 2007]; given the

substantially smaller space of possibilities in tic-tac-toe its common for people to

independently figure out, before adulthood, that players can always force a draw.

What gives depth and uncertainty to the darts version of tic-tac-toe is the difficulty

and bodily coordination necessary to consistently throw a dart to any intended

spot. Subtle variations in how a dart is held, when the dart is released, and how

the arm moves for the throw can make the difference between landing in different

scoring sections, even at a very high level of player ability. Given the certainty of

at least some deviation from the intended destination point, players need to aim

for a point based not only on what will happen if the dart flies true, but also withconsideration for what will happen if they miss by some approximate amount in

any given direction [Tibshirani et al. 2011].

If both players somehow possessed superhuman skill at dart throwing, such that

darts would always hit precisely the intended spot on the board, then dart Tic-

Tac-Toe would becomes no different than traditional tic-tac-toe as played on

paper. More realistically, if one of the two players is much better than the other at

accurately throwing darts, then theres a good chance they will simply dominate

the game, without much need for clever strategy. In that case the winner is

effectively decided by a variation of tic-tac-toe in which one side can occasionallytake multiple turns in a row.

The more reliably and evenly the participants can execute bodily maneuvers with

consistency and accuracy, the more strategic the game becomes, leading to loss

or failure being due to the decisions made, more so than the failure to execute on

those plans. Conversely, the less capable both sides are of reliably executing

those maneuvers, the more the outcome degrades into pure chance, going to

5


6/18

whichever player happens to first hit targets that work out in their favor.

This distinction partly explains why, for athletic competitions with room for

strategy, professional athletes are more exciting to spectators: besides

everything simply happening on a more impressive scale, strategy becomes

relevant at that level which non-expert athletes lack the ability necessary torealize. Young children playing American football are so unlikely to catch a pass

that it would be fruitless to expend much energy planning out elaborate

strategies; like two horrible dart players giving the Tic-Tac-Toe variation a try, the

game will likely favor whichever team gets lucky the most times regarding the

successful completion of even the most basic actions. Or from another angle, the

effective range at which a youth pass can be completed severely limits the

variety in strategies possible. For professional football, since the players are

exceptionally adept at the games fundamentals, the number of strategies that

can realistically be executed greatly expands. It becomes more about the tic-tac-

toe board, rather than who can simply throw the dart better.

Because football takes place in continuous time and space, as opposed to turns

on a board of discrete positions, it doesnt really make sense to consider whether

football at a strategic level can be absolutely solved in the same way that tic-tac-

toe and checkers have been solved. The absurdity of imagining players capable

of running any speed, throwing any distance, catching any pass, kicking field

goals from any range, and plowing clean through the offensive line or,

alternatively, halting every defensive linemen, reveals that while relevant

strategies can be increased by higher levels of ability, the strategic part of the

game cannot be as cleanly separated out as tic-tac-toe can be from dart Tic-Tac-

Toe. Short of such physiologically impossible levels of perfection, there is always

room for advantage by outperforming the other team with superior fundamentals,

diluting the significance of the other teams strategic choices by utilizing

maneuvers that they cannot keep up with.

This seemingly boundless potential for game-relevant increase in ability

highlights another quality elevating professional sports for spectators. Recall that

in dart Tic-Tac-Toe, a significant gap in player ability to execute intention could

overwhelm any strategy by the other player by simply giving one side more turns

at the tic-tac-toe board. In the same way, an especially talented player at the

height of their career makes their significance as a statistical outlier even moreevident by their ability to simply outperform whatever strategy the other player or

team may attempt. The achievements of Andre Agassi [Wallace 2006], Michael

Jordon, Mike Tyson or other sports legends stand out as the very best in the

world because unusually often it seems as though their opponents are simply not

capable of any strategy that could make up for their gap in sheer ability.

6


7/18

Snap Judgment in Real-Time Gameplay

One significant gameplay difference between dart Tic-Tac-Toe or ordinary golf

versus activities such as soccer or racing is that these latter sports enable the

competitors to perform simultaneously, in immediate opposition, rather than in

discrete turns. This type of distinction isnt as clean a split as it first appears, with

sports like baseball and football showing hybrid examples that mix turn taking

with real-time response, but in such cases its still possible to study the real-time

response portion separately from the turn-based structure in which it takes place.

Anyone literate in how a particular sport is played could watch a replay in slow

motion and point out what the losing player ought to have done differently - the

batter should have swung sooner, the basketball player should have thrown their

shot a little harder, and so on. The difficulty isnt necessarily a problem with

making the right decisions, but is instead due to scarcely having time to make

decisions at all, or in failure of practiced actions to consistently achieve intention.

This time element is a constraint with cognitive and motor limitations, but

practically speaking can become competitive at increasingly small intervals

seemingly without bound. When reflex appears to exceed the minimum time

separating perception from motor action, its accomplished by a combination of

predictive estimation and chance [Swink 2009; Polin and Rain 1979]. One simple

example of this is to picture a major league batter determining and timing their

swing based on the pitchers leading movements, rather than trying to swing

when the ball is where theyd like to hit it (at which point it would obviously be too

late). The illusion of professional athletes simply having superior reaction timehas been shed by research revealing no significant advantage in reaction time

outside of their particular domain [Ericsson 2009]. Experience has trained these

athletes to be able to read and properly respond to earlier stimuli, creating an

advantage functionally equivalent to responding faster and more accurately2.

2

Earlier work by Keith Anders Ericsson investigated this concept indirectly bystudying how experience level affects SWAT member responses to unfolding

scenarios [Foer 2011]. More experienced SWAT officers perceived, diagnosed,

and properly handled situations in response to earlier warning signals that

inexperienced officers overlooked until it was too late. The effect of mastery on

timeliness was not on account of superior reaction time at a motor level, but from

being able to identify earlier signs with greater certainty, initiating the appropriate

reaction without hesitation.

7


8/18

Probabilities in Personal Limitations

The most obvious way that players improve the consistency and timing of their

fundamentals is through drilling and play experience. Depending on the sport,

these might include taking shots, catching, passing, dribbling, skating, running, or

any of dozens of other actions. When athletes perform these actions during a

game of simultaneous or direct competition, they do so under intense time

pressure. The player needs to execute without thinking about the maneuvers,

trusting their drills and experience to produce the motions needed for their

desired result. Hockey and soccer players need to be able to concentrate on

what part of a goal they ought to shooting for, not on how to get their shot to go

where they intend. Along the earlier example, this would translate to playing darts

masterfully enough to focus on the tic-tac-toe aspects of the game.

But, like dart throwing, and especially with the potential for one-upmanship in

timing, those skills will never be absolutely perfect. Athletes need to accuratelyunderstand and account for their own particular degrees of imperfection. For the

soccer player figuring out which part of the goal to shoot for, theres not a fixed

probability of scoring regardless of where the player aims. The player in control

of the ball cannot simple aim for whichever area of the goal is farthest from the

opposing goalie, but must also account for the risk of missing the net if they

overestimate their own accuracy under the circumstances.

Alternatively, whether a basketball player should take a shot from any particular

position at any given time is not only a factor of the angles, and the other teams

current activity, but also of the shooters confidence (hopefully roughly fittingreality) that they personally have a good chance of making the available shot. Or

and this is where strategy comes into play the player has to believe that the

risk of taking a given shot is justified by the likelihood of earning a certain number

of points, weighed against the other opportunities that play.

Every athlete has different capabilities when it comes to accomplishing various

fundamentals. Buzzer shots aside, many sports have negative consequences for

overreaching, most frequently in the form of giving up (or greatly risking giving

up) possession to the other team upon missing a shot, missing a pass, losing

control of a puck while skating, or tripping over a soccer ball.

This type of concern doesnt happen in tic-tac-toe, nor in games like blackjack or

chess, because in these types of games the player intentions fully account for the

fundamental operations in those games. Arguably the execution in such games

reflects mediated input in the most extreme form of the idea, since no matter

whether the player is moving a piece via mouse, by typing in g1 f3, using their

hands, or having someone else make the actual board movements on their

8


9/18

behalf, the essence of these games is kept focused on the activity of pure

decision making. No one loses at chess from meaning to move their piece from

g1 to f3, but messing up that movement and thereby landing partly or completely

on other tiles, in the way that someone might miss a shot taken in basketball.

Timing in Mediated Input

Mediated input, whether driving a car, playing a real-time videogame, or playing

pinball, falls into a strange place in the middle between coordinated athletic

execution and a purely mental game. It removes the rich complexity of athletic

motion, though so long as the game in question involves response to real-time

stimuli, the timing of the maneuver itself and the decision of which maneuver is

appropriate are still relevant elements of skill.

The batting portion of baseball is a good place to highlight this distinction. In itsfull physical form, hitting the ball requires the player to both correctly time and

correctly execute a particular motion. The combination of these factors in relation

to the current pitch will determine the success of that swing, and making too

severe an error along either dimension can render the other irrelevant. The ideal

swing executed with improper timing is just as much a problem as an improper

swing with ideal timing.

From left to right: (1) real batting, in which the player is responsible for both

swing movement and timing; (2) T-ball, in which the player is responsible for

swing movement, but not for timing the player can miss the ball, but its

impossible to swing too early or late; (3) virtual baseball using mediated input,

pre-motion control era, in which the player is responsible for timing, but not

the swing movement the player can swing too early or too late, but with

proper timing its virtually impossible to miss a ball thrown through the strike

zone. Image sources in bibliography.

In addition to timing the bat swing, many videogames provided a little extra input

fidelity. In the videogame shown above, Ken Griffey Jr. Major League Baseball

9


10/18

(Software Creations 1994), the player at bat can use the directional pad to slide

around within the batters box to better line up the sweet spot of the bat with

where the batter predicts the pitchers throw will pass. And, as in real baseball

(but unlike T-ball) swinging a little too early or too late can cause the ball to veer

left or right, resulting in a foul ball if overly so. However unlike real baseball and

T-ball, in virtual baseball with mediated input the batters swing is exactlyreplicated every time the swing button is pressed. This always puts the mass of

the bat at the same distance from the player every swing, and at precisely the

same lapse in time (matching the batting animation) relative to each press.

Returning to the earlier idea about why a racing or flight simulator can achieve

real results, whereas simulators of sports are unlikely to play any real role in

training, the mediation of input is the most important difference here, not the

videogames virtual, digital nature. Evidence supporting this claim can be seen in

electromechanical novelty games that were designed to crudely simulate the

basic mechanics of batting in baseball.

This is a flyer for Upper Deck, a 1973 electromechanical game by Williams.This is one of many novelty games utilizing a pinball-like flipper action to

mediate input, involving player skill similar to pre-motion control virtual

baseball games: timing only, without responsibility for motion of the swing.

Pressing the left button releases a pinball from the pitchers mound, pressing

the right button powers a solenoid assembly beneath the playfield to cause

the bat-like flipper on the surface to swing. Image source in bibliography.

10


11/18

Another sign of this separation between timing and execution is that while we

might find the athletic execution of a real batter noteworthy and beautiful, to be

impressed at a game with mediated input we would have to be impressed

instead at the overall play, since isolated events like batting or layups just

happen as canned, consistent animations played in response to the users input.

Whether by designer intention or as an economically reinforced coincidence, a

major advantage to gameplay based on button mediated input rather than

athletic action comes in the form of broad and immediate accessibility. When it

comes to pressing a button to produce an action, there is no preference for body

type, no requirement to have a particularly fit body, and it virtually eliminates

concerns for injury. By comparison, many sports require or strongly favor a

particular type of body [Markovits et. al. 2010], and can be a common source of

serious injuries [Paolantonio 2008]. Lastly, by using mediated input the player

does not need to be shown how to correctly perform whatever task is needed,

they only need to figure out when to make the task happen; because the actionhappens consistently upon the press of a button, it becomes impossible for the

player to execute the task incorrectly, only to time the execution incorrectly.

Tools

As argued earlier, simulations of activities that require mediated input for the real

tasks, as in driving simulators and flight simulators, can have much more in

common at a skill level with the activity itself. But, someone in disagreement

might wonder, why doesnt a baseball bat, golf club, or hockey stick qualify asmediating those real activities that I claim rely upon bodily coordination? Are

racecars and jets really so different from tennis racquets and fencing blades?

Although all of these objects are alike in extending or amplifying the players

activities, what the simpler tools fail to capture in terms of mediating the players

input is the consistent and automatic translation between minimal expression of

intention and the consistent, automatic performance of the fundamental act.

When directly handling a bat, golf club, hockey stick, racquet, or fencing blade, a

slight twist of the wrist or shift in balance or tightness of the grip may be the

difference between successful and failed execution. The central quality

distinguishing mediated input from athletic activity is the use of controls to render

irrelevant any subtlety in execution besides timing (and in more complex cases

for analog controls, degree and/or an isolated angle).

11


12/18

Skill in Mediated Input

For an exploration of the many ways that skill factors into mediated input, please

refer to my HobbyGameDev.com blog entry onSkill in Mediated Input3 [DeLeon

2011]. Though originally covered in this section of the paper, for this extended

version I expanded the topic into a standalone write-up due to its relative utility

and conceptual separation.

Practical Application: Design for Motion Control

Whereas natural mappings were previously only relevant to directional inputs

left meant left, and up meant up - motion controls offered a way for action inputs

to be naturally mapped, too. With traditionally mediated input, Y button might

signal a players intention to swing the in-game bat. With motion control, the

player swinging an imaginary bat became the signal to swing the in-game bat.

Motion control, as provided in various ways by the Wii Remote, PlayStation

Move, Microsoft Kinect, and smartphones, on the surface seems to do away with

mediated input. A more thorough inspection reveals a more complicated

relationship between motion controls and mediated input, providing along the

way clues about which contexts and approaches work better for motion control.

Before delving into that question, I need to acknowledge that there are several

ways that motion controllers are used other than motion control in the sense of

acting out player actions. Several strategies sidestep the main challenges andbenefits of motion control by using the devices instead for more traditional

mediated input. When a Wii remotes IR camera is pointed at the TV screen to

control a reticle or cursor, serving as a hybrid light-gun mouse, that usage

constitutes mediated input by giving the user a way to efficiently express

intention through natural mapping. When an iPhones or Wii remotes

accelerometers are used to gauge orientation while otherwise stationary in

space, as when tilting to provide angular input, those rotational inputs are really

no different than a steering wheel or an old Atari 2600 paddle dial.

With regard to motion control in its widely hyped variety, that being the type inwhich players act out their intentions, detecting the action at all requires

substantial enough movement to be distinguished from unintended noise data.

Since casually reorienting and repositioning accelerometers in space cause

some minor turbulent readings mostly on the order of 1G, on account of being

held still against Earths gravity, the most trivial checks are whether the total

3 http://www.hobbygamedev.com/adv/skill-in-mediated-input/

12
http://www.hobbygamedev.com/adv/skill-in-mediated-input/http://www.hobbygamedev.com/adv/skill-in-mediated-input/


13/18

acceleration is much different than 1G, signifying a rapid movement.

Unambiguously recognizing a motion takes longer than pressing a button. Full

recognition would require waiting until the end of an action gets signaled by lack

of further change or by returning to a neutral position. Although an action can

often be predicted partway through, logically the certainty of such predictions areroughly proportional to how much of the gesture has already been performed.

Rather than waiting until an action is completed, which might shatter the illusion

of connection between the player and on-screen response, interpretation typically

occurs midway through, assuming (ignoring) motions that follows4.

Its difficult to push an action button differently than intended. On the other hand

there are countless ways to act out a swing, a throw, or any other action that

might be required by a game. Subtle differences in orientation can yield

significant deviations in accelerometer or Kinect data from motions that a human

might otherwise perceive as roughly similar. In both cases a wide range ofaccidental difference have to be accounted for. Because that range is largely a

product of unintended differences, this wide range is often handled as equivalent,

possibly with deviation on account of user assistance combined with minor

randomization. Even with a range of gestures conveying the same intention to

the game, there are nevertheless other ways to perform actions that will not

register the intention. The motion controls in this case become like a button,

albeit one that takes the player longer to press and which may fail to respond.

One way to eliminate that problem of inconsistent triggering is for the software to

treat every action measurable by the hardware as the same intention. The only

way to make this possible is for motion control to signify only one particular type

of action in any given context. To someone unfamiliar with motion control, this

may sound like a baffling limitation; to readers that have played motion control

games, its likely obvious by this point that what I am describing largely applies to

Wii Sports(Nintendo 2006) and many other motion control games.

To make up for action recognition taking longer than a button press, time-

sensitive response needs to be eliminated or telegraphed. Action needs to either

be initiated by the user (as in golfing, bowling) or timed in response to stimulus

that gives ample warning (as with the slow pitches and floaty hits in Wii Sports

baseball and tennis respectively). Boxing works due to the symmetry of inputdelays and inconsistencies between both players.

4 For context: my specific experiences with motion control development are from

working on Boom Blox(EA, 2008) on Wii, Topple(ngmoco, 2008) and feelforit

(DeLeon, 2010) on iPhone, and Kinect programming for the Digital Improv project

at Georgia Tech. Beyond that my understanding is based on observation and

experience playing other games using these devices for motion control.

13


14/18

While working within those constraints worked incredibly well for Wii Sports, it

also helps highlight why motion controls have widely proved problematic for

many other types of videogames. The lag necessary to identify action has made

motion control unsuitable for quick-reflex games. The need to treat every motion

as effectively the same motion had meant that motion controls really only workwell for games that would use motion to only mean one action, done one way,

which doesnt map well onto games that depend upon how long a button or

direction gets held to have certain significance.

Motion controls shine when, rather than being interpreted to signify a discrete

action, a range of success becomes possible depending upon how the action is

carried out. When the quality of the outcome varies on a complex continuum, it

becomes more like an athletic action than a case of mediated input.

Wii Sportsdoes incorporate limited range of this sort, within which actions can bedone better or worse. For example, rotating the remote unintentionally during the

bowling or golf swinging results in impaired accuracy. Though these

discrepancies are not treated with a simulation-level of accuracy, this

demonstrates one form of unmediated input (players arm twist during swing,

dealt with as such) mapping roughly to another (arm twist for the action). The Wii

MotionPlus adapter included with Wii Sports Resort(Nintendo, 2009) enabled

that game to come closer to a 1:1 mapping for movements, creating a richer

range for actions to be performed well or poorly, coming somewhat closer to the

athletic-complexity5.

Dodge ball games work well on Kinect because the positions of the users limbs

are the meaningful input, directly. Body positions are not interpreted as a gesture

to initiate some action. Kinect dancing games have likewise likely fared well on

account of their ability to gauge success by degrees, by comparison to full body

position.

By contrast, games using motion control to initiate a discrete action the same

way every time, rather than using that range of input fidelity, may be better

served with buttons than motion controls. A concrete example of this

phenomenon occurred when Zelda: Twilight Princess(Nintendo 2006) came out

with nearly identical ports for both GameCube (traditional button controller) andWii (motion control). A distinction made by players and reviewers comparing the

two versions is that whereas the player could rapidly initiate an attack with a

button on the GameCube, performing the same attack on Wii was delayed by

5 These titles are obviously not meant to be accurate simulations. Full-body

movements get simplified into hand gestures. Athleticism and conditioning also

remain completely separated from the experience, as is likely intended.

14


15/18

needing to waggle the remote [Zing 2009].

To be clear: these design constraints are not a problem of the software or

hardware improperly accounting for the data. These issues are inherent in

gesture recognition, and often result in it being little more than a silly-looking but

otherwise inferior form of mediated input6.

Consumer response after the release of Wii and Kinect has at times incorrectly

assumed an industry-wide learning curve akin to graphics on each console

generation, as though developers just had to figure out how to use the hardware

to its fullest advantage. The difference is that in the case of motion controls, what

motion controls are good for was figured out very early, after which weve seen a

mixture of repeats, games that utilize the motion control devices to provide input

unrelated to motion control (light gun, mouse, D-pad, waggle stick), and games

being released in genres that are deeply incompatible with the particular

strengths and weaknesses of motion controls but nevertheless get madebecause they appeal to consumers by their recognizability.

Closing

Ive attempted here to outline a number of distinctions about the nature of

mediated input. The elevated role of precision timing and the lowered importance

of coordinated muscular action are essential to the core gameplay of many real-

time videogames dating back to the early classics. Mediated input qualitatively

differentiates these types of play experiences from traditional games and sportactivities. Lastly, contrasts were drawn to contemporary motion control,

illustrating how concepts introduced and discussed elsewhere in this paper can

be of use in discourse about more modern and varied forms of input.

6 The entertainment value of silly-looking should not be underestimated.

15


16/18

Bibliography

"Actions per Minute - StarCraft and StarCraft II Wiki." StarCraft Wiki. Web. 01 Jan. 2012.

Aldrich, Clark. Learning by Doing: a Comprehensive Guide to Simulations, Computer

Games, and Pedagogy in E-learning and Other Educational Experiences. San

Francisco: Pfeiffer, 2005. Print.

Best, David. Philosophy and Human Movement. London: G. Allen & Unwin, 1978. Print.

Bogost, Ian. "Lockdown: A School Shooting Game." Ian Bogost. Water Cooler Games,

17 July 2008. Web. 16 Dec. 2011.

Connor, Steven. A Philosophy of Sport. London, England: Reaktion, 2011. Print.

Crawford, Garry. Consuming Sport: Fans, Sport and Culture. London: Routledge, 2004.

Print.

DeLeon, Chris. Skill in Mediated Input. HobbyGameeDev.com, 30Dec. 2011. Web. 15

Jan. 2012.

Eichberg, Henning. Bodily Democracy: towards a Philosophy of Sport for All. London:

Routledge, 2011. Print.

Elias, Norbert, and Eric Dunning. Quest for Excitement: Sport and Leisure in the

Civilising Process. Dublin: University College Dublin, 2008. Print.

Elwin, Keith. Pinball 101. pinballvideo.com, 2009. DVD.

Ericsson, Karl Anders. The Road to Excellence: the Acquisition of Expert Performance

in the Arts and Sciences, Sports, and Games. Mahwah, NJ: Lawrence Erlbaum

Associates, 1996. Print.

Foer, Joshua. Moonwalking with Einstein: the Art and Science of Remembering

Everything. New York: Penguin, 2011. Print.

Frideswag. Baseball batter. Photo. Baseball Hitting A Beginner Basic Need | Sports |

Games | Adventure. 6 Oct. 2011. Dec. 14, 2011.

Gee, James Paul. What Video Games Have to Teach Us About Learning and Literacy.

New York: Palgrave Macmillan, 2007. Print.

16


17/18

Gumbrecht, Hans Ulrich. In Praise of Athletic Beauty. Cambridge, MA: Belknap of

Harvard UP, 2006. Print.

Holt, Richard. Sport and the British: a Modern History. Oxford: Clarendon, 1993. Print.

Jupiterimages. T-ball image. Photo. Batting Tips For T-ball | LIVESTRONG.COM. 21,Dec. 2010. Dec 14, 2011.

King of Kong: A Fistful of Quarters. Dir. Seth Gordon. Perf. Billy Mitchell and Steve

Wiebe. New Line Home Video, 2008. DVD.

Kuper, Simon, and Stefan Szymanski. Soccernomics: Why England Loses, Why

Germany and Brazil Win, and Why the U.S., Japan, Australia, Turkey and Even

Iraq Are Destined to Become the Kings of the World's Most Popular Sport. New

York: Nation, 2009. Print.

Kyle, Donald G.Sport and Spectacle in the Ancient World

. Malden, MA: Blackwell Pub.,

2007. Print.

Link, Holden. "Why I Can't Finish Skyward Sword." Web log post. HoldenLink.com. 21

Dec. 2011. Web. 22 Dec. 2011.

Markovits, Andrei S., and Lars Rensmann. Gaming the World: How Sports Are

Reshaping Global Politics and Culture. Princeton: Princeton UP, 2010. Print.

Markovits, Andrei S., and Steven L. Hellerman. Offside: Soccer and American

Exceptionalism. New Jersey: Princeton UP, 2001. Print.

NASCAR. NASCAR Will Restrict Gear Ratios in '05 for Cup, Busch, Trucks. The Official

Site of NASCAR News, Video, Drivers, Tracks, Schedules & Fantasy. 18 Aug.

2004. Web. 28 Dec. 2011.

Norman, Donald A. The Design of Everyday Things. New York: Basic, 2002. Print.

Paolantonio, Sal. How Football Explains America. Chicago: Triumph, 2008. Print.

Polin, Robert, and Michael Rain. Pinball Wizardry: the Theory and Practice of the Art

and Science of Pinball. Englewood Cliffs, NJ: Prentice-Hall, 1979. Print.

RawmeatCowboy. "GameSpot Reviewer Stands by His Skyward Sword Review,

Controls Impressions...and I'm Still Confused." GoNintendo. GoNintendo.com, 17

Nov. 2011. Web. 27 Dec. 2011.

Read, Dan. "Geek, Rebooted - BBC Top Gear." Cars, Driving, Supercars and the TV

Show - BBC Top Gear - BBC Top Gear. Top Gear, 26 Nov. 2010. Web. 16 Dec.

2011.

17


18/18

[email protected]. "A1Darts.com - Dart Games & Dart Rules, Tic Tac Toe."

A1Darts.com - Darts, Dart Boards, Dart Flights and Dart Supplies.

Mostdartgames.com. Web. 15 Dec. 2011.

Ryan J. Tibshirani, Andrew Price, and Jonathan Taylor (January 2011) "A statistician

plays darts", Journal of the Royal Statistical Society, series A, vol. 174, no. 1,pages 213-226 PDF: http://www-stat.stanford.edu/~ryantibs/papers/darts.pdf

Sharpe, Roger. Pinball!. New York: Dutton, 1977.

Suhas Sreedhar. "Checkers, Solved! - IEEE Spectrum." IEEE Spectrum: Technology,

Engineering, and Science News. IEEE, July 2007. Web. 15 Dec. 2011.

Swink, Steve. Game Feel. Burlington: Morgan Kaufmann, 2009.

Tilt: The Battle to Save Pinball. 2008. DVD.

Uploader unknown, original by Williams Mfg. Upper Deck Image Title: Flyer Scan of

print ad. Internet Pinball Machine Database: Williams Upper Deck Images. 14

Dec. 2011.

Vaughnsbar. Ken Griffy Jr videogame baseball Image. Screenshot. Vaughns Bar and

Grill: game ninety five: daddys drinking, son. 20 July 2007. Dec 14 2011.

Zeigler, Earle F. Sport and Physical Education in the Middle Ages. Victoria, B.C.:

Trafford, 2006. Print.

Zing. "The Definitive Twilight Princess Wii vs. Gamecube Thread." The Videogame

Database. Digital Press, 7 Oct. 2009. Web. 20 Dec. 2011.

18

mediated input

Documents