PENny: An Extremely Low-Cost Pressure-Sensitive Stylusfor Existing Capacitive Touchscreens

Johnty WangMedia and Graphics

Interdisciplinary CentreUniversity of British Columbia

Vancouver BC, Canadajohnty@ece.ubc.ca

Nicolas d’Alessandronumediart Institute for New

Media Art TechnologyUniversity of Mons

B-7000, Mons, Belgiumnicolas@dalessandro.be

Aura PonInteractions Lab. Depts. of

Music and Computer ScienceUniversity of CalgaryCalgary, AB, Canadaaapon@ucalgary.ca

Sidney FelsElectrical and ComputerEngineering & MAGIC

University of British ColumbiaVancouver BC, Canada

ssfels@ece.ubc.ca

ABSTRACTBy building a wired passive stylus we have added pressuresensitivity to existing capacitive touch screen devices forless than $10 in materials, about 1/10th the cost of exist-ing solutions. The stylus makes use of the built-in audiointerface that is available on most smartphones and tabletson the market today. Limitations of the device include thephysical constraint of wires, the occupation of one audioinput and output channel, and increased latency equal tothe period of at least one audio buffer duration. The sty-lus has been demonstrated in two NIME applications thusfar: a visual musical score drawing and a singing synthesisapplication.

Keywordsinput interfaces, touch screens, tablets, pressure-sensitive,low-cost

1. INTRODUCTIONThe growth of increasingly powerful mobile devices haveprovided a new platform for the implementation of NIMEs[8]. The opportunities provided by mobile devices lie notonly in the processing and multimedia capabilities, but alsothe wide potential user base due to the ubiquity of suchdevices.

However, one area of limitation for mobile devices, espe-cially for NIME, lies in their input capabilities. The primaryform of input on such devices is the touch screen. Whilethe act of drawing and writing is an expressive mediumof expression that has been used throughout history, mostmodern devices do not adequately simulate the experience.Drawing and writing is implemented through the detectionof finger or stylus position on a sensing display, but most de-vices do not capture the pressure information, a crucial com-ponent of the process that every person learns from a youngage. Only a small portion of devices on the market today in-

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.NIME’13, May 27 – 30, 2013, KAIST, Daejeon, Korea.Copyright remains with the author(s).

tegrate pressure sensing technology into the display1. Whiledeveloping new instruments on mobile devices, we have de-vised an extremely simple pressure sensitive addition thatmakes use of as much existing hardware as possible, therebyreducing the cost and number of new components used.

2. RELATED WORKThe Melody Easel from the Brain Opera [9] is one of theearlier examples of an existing display augmented with lo-cation and pressure sensitivity for gestural control of music.The Wacom digitizer [3] consists of an inductively poweredactive stylus with sensing circuitry that transmits pressureinformation wirelessly to the receiver circuitry. It has beenused as an expressive musical controller on a number of oc-casions [12] [6] [5]. [11] is an example of a physical interfacethat detects multiple pressure points and widget locationand orientation. While the interface allows a richer set ofinputs to be captured, the active electronics and compo-nents add to the bulk and cost of the device.

A number of manufacturers have addressed the lack ofpressure sensitive touch displays on modern devices throughthe implementation of active (powered) pens and styluses.The Adnoit Touch [1] is an example of a pressure sensi-tive pen designed to be used for bluetooth enabled devices.An active pressure sensing circuit is built into a pen pack-age with a conductive tip to detect the pen’s location onscreen. The analog to digital conversion is performed bythe onboard electronics, and then sent to the host devicevia bluetooth.

The Hex3 Jaja [7] pen is an active device that emits ahuman-inaudible analog tone that varies according to thepressure applied. The audio signal is picked up by the mi-crophone input on the host device and decoded into pres-sure levels. By eliminating a digital wireless connection thecircuitry inside the pen is reduced. One of the major lim-itations of the device is that since each pen radiates thesame tone, multiple pens cannot be used in close proximitywith each other without interference. Additionally, perhapsironically, the cost of this pen is the same as the AdnoitTouch mentioned above at around $90 despite the lack ofdigital transmission circuitry.

1Samsung Galaxy Note and Microsoft Windows ProTablets, for example.

3. SYSTEM OVERVIEWPENny is a pressure sensitive stylus for a smartphone ortablet device. PENny uses a similar concept of using audiosignals to represent pressure data as the Hex3 jaja [7], butwith an additional simplification: even the production ofaudio is done by the parent device, and as such no activecircuitry is required at all, eliminating most of the circuitcomponents, the need for batteries and drastically reducingcost. Additionally, the wired interface eliminates the possi-bility of wireless interference and multiple pens can be usedwithin close proximity.

3.1 HardwareThe central component of the PENny system is a passiveresistor network wired to the mobile device’s audio inputand output ports. The resistor network consists of a fixedand variable (sensing) resistor set up in a voltage dividerconfiguration as shown in Figure 1 and is fed with an au-dio signal from the device through the network at Vin, theoutput at Vout is dependent on the value of R2 given byEquation 1.

R1 R2

Vin Vout

Figure 1: Two resistors in a voltage divider configuration

Vout = Vin × R2

R2 + R1(1)

If the resistor R2’s resistance changes with applied pres-sure (such as a flex or force sensor), the output voltage willchange as well and this corresponds to the force applied.For the first prototype, we have placed a force sensitive re-sistor (FSR) on the tip of a conductive tube 2 cushionedwith a small piece of foam. A strip of copper tape wasplaced over the outer surface to ensure conductivity for thecapacitive sensing of XY location (touch). Since most audiocircuitry is DC filtered (to prevent a constant voltage offsetfrom damaging speakers), we choose a sinusoidal signal asthe input activation. A diagram of the hardware layout isshown in Figure 2 along with a photo of the prototype.

Figure 2: PENny Hardware Layout

The total cost of the materials, using readily availablecomponents from an electronics store at retail price, is un-

2A used AA battery was employed in this case

der $10 (Compared to around $100 for styluses with activeelectronics). Table 1 shows a cost breakdown of the device.

Table 1: Cost Breakdown of the PENny Stylus

ITEM COSTFSR $5.95

4-pole 3.5mm Plug $1.5010k Resistor $0.15

Wire $0.10Conductive Tube $0.50

Scrap foam $0.105cmx5mm Copper Tape $0.15

Total $8.45

3.2 SoftwareIn terms of software, the application responsible for pres-sure sensing has to generate an audio signal to the soundcard and then receive it. The power of the received sig-nal can be calculated to determine the force applied. Theaudio system runs at 44.1kHz with a buffer size of 512 sam-ples (11ms). The excitation signal is a sine wave generatedat 8Khz. We take the power measurement of the receivedbuffer from the audio input, and run a moving average of 4buffers for smoothing. The software is implemented usingopenFrameworks [2] which allows applications to be com-piled for all iOS devices with audio in and out capabilities.

4. APPLICATION EXAMPLES4.1 Portable VuzikVuzik is an interactive, visual musical score interface [10].Originally implemented on a large screen display augmentedwith drawing input, Vuzik allowed the creation of a scorethrough painting gestures. While the large screen allowedlarger arm movements, the system required specialized hard-ware and was not readily available to the public. We are in-terested in exploring the use of the visual drawing metaphoron smaller and more accessible devices such as the iPad.While the level of precision on a smaller screen may belower, the sheer number of devices on the market allows thesystem to be deployed to a larger audience. We have be-gun porting the application into iOS/openFrameworks andhence it was easy to provide a proof of concept using thePENny stylus as input. In Vuzik, the thickness of the drawnlines represent note velocity and due to the lack of pressureinput, this parameter had to be selected using a menu. Bymapping the line thickness to the input pressure, not only isit more convenient and intuitive (consistent with the paint-ing metaphor of thicker lines equating to brush stroke pres-sure), the pressure to line thickness mapping also eliminatesthe need for extra UI widgets that take up screen space.

Portable Vuzik with PENny:- pressure sensitive input- line thickness determined by stylus pressure

Vuzik:- non-pressure sensitive input- line thickness selected by menu

thin thick

ss

Figure 3: PENny as Input for the Mobile Vuzik Prototype

Tilt: Vowel Interpolation

Y: Position Pitch

X Position: Volume and Vocal Effort

Tilt: Vowel Interpolation

Y: Position Pitch

X Position: Volume

Pressure: Vocal Effort

(A) ChoirMob Mapping (B) ChoirMob+ Mapping 1

Tilt: Available for other params

Y: Position Pitch

X Position: Vowel Interpolation

(C) ChoirMob+ Mapping 2

Pressure: Volume and Vocal Effort

Figure 4: ChoirMob and ChoirMob+ Mappings

4.2 ChoirMob+ChoirMob is an iOS-based singing synthesis application [5]and is based on the HandSketch instrument [4] that useda Wacom Tablet an input controller. When moving theinterface onto a smaller screen without pressure sensitivity,sacrifices were made including the reduction of pitch rangeand the remapping of volume/vocal tension to one of thetouch screen axis, thereby occupying one of the position axisthat was originally used for vowel mapping. A less precisevowel interpolation was assigned to the tilt angle of thedevice. We have developed ChoirMob+ with the addition ofthe PENny stylus that brings back pressure sensitivity intothe system. We are experimenting with reimplementing thevowel mapping and comparing that with the decoupling ofvolume and vocal tension parameters and opening up thetilt to control other vocal parameters, as shown in Figure 4.While a smaller capacitive touch screen does not provide thesame degree of precise control as a Wacom tablet, the sizeand ubiquity of portable devices allow new opportunitiesfor exploration.

5. LIMITATIONSThe PENny stylus requires the use of the audio jack, whichdisables the internal speakers of the device and occupies oneaudio input and one audio output. If the remaining chan-nel of audio output is required by the application, the extrachannel can be broken out from the stylus through a sepa-rate jack, but only external speakers can be used. In gen-eral, since the built-in speakers on portable devices requireamplification in most performance settings, this constraintis not unreasonable.

Compared to wireless styluses, the wired device does con-strain movement in return for the extremely simple design.Another potential issue is input latency introduced by theaudio input/output system. Since the sensing of the pres-sure requires both the transmission and receiving of audio,at least one additional buffer period is added to the process.

6. CONCLUSIONBy using a passive stylus driven by the built-in sound cardwe have demonstrated the possibility of adding pressuresensing capability to a large number of tablets and smart-phones at less than 1/10 the cost of existing solutions. Thewired interface dramatically reduces the cost of the designthrough sacrificing an audio input/output and some free-dom of movement. However, as an input interface it is anextremely simple way to add pressure sensitivity and canprovide more expressive input suitable for NIME as wellas other artistic applications, especially for the casual enduser where many mobile apps are targeted. The interfacehas been used in two application examples to demonstrateits effectiveness. We hope that this method can be usedby the community to extend the functionality of existing

devices for creative applications in a simple, cheap and effi-cient manner.

7. REFERENCES[1] Adonit jot touch. http://adonit.net/jot/touch/,

2013.

[2] Openframeworks. http://www.openframeworks.cc,2013.

[3] Wacom interactive pen displays and digital drawingtablets. http://www.wacom.com, 2013.

[4] N. d’Alessandro and T. Dutoit. Handsketch bi-manualcontroller: investigation on expressive control issues ofan augmented tablet. In Proceedings of the 7thinternational conference on New interfaces formusical expression, pages 78–81. ACM, 2007.

[5] N. d’Alessandro, B. Pritchard, J. Wang, and S. Fels.Ubiquitous Voice Synthesis : Interactive Manipulationof Speech and Singing on Mobile DistributedPlatforms. In Proceedings of the 2011 AnnualConference Extended Abstracts on Human factors incomputing systems, pages 335–340, 2011.

[6] A. Forsberg, M. Dieterich, and R. Zeleznik. Themusic notepad. In Proceedings of the 11th annualACM symposium on User interface software andtechnology, pages 203–210. ACM, 1998.

[7] Hex3. Jaja stylus. http://hex3.co/products/jaja,2013.

[8] J. Oh, J. Herrera, N. Bryan, L. Dahl, and G. Wang.Evolving the mobile phone orchestra. In Proceedingsof the International Conference on New Instrumentsfor Musical Expression, 2010.

[9] J. A. Paradiso. The brain opera technology: Newinstruments and gestural sensors for musicalinteraction and performance. Journal of New MusicResearch, 28(2):130–149, 1999.

[10] A. Pon, J. Ichino, E. Sharlin, D. Eagle,N. d’Alessandro, and S. Carpendale. Vuzik: APainting Graphic Score Interface for Composing andControl of Sound Generation. In InternationalComputer Music Conference, Ljubljana, Slovenia,2012, 2012.

[11] J. Wang, N. d’Alessandro, B. Pritchard, and S. Fels.Squeezy: Extending a multi-touch screen with forcesensing objects for controlling articulatory synthesis.In Proceedings of the 2011 Conference on NewInterfaces for Musical Expression (NIMEO11), pages531–532, 2011.

[12] M. Wright, D. Wessel, and A. Freed. New musicalcontrol structures from standard gestural controllers.In International Computer Music Conference, pages387–390, Thessaloniki, Hellas, 1997. InternationalComputer Music Association.