PIN Skimming: Exploiting the Ambient-Light Sensor in Mobile Devices

Raphael Spreitzer∗

IAIK, Graz University of Technology, Austriaraphael.spreitzer@iaik.tugraz.at

Abstract

In this paper, we propose a new type of side channelwhich is based on the ambient-light sensor employed intoday’s mobile devices. The pervasive usage of mobiledevices, i.e., smartphones and tablet computers and theirvast amount of sensors represent a plethora of side chan-nels posing a serious threat to the user’s privacy and se-curity. While recent advances in this area of researchfocused on the employed motion sensors and the cam-era as well as the sound, we investigate a less obvioussource of information leakage, namely the ambient light.We successfully demonstrate that minor tilts and turnsof mobile devices cause variations of the ambient-lightsensor information. Thus, we are the first to show thatthis sensor leaks sensitive information. Furthermore, wedemonstrate that these variations leak enough informa-tion to infer a user’s personal identification number (PIN)input based on a set of known PINs. Our results showthat we are able to determine the correct PIN—out of aset of 50 random PINs—within the first ten guesses about80% of the time. In contrast, the chance of finding theright PIN by randomly guessing ten PINs would be 20%.Since the data required to perform such an attack can begathered without any specific permissions or privileges,the presented side channel seriously jeopardizes the se-curity and privacy of mobile-device owners.

1 Introduction

Mobile devices such as smartphones and tablet com-puters have become a ubiquitous part of our everydaylife. Powerful processors and a variety of sensors ledto manifold applications being developed on these de-vices. Besides more general applications that allow usersto browse the Internet as well as to take and view pic-

∗This work has been supported by the Austrian Research PromotionAgency (FFG) and the Styrian Business Promotion Agency (SFG) un-der grant number 836628 (SeCoS). We would also like to thank PeterTeufl and Horst Possegger for helpful discussions about machine learn-ing. Last but not least, we would like to thank the volunteers for theirparticipation in our experiments.

tures, more sophisticated applications like augmented-reality applications and games also exist. However, thesedevices are not solely employed for entertainment appli-cations but also for handling business applications suchas banking transactions and payment services. This inturn leads to sensitive information being processed onthese devices, which also attracts the attention of crimi-nals and imposters aiming to steal money from users andto spy on specific people through malicious applications.Hence, the investigation of security and privacy compro-mising threats is of utmost importance.

One potential source of security and privacy compro-mising threats is denoted to side channels. Side chan-nels represent unintended information leakage duringthe operation of a device and potentially allow attack-ers to recover secret information, as demonstrated byKocher [32] in 1996. Mobile devices include a myr-iad of features and sensors that allow for many dif-ferent attacks. While traditional side-channel attacksagainst smartcards require expensive hardware, smart-phones employ many different types of sensors that al-ready represent a plethora of side channels themselves.Thus, the “hardware” required to perform side-channelattacks is provided for free.

In other words, besides providing useful information,these sensors also represent a variety of threats to theuser’s security and privacy. Though Android employs apermission system to prevent malicious access to specificdevice resources, many of the employed sensors do notrequire any permission at all. This exacerbates the sever-ity of these vulnerabilities since applications without anyspecific permissions might be able to exploit these sidechannels, something that has already been demonstratedimpressively. For instance, Marquardt et al. [34] devel-oped an application intended to run on a smartphone thatis capable of recovering keyboard inputs of nearby key-boards. In this case, the application exploits the vibra-tions recorded via the accelerometer sensor. In 2012,Aviv et al. [8] demonstrated the possibility of extractingthe personal identification number (PIN) and the patternunlock mechanism by exploiting the information pro-vided by the accelerometer sensor in smartphones. In

1

arX

iv:1

405.

3760

v1 [

cs.C

R]

15

May

201

4

addition, Miluzzo et al. [36] have shown how to extractpressed keys from the accelerometer sensor in combina-tion with the gyroscope sensor. In 2013, Simon and An-derson [45] demonstrated how to infer PINs through thecamera and the microphone. However, compared to thework of Simon and Anderson, our attack does not re-quire any permission and it does not signal the capturingof sensor data via audio-visual feedback. For instance,like the shutter sound or the LED while taking imageswith the camera.

The importance of research in this area has been em-phasized by Becher et al. [13]. In order to raise aware-ness about such attacks and to develop effective coun-termeasures, these vulnerabilities must be reported andanalyzed extensively. To this end, we show how the am-bient light of the user’s environment seriously affects thesystem’s security and privacy. This is due to the factthat today’s mobile devices are also equipped with a lightsensor that provides information about the ambient light.The most common usage of this sensor is to adjust thescreen brightness depending on the light intensity. How-ever, due to slight tilts and turns during the operation ofthe device, the information provided by the sensor al-lows an attacker to infer the input provided by the user.In this paper, we investigate the sensitive informationleaked through this sensor.

Contribution. The contributions of this work can besummarized as follows. First, we show that the lightsensor employed in today’s mobile devices actually rep-resents a new type of side channel that leaks the user’sinput like the secret PIN input. Second, we observedthat the light sensor of modern smartphones also cap-tures the red, green, blue, and white (RGBW) intensitieswhich leaks even more information and improves the ac-curacy of the attack. Third, we provide practical insightsinto the ways in which this side channel can be exploitedto gather the secret PIN input of the user by employingmachine-learning algorithms. Fourth, we discuss poten-tial mitigation techniques to prevent the exploitation ofsensor-based side channels.

Outline. The remainder of this paper is structured asfollows. In Section 2 we start with a basic investigationof the ambient-light sensor and provide an insight intothe information leaked through this sensor. Section 3 out-lines one potential attack scenario which takes advantageof this side channel to recover the secret PIN input pro-vided by the user. Later on, in Section 4, we detail howthe leaked information is actually exploited by employ-ing machine-learning algorithms. In Section 5 we ex-tensively analyze the information gathered and providedetailed evaluations regarding the applicability and re-liability of this attack. We cover limitations of the pre-

Figure 1: Samsung Galaxy SIII with (1) the proximityand ambient-light sensor as well as (2) the front camera.

Rate parameter Sample rate

SensorManager.SENSOR DELAY FASTEST (0) ∼ 750 HzSensorManager.SENSOR DELAY GAME (1) ∼ 49 HzSensorManager.SENSOR DELAY UI (2) ∼ 15 HzSensorManager.SENSOR DELAY NORMAL (3) ∼ 5 Hz

Table 1: Sampling rates on the Samsung Galaxy SIII.

sented attack in Section 6 and we provide a brief analysisof countermeasures to prevent such attacks in Section 7.Last but not least, we discuss related work in Section 8and present a conclusion for this paper in Section 9.

2 Ambient-Light Sensor

The ambient-light sensor employed in many of today’smobile devices provides information about the intensityof the surrounding illumination. Figure 1 illustrates theproximity and ambient-light sensor as well as the frontcamera on a Samsung Galaxy SIII, which acts as ourtest device. The information reported from the ambient-light sensor is given in SI lux units, which measuresthe intensity of illumination of a surface. This informa-tion is used to adapt the screen brightness appropriately.For instance, outside in direct sunlight the screen bright-ness must be increased to remain readable, whereas indarker surroundings the screen is dimmed to reduce eyefatigue [44].

The ambient-light sensor can be accessed viathe Android Sensor API [5] that allows appli-cations to register listeners which are notifiedabout changes of the sensor values. The an-droid.hardware.SensorManager.registerListener(...)method accepts a rate parameter which determineshow fast the events should be reported. Table 1 showsthe resulting sampling frequencies for the differentrate parameters according to our observations on theSamsung Galaxy SIII smartphone running Android 4.3.We determined the number of reported values over aperiod of 10 seconds for each of the listed parametersand we observed that about 750 measurement samplescan be gathered per second. We also observed a sensorresolution of 1 lux, i.e., the smallest detectable change

2

Figure 2: PIN input mask to gather test samples.

is 1 lux. Furthermore, the ambient-light sensor can beaccessed without any specific permission which allowsmalicious applications to access this sensor informationwithout raising any suspicion.

The next step in determining the applicability of thelight-sensor information regarding side-channel attacksis to test the sensitivity, i.e., whether operating the smart-phone actually results in changes of the information pro-vided by the sensor. Therefore, we developed an Androidapplication that randomly generates a 4-digit PIN and ex-pects the user to enter this PIN on the provided PIN pad.Figure 2 illustrates a screenshot of this application. Dur-ing the input of the PIN the application collects the infor-mation provided by the ambient-light sensor as well asthe corresponding timestamp of the reported value. Fur-thermore, we store a timestamp when a button is clickedas well as the digit of the clicked button itself. We col-lected this information for five consecutive PIN inputsand visualized the gathered information in Figure 3. Forthe sake of clarity we also plotted the corresponding dig-its of the PIN and, as can be seen in this plot, a recurringpattern for different digits of the PIN (1-5-9-0) can beobserved. These differences in the light intensity dur-ing the input of the PIN occur inevitably due to slighttilts and turns while operating the smartphone. Figure 4shows a schematic illustrating the tilts and turns leadingto variations of the reported sensor information. For in-stance, assuming the light bulb being the main source oflight, then tilting the device to the left causes a decreaseof the reported lux value. Although we illustrate a point-like light source, the attack also works for environmentsthat are uniformly lit via tube lights.

Note that the PIN input mask can be aligned on thetop or on the bottom of the screen. For instance, on our

0 2 4 6 8

x 109

0

100

200

300

400

500

600

700

1

5

90

1

5

90

1

5

9

0

1

5

90

1

5

90

Lig

ht−

se

nso

r in

form

atio

n [

lx]

Time [ns]

Figure 3: Light-sensor information for five consecutivePIN inputs (1-5-9-0).

Figure 4: Schematic of information leakage caused bytilts and turns of the smartphone.

test device the SIM-unlock PIN mask is aligned on thetop while the standard phone pad is aligned on the bot-tom. However, our observations showed that the align-ment does not influence the information leakage.

Accessing the ambient-light sensor does not requirespecial permissions and, hence, any malicious applica-tion can gather the required information without raisingany suspicion. Contrary to the shutter sound or a LEDthat indicate an active camera [4], there is not even anaudio-visual feedback that signals the user that data isbeing collected with the sensor. Thus, we conclude thatthe information provided by the ambient-light sensor canbe exploited in side-channel attacks, which means thatan attacker is able to determine the corresponding inputbased on this information.

3

0 2 4 6 8

x 109

0

50

100

150

200

250

300

350

400

17 3

0

17 3 0

1 7 3 0

17

3

0

17

30

Lig

ht−

se

nso

r in

form

atio

n

Time [ns]

LuxRedGreenBlueWhite

Figure 5: RGB(W)-sensor information for five consecu-tive PIN inputs (1-7-3-0).

2.1 RGB(W) Sensor

More recent smartphones, like the Samsung GalaxySIII [42] as well as the Samsung Galaxy S4 [41], alsoemploy an RGB(W) sensor which is capable of reportingthe red, green, blue, and white (RGBW) intensities of theambient light. Samsung refers to this sensor as an RGBsensor, but since it also reports the white intensity of theambient light we refer to it as RGB(W) sensor. Accord-ing to the official Blog of Samsung Electronics [44], theRGB sensor is used to optimize the screen brightness andsharpness to prevent eye fatigue.

The Android API does not yet support RGB(W)sensors, not even the latest version (Android 4.4KitKat [3]). Thus, a workaround is necessary to re-trieve the desired sensor values from the virtual filesystem directly. On the Samsung Galaxy SIII reading/sys/devices/virtual/sensors/light sensor/raw data yieldsthe RGBW values of the color sensor. We verifiedthe correctness of these values through the ServiceMenu [26]—accessible by entering *#0*# on the phonepad—which reports the same values.

Figure 5 illustrates the data of the same experiment asmentioned above, but this time we also include the infor-mation provided by the RGB(W) sensor. Due to reasonsof readability, we plotted only every 10-th value of thered, green, blue, and white intensities. We observe thatall curves, i.e., red, green, blue, and white as well as thelux values, show a similar curve. However, the intensityof the blue light seems to provide a smoother curve thanthe other four values. Again, we plotted the event of adigit input for the sake of clarity and labeled each eventappropriately. For this specific plot we observe that the“OK” button leads to a rather heavy decrease of the luxvalue as well as the RGBW intensities.

Our observations show that the RGBW information

provides additional information that can be exploited,i.e., additional features to be used for the machine-learning algorithm later on. Again, no specific permis-sions or super-user privileges are required, even thougha workaround is necessary to gather the RGB(W)-sensorinformation.

3 Attack Scenario

In this section, we outline one possible attack scenario.Based on this scenario, we will later on illustrate the in-dividual steps to reproduce the actual exploitation of thelight-sensor information by employing machine-learningtechniques. Since the captured light-sensor informationreflects the user’s ambient-light conditions, a generic at-tack is not possible. More formally, the user needs toprovide the data to be used to train the machine-learningalgorithm in the same environmental setting as the ac-tual data which is to be classified later on. Hence, weconsider the following situation as a possible attack sce-nario.

3.1 Training Phase

A hand-crafted application, i.e., an addicting game, isused to gather the light-sensor information during theuser’s interaction with the smartphone. The applicationin question tricks the user into operating the smartphonein a way that is similar to the input of multiple PINs.For example, an application similar to Math Trainer [46]could be used, where the user is supposed to solve math-ematical puzzles and to enter numbers which can be seenas PINs. During our tests we even observed that a gamewhere the number of correctly entered PINs in the short-est period of time could also serve as an addicting game.

We assume that users play such games in the livingroom while watching TV, in a waiting room while wait-ing for an appointment, or during a train ride. Hence,even though mobile devices are carried around all thetime, there are still many opportunities to gather the re-quired data from users operating the smartphone with-out walking around. Furthermore, a recent study per-formed by the UK’s Office of Communications [48]coined the term media stacking, which refers to the factthat about half of UK’s adults conduct their smartphoneor tablet computer while watching TV. Thus, our as-sumption seems to be reasonable and the outlined attackscenario is rather realistic.

The game, as outlined above, might only be able tocapture a limited number of “PINs” to be used for theclassification of the unknown PIN. However, as has beenshown in 2012 by Bonneau et al. [17] as well as Jakob-sson and Liu [29], people tend to choose specific PINs

4

like dates as well as PINs that represent common four-digit words, e.g., “love” (5683). Thus, taking researchabout the users’ tendency to choose PINs into consider-ation, a set of commonly used PINs could be establishedand might be enough to determine the correct one. Theabove mentioned game can be used to learn this set ofcommonly used PINs.

In case computing power for the machine-learning al-gorithm is required, i.e., a powerful server, the applica-tion requires an Internet connection to transmit the gath-ered data to this powerful intermediary. Nevertheless,we argue that convincing the user that the application re-quires the Internet permission is easy. For instance, toretrieve high scores or information about new updates,new games, etc. Thus, we claim that the proposed attackcan be performed without raising the user’s suspicion.

3.2 Exploitation PhaseAfter gathering enough samples, the application tricksthe user into restarting the device or starting the desiredapplication, e.g., the banking application, just to capturethe light-sensor information during the input of the au-thentication PIN. If one considers to attack the smart-phone’s PIN, a service can be implemented to be startedon boot time. Afterwards the sensor information cap-tured during the game play is used to deduce the un-known PIN input by means of machine learning. Notethat the game, e.g., Math Trainer, might also trick theuser into buying a specific add-on, a “new level”, or a“new stage”. When the user performs the in-app billingvia Google Wallet, the game skims the corresponding au-thentication PIN.

Now one might question whether the revealed PIN isof any value for the attacker. In fact, if the attacker latergains physical access to the mobile device, she mightgain access to the mobile device by unlocking the phone,or even worse, might cause financial damage by authenti-cating herself to the corresponding application using thecorrect PIN. Furthermore, Aviv et al. [8] argued that thelearned PIN might be valuable in case the user reusesthe PIN, for instance, as the ATM PIN. In addition, Si-mon and Anderson [45] predict that the number of smart-phone applications requiring an authentication PIN willincrease over time. Hence, users might be tempted toreuse one PIN across different applications, which exac-erbates PIN-skimming vulnerabilities. Overall, we con-sider the presented attack as a serious threat for today’smobile devices.

3.3 Security ImplicationsSimon and Anderson [45] state that sensor-based side-channel attacks are capable to overcome strong separa-

tion mechanisms like BlackBerry Balance [15] or Sam-sung KNOX [43]. These mechanisms try to separate the“private” world from the “business” world in order toprovide greater protection of corporate data on smart-phones. However, based on the leaking sensor informa-tion, an unprivileged application running in the “private”world of the smartphone might gain knowledge of sen-sitive information entered in the “business” world. It isimportant to note that the presented attack is also capa-ble to overcome these separation mechanisms, if unpriv-ileged applications running in the “private” world are al-lowed to access the ambient-light sensor during the oper-ation of the “business” world. Furthermore, since Black-Berry smartphones also support the execution of Androidapplications and the BlackBerry Runtime 10.0 [16] alsosupports the ambient-light sensor, a malicious applica-tion developed for Android smartphones might also harmBlackBerry-based devices.

3.4 Observations and AssumptionsThe scenario outlined above is based on some observa-tions and assumptions which are detailed within the fol-lowing paragraphs. Considering a PIN-input mask as il-lustrated in Figure 2, and a user operating the smartphonewith only one hand, using the thumb to enter the digits,we make the following observations. Left-handed per-sons tilt the smartphone slightly to the left side when en-tering PIN digits in the middle and right column of thekey pad, i.e., 2, 3, 5, 6, 8, 9. In contrast, right-handedpersons tilt the smartphone slightly to the right side whenentering PIN digits in the left and middle column, i.e., 1,2, 4, 5, 7, 8. We attribute this observation to the fact thatpeople might possibly drop the device if operating it theother way round. Furthermore, this slightly moves thedisplay towards the thumb, i.e., the users slightly pushthe display towards their thumb. These tilts of the devicecause the variations in the captured light-sensor informa-tion. Similarly, we assume fewer tilts and turns of themobile device if it is held with one hand and operatedwith the index finger of the other hand or a stylus pen.

Assumption 1. We assume the user is holding the mo-bile device in his hands rather than laying it onto a flatsurface while operating it. If we would assume the mo-bile device is lying on a stable surface, i.e., a table, thelight-sensor information would not change during the op-eration of the device, unless the user’s hand causes thelight-sensor changes.

Assumption 2. Furthermore, we assume that the PINis entered on a key pad similar to the one illustrated inFigure 2 rather than a QWERTY keyboard with a sin-gle row of numbers. Examples of applications that are

5

“protected” with an authentication PIN are, for instance,AppLock [21], Evernote [22], and KeepSafe [30], aswell as mobile banking applications, e.g., Barclays [12],and NAB [37], just to name a few of them. Screen-shots of these applications—provided by the correspond-ing developers—clearly show that the authentication PINis entered on a key pad as illustrated in Figure 2. Hence,this seems to be a rather fair assumption which does nothave a negative impact on the attack scenario.

Assumption 3. We also assume that the user operatesthe mobile device in an environment where the ambient-light sensor faces a sufficiently large variance of the am-bient light during the operation of the device. This isnot the case in completely dark environments. However,also rather gloomy environments, i.e., a room in the lateafternoon without any artificial light source, can be con-sidered for potential attack scenarios, at least in case thelux values vary slightly during the handling of the device.

4 Attack Approach

In this section, we detail the steps for the exploitation ofthe light-sensor information. As outlined in the scenarioabove, we perform a matching of sensor values capturedduring the input of an unknown PIN to the sensor valuesof known PINs. In terms of machine learning this rep-resents a classification problem, where a so-called fea-ture vector is mapped to a finite number of labels or cat-egories, i.e., PINs in our case. The required steps areas follows: (1) gathering the sensor values under knownPINs as well as the sensor values under unknown PINs,and (2) employing machine-learning techniques to deter-mine the unknown PINs based on the set of known PINs.

In order to perform the classification, sensor val-ues and the corresponding PINs are used to train themachine-learning algorithm. This data is referred to astraining data. The actual data that is to be classified isknown as test data. We stick to the notation of Alpay-din [2] and Bishop [14], i.e., bold letters denote vec-tors and a superscript T denotes the transpose of a vec-tor. Furthermore, uppercase bold letters denote matrices.Vectors are assumed to be column vectors by default.

4.1 Gathering the Required DataThe gathering of the required training data during gameplay can be formalized as follows. The malicious ap-plication captures a list of tuples (t,L,R,G,B,W ), with tbeing the timestamp and L, R, G, B, and W representingthe lux information, as well as the red, green, blue, andwhite intensities of the ambient light. Furthermore, wecapture a list of tuples (tp,d), with tp being the times-tamp of a pressed digit d ∈ {0, . . . ,9} of the p-th PIN. In

our scenario one PIN consists of four consecutive tuplesin this list. The two timestamps t and tp allow us to prop-erly align the event of a pressed digit with the data of theambient-light sensor later on.

For each PIN we extract the sequence of sensor valueswithin the period defined by the timestamp of the firstdigit and the timestamp of the last digit of the PIN. Inaddition, one might consider a timeframe of a few mil-liseconds before and after the input of the first and thelast digit of one PIN, which covers additional informa-tion. The resulting matrix M for one PIN is as follows.

M =

(t,L,R,G,B,W )1...

(t,L,R,G,B,W )l

Every column within matrix M represents the corre-sponding sensor information during the input of one spe-cific PIN, except the first one which represents the times-tamp at which the information was captured. Before thegathered information can actually be exploited, we nor-malize the sensor values appropriately. This normaliza-tion of the data is done either by dividing each value (L,R, G, B, W ) in one column by the norm of the corre-sponding column vector, or by rescaling the values via,e.g., Li = (Li−min(L))/(max(L)−min(L)).

The data of matrix M is then used to derive the actualfeature vectors. While Owusu et al. [38] consider a largenumber of features and employ special feature selectorsto determine the actual set of features to be used, we keepthe feature space simple and stick to a limited numberof features for a first investigation. These features areoutlined briefly within the following paragraphs.

Lux Values Only. The first set of feature vectors weconsider are the exact lux values at the input of each spe-cific digit of the PIN. Therefore, we use the timestampof each input event during one PIN input to look-up thecorresponding values in matrix M. We represent thesevalues as a vector x = [L1,L2,L3,L4]

T, where the sub-script refers to a specific digit of the PIN.

Lux Values Including RGBW Values. The second setof feature vectors we consider are the exact lux values in-cluding the red, green, blue, and white (RGBW) intensi-ties at the input of each digit for one specific PIN, repre-sented as x = [L1,R1,G1,B1,W1, . . . ,L4,R4,G4,B4,W4]

T.In the following we refer to the feature vector comprisedof these five features as LRGBW.

Polynomial of Degree 3. The third possibility we con-sider is fitting a polynomial of degree 3 through the sec-ond column of M. More formally, through all lux values

6

during one PIN input. The coefficients of this polyno-mial f (x) = ax3+bx2+cx+d are then considered as thefeatures of a specific PIN, i.e., x = [a,b,c,d]T. The co-efficients for the red, green, blue, and white intensities—the remaining columns of M—are obtained in the samemanner and appended to the feature vector x.

After gathering the data as outlined above, any setof the above outlined feature vectors is then com-bined into a matrix Fn with n rows, i.e., one rowfor each PIN. Furthermore, a label or class vectorcn = [(d1,d2,d3,d4)1, . . . ,(d1,d2,d3,d4)n] of tuples cor-responding to the PIN-digits can be derived.

Fn =

x1...

xn

, cn =

(d1,d2,d3,d4)1...

(d1,d2,d3,d4)n

The matrix Fn as well as the label vector cn are then usedto train the classification algorithm.

4.2 Detecting PIN InputsThe above outlined approach of gathering the featurevectors can be performed in a real attack because thedata for training the machine-learning algorithm can becaptured during game play, in which situation all of thisinformation is available, including the timestamp of apressed digit. However, during the input of the unknownPIN the timestamp is not known and hence we need amechanism to determine the PIN input on a sequenceof sensor values. Therefore, the approach mentioned byMiluzzo et al. [36] might be used. Their idea is to move“windows” of a fixed length (the length of an averagePIN input time) over the sensor data and try to detect theinput. Another approach by Simon and Anderson [45]suggests to use the microphone to capture the vibrationsof the haptic feedback to determine when a button onthe touchscreen has been pressed. While the former ap-proach does not require any specific permission, the lat-ter approach requires a permission to access the micro-phone. However, we consider the detection of PIN inputson a sequence of sensor values as solved and beyond thescope of this work. Thus, we gather the test data, i.e., thedata to be classified, in the same way as the training data.

4.3 Determining the Unknown PINAfter we gathered the required light-sensor informationfor all the PINs (Fn and cn) to be used for the train-ing phase of the machine-learning algorithm, we startthe actual training phase. Therefore, we employ Mat-lab’s Statistics Toolbox [35] with its extensive featuresand machine-learning algorithms. We perform the out-lined attack by employing a supervised learning algo-rithm, which tries to learn a function and its parameters

based on labeled training data. This function is later onused to determine the label of unseen data. More for-mally, given a set of tuples (xi,ci), with xi ∈ Rn being afeature vector and ci the corresponding label of the obser-vation, the algorithm tries to infer a function f : X →C,where X ∈ Rn represents the feature vector of an obser-vation and C the inferred label.

Choosing the right classification algorithm seems to besome kind of “mystery”. For instance, related work byMiluzzo et al. [36]—who try to infer input from the ac-celerometer sensor and the gyroscope sensor—takes theapproach of employing ensemble techniques. This meansthat they train several different classification algorithms,and finally they employ a majority-voting scheme tochoose the final classification result. For the sake of sim-plicity, we investigate three different classification algo-rithms and compare their results afterwards. We brieflyoutline the chosen classification algorithms in the follow-ing paragraphs. For further information about these algo-rithms we refer to [2, 14].

Multiclass Logistic Regression. The classifier tries tolearn parameters wk for every class label k ∈C, such thata vector x is assigned to label k in case p(k|x) > p( j|x)for all j 6= k, with p defined as below.

p(k|x) =exp(wT

k ·x)

∑i exp(wiT ·x)

Discriminant Analysis. The classifier tries to learn pa-rameters wk for every class label k ∈C, such that a vectorx is assigned to label k in case fk(x)> f j(x) for all j 6= k,with f defined as below.

fk(x) = wTk ·x+wk0

More formally, when talking about discriminant analy-sis we refer to the linear discriminant analysis, whereclasses are separated linearly.

K-Nearest Neighbor Algorithm. The algorithm as-signs the input vector x a label which is determined bythe majority of the K nearest neighbors. In our case, weused K = 5. Usually, the Euclidean distance is used todetermine the distance between two vectors.

5 Evaluation and Results

We engaged a total of ten users to acquire the necessarydata for the evaluation of this information leakage. Eachuser was asked to enter at least one set of random PINswith cardinality k ∈ {15,30,50}, each PIN for a spe-cific number of times N ∈ {3, . . . ,10}. If a PIN was en-tered incorrectly, we ignore the corresponding input and

7

prompt the user to enter the PIN again. Unless explicitlystated, we always use the largest set of 50 PINs withinthe following analysis. We basically follow the approachof Aviv et al. [8] who also measured the performance oftheir attack on a set of 50 PINs. In total we use the dataof 29 test runs gathered by a total of 10 users.

Evaluation Methodology. Since the performance ofthe learned classifier could be distorted—either posi-tively or negatively—when determined solely with thegathered test data, we apply the concept of k-fold crossvalidation. The purpose of this concept is to estimatethe average success rate of classifying unknown data intoone of the learned categories. Therefore, k-fold cross val-idation partitions the set of training data into k equally-sized sets. Afterwards, k−1 sets of the training data areused to train the classifier and the remaining set is clas-sified according to the learned categories to estimate theperformance. This process is repeated until all k possi-ble sets have been used once as a test set. Afterwardsthe average performance can be estimated. Compared tothe performance based solely on specific test data, theconcept of cross validation provides more reliable esti-mations regarding the performance of a classifier [14].

Setup. We performed the experiments in rather uncon-strained environments regarding the lighting conditions.This means that tests were performed in office rooms thatwere uniformly illuminated via tube lights, in a livingroom with a standard ceiling lamp, and in rooms wherethe only light source was a window. In the room wherethe only light source was a window, we even considereddifferent daytimes, e.g., during the day and in the late af-ternoon. However, we asked the users not to walk aroundwhile entering the presented PINs, which is compliantwith the above outlined attack scenario. Furthermore,we did not insist on a specific input method, but onlythat the user holds the mobile device during the opera-tion. As already explained above, operating the mobiledevice while it is laying on a stable surface is unlikelyto cause variations of the light-sensor information. Al-lowing users to choose their desired input method freelyyields more generic results because the users operate themobile device in their usual manner.

We watched the users during the gathering of themeasurement samples and observed the following inputmethods:

1. Holding the smartphone in one hand and enteringthe digits with the thumb of the same hand.

2. Holding the smartphone in one hand and enteringthe digits with the thumb of the other hand.

3 4 5 6 7 80

0.2

0.4

0.6

0.8

1

Number of samples per PIN

Ra

te o

f co

rre

ctly c

lassifie

d P

INs

Logistic regression (L)Logistic regression (LRGBW)Discriminant analysis (L)Discriminant analysis (LRGBW)KNN (L)KNN (LRGBW)

Figure 6: Average rate of correctly classified PINs overmultiple runs with a set of 15 PINs each.

3. Holding the smartphone in one hand and enteringthe digits with the index finger of the other hand.

Within the following paragraphs we analyze the gath-ered data with the proposed feature vectors and machine-learning algorithms in order to determine whether the se-cret PIN can be recovered based on a set of known PINs.

Comparison of Classification Algorithms. As out-lined above, we employ three different classification al-gorithms. Thus, our first intention is to determine theoverall classification rate based on different classifica-tion mechanisms: (1) logistic regression, (2) discrimi-nant analysis, and (3) k-nearest neighbor classification.These classifiers are fed with the feature vectors: (a) thelux values only (L), and (b) the lux values including theRGBW values (LRGBW).

We applied a 10-fold cross validation on all three clas-sifiers and evaluated the performance of the suggestedfeatures for different numbers of samples (repetitions)per PIN. Figure 6 illustrates the average rate of correctlyclassified PINs out of a set of 15 known PINs for the dif-ferent classifiers and the proposed feature vectors. They-axis represents the average rate of correctly classifiedPINs, and the x-axis illustrates the number of gatheredsamples (repetitions) per PIN. Given this plot, we ob-serve that the feature vector comprised of the LRGBWvalues outperforms the lux value only feature vector mostof the time for all classifiers. Thus, we conclude that theadditional information leaked through the RGB(W) sen-sor leads to a better attack performance. We also observethat the discriminant analysis provides better results thanthe other two classifiers. Furthermore, the average rateof correctly classified PINs increases with the numberof samples per PIN. For instance, if we perform a lin-ear discriminant analysis with a training set of 15 PINs,each repeated 8 times, then we are able to classify more

8

3 4 5 6 7 8 9 100

0.2

0.4

0.6

0.8

1

Number of samples per PIN

Ra

te o

f co

rre

ctly c

lassifie

d P

INs

Logistic regression (L)Logistic regression (LRGBW)Discriminant analysis (L)Discriminant analysis (LRGBW)KNN (L)KNN (LRGBW)

Figure 7: Average rate of correctly classified PINs overmultiple runs with a set of 50 PINs each.

3 4 5 6 7 80

0.2

0.4

0.6

0.8

1

Number of samples per PIN

Ra

te o

f co

rre

ctly c

lassifie

d P

INs

Logistic regression (LRGBW)Logistic regression (poly LRGBW)Discriminant analysis (LRGBW)Discriminant analysis (poly LRGBW)KNN (LRGBW)KNN (poly LRGBW)

Figure 8: Average rate of correctly classified PINs overmultiple runs with a set of 15 PINs each.

than 80% of the PINs correctly. Note that the chance ofcorrectly guessing the right PIN from a set of 15 PINsrandomly is 1

15 = 6.7%.Figure 7 illustrates the average rate of correctly clas-

sified PINs out of a set of 50 PINs. Again, the lineardiscriminant analysis outperforms the other two classi-fiers and the additional information from the RGB(W)sensor increases the performance compared to the luxvalue only. At first glance, an average rate of correctlyclassified PINs of 40 to 50% seems to be quite moder-ate. However, the chance of correctly guessing the rightPIN from a set of 50 PINs is 1

50 = 2%. Thus, our attackoutperforms random guessing by a factor of 20 to 25.

Comparison of Feature Vectors. Since we introduceda feature vector which is comprised of a degree 3 poly-nomial fitted through every value (e.g., lux and RGBWvalues) during one PIN input, we need to compare theperformance of the plain features and the approximated

2 4 6 8 100

0.2

0.4

0.6

0.8

1

Number of guesses

Ra

te o

f co

rre

ctly c

lassifie

d P

INs

Discriminant analysis (L)

Discriminant analysis (LRGBW)

Random guessing

Figure 9: Average rate of correctly classified PINs formultiple guesses.

features. Figure 8 illustrates the respective performancesfor these feature vectors based on a 10-fold cross valida-tion over multiple runs with a set of 15 PINs. We observethat both feature vectors yield a similar performance forthe different classification algorithms, with the discrimi-nant analysis performing best. Based on this observationwe only focus on the discriminant analysis within thefollowing investigations.

Guessing PINs According to their Probability. Aninteresting approach is to consider the fact that an adver-sary is able to enter PINs for a specific number of times,i.e., to guess possible PINs according to their probabilityfor being the correct one. In this case the probability offinding the correct PIN increases with every tested PIN.Thus, we sort potential PINs according to their probabil-ity for being the correct one and illustrate the rate of cor-rectly classified PINs after a specific number of guesses.

Figure 9 shows the average rate of correctly guessinga PIN out of a set of 50 random PINs for a specific num-ber of guesses. The additional information provided bythe RGB(W) sensor yields better results and seems toincrease the success rate by about 10 percentage pointscompared to the lux value only feature vector. We also il-lustrate the success rate if one were trying to guess PINsrandomly, which clearly shows the advantage of our at-tack compared to random guessing.

When comparing our results to the results of Aviv etal. [8], we observe that the ambient-light sensor pro-vides results at least as good as those achieved by the ac-celerometer sensor. Comparing our results to the work ofSimon and Anderson [45], we observe that the ambient-light sensor provides even better results than the ap-proach of exploiting the camera to infer PINs. For in-stance, Simon and Anderson claim to infer more than30% of the PINs after two guesses and more than 50% of

9

2 4 6 8 100

0.2

0.4

0.6

0.8

1

Number of guesses

Ra

te o

f co

rre

ctly c

lassifie

d P

INs

Index finger

Left hand and right thumb

Right hand and right thumb

Random guessing

Figure 10: Average rate of correctly classified PINs fordifferent input methods.

the PINs after five guesses. In contrast, the ambient-lightsensor allows us to infer about 50% of the PINs after twoguesses and about 65% of the PINs after five guesses.

The presented results indicate that guessing PINs ac-cording to their probability provides an effective meansof finding the correct one. On average we are able to de-termine the correct PIN with a probability of 80% whenconsidering the ten most probable PINs. In contrast,guessing PINs randomly from a set of 50 PINs wouldresult in a success rate of 20% after ten guesses.

Impact of Different Input Methods. Since users em-ploy different input methods, we investigate the impactof an input method on the classification rate. To this end,we compare the three major input methods:

1. Holding the device in one hand and using the thumbof the same hand to operate it.

2. Holding the device in one hand and operating it withthe thumb of the other hand.

3. Holding the device in one hand and using the indexfinger of the other hand to operate it.

Figure 10 illustrates the average rate of correctly clas-sified PINs for the three different input methods afterguessing a specific number of the most probable PINs.The plot is based on a 10-fold cross validation consider-ing a discriminant analysis on the LRGBW values. Theunderlying set of PINs had a cardinality of 15. Accordingto this plot the two input methods involving the thumb,i.e., left hand and right thumb as well as right hand andright thumb, seem to be more vulnerable to this attackthan the one with the index finger. This is due to thefact that the mobile device usually undergoes only mi-nor movements when the index finger is used, because

User Input method

User 1 Left hand and index fingerUser 2 Right hand and right thumbUser 3 Left hand and index finger

Table 2: Input methods of three users.

2 4 6 8 100

0.2

0.4

0.6

0.8

1

Number of guesses

Ra

te o

f co

rre

ctly c

lassifie

d P

INs

User 1

User 2

User 3

Random guessing

Figure 11: Average rate of correctly classified PINs forthree specific users on a set of 30 PINs.

one hand is solely used to hold the mobile device. How-ever, this is not entirely correct because also for the inputmethod involving the left hand and the right thumb onehand is solely used to hold the mobile device.

To gain further insight into factors affecting the rateof correctly classified PIN inputs, we compare the pro-vided data of three different users. The correspondinginput methods of these users are illustrated in Table 2.All three users entered 30× 3 PINs, i.e., each of the 30PINs was entered 3 times, within the same room underthe same environmental conditions regarding the ambi-ent light. Figure 11 illustrates the result of the 10-foldcross validation of the three data sets provided by User1, User 2, and User 3, respectively. The input methodof User 2 seems to leak the most information. This ap-pears to be due to the fact that she rested her upper armagainst her upper body in a relaxed manner and operatedthe smartphone in a very comfortable way. User 1 placedher elbows on her knees and also operated the device in avery relaxed way. In contrast, User 3 tightly pressed herupper arm against the upper body and held the devicevery firmly in her hand while entering the digits with theindex finger. Based on the investigations of these threeusers, we observe that though the input method seems tohave an impact on the classification rate, a general state-ment regarding the security or insecurity of a specific in-put method is difficult to make. Nevertheless, we claimthat the tighter and more firmly one holds the device, theless information is leaked. To put it more generally, the

10

2 4 6 8 100

0.2

0.4

0.6

0.8

1

Number of guesses

Ra

te o

f co

rre

ctly c

lassifie

d P

INs

Discriminant analysis (LRGBW 5Hz)

Discriminant analysis (LRGBW 50Hz)

Random guessing

Figure 12: Average rate of correctly classified PINs fordifferent sampling rates based on a set of 15 PINs.

more movements the mobile device undergoes during theoperation, the more information is leaked.

Impact of the Sampling Frequency. As outlined inSection 2, the ambient-light sensor can be configuredto operate with a variety of different sampling frequen-cies. With a sampling frequency of 750 Hz the SamsungGalaxy SIII provides an immense number of measure-ment samples per second, far more than is necessary fora successful attack. In fact, most of our attacks were per-formed with a sampling frequency between 5 and 50 Hz,though we performed successful attacks with all possiblesampling frequencies.

Figure 12 indicates that the lowest sampling frequencysupported by our device (5 Hz) is actually enough to per-form the presented attack. The plot illustrates that theperformance does not even decrease when sampling withthe lowest frequency of 5 Hz.

Demonstrating that an attack can be performed with alow sampling frequency is more important than demon-strating its success on a high sampling frequency. This isdue to the fact that multiple devices are already equippedwith such a sensor and upcoming mobile devices aremost likely equipped with more powerful RGB(W) sen-sors, i.e., supporting higher sampling frequencies andresolutions.

Though we did not observe any problems with thelowest sampling frequency on the Samsung Galaxy SIII,we note that even lower sampling frequencies potentiallymight prevent this attack. This is due to the fact that forsampling frequencies below 5 Hz too few measurementsamples might be gathered if one enters the PIN too fast.

6 Limitations

The presented attack also has some limitations. First,our model does not consider mistyped PINs. If a userdeletes an incorrect digit and enters the correct digit af-terwards we are not able to infer the correct PIN any-more. Nevertheless, related work does not cover this caseneither. Second, we did not evaluate our attack outsideunder sunny, foggy or cloudy light conditions. However,we evaluated the attack in a room—where the only lightsource was a window—during different daytimes. Fur-thermore, the data used to train the machine-learning al-gorithm must be gathered in the same environment as theactual data that should be classified. Nevertheless, our at-tack scenario is based on the assumption that people usetheir smartphone while watching TV, while waiting in awaiting room, or during a train ride, which seems to bea reasonable assumption. Future work, however, mightinvestigate whether a more general model can be estab-lished in order to decouple the training phase from theactual attack phase. For instance, a “calibration” phasemight be used to determine the overall light conditionsin the user’s environment to speed-up the training phase.Third, due to the fact that the presented attack is basedon the ambient-light sensor, it does not work in case theuser operates the mobile device in a completely dark en-vironment. Though, a completely dark environment alsoprevents an attack that exploits the camera [45].

7 Analysis of Countermeasures

In this section, we discuss potential mitigation tech-niques to prevent the exploitation of sensor information.

UI and API Modifications. Aviv et al. [8] argue thatan effective security mechanism would be to prevent un-trusted applications from accessing motion sensors, atleast during the input of sensitive information. However,the crucial question is: When is an input considered asbeing sensitive? Clearly, the input of an authenticationPIN or a password represents a sensitive input. But whatabout the input while writing an e-mail or the data en-tered in forms on websites? Perhaps the sensors shouldbe disabled as soon as the virtual keyboard is displayed?However, this renders applications that rely on these sen-sors completely useless as is also stated by Aviv et al. [8].

Owusu et al. [38] suggest to vary the keyboard layoutfor sensitive inputs, which means that buttons are rear-ranged on the virtual keyboard prior to every sensitiveinput. The drawback of such countermeasures is the dra-matic decrease of usability. While this might be appli-cable for a PIN pad, it would definitely undermine theusability of the QWERTY keyboard layout.

11

Limiting the resolution and the sampling frequencyof the sensor might be another possible countermeasure.For instance, the ambient-light sensor is currently usedto adapt the screen brightness. For such an application amore coarse resolution as well as a lower sampling fre-quency should be sufficient. We are not aware of any sce-nario that requires a sampling frequency of 50 Hz or even750 Hz. As we have shown in this paper, even the lowestsampling frequency of 5 Hz on the Samsung Galaxy SIIIdoes not prevent the presented attack. However, reducingthe sampling frequency to 1-2 Hz should suffice for thetask of adapting the screen brightness and to hedge thepresented attack. Furthermore, since only the OS per-forms the task of adapting the screen brightness, accessto this sensor might be restricted to the OS exclusively.While these countermeasures might be quite effective,they limit the functionality of specific applications, e.g.,games that heavily rely on the usage of sensors.

Rethinking the Permission Model. A quite sophisti-cated countermeasure might be a fine-grained permis-sion system in mobile operating systems. Felt et al. [24]evaluated different permission-granting mechanisms in-cluding automatic granting, trusted UIs (cf. Roesner etal. [40]), runtime-consent dialogs, and install-time warn-ings. After considering their model we conclude that aneffective countermeasure would be an install-time warn-ing, i.e., to pause the installation process and to explicitlyinform or warn the user about the requested permission.

Specific risks that might arise from permissions mustbe communicated to the user, as has also been reportedby Felt et al. [25], especially since the manifold permis-sions confuse many users [25, 31]. However, excessivewarnings lose effectiveness and might cause users to ig-nore these warnings again. In order to overcome thisproblem Peng et al. [39] suggest to rank applications ac-cording to their risks rather than using a binary decisionfor the classification of vulnerable applications. Thisranking decision is based on the requested permissionsof applications that are known to be malware. Based onsuch a ranking users might make more deliberate deci-sions regarding the installation of applications. However,such rankings are only applicable if the motion sensors aswell as the ambient-light sensor are considered within thepermission system of the OS, which calls for security-specific actions of operating-system developers.

Application Analysis. A similar approach might beachieved by extending AppGuard [11]—an Android ap-plication to enforce security policies—to support the de-tection of possibly unwanted sensor accesses by mali-cious applications. AppGuard could scan applicationsduring the installation and inform the user about sen-sor accesses that potentially leak sensitive information.

Other malware-analysis applications such as static ana-lyzers, e.g., Stowaway [23] or AndroidLeaks [27], or ap-plications like VirusTotal [49] could also be extended tocheck applications for malicious sensor accesses.

User Behavior. Another possible countermeasuremight be to enter sensitive data only in environmentswithout any light source and with the index finger or astylus pen. However, in this case other sensors, e.g., mo-tion sensors, might still be exploitable. So, for really sen-sitive data, the user might cover the ambient-light sensoras well as the camera, e.g., with her finger, and place themobile device on a flat surface while providing the input.

Encouraging users to choose longer PINs and pass-words might also increase the security [45]. However,some applications do not even allow PINs with more than4–5 digits. Again, the drawback of such countermeasuresis the decreasing usability.

Last but not least, awareness must be raised amongstusers. Applications should not be able to gather informa-tion without knowledge of the user and users must be en-couraged to be wary when installing applications, whichis why studies like this one are essential.

8 Related Work

Side-channel attacks on input devices in general havebeen shown to occur both directly [6, 33, 47] as wellas indirectly, e.g., through oily residues on the touch-screen [7], or through reflections of monitors [9, 10].More specific investigations of threats represented bysensor-based side channels on mobile devices include,for instance, the work of Cai et al. [20]. They raised theawareness regarding the camera, the microphone, and theGPS signal in modern smartphones. In 2011, Cai andChen [18] claimed to be the first to show the privacyrisk of motion sensors utilized in smartphones. Han etal. [28] used the accelerometer sensor to infer the loca-tion of the device owner, even with the location-basedservices deactivated. In 2012, Owusu et al. [38] em-ployed the accelerometer sensor to infer passwords en-tered on touchscreens. Miluzzo et al. [36] and Xu etal. [50] made use of the accelerometer and motion sen-sors in general to infer the locations of taps on touch-screens. Recently, attacks have been presented to in-fer unknown PIN inputs based on a set of learned PINs.For instance, Aviv et al. [8] demonstrated such an attackby exploiting the accelerometer and Simon and Ander-son [45] demonstrated such an attack by exploiting thecamera and the microphone. We briefly compare relatedattacks to our attack within the following paragraphs.

12

Aviv et al. [8] Simon and Anderson [45] Ours

Sensor Accelerometer Camera Ambient-light sensorPermissions Internet Camera, Internet InternetTraining Independent of user/location For each user individually For each user/environment individuallyDrawbacks - LED and shutter sound on non-rooted devices Does not work in completely dark environmentsInput method No constraints Thumb of holding hand No constraintsAccuracy 43% within 5 guesses 50% within 5 guesses 65% within 5 guesses

Table 3: Comparison of related work targeting a set of 50 PINs.

Comparison. Compared to attacks based on motionsensors, a major advantage of the ambient-light sen-sor is the power consumption. The method an-droid.hardware.Sensor.getPower() returns the power inmA used by the corresponding sensor while in use [5].On our Samsung Galaxy SIII this method returns 0.2 mAfor the ambient-light sensor, 0.23 mA for the accelerom-eter, and 6.1 mA for the gyroscope. Hence, the ambient-light sensor consumes a factor of 30 less power than thegyroscope, which means that our attack is less prone togain the user’s attention through battery drainage.

Table 3 provides a comprehensive comparison of therelated work that is similar to ours, i.e., attacks targetinga specific set of PINs. All attacks assume to have Internetaccess in order to transfer the gathered data to a power-ful server that performs the machine learning. However,as argued in Section 3, the Internet permission can begained rather easily without raising the user’s suspicion.

The main drawback of our attack is that users are notallowed to walk around while entering the PINs becausethe data is only exploitable for one specific environment.Hence, the data cannot be reused for multiple attacks asin case of Aviv et al. [7]. Though, their results indicatea rather low success rate of 20% within 5 guesses wheninferring PINs that were entered while walking around.Furthermore, their attack also works in completely darkenvironments which does not hold for the ambient-lightsensor. However, our results indicate a better accuracy of65% within 5 guesses when inferring unknown PINs.

The work of Simon and Anderson [45] additionallyrequires the Camera permission which potentially gainsthe user’s suspicion. In addition, their attack must dealwith the problem of audio-visual feedback, e.g., the shut-ter sound or the LED, while capturing the required data.Compared to their work we do not restrict our study tospecific input methods as long as the user holds the de-vice while operating it. Furthermore, they need to trans-fer image data to the server, which cannot be representedas compact as sensor values.

To summarize this comparison, an attacker trades ahigher classification rate for two minor drawbacks whenusing the ambient-light sensor.

9 Conclusion and Future Work

In this paper, we investigated a new type of side channelwhich is based on the ambient-light sensor employed intoday’s mobile devices. To the best of our knowledge, weare the first to show that the ambient-light sensor indeedleaks sensitive information about the user’s input on thetouchscreen. We developed a proof-of-concept applica-tion that allows us to infer unknown PINs, when givena set of already known PINs. This application clearlydemonstrates that the leaked information represents a vi-able side channel for compromising the user’s privacyand security. Since no specific permission is requiredto access this sensor, an adversary is able collect sen-sitive inputs from mobile-device owners without raisingany suspicion and, thus, remain unrecognizable.

Any mobile device equipped with an ambient-lightsensor that provides a sufficiently high sampling rate andresolution can be exploited. Our investigation showedthat state-of-the-art smartphones—from all major devicemanufacturers—include an ambient-light sensor and therapid technical progress in combination with demandingusers possibly leads to an increasing resolution and sam-pling frequency of this sensor. For a list of potential vul-nerable devices see Appendix A.

There are many different dimensions among the infor-mation leakage of sensors can be investigated. Examplesof factors affecting the applicability and performance ofsensor-based side-channel attacks are, for instance, thesensor hardware itself, the screen dimension, the deviceorientation, the keyboard layout, the user’s behavior andhis typing style, the different classification algorithmsand the employed feature vectors, the actual environ-ment (e.g., indoor and outdoor), etc. These examplesdemonstrate that further research is necessary to evalu-ate the performance of all available sensors under differ-ent settings in order to determine the best sensor for aspecific attack scenario. For instance, Cai and Chen [19]as well as Al-Haiqi et al. [1] performed a comparison ofthe gyroscope and the accelerometer for multiple userson different devices in different settings to determinethe best motion-based sensor. It remains an open ques-tion whether a combination of motion sensors with theambient-light sensor might lead to better performances

13

than reported so far. However, the intention of this workwas to provide a first feasibility study. By comparingthree classification algorithms, different feature vectors,different input methods, different environments, and theimpact of different sampling frequencies, we showed thatthe ambient-light sensor provides a viable side channel.

Related work on the leakage of motion sensorsclaimed that access to these sensors must be limited witha fine-grained permission system. As shown in this work,access to the ambient-light sensor must also be protectedthrough such a permission system and, thus, operating-system developers need to deal with this problem. Prob-ably even more important is the fact that users need to beaware of such threats and be wary when installing appli-cations that require permissions to access sensors. Thisis actually why studies like this one are essential.

References[1] AL-HAIQI, A., ISMAIL, M., AND NORDIN, R. Keystrokes In-

ference Attack on Android: A Comparative Evaluation of Sensorsand Their Fusion. Journal of ICT Research and Applications 7, 2(2013), 117–136.

[2] ALPAYDIN, E. Introduction to Machine Learning (AdaptiveComputation and Machine Learning, 2nd ed. The MIT Press,2010.

[3] ANDROID DEVELOPERS. Android KitKat. https://

developer.android.com/about/versions/kitkat.html.

[4] ANDROID DEVELOPERS. Camera. http://developer.

android.com/reference/android/hardware/Camera.

html.

[5] ANDROID DEVELOPERS. Sensors Overview. http:

//developer.android.com/guide/topics/sensors/

sensors_overview.html.

[6] ASONOV, D., AND AGRAWAL, R. Keyboard Acoustic Emana-tions. In IEEE Symposium on Security and Privacy (2004), pp. 3–11.

[7] AVIV, A. J., GIBSON, K., MOSSOP, E., BLAZE, M., ANDSMITH, J. M. Smudge Attacks on Smartphone Touch Screens. InUSENIX Conference on Offensive Technologies (WOOT) (2010),pp. 1–7.

[8] AVIV, A. J., SAPP, B., BLAZE, M., AND SMITH, J. M. Prac-ticality of Accelerometer Side Channels on Smartphones. InAnnual Computer Security Applications Conference (ACSAC)(2012), pp. 41–50.

[9] BACKES, M., CHEN, T., DURMUTH, M., LENSCH, H. P. A.,AND WELK, M. Tempest in a Teapot: Compromising ReflectionsRevisited. In IEEE Symposium on Security and Privacy (2009),pp. 315–327.

[10] BACKES, M., DURMUTH, M., AND UNRUH, D. CompromisingReflections-or-How to Read LCD Monitors around the Corner. InIEEE Symposium on Security and Privacy (2008), pp. 158–169.

[11] BACKES, M., GERLING, S., HAMMER, C., MAFFEI, M., ANDVON STYP-REKOWSKY, P. AppGuard - Enforcing User Require-ments on Android Apps. In Tools and Algorithms for the Con-struction and Analysis of Systems (TACAS) (2013), pp. 543–548.

[12] BARCLAYS PLC. Mobile Banking Services. http:

//www.barclays.co.uk/Mobile/BarclaysPingit/

P1242603570446.

[13] BECHER, M., FREILING, F. C., HOFFMANN, J., HOLZ, T.,UELLENBECK, S., AND WOLF, C. Mobile Security CatchingUp? Revealing the Nuts and Bolts of the Security of MobileDevices. In IEEE Symposium on Security and Privacy (2011),pp. 96–111.

[14] BISHOP, C. M. Pattern Recognition and Machine Learning (In-formation Science and Statistics). Springer-Verlag New York,Inc., Secaucus, NJ, USA, 2006.

[15] BLACKBERRY. BlackBerry Balance. http:

//us.blackberry.com/business/software/

blackberry-balance.html.

[16] BLACKBERRY. BlackBerry Runtime 10.0. http://

developer.blackberry.com/android/.

[17] BONNEAU, J., PREIBUSCH, S., AND ANDERSON, R. A Birth-day Present Every Eleven Wallets? The Security of Customer-Chosen Banking PINs. In Financial Cryptography (FC) (2012),pp. 25–40.

[18] CAI, L., AND CHEN, H. TouchLogger: Inferring Keystrokes onTouch Screen from Smartphone Motion. In USENIX Conferenceon Hot Topics in Security (HotSec) (2011).

[19] CAI, L., AND CHEN, H. On the Practicality of Motion BasedKeystroke Inference Attack. In Trust and Trustworthy Computing(TRUST) (2012), pp. 273–290.

[20] CAI, L., MACHIRAJU, S., AND CHEN, H. Defending AgainstSensor-Sniffing Attacks on Mobile Phones. In ACM SIGCOMMWorkshop on Networking, Systems, and Applications for MobileHandhelds (MobiHeld) (2009), pp. 31–36.

[21] DOMOBILE. AppLock. https://play.google.com/store/apps/details?id=com.domobile.applock.

[22] EVERNOTE CORPORATION. Evernote. https://play.

google.com/store/apps/details?id=com.evernote.

[23] FELT, A. P., CHIN, E., HANNA, S., SONG, D., AND WAGNER,D. Android Permissions Demystified. In ACM Conference onComputer and Communications Security (CCS) (2011), pp. 627–638.

[24] FELT, A. P., EGELMAN, S., FINIFTER, M., AKHAWE, D., ANDWAGNER, D. How to Ask for Permission. In USENIX Confer-ence on Hot Topics in Security (HotSec) (2012).

[25] FELT, A. P., HA, E., EGELMAN, S., HANEY, A., CHIN, E.,AND WAGNER, D. Android Permissions: User Attention, Com-prehension, and Behavior. In Symposium On Usable Privacy andSecurity (SOUPS) (2012), p. 3.

[26] GARY MAZO. How to uncover and use the hidden Servicemenu on the Galaxy S3. http://www.androidcentral.com/how-uncover-and-use-hidden-service-menu-galaxy-s3.

[27] GIBLER, C., CRUSSELL, J., ERICKSON, J., AND CHEN, H. An-droidLeaks: Automatically Detecting Potential Privacy Leaks inAndroid Applications on a Large Scale. In Trust and TrustworthyComputing (TRUST) (2012), pp. 291–307.

[28] HAN, J., OWUSU, E., NGUYEN, L. T., PERRIG, A., ANDZHANG, J. ACComplice: Location Inference using Accelerom-eters on Smartphones. In Communication Systems and Networks(COMSNETS) (2012), pp. 1–9.

[29] JAKOBSSON, M., AND LIU, D. Your Password is Your New PIN.In Mobile Authentication, SpringerBriefs in Computer Science.Springer New York, 2013, pp. 25–36.

[30] KEEPSAFE. KeepSafe. https://play.google.com/store/

apps/details?id=com.kii.safe.

[31] KELLEY, P. G., CONSOLVO, S., CRANOR, L. F., JUNG, J.,SADEH, N. M., AND WETHERALL, D. A Conundrum of Per-missions: Installing Applications on an Android Smartphone. InFinancial Cryptography Workshops (2012), pp. 68–79.

14

[32] KOCHER, P. C. Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems. In Advances in Cryp-tology (CRYPTO) (1996), pp. 104–113.

[33] KUNE, D. F., AND KIM, Y. Timing Attacks on PIN Input De-vices. In ACM Conference on Computer and CommunicationsSecurity (CCS) (2010), pp. 678–680.

[34] MARQUARDT, P., VERMA, A., CARTER, H., AND TRAYNOR,P. (sp)iPhone: Decoding Vibrations From Nearby Keyboards Us-ing Mobile Phone Accelerometers. In ACM Conference on Com-puter and Communications Security (CCS) (2011), pp. 551–562.

[35] MATHWORKS. Statistics Toolbox. http://www.mathworks.

com/products/statistics/.

[36] MILUZZO, E., VARSHAVSKY, A., BALAKRISHNAN, S., ANDCHOUDHURY, R. R. Tapprints: Your Finger Taps Have Finger-prints. In Mobile Systems, Applications, and Services (MobiSys)(2012), pp. 323–336.

[37] NAB. NAB. https://play.google.com/store/apps/

details?id=au.com.nab.mobile.

[38] OWUSU, E., HAN, J., DAS, S., PERRIG, A., AND ZHANG, J.ACCessory: Password Inference using Accelerometers on Smart-phones. In Mobile Computing Systems and Applications (HotMo-bile) (2012), p. 9.

[39] PENG, H., GATES, C. S., SARMA, B. P., LI, N., QI, Y.,POTHARAJU, R., NITA-ROTARU, C., AND MOLLOY, I. UsingProbabilistic Generative Models for Ranking Risks of AndroidApps. In ACM Conference on Computer and CommunicationsSecurity (CCS) (2012), pp. 241–252.

[40] ROESNER, F., KOHNO, T., MOSHCHUK, A., PARNO, B.,WANG, H. J., AND COWAN, C. User-Driven Access Control:Rethinking Permission Granting in Modern Operating Systems.In IEEE Symposium on Security and Privacy (2012), pp. 224–238.

[41] SAMSUNG. Samsung Galaxy S4 Specifications. http://www.

samsung.com/global/microsite/galaxys4/.

[42] SAMSUNG. Samsung Galaxy SIII Specifications. http://www.samsung.com/global/galaxys3/specifications.html.

[43] SAMSUNG. Samsung KNOX. http://www.samsung.

com/global/business/mobile/solution/security/

samsung-knox.

[44] SAMSUNG. What You May Not Know About GALAXY S4.http://global.samsungtomorrow.com/?p=23610.

[45] SIMON, L., AND ANDERSON, R. PIN Skimmer: Inferring PINsThrough The Camera and Microphone. In ACM Workshop onSecurity and Privacy in Smartphones & Mobile Devices (SPSM)(2013), pp. 67–78.

[46] SOLIRIFY. Math Trainer. https://play.google.com/

store/apps/details?id=com.solirify.mathgame.[47] SONG, D. X., WAGNER, D., AND TIAN, X. Timing Analysis

of Keystrokes and Timing Attacks on SSH. In USENIX SecuritySymposium (2001).

[48] UK OFFICE OF COMMUNICATIONS. Communications MarketReport 2013. http://media.ofcom.org.uk/2013/08/

01/the-reinvention-of-the-1950s-living-room-2/,2013.

[49] VIRUSTOTAL. VirusTotal. https://play.google.com/

store/apps/details?id=com.virustotal.[50] XU, Z., BAI, K., AND ZHU, S. TapLogger: Inferring User Inputs

On Smartphone Touchscreens Using On-Board Motion Sensors.In Conference on Security and Privacy in Wireless and MobileNetworks (WISEC) (2012), ACM, pp. 113–124.

A Mobile Devices Potentially at Risk

Table 4 lists Android-based mobile devices as well as theempirically determined sampling frequencies and resolu-tions of the light sensor. For the Google Nexus S runningAndroid 2.3.6 and the Samsung Galaxy SII running An-droid 2.3.4 we state the observed lux values instead ofthe actual resolutions.

Except the HTC Nexus One, all investigated devicessupport a sampling rate that is high enough for the pre-sented attack. For two devices, i.e., the Google NexusS running Android 2.3.6 and the Samsung Galaxy SIIrunning Android 4.3, the limiting factor seems to be theresolution. The limited resolution in turn seems to be aresult of the employed operating system rather than thehardware itself. For instance, the Google Nexus S run-ning Cyanogenmod 4.0.4 provides a more fine-grainedresolution than the Google Nexus S running Android2.3.6. This also complies with our observation that read-ing the sensor values directly from the virtual file sys-tem, e.g., /sys/devices/virtual/lightsensor/switch cmd onthe Samsung Galaxy SII, provides a better resolution thanusing the official Android Sensor API.

Since this table contains only devices we had at hand,we expect far more devices to be vulnerable to attacksbased on the ambient-light sensor.

Device Operating system Sample rate [Hz] Resolution [lux]

Google Nexus S Android 2.3.6 ∼ 20 {10, 160, 320, 640}Google Nexus S Cyanogenmod 4.0.4 ∼ 140 1HTC Nexus One Android 2.3.6 ∼ 1 1LG Optimus G Android 4.1.2 ∼ 7 1LG Optimus G Pro Android 4.1.2 ∼ 7 1Samsung Galaxy SII Android 2.3.4 ∼ 10 {10, 100, 1000}Samsung Galaxy SIII Android 4.3 ∼ 750 1Samsung Galaxy S4 Mini Android 4.2.2 ∼ 100 1Samsung Galaxy Note II Android 4.3 ∼ 100 1

Table 4: Mobile devices and their corresponding operating system as well as the observed sampling rate and theresolution of the ambient-light sensor.

15