bayesian adaptive estimation of threshold versus … adaptive estimation of threshold versus...

www.elsevier.com/locate/visres

Vision Research 46 (2006) 3160–3176

Bayesian adaptive estimation of threshold versus contrastexternal noise functions: The quick TvC method q

Luis Andres Lesmes a,*, Seong-Taek Jeon b, Zhong-Lin Lu b,*, Barbara Anne Dosher c

a Vision Center Laboratory (VCL), Salk Institute for Biological Studies, La Jolla, CA 92037, USAb Laboratory of Brain Processes (LOBES), Departments of Psychology and Biomedical Engineering, and Neuroscience Graduate Program,

University of Southern California, Los Angeles, CA 90089-1061, USAc Memory, Attention, and Perception (MAP) Laboratory, Department of Cognitive Sciences and Institute of Mathematical Behavioral Sciences,

University of California, Irvine, CA 92697-5100, USA

Received 12 July 2005; received in revised form 31 March 2006

Abstract

External noise paradigms, measuring contrast threshold as a function of external noise contrast (the ‘‘TvC’’ function), provide a valu-able tool for studying perceptual mechanisms. However, measuring TvC functions at the multiple performance criteria needed to con-strain observer models has previously involved demanding data collection (often > 2000 trials). To ease this task, we developed a novelBayesian adaptive procedure, the ‘‘quick TvC’’ or ‘‘qTvC’’ method, to rapidly estimate multiple TvC functions, by adapting a strategyoriginally developed to estimate psychometric threshold and slope [Cobo-Lewis, A. B. (1996). An adaptive method for estimating multi-ple parameters of a psychometric function. Journal of Mathematical Psychology, 40, 353–354; Kontsevich, L. L., & Tyler, C. W. (1999).Bayesian adaptive estimation of psychometric slope and threshold. Vision Research, 39(16), 2729–2737]. Exploiting the regularitiesobserved in empirical TvC functions, the qTvC method estimates three parameters: the optimal threshold c0, the critical noise levelNc, and the common slope, g, of log-parallel psychometric functions across external noise conditions. Before each trial, the qTvC usesa one-step-ahead search to select the stimulus (jointly defined by signal and noise contrast) that minimizes the expected entropy of thethree-dimensional posterior probability distribution, p(Nc,c0,g). The method’s accuracy and precision, for estimating TvC functions atthree performance criteria (65%, 79%, and 92% correct), were evaluated using Monte-Carlo simulations and a psychophysical task. Sim-ulations showed that less than 300 trials were needed to estimate TvC functions at three widely separated criteria with good accuracy(bias < 5%) and precision (mean root mean square error <1.5 dB). Using an orientation identification task, we found excellent agreement(weighted r2 > .95) between TvC estimates obtained with the qTvC and the method of constant stimuli, although the qTvC used only 12%of the data collection (240 vs 1920 trials). The qTvC may hold considerable practical value for applying the external noise method tostudy mechanisms of observer state changes and special populations. We suggest that the same adaptive strategy can be applied to direct-ly estimate other classical functions, such as the contrast sensitivity function, elliptical equi-discrimination contours, and sensory memorydecay functions.� 2006 Elsevier Ltd. All rights reserved.

Keywords: External noise; Adaptive; Psychophysics; Bayesian estimation; Noise

0042-6989/$ - see front matter � 2006 Elsevier Ltd. All rights reserved.doi:10.1016/j.visres.2006.04.022

q MATLAB code for the method can be downloaded from http://lobes.usc.edu/qMethods.

* Corresponding authors.E-mail addresses: [email protected] (L.A. Lesmes), [email protected]

(Z.-L. Lu).URL: http://lobes.usc.edu (Z.-L. Lu).

1. Introduction

External noise methods measure sensory thresholds forstimuli embedded in external noise (Pelli & Farell, 1999).The classical external noise function, the threshold versusnoise contrast (TvC) function (Fig. 1), describes how signalcontrast thresholds change with external noise strength. Ini-tially used to characterize fundamental properties of the

http://dx.doi.org/10.1016/j.visres.2006.04.022


mailto:[email protected]

mailto:[email protected]

http://lobes.usc.edu

-4 -3 -2 -1

-5

-4

-3

-2

-1

-4

-3

-2

-1-6 -5 -4 -3 -2 -1 0

0.5

0.6

0.7

0.8

0.9

1

log c

log Next

(Nc, c0, 79%)

% C

orre

ct

log Next

log c

Nc

c0

a

b

Fig. 1. In external noise studies, the family of psychometric functions measured across different external noise levels can be characterized using threeparameters: (1) c0 defines the contrast level corresponding to the 79% threshold, over the low external noise region (2) Nc defines the critical noise level atwhich thresholds start increasing from c0, and (3) g defines the psychometric function’s slope, which determines the constant threshold ratio observedacross external noise levels. (a) The psychometric surface (percent correct as a function of noise and signal contrast) defined by these three parameters; (b)the typical 2-D presentation of multiple TvC functions (Lu & Dosher, 1998).

1 TvC functions measured at three widely separated performance levelsprovide excellent proxies for full psychometric functions over all externalnoise conditions.

L.A. Lesmes et al. / Vision Research 46 (2006) 3160–3176 3161

perceptual system (Ahumada, 1987; Ahumada & Watson,1985; Barlow, 1956; Burgess, Wagner, Jennings, & Barlow,1981; Legge, Kersten, & Burgess, 1987; Legge, Rubin, Pelli,& Schleske, 1988; Nagaraja, 1964; Pelli & Farell, 1999; Pelli,1990), external noise methods have also been used to assaymechanisms of attention (Eckstein, Pham, & Shimozaki,2004; Lu & Dosher, 1998; Lu & Dosher, 2004b), perceptuallearning (Chung, Levi, & Tjan, 2005; Dosher & Lu, 1998;Gold, Bennett, & Sekuler, 1999), object recognition (Tjan,Braje, Legge, & Kersten, 1995), and adaptation (Dao, Lu,& Dosher, 2006). Recently, the external noise method hasbeen applied to special populations, to characterize percep-tual deficits accompanying amblyopia (Huang, Tao, Zhou,& Lu, submitted; Levi & Klein, 2003; Pelli, Levi, & Chung,2004; Xu, Lu, Qiu, & Zhou, under review), dyslexia (Sper-ling, Lu, Manis, & Seidenberg, 2005), prosopagnosia (Man-gini & Biederman, 2002), development (Kiorpes &Movshon, 1998; Skoczenski & Norcia, 1998), and senes-cence (Bennett, Sekuler, & Ozin, 1999; Pardhan, 2004).

Several observer models have been developed to inter-pret TvC functions, including a linear amplifier model (Pelli,1981), a multiplicative noise model (Burgess & Colborne,1988), a multiplicative noise plus uncertainty model (Eck-stein, Ahumada, & Watson, 1997), and a perceptual tem-plate model (Lu & Dosher, 1999). With only twoparameters describing equivalent internal noise and efficien-cy, the simplest linear amplifier model is fully constrainedby measuring a TvC function at a single performance crite-rion (e.g., 75% correct). However, because a particularparameterization of the linear amplifier model only charac-terizes the observer for that given performance level, addi-

tional parameters are needed to characterize the observerfor other performance levels (Chung et al., 2005; Lu &Dosher, 2004a; Tjan, Chung, & Levi, 2002). To modelTvC functions across the full performance range with a sin-gle set of parameters, the linear amplifier model needs elab-oration, to include multiplicative noise (Burgess &Colborne, 1988) and either nonlinear transducer functions(Lu & Dosher, 1999) or channel uncertainty (Ecksteinet al., 1997; Pelli, 1985). We have shown that the perceptualtemplate model (PTM) accommodates the known standardproperties of TvC functions, and provides the best qualita-tive and quantitative account of data across a range of par-adigms (Lu & Dosher, 2002; Lu & Dosher, in preparation).To fully constrain the PTM, and thereby comprehensivelyaccount for TvC functions across the full performancerange, it is necessary and sufficient to measure TvC func-tions at three performance criteria (Lu & Dosher, 1999).1

For other noisy observer models, measurements of TvC

functions are also needed to characterize the perceptual sys-tem over a wide performance range. Lastly, measuring TvC

functions at multiple performance criteria is critical for dis-tinguishing mechanism mixtures in the study of attentionand perceptual learning (Dosher & Lu, 1999; Dosher &Lu, 2000; Lu & Dosher, 2000).

Unfortunately, measuring multiple TvC functions has sofar required a large investment in data collection. Forexample, adequate threshold estimates at three perfor-

3162 L.A. Lesmes et al. / Vision Research 46 (2006) 3160–3176

mance criteria, over eight external noise conditions—usingthe method of constant stimuli to measure psychometricfunctions at each noise level—typically demand more than2000 trials. Because such data collection is cumbersome forall applications of the external noise paradigm—and unten-able for applications in special populations—we havedeveloped a new adaptive procedure to estimate TvC func-tions with good accuracy and precision, using relatively fewexperimental trials.

2. The qTvC procedure

Our procedure, named the ‘‘quick TvC’’ or ‘‘qTvC’’method, builds on recent developments in Bayesian adaptiveprocedures for estimating the threshold and slope of psycho-metric functions (Cobo-Lewis, 1996; Kontsevich & Tyler,1999). Exploiting the regularities observed in empiricalTvC functions, the qTvC method uses only three parametersto characterize TvC functions over the full performancerange: the optimal threshold c0, the critical noise level Nc,and the slope, g, of the psychometric function. The methoduses responses to stimuli, jointly specified by the contrast ofsignal and external noise,~x ¼ ðs;N extÞ, to estimate a three-di-mensional probability distribution, pð~tÞ, defined over aspace of possible TvC functions; the posterior probabilitydistribution pð~tÞ describes the probability that the vector~t ¼ ðN c; c0; gÞ represents the observer’s TvC functions.

The adaptive psychophysical strategy, namely, using thesubject’s previous responses to present the most ‘‘informa-tive’’ stimulus on each trial (Leek, 2001; Treutwein, 1995)is formulated using the entropy of pð~tÞ. By simulating thenext trial’s outcome for all possible stimulus conditions,the qTvC method uses a one-step-ahead search (Cobo-Lewis,1996, 1997; Kontsevich & Tyler, 1999) to choose the stimulus~x providing the minimum expected entropy of pð~tÞ. Becauseentropy reflects the level of uncertainty in a probability distri-bution, minimizing the entropy of pð~tÞ corresponds to gain-ing information about the underlying TvC functions.

2.1. Parameterizing TvC functions

Several properties of TvC functions (for reviews, see Lu& Dosher, in preparation; Pelli, 1990) have been reportedin the classical external noise literature: (1) over low exter-nal noise levels, thresholds are nearly constant2 (Nagaraja,1964; Pelli, 1981); (2) over high external noise levels, con-trast thresholds increase with noise contrast, withslope � 1.0 in log–log space3 (Griffiths & Nagaraja, 1963;

2 The dip in TvC functions observed in contrast-pedestal paradigms(Legge & Foley, 1980) results from a phase-interaction between the targetstimulus and the pedestal. Because such interaction does not occurconsistently between the target stimulus and broadband noise, the dip isusually not observed in external noise paradigms.

3 This property, observed in nearly all paradigms using white noise(Pelli, 1990), is also predicted by all the existing noisy observer models(Lu & Dosher, in preparation).

Nagaraja, 1964; Stromeyer & Julesz, 1972; van Meeteren& Boogaard, 1973; as cited by Pelli, 1981), and (3) acrossall noise levels, the ratio between thresholds at two perfor-mance levels (e.g., 79% vs 65% correct), is constant (Chunget al., 2005; Lu & Dosher, 1999; Lu & Dosher, 2002). Col-lectively, these empirical regularities allow us to approxi-mate TvC functions across the full performance rangewith three parameters: (1) the optimal threshold c0—thethreshold contrast measured in low external noise levels(2) the critical noise level Nc—the external noise contrastat which thresholds start increasing from c0, and (3) theslope of the psychometric function, g.

The third property reported above—invariance ofthreshold ratios across noise contrasts—is consistent withthe observation that psychometric functions measured indifferent noise conditions are log-parallel (Burgess, Hum-phrey, & Wagner, 1979; Cohn, 1976; Gold, 2001; Griffiths& Nagaraja, 1963; Lu & Dosher, 1999; Pelli, 1981; Pelli,1985). Therefore, a psychometric function that translateson log-axes (e.g., log-Weibull or log-Gaussian; see Klein,2001; Strasburger, 2001a, 2001b; Treutwein, 1995; Watson& Pelli, 1983), can be fit to data obtained at different noiselevels with a common slope g, although thresholds, aN ext ,depend on external noise: wN ext

¼ f ðaN ext ; gÞ.Properties 1 and 2 describe how log contrast thresholds,

aN ext , approximately change with external noise contrast.Over low noise levels, (below the critical noise level Nc), logthresholds can be approximated by the optimal threshold:

aN ext � a0 ¼ logðc0Þ; if N ext 6 N c: ð1aÞProperty 2 reflects that, when external noise exceeds the crit-ical noise level, Nc, the corresponding increase in psycho-metric threshold (in log units), Da ¼ aN ext � a0, isapproximately equal to the log difference between the exter-nal noise contrast, Next, and the critical noise contrast, Nc:

aN ext ¼ a0 þ Da

� logðc0Þ þ logðN extÞ � logðN cÞ; if N ext > N c: ð1bÞ

Therefore, wN extðsÞ, describing the expected percent correct

as a function of signal and external noise contrast, s andNext, can be defined using the log-Weibull psychometricfunction4 as:

wNextðsÞ¼

cþð1�c�k=2Þ�f1�expð�expðgðlogðsÞ�a0ÞÞÞg; if N ext6N c

cþð1�c�k=2Þ�f1�expð�expðgðlogðsÞ�a0

� logðN extÞþ logðN cÞÞÞÞg; otherwise

8>>>>>><>>>>>>:

ð2Þ

where c and k are nuisance parameters determining thelower and upper asymptotes of the psychometric function

4 Although the theoretical basis and implication of fitting Weibullfunctions to psychometric data has been recently debated (Mortensen,2002; Tyler & Chen, 2000), the Weibull is used here as a good descriptionof psychometric data, not for its inference of underlying noise distribu-tions or nonlinearities.


(Treutwein & Strasburger, 1999; Wichmann & Hill, 2001a).These parameters, respectively, describing the chance per-formance level in the absence of a stimulus (for example,1/m in mAFC), and the proportion of lapsed trials at thehighest stimulus intensities, are assumed to be independentof external noise level.

Fig. 1 presents an example of the psychometric surface,percent correct as a function of signal and noise contrast,defined by Eq. (2). This surface, illustrating the family ofpsychometric functions typically measured in externalnoise paradigms (see Fig. 1, inset), is fully specifiedby the three-dimensional vector ~t ¼ ðN c; c0; gÞ ¼ ð10%;5%; 2:5Þ. Therefore, given an estimate of ~v ¼ ðN c; c0; gÞ,Eq. (2) can be reversed to estimate TvC functions at anyperformance criterion. The qTvC method is designed toestimate these three parameters using a minimum expectedentropy criterion (Cobo-Lewis, 1997; Kontsevich & Tyler,1999) to adaptively place stimuli at signal and noise levelsproviding the most information about defining featuresof multiple TvC functions.

2.2. Estimating TvC functions using the qTvC

We define a parameter space T~t, where ~t ¼ ðN c; c0; gÞspecifies the TvC functions defined by the particular valuesof Nc, c0, and g. The entire parameter space therefore repre-sents all the possible TvC functions specified by all the pos-sible~t’s in T~t. A stimulus space, X~x, is defined, containing thecombinations of signal and noise contrast, ~x ¼ ðs;N extÞ,which will possibly be presented in the experiment. Follow-ing the observer’s response to stimulus~xt ¼ ðs;N extÞ on trialt, the qTvC method uses Bayes’ rule to evaluate the posteriorprobability, ptþ1ð~tÞ, that a vector ~t in parameter space T~trepresents the TvC functions underlying the observer.Before running trial t + 1, the qTvC calculates ptþ1ð~tÞ forall possible stimulus conditions (i.e., combinations of signaland external noise contrast in the stimulus space X~x), andcalculates the expected entropy of ptþ1ð~tÞ, representing thedegree of uncertainty about the particular~t best describingthe observer. On trial t + 1, the method presents to theobserver the stimulus condition providing the minimumexpected entropy for ptþ1ð~tÞ and therefore the most informa-tion about the observer’s TvC functions.

To implement the qTvC method, it is useful to initializea conditional probability lookup table,5 pðcorrectj~x;~tÞ,specifying the probability of a correct response for all thepossible stimulus conditions and all the possible TvC func-tions defined in the parameter space T~t. Before the experi-ment, we define an a priori probability distribution poð~tÞ,representing our knowledge of pð~tÞ. Following the generalsequence prescribed by Kontsevich and Tyler (1999),before trial t + 1 (t = 0, . . .), the qTvC method estimates:

5 A lookup table is computed over a discretely sampled space of possiblestimuli X~x, and a sampled space of possible TVC functions, T~t (see Section3.2.5 for discussion of issues related to parameter and stimulus spacesampling).

(1) The probability of a correct response in a given stim-ulus condition~x ¼ ðs;N extÞ:

ptþ1ðcorrectj~xÞ ¼X

t

pðcorrectj~t;~xÞptð~tÞ:

(2) The posterior probability distributions ptþ1ð~tÞ follow-ing a correct and an incorrect response to each possi-ble stimulus ~x in trial t + 1 (the Bayes’ rule):

ptþ1ð~tj~x; correctÞ ¼ ptð~tÞpðcorrectj~x;~tÞPt

ptð~tÞpðcorrectj~x;~tÞ ;

ptþ1ð~tj~x; incorrectÞ ¼ ptð~tÞ½1� pðcorrectj~x;~tÞ�Pt

ptð~tÞ½1� pðcorrectj~x;~tÞ� :

(3) The entropies of the estimated posterior probabilityptþ1ð~tÞ following a correct and an incorrect responseto~x:

Htþ1ð~x; correctÞ ¼ �X

t

ptþ1ð~tj~x; correctÞ

� logðptþ1ð~tj~x; correctÞÞ;

Htþ1ð~x; incorrectÞ ¼ �X

t

ptþ1ð~tj~x; incorrectÞ

� logðptþ1ð~tj~x; incorrectÞÞ:

(4) The expected entropy after a trial with stimulus ~x:

E½H tþ1ð~xÞ� ¼ H tþ1ð~x; correctÞptþ1ðcorrectj~xÞþ Htþ1ð~x; incorrectÞptþ1ðincorrectj~xÞ:

(5) The stimulus condition providing the lowest expectedentropy:

~xtþ1 ¼ arg minx

E½H tþ1ð~xÞ�:

After the observer finishes trial t + 1,the posterior prob-ability function corresponding to ~xtþ1 and the observedresponse (correct or incorrect) is saved and used as the pri-or probability function for the subsequent trial.

In the next two sections, we describe Monte-Carlo sim-ulations and a psychophysical validation of the qTvC

method.

3. Testing the qTvC Method: Simulations

3.1. Method

Monte-Carlo simulations (1000 iterations) were used toestimate the precision and accuracy of the qTvC method.Each simulated experiment consisted of 1000 trials. Possiblesignal contrasts (s) ranged from 1.5% to 100% in 1 dB steps.Possible noise contrasts (Next) ranged from 2% to 33%, sam-pled in 3 dB steps. We selected the range of possible ~t’s(defining the parameter space T~t) based on the results ofprevious external noise studies (Chung et al., 2005; Daoet al., 2006; Gold, 2001; Lu & Dosher, 1998, 1999, 2002,2004a, 2004b, in preparation; Lu, Jeon, & Dosher, 2004;


Pelli, 1990): Nc ranged from 2.5% to 25%; c0 ranged from2.5% to 33%; and g ranged from .5 to 7, with 1 dB samplingover each parameter’s range. Like other Bayesian methods,the efficiency of the qTvC method can be improved by usinga better informed a priori probability distribution poð~tÞ. Thepresent simulations assumed that the a priori probabilitydistribution poð~tÞ was uniform over T~t. In certain applica-tions, better informed priors can be used.

The simulated observer, specified by~t0 ¼ ð10%; 12%; 2:5Þ, matched a typical observer in a2AFC orientation identification experiment (Lu & Dosher,1999). With a lapse rate of 4%, the observer’s psychometricfunction exhibited a lower asymptote of 50% and an upperasymptote of 100� 4

2¼ 98%. Based on these parameters

and Eqs. (1) and (2),~t0 completely specified the observer’sperformance (percent correct) in all possible stimulusconditions. For each simulated trial,~t0 and Eqs. (1) and (2)were used to compute the expected percent correct p, basedon signal and noise contrast. After drawing a randomnumber r from a uniform distribution over [0,1], theobserver’s response was marked correct if r < p, and markedincorrect otherwise.

Every 10 trials, t was estimated from pð~tÞ using themean of the posterior’s marginal probability distributions(King-Smith & Rose, 1997; Kontsevich & Tyler, 1999).TvC functions at three performance criteria were estimatedfrom t. Reversing Eq. (2), the threshold estimate sij wascalculated for each external noise condition, N ðjÞext

(j = 1, . . . , 9), and each performance criterion, Pi = 65%,79%, and 92% (i = 1,2,3):

sij ¼ exp aN ðjÞextþ 1

glog � log 1� P i

1� c� k=2

� �� ;

ð3Þwhere c = .5, k = .04, and aN ext , describing psychometricthreshold as a function of external noise contrast, is definedby Eq. (1). The influence of such factors as the assumedlapse rate, sampling range and grain of the stimulus space,X~x, and parameter space, T~v, and the functional form of thepsychometric function is examined later, (Sections 3.2.5and 3.2.6), with more conceptual detail and comprehensivesimulations.

3.2. Results

We present the qTvC method’s stimulus placement strat-egy and its accuracy and precision for estimating (a) theparameter vector ~t0, and (b) TvC functions (sij generatedby t). Estimates of TvC functions obtained with the qTvC

are compared with estimates obtained with the method ofconstant stimuli (MCS).

3.2.1. Stimulus placement

Fig. 2 illustrates the qTvC’s sampling of the stimulusspace, X~x. Presentation frequency, as a function of noiseand signal contrast, is presented for different experimentalepochs. To aid evaluation, the simulated observer’s ‘‘true’’

TvC functions (at 65%, 79%, and 92%) are overlaid. Overthe first 50 trials, stimuli are exclusively presented at twonoise levels (one low and one high), with a unimodal distri-bution of signal contrasts across a wide range of perfor-mance levels on both psychometric functions. Forsubsequent epochs of the experiment (for example, trials51–240), the distribution of signal contrasts is bimodal:presentation alternates between two signal contrasts,roughly corresponding to performance at 70% and 90%(Kontsevich & Tyler, 1999). As the session progresses,stimuli are placed at intermediate noise levels, over whichsignal contrasts alternate between 90% performance (fornoise levels on the elbow’s left side) and 70% performance(for levels on the elbow’s right side).

Two more notable properties of the qTvC’s stimulusplacement strategy are (1) relatively infrequent samplingof the stimulus space region corresponding to the elbowof the TvC function, (Nc, c0); and (2) relatively frequentsampling of high signal contrasts: observer performanceaverages 85%. In fact, over 50% of trials occur in stimulusconditions corresponding to >90% correct performance.This is consistent with the observation that large numbersof trials near the upper asymptote of the psychometricfunction are needed to accurately estimate its thresholdand slope (Green, 1990; Kontsevich & Tyler, 1999; Wich-mann & Hill, 2001a, 2001b). This feature of the qTvC

method bodes well for applications to special populations:such a high rate of correct response is comfortable for non-experienced psychophysical observers.

3.2.2. Accuracy

The qTvC’s accuracy for estimating the parameters Nc,c0, and g, and TvC functions, is evaluated. The bias(in dB) between the actual parameter,~tðiÞ0 , and its estimate,tðiÞ, for i = 1,2,3, is calculated from N = 1000,(k = 1, . . .,N), simulation repetitions:

biasðtðiÞ0 Þ ¼P

20log10 tðiÞk

� �� 20log10 tðiÞ0

� �h iN

: ð4Þ

Fig. 3a presents bias estimates for each parameter, (Nc, c0,and g), as a function of completed trial number. For eachqTvC parameter, estimation bias decreases to less than±.2 dB (<2.5%) after 300 trials, and thereafter continuesdecreasing. Using Eq. (3), the qTvC estimate, t, obtainedin each of N simulations provided a threshold estimate,sij, for each external noise condition, N ðjÞext (j = 1, . . ., 9),and each performance criterion, Pi = 65%, 79%, and 92%(i = 1,2,3). The mean bias of TvC estimates (for J = 9noise conditions), averaged over N = 1000 simulations, iscalculated for each performance criterion:

biasðsiÞ ¼P

j

P20log10ðsijÞ�20log10 sk

ijð Þ½ �N

J: ð5Þ

Fig. 4a illustrates the mean bias for TvC estimates as afunction of the number of simulated trials. Bias of thresh-

Fig. 2. The minimum entropy criterion used by the qTvC presents stimuli (defined by signal and external noise contrast) that maximize the informationgained about the observer’s TvC functions on each trial. The frequency of stimulus presentation, as a function of signal and noise contrast, is presented fordifferent epochs (trials 1–50, 51–240, 241–480, and 481–1000) of a qTvC experiment, overlaid on a simulated observer’s TvC functions (65%, 79%, and92%). The inset circles provide a standard for estimating stimulus presentation frequency as a function of signal and noise contrast: the diameter of thecircle presented at each point in the discretely sampled stimulus space, X~x, is proportional to stimulus presentation frequency.

0

1

2

3

4

bias

(dB

)

100 1000trials

Bias

100

1000

Confidence Intervals: 65%

c0c0

NcNc

Interval W

idth

(dB

)

0

2

4

6

8

10

12

14a b

trials

Fig. 3. Accuracy and precision for estimating the three qTvC parameters: Nc, c0, and g. The true qTvC parameter vector generating the simulated data was~t ¼ ð10%; 12%; 2:5Þ. (a) Bias for Nc, c0, and g estimates, as a function of completed trials. (b) The mean width of 65% confidence intervals (in dB) for Nc,c0, and g estimates, as a function of completed trial number. These simulations suggest that qTvC parameter estimates reach reasonable accuracy(bias < 1 dB) and precision (65% confidence interval width < 4 dB) in the region of 200–300 trials.


10 30 100 300 1000

-3

-2

-1

0

1

2

bias

(dB

)

log(noise contrast)

log(signal contrast)

240 trials

-1-3-50

-1

-1.5

-2

-2.5

92%

79%

65%

92%

79%

65%

trials

a b

Fig. 4. Accuracy and precision of TvC estimates, at three performance criteria (65%, 79%, and 92% correct), obtained with the qTvC. (a) Bias estimates forTvC functions at 65%, 79%, and 92% correct are presented as a function of completed trials. Bias decreases to less than ±.5 (<5%) dB after 200–300 trials.(b) The solid lines represent the ‘‘true’’ TvC functions defined by the stimulated observer,~t ¼ ð10%; 12%; 2:5Þ. Circles mark the mean threshold estimatesobtained from the qTvC with 240 trials, and error bars represent ±1.0 SD. For the three criteria, these error bars reflect threshold variability (1 SD)between 1 and 1.5 dB.


old estimates for each performance criteria falls below±.5 dB, (<5%), after 240 trials. It’s apparent that the smallbias in the psychometric slope estimate (bias(g) � 2.5%) at240 trials does not greatly bias estimates of TvC functions.

3.2.3. Precision

Fig. 3b presents the mean width (in dB) of the 65% con-fidence intervals, for estimated parameters N c, c0, and g asa function of the number of completed trials. By 300 trials,the mean width of the 65% confidence intervals decreasesto 2 dB for N c and c0, and to 4 dB for g. These intervalwidths correspond to standard errors of ±11.5% and±23%, which are typically acceptable stop criteria for psy-chophysical experiments (Kontsevich & Tyler, 1999).

The standard deviation (in dB) of each threshold esti-mate, eðsijÞ, was calculated from N = 1000 (k = 1, . . .,N)simulations, for each external noise condition, N ðjÞext

(j = 1, . . ., 9), and each performance criterion, Pi = 65%,79%, and 92% (i = 1,2,3):

eðsijÞ ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiP20log10ð�sijÞ � 20log10 sk

ij

� �� 2

N � 1

s: ð6Þ

Fig. 4b presents the average TvC estimates (over N = 1000simulations) provided by 240 qTvC trials overlaid on the‘‘true’’ TvC functions. The error bars represent standarddeviation estimates calculated with Eq. (6).

The precision for estimating multiple TvC functions issummarized by the mean error of threshold estimates, overJ = 9 noise levels and I = 3 performance criteria:

�eTvC ¼P

i;jeðsijÞI � J

: ð7Þ

Fig. 5a, presenting the mean error, �eTvC, of TvC estimates asa function of completed trial number, demonstrates thatstandard error falls below 1.5 dB after 200 trials anddecreases to 1 dB after 300 trials. Simulations showed thatthis precision for estimating TvC functions agrees with thatpredicted by the precision of qTvC parameter estimation.

Together with the previous bias analysis, these resultssuggest that about 300 trials are sufficient for accurate(bias < 5%) and precise estimates (error < 1.5 dB) of bothqTvC parameters and TvC functions at three performancecriteria (65%, 79%, and 92%), although fewer trials may beneeded when estimating TvC functions.

3.2.4. Simulated comparison with the method of constant

stimuli

We compared the precision of TvC estimates, at threeperformance levels, obtained with the qTvC and the methodof constant stimuli (MCS). The same simulated observerwas tested using the method of constant stimuli, using upto 3600 trials per simulation. Constant stimuli were placedat five contrast levels: 58%, 78%, 100%, 129%, and 150%of threshold for each external noise level, based on the spec-ification of the simulated observer, ~t0 ¼ ð10%; 12%; 2:5Þ;these stimulus levels were near optimal for estimatingthreshold and slope of psychometric functions at each noisecontrast (Green, 1990; Wichmann & Hill, 2001b). The sim-ulated results were fit with the psychometric functionsdefined in Eq. (2), although aN ext , describing the relationshipbetween thresholds at different noise contrasts, was definedtwo ways: aN ext was assumed to (1) follow the bilinearapproximation, defined by Nc and c0, described in Eqs.(1), or (2) vary independently across external noise condi-tions. Following Eq. (2), both models assumed a commonpsychometric slope across noise conditions. Hereafter, theestimates of TvC functions provided by these models willbe referred to as TvCMCS_constrained and TvCMCS estimates.

For each MCS method, we calculate the root meansquare error (RMSE) of threshold estimates, Eq. (6), as afunction of completed trial number. The results are shownin Fig. 5a. Examining the relative shape and position ofthe error curves presented in Fig. 5a suggests that reachinga given precision level (ranging from .5 to 2 dB) with theqTvC requires 25% of the trials needed by TvCMCS, and80% of the trials needed by TvCMCS_constrained estimates.Not surprisingly, combining the qTvC parameterization

10 100 10000

1

2

3

mea

n R

MS

E (

dB)

0

4

1

2

3

qTvC

TvCMCS_constrained

TvCMCS

a b

10 100 1000

mean R

MS

E (dB

)

10 30 100 300

14%

12%

8%

4%

1%

c

Trials

Fig. 5. Precision of TvC function estimates, when (a) compared to TvC estimates obtained with the method of constant stimuli (MCS); (b) the samplinggrain of the parameter space Tv is varied, and (c) the observer’s lapse rate varies (from 1 to 14%), from the qTvC’s assumed lapse rate (8%). For thesimulations presented in (a), psychometric data was simulated using constant stimulus levels providing near optimal sampling of signal contrast at each ofnine noise levels. To generate TvC estimates, the MCS psychometric data obtained at each noise level were fit with two alternative models, usingmaximum-likelihood: (1) independent thresholds (9 thresholds) and single slope parameter (TvCMCS), or (2) the three parameter description of TvC

functions (Eq. (2)) used by the qTvC (TvCMCS_constrained). For each simulated data set, threshold estimates at three performance criteria, for nine noiselevels, were estimated from the best-fiting models. We calculate the root mean squared error (RMSE) for each threshold over all simulations, and presentthe mean RMSE over all the threshold estimates in (a). The relative position of the curves for the three procedures suggests that the qTVC

parameterization offers its own advantage, independent of adaptive stimulus placement. However, it should be noted that the precision of MCS estimatesis probably overestimated, as they depend on optimal stimulus placement, which is difficult when attempting to measure nine psychometric functions. (b)The precision of TvC estimates is presented, for simulations varying the sampling grain (in dB) of the parameter space, T~t. For sampling schemes (in dB)ranging from fine, (.5, .5,1.25), to coarse (1.5,1.5,2), the qTvC’s precision, as a function of completed trial number, was unchanged. (c) The precision ofTvC estimates as a function of the observer’s lapse rate. With the lapse rate assumed by the qTvC fixed at 8%, the precision of TvC estimates depended noton the lapse rate mismatch, but on the lapse rate magnitude (lower lapse rates provided better precision).


with the near optimal stimulus placement assumed by ourMCS simulations results in very efficient TvCMCS_contsrained

estimates. However, these simulations overestimate the effi-ciency of TvC estimation using constant stimuli; becauseoptimal sampling of the psychometric function, dependenton a priori specification of thresholds over all external noiseconditions, is usually impossible in an experimental setting,the practical efficiency of the method of constant stimuli islower. In contrast, the only a priori knowledge the experi-menter needs to apply the qTvC method is readily availablefrom the literature (or minor preliminary data collection).

3.2.5. The qTvC’s stimulus and parameter space: Sampling

grain and range

The basic experimenter input needed by the qTvC

method is the specification of sampling range and grainfor the stimulus space X~x and the parameter space T~t. Sig-nal contrasts used in external noise paradigms, selected tospan psychometric functions in low and high noise condi-tions, can vary widely across tasks (e.g., from .004 to18% in motion discrimination and 3–99% for orientationdiscrimination); however, within a task, the range of signalcontrasts is typically 30–35 dB. For experiments estimatingthresholds that may change over multiple measurements(e.g., perceptual learning), a signal contrast range of40 dB, with 1 dB sampling, is appropriate.6 Capping the

6 This sampling grain for stimulus intensity is standard for adaptiveprocedures. A 40 dB sampling range is wide enough that, given aparticular task, an experimenter should have a clear sense of theappropriate range.

sampling range of external noise contrast at 33% (for dis-plays with a ±100% contrast range) maintains a goodapproximation of a Gaussian distribution in the maximumexternal noise condition. If the minimum noise contrastused is 2%, then 3 dB sampling over this noise range(2–33%) corresponds to nine noise levels.

Similar issues apply to sampling the parameter space,T~t. Inter-observer variability suggests a 20 dB samplingrange is appropriate for optimal thresholds, c0. There is lessinter-observer and inter-task variability for critical noise,Nc: dozens of experimental data sets show critical noise lev-els typically fall between 5 and 10% (Chung et al., 2005;Dao et al., 2006; Gold, 2001; Lu & Dosher, 1998, 1999,2002, 2004a, 2004b, in preparation; Lu et al., 2004; Pelli,1990). Therefore, if 2.5–25% is used as the parameter rangefor critical noise Nc, the range includes many critical noisevalues that have not been previously observed. In Fig. 7c,the lightly shaded region marks the recommended samplingrange for optimal threshold, c0 (2.5–33%), and criticalnoise (2.5–25%); the darkly shaded region marks the rangeof normally observed critical noise values (5–10%) Therange of psychometric slope (.5–7) was suggested by previ-ous investigations (King-Smith & Rose, 1997; Kontsevich& Tyler, 1999).

Additional simulations examined how the qTvC’s preci-sion is affected by the sampling grain of the parameterspace T~t. The same simulated observer was tested withthe qTvC, using five different sampling schemes s, wheres = (s1, s2, s3) defines the respective sampling grain (indB) for Nc, c0, and g: (.5, .5, 1.25), (1, .5, 1.5), (1,1,1.5),(1.5,1,2), and (1.5,1.5, 2). Fig. 5b demonstrates the negligi-


ble effects of sampling scheme on the precision of theestimated TvC functions: the number of trials (250–350)needed to reach TvC estimates of good precision (RMSE�1–1.25 dB) does not depend on sampling grain. Theindependence of parameter sampling grain (over the rangestudied) makes it possible to implement the qTvC methodon even relatively old equipment: with a PowerMac G4(running OS 9 and MATLAB with 512 MB RAM), thetrial-to-trial computing needed by the qTvC method takesless than 500 ms.

3.2.6. Observer-method mismatches: Lapse rate and

parameter range

When applying adaptive Bayesian methods, it is impor-tant to consider the possible bias and imprecision intro-duced by mismatches between the true properties of theobserver and those assumed by the method (Alcala-Quinta-na & Garcia-Perez, 2004). In the current study thus far,simulations have assumed complete agreement betweenthe simulated observer’s veridical lapse rate (ktrue) and thatassumed by the qTvC: kqTvC = 4%. Because such consisten-cy is rare in experimental applications, an additional set of

Fig. 6. Simulations examined the bias introduced by the mismatch betweenkqTvC = 8%. (c) Presents the bias for qTvC parameters (top row) and TvC estimfor five ‘‘veridical’’ lapse rates: ktrue = 1%, 4%, 8%, 12%, and 14%. The asymptoin (a and b). The lapse mismatch introduces biases estimates of optimal thresholbiased estimates of c0 and g apparently tradeoff, so that bias is low, (< ± 5 dB

simulations examined how qTvC estimates were affected bylapse mismatch (Alcala-Quintana & Garcia-Perez, 2004;Green, 1995; Kontsevich & Tyler, 1999; Wichmann & Hill,2001a). The same simulated observer,~t0 ¼ ð10%; 12%; 2:5Þ,was used, but her veridical lapse rate, ktrue, could take val-ues of 1%, 2%, 8%,12%, or 14%. To increase the possiblerange of lapse mismatch (both over- and under-estima-tion), the qTvC’s assumed lapse rate was increased:kqTvC = 8%. Fig. 6c (top row) presents bias estimates foreach qTvC parameter as a function of trial number for eachlapse rate tested. Fig. 6a summarizes the parameter biasresults, presenting the asymptotic bias for each qTvC

parameter as a function of the observer’s veridical lapserate, with dotted line signifying the method’s assumed lapserate. Similarly, Fig. 6b summarizes the lapse mismatcheffects for estimating TvC functions: asymptotic bias foreach performance criterion (65%,79%, and 92%) as a func-tion of the observer’s veridical lapse rate.

As suggested by previous simulations, there is no system-atic bias for qTvC parameters when the observer’s lapse ratematches the method’s assumed lapse rate: ktrue = kqTvC.Under conditions of lapse mismatch, ktrue 5 kqTvC, the bias

the observer’s lapse rate, ktrue, and the method’s assumed lapse rate:ates at 65%, 79%, and 92% (bottom row) as a function of completed trials,tic bias estimates for qTvC parameters and TvC estimates are summarized

d, c0, and psychometric slope, g, but not for critical noise, Nc. Interestingly,), for TvC estimates over a broad range of lapse rates (1–14%).


pattern for psychometric slope, g, and optimal threshold, c0,is clearly illustrated in Fig. 6a: when the observer’s lapserate is underestimated (kqTvC < ktrue), c0 is likewise underes-timated (bias < 0), and psychometric slope, g, is overesti-mated (bias > 0). The opposite pattern emerges whenlapse rate is overestimated (kqTvC > ktrue): that is, c0 is over-estimated and g is underestimated. Furthermore, consistentwith the conclusions of Alcala-Quintana and Garcia-Perez(2004), the magnitude of bias was greater when the assumedlapse rate underestimated the observer’s true lapse rate.Therefore, a conservative approach to achieving accurateestimates of qTvC parameters, especially in applicationswith lapse-prone observers, (e.g., special populations)would be to slightly overestimate lapse rate.

When the lapse rate of a given observer ~t0 increasesfrom 1 to 14%, the corresponding change in the psychomet-ric function’s upper asymptote can change the observer’sTvC function at 92% correct by up to 2.5 dB.7 Therefore,for each lapse rate, the bias of threshold estimates at eachperformance criterion is calculated relative to the empirical(non-guessing-corrected) TvC functions generated by thelapsing observer (see Fig. 6c, bottom row). Unlike the pat-terns observed for qTvC parameters (top row), the bias pat-tern in the bottom row of Fig. 6c is remarkably consistentover conditions of lapse rate mismatch (ktrue = 1–14% vskqTvC = 8%). Fig. 6b demonstrates that the asymptotic biasin TvC estimates (over the three criteria: 65%, 79%, and92%) does not exceed ±.5 dB across the range of lapse ratesstudied.

Although the accuracy of TvC estimates is not affectedby high lapse rates, there are effects on the precision ofTvC threshold estimates (see Fig. 5c). The laminarityexhibited by precision curves in Fig. 5c clearly demon-strates that the qTvC’s precision improves as the observer’slapse rate decreases, even when the observer’s low lapserate mismatches the assumed lapse rate (8%).

Bias and imprecision for adaptive Bayesian estimatescan also result when the assumed parameter space, T~t, doesnot sufficiently encompass the observer’s psychometricparameters (King-Smith & Rose, 1997). The parameterspace, T~t, specified for previous simulations, was basedon empirical results reported in the literature (see Section3.2.5). Though the range of the recommended parameterspace was conservative, there remains the possibility ofan atypical observer t = (4%, 3%, 1.0), corresponding tothe edges of the parameter space, (see Fig. 7c). The observ-er’s odd TvC functions, and their relation to the parameterrange, are apparent by inspection of Fig. 7c’s inset, whichdisplays the same parameter range relative to typical TvC

functions. Simulations examined the accuracy and preci-sion of qTvC estimates when the assumed parameter rangedid not adequately match the TvC functions of an atypical,

7 The parameter vector ~t0 specifies the observer’s guessing-correctedpsychometric function. However, an increasing lapse rate truncates thepsychometric function at the upper asymptote, thereby shifting TvC

functions defined by the same~t0 downwards.

aberrant observer. Although the estimates of the observer’sqTvC parameters and TvC functions are biased (see Figs.7a and b), the TvC functions estimated with only 100 qTvC

trials (see Fig. 7c) clearly reflect the aberrance of theobserver’s functions. At 100 trials, the bias of qTvC param-eters and TvC estimates are less than 2 dB. One possibleway to avoid the effects of parameter range mismatch8 isa stopping criterion: if the marginal probabilities at theborders of p(Nc, c0, g) exceed 15%, qTvC sessions can beinterrupted and rerun (with a properly specified parameterrange). This problem should recede as computing powerincreases: it should become possible to specify more expan-sive parameter spaces that easily encompasses nearly allTvC functions likely to be empirically observed.

A last set of simulations examined the effects of anotherparametric mismatch: the specific form of the psychometricfunction. The simulated observer’s performance (TvC func-tions) was generated by a log-Gaussian psychometric func-tion, which mismatched the log-Weibull psychometricfunction assumed by the qTvC. As expected from the smallpractical differences between these psychometric functions(Pelli, 1987), psychometric function mismatch only intro-duced a small systematic bias (�5%) for TvC estimates atthe 65% percent correct criterion.

Taken together, these simulation results suggest that themost precise and accurate estimates of qTvC parametersand TvC functions occur when the observer’s lapse rate isboth (1) low and (2) equal to the method’s assumed lapserate. Surprisingly, when the observer’s lapse rate mismatch-es the assumed rate, the resulting biases in qTvC parame-ters trade-off, and the accuracy of TvC functions isrelatively maintained. Furthermore, even under conditionswhen the parameter range or functional form of the psy-chometric function is misinformed, the qTvC delivers rea-sonably accurate (bias < 1 dB) and precise (error < 2 dB)estimates of TvC functions with about 300 trials. Thesesimulations demonstrate the robustness of the qTvC

method under non-optimal conditions likely to be encoun-tered in experimental and clinical settings.

4. Testing the qTvC method: Psychophysical validation

We conducted a psychophysical experiment to directlycompare TvC estimates obtained with the qTvC procedureand the method of constant stimuli (MCS).

4.1. Method

4.1.1. Apparatus

The experiment was conducted on a Macintosh PowerG4 computer running Psychtoolbox extensions (Brainard,1997; Pelli, 1997). The stimuli were presented on a HewlettPackard hp91 color monitor at a 120 Hz refresh rate.

8 Another possibility is to perform post hoc fitting of psychometric datawith maximum-likelihood or Bayesian fitting procedures (Jeon, Lu,Lesmes, & Dosher, in preparation; Treutwein & Strasburger, 1999.

Fig. 7. The results of simulations testing the qTvC’s accuracy and precision when the method’s assumed parameter range does not adequately contain theobserver’s TvC functions. The above figure presents the bias of (a) qTvC parameters and (b) TvC threshold estimates as a function of trial number for anaberrant observer. (c) The observer’s true TvC functions are overlaid and the inset demonstrates the coverage of normal TvC functions provided by theassumed parameter range for optimal threshold, c0, and critical noise, Nc. TvC estimates, obtained with 100 qTvC trials, are represented by circles. Eventhough 100 trials provide qTvC parameter and TvC threshold estimates with bias � 2 dB, the method does provide a good sense of the observer’s aberrantfunctions. One way to avoid such mismatch may be to stop the experiment if some probability level is exceeded by the parameters at the edge of theparameter space. If the marginal probabilities observed at the borders of the parameter range are too large (e.g., >15%) the qTvC session should be re-run.


A special circuit, (Li, Lu, Xu, Jin, & Zhou, 2003), changedthe display to a monochromatic mode, with high grayscaleresolution (>12.5 bits). A lookup table was used to linear-ize the luminance levels. Stimuli were viewed binocularlywith natural pupil at a viewing distance of approximately72 cm in dim light. Observers used a chinrest to maintainhead position and aid fixation throughout the experiment.

4.1.2. Participants

Two naıve observers (C.C. and W.C.) and the secondauthor (S.J.) participated in the experiment. All observershad corrected-to-normal vision and were experienced inpsychophysical studies.

4.1.3. Stimuli

The signal stimuli were Gaussian-windowed sinusoidalgratings, oriented h = ±45 degrees from vertical. The lumi-nance profile of the Gabor stimulus is described by:

Lðx; yÞ ¼ L0 1:0þ c sin½2pf ðx cos hþ y sin hÞ� exp �ðx2 þ y2Þ2r2

� �ð8Þ

where c is the signal contrast, r = 0.57� is the standarddeviation of the Gaussian window, and the backgroundluminance L0 was set in the middle of the dynamic rangeof the display (Lmin = 1 cd/m2; Lmax = 55 cd/m2).

The signal stimuli were rendered on a 64 · 64 pixel grid,extending 2.78 · 2.78 deg of visual angle. External noiseimages were constructed using 2 by 2 pixel elements

(0.087 · 0.087 deg). Each noise element’s contrast levelwas drawn independently from a Gaussian distributionwith mean of 0 and standard deviation ranging from .01to .33. Because the maximum achievable contrast is±100%, a noise sample with standard deviation of 0.33conforms reasonably well to a Gaussian distribution. In agiven trial, external noise images were made of elementswith jointly independent, identically distributed contrasts.Eight external noise levels (0%, 3%, 4.5%, 6.7%, 10%,15%, 22%, and 33%) were used in the experiment.

4.1.4. Design

TvC functions were obtained with the qTvC method andthe method of constant stimuli (MCS). The MCS psycho-metric functions (for each of eight external noise levels)were sampled at five signal contrast levels, specified foreach observer based on preliminary data. For the qTvC

method, signal contrast was sampled from a pool of 40possible contrast levels, ranging from 2 to 100% with1 dB step size. For the parameter space T~t : N c ranged from2.5 to 25%; c0 ranged from 2.5 to 33%; and g ranged from.5 to 7, with 1.5 dB sampling over each parameter’s range.For two observers (W.C. and C.C.), the lapse rate wasassumed to be 1%. For the other observer (S.J.), the lapserate was assumed to be higher: 8%.

Each observer completed four sessions of 960 trials. Ineach session, observers ran 480 qTvC trials and 480 con-stant stimulus trials, randomly interleaved. An experimen-tal session lasted about 1 h.


4.1.5. Procedure

In the beginning of each trial, a fixation-cross was pre-sented in the center of the screen for 500 ms. The subse-quent stimulus sequence consisted of three 8.3 ms frames:a noise frame, a signal frame, and another (independent)noise frame. Observers were instructed to identify the ori-entation of the Gabor stimulus by pressing different keyson the computer keyboard. A beep immediately followedeach incorrect response. The intertrial interval (3 s), heldconstant across the qTvC and constant stimulus conditions,included the time (�1 s) needed for the qTvC to computethe stimulus prescribed for the next trial.

4.2. Results

4.2.1. Re-checking the regularities of the TvC functions

To verify the assumptions underlying the qTvC parame-terization, the data collected with the method of constantstimuli was analyzed for the TvC function regularitiesdescribed in Eqs. (1) and (2). Data collected at each noiselevel were fit with log-Weibull psychometric functions:

wN extðsÞ ¼ :5þ ð1� :5� k=2Þ

� f1� expð� expðgN extðlogðsÞ � aN extÞÞÞg; ð9Þ

Fig. 8. To justify the qTvC’s assumptions concerning external noise functionsregularities. (a) The thresholds measured over low noise levels, alow

Next, are present

that psychometric thresholds over low noise levels are constant (Property 1). (bfunction of low noise threshold, alow

N ext, plus the log ratio of external noise to the c

excellent correspondence (r2 = .97) suggests that, at high noise levels exceedin(Property 2). (c) Signal contrasts (s) are normalized by each noise level’s psychoprovides evidence for slope invariance of psychometric functions measured ov

with threshold and slope parameters depending on noise le-vel (gN ext

, aN ext ) and a lapse rate parameter, k, independentof noise level.

Psychometric thresholds were classified into low noisethresholds (alow

N ext, when Next < 8%) and high noise thresh-

olds (ahighN ext

, when Next > 8%). Fig. 8a presents low noisethresholds, alow

N ext, measured for each observer, with error

bars signifying standard error estimates (1 SD) obtainedwith a resampling procedure, and each observer’s meanthreshold (across the four noise levels) signified by a dottedline. A nested v2 test, based on the likelihood ratio, com-pared the fit of two models. The full model posited inde-pendent psychometric threshold and slopes for each lowexternal noise level. The alternative (reduced) model posit-ed that a single psychometric function (with a single set ofthreshold and slope parameters) fits psychometric dataover the low noise levels. The appropriate v2 statistic isdefined as:

v2ðdf Þ ¼ 2 logmax likelihoodfull

max likelihoodreduced

� �; ð10Þ

where df = kfull � kreduced, the difference of parameter num-ber between the two models. A nested v2 test suggested thatfor all three observers, the model positing a single psycho-

, the psychophysical data collected with the MCS is analyzed for the TvC

ed for the three subjects. A v2 test supports what is apparent by eye, namely) The thresholds measured over high noise levels, ahigh

Next, are presented as a

ritical noise level: ahighNext

vs alowNextþ logðN extÞ � logðN cÞ, where Nc = 10%. The

g the critical noise levels, psychometric thresholds increase proportionallymetric threshold. The resulting overlap of the raw psychometric functionser external noise levels (Property 3).


metric function over low noise levels provided the optimalfit (p > .25 for each observer). Thus, this analysis verifiesthe proposition that psychometric thresholds over lownoise contrasts are constant (Property 1).

To verify that thresholds increase proportionally atexternal noise levels exceeding the critical noise level (Prop-erty 2), high noise thresholds, ahigh

N ext, were analyzed. Because

preliminary examination suggested that the critical noise,Nc, differed little across the three subjects (Nc � 9–11%),the analysis was simplified by fixing the Nc estimate(10%) across the three observers. In Fig. 8b, high noisethresholds, ahigh

N ext, are plotted against the mean threshold

measured over low noise levels, alowN ext

(calculated in the pre-

vious analysis), plus the log difference between the corre-sponding noise contrast and Nc; that is, ahigh

N extvs

alowN extþ logðN extÞ � logðN cÞ (see Eq. (1b)). The excellent cor-

respondence evident in Fig. 8b (r2 = .97) provides evidencethat thresholds increase proportionally above the criticalnoise level; or equivalently, that the rising slope of theTvC function over high noise levels is 1.0 (Property 2).

To verify psychometric slope invariance across noise lev-els (Property 3), Fig. 8c presents psychometric data, re-cal-culated and presented as raw psychometric functions onlog-relative-contrast (Strasburger, 2001a, 2001b). That is,for each noise level, signal contrasts, s, were transformedto log-threshold units, s0 ¼ logðsÞ � aN ext , by normalizingsignal contrast by the noise level’s psychometric threshold.Fig. 8c demonstrates excellent correspondence between thetransformed psychometric data measured across differentnoise levels. The psychometric function defined on log-rel-ative-contrast, s 0, can be fit with only a slope parameter, g,and a lapse rate parameter, k:

wðs0Þ ¼ :5þ ð1� :5� k=2Þf1� expð� expðg� s0ÞÞg: ð11Þ

The best-fitting psychometric functions (overlaid in thefigure) account well for the raw psychometric data,(mean r2 = .89; SD = .025), across noise levels. Further-more, a nested v2 test, based on the likelihood ratio be-tween full and reduced models, in which the full modelproposes distinct slopes across all noise levels and theformer uses a single slope parameter across all levels,suggests that additional slope parameters do not appre-ciably improve the fit to psychometric data (p > .15 foreach observer). Taken together, these analyses of theMCS data verify the regularities of TvC functions andjustify the parameterization forming the basis of theqTvC method.

4.2.2. Comparing the qTvC and MCS

A previously fit psychometric model (independentthresholds and a single slope parameter), provides theempirical TvC functions presented in Fig. 9. Thresholdsobtained with the MCS, for three performance criteria(65%, 79%, and 92% correct), across eight noise conditions,are plotted with circles, with error bars reflecting bootstrap-ping estimates of threshold variability (1 SD) (Wichmann &

Hill, 2001b). The constant stimulus TvC functions (sMCS)are repeated in the top and bottom rows of Fig. 9.

Threshold variability of qTvC threshold estimates wasestimated using a bootstrapping procedure. The mean andstandard deviation of qTvC parameters, (measured acrossfour qTvC runs of 240 or 480 trials), provided estimatesof the mean and standard error for sampling distributionsof each subject’s qTvC parameters. Independently samplingthese distributions 5000 times provided 5000 sets of simulat-ed threshold estimates. The variability of each thresholdestimate (in dB) was calculated from the simulation results:

rs ¼

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiPð20log10ð�sÞ � 20log10ðskÞÞ2

N � 1

s: ð12Þ

The shaded region provides the ±1 SD region of qTvC

threshold estimates for three performances levels (65%,79%, and 92% correct). The qTvC threshold estimatesobtained with 240 and 480 trials are, respectively, presentedin the top and bottom rows of Fig. 9. The mean thresholdvariability at 65%, 79%, and 92% correct was 1.49, 1.13,and 1.35 dB across the external noise conditions for qTvC

estimates obtained with 240 trials and 1.24, .91, and1.11 dB for those obtained with 480 trials. The mean thresh-old variability, for the method of constant stimuli (with1920 trials), was .81, .72, and .76 dB for the three perfor-mance levels. Although threshold variability was lower forthe method of constant stimuli, the qTvC’s threshold vari-ability compares favorably, considering the adaptive proce-dure used only 12–25% of number of experimental trials.

To illustrate the consistency of individual qTvC runs,and their correspondence with MCS results, Fig. 10 pre-sents the aggregate TvC estimates (for 4 qTvC runs for eachof 3 observers). TvC estimates defined at three performancecriteria (65, 79, and 92%) obtained with the qTvC (4 runs of240 or 480 trials), are plotted against MCS estimatesobtained with 1920 trials. A regression analysis was per-formed to examine correspondence between mean TvC esti-mates obtained with the qTvC and MCS. Due to differencesin variability between threshold estimates, both within andbetween estimation methods, a Deming (type II) weightedregression analysis (Linnet, 1998; Martin, 2000) was usedto analyze the agreement between thresholds obtained withthe qTvC procedure and the MCS. This analysis, account-ing for measurement errors in both methods, finds the lin-ear relation:

logðsqTvCÞ ¼ aþ b logðsMCSÞ: ð13ÞA b close to 1.0 would imply excellent agreement betweenthe two different procedures and provide an experimentalvalidation of the qTvC procedure.

The analysis was applied to the mean aggregate data, torelate mean qTvC estimates obtained with 240 and 480 tri-als with MCS estimates obtained with 1920 trials (seeFig. 9). There was excellent agreement between TvC esti-mates: for 240 trials, slope b = 0.974 (rb = 0.024), intercepta = �0.139 (ra = 0.062), and weighted correlation coeffi-cient r = .984; for 480 trials, b = 0.937 (rb = 0.021),

Fig. 9. Empirical TvC functions (obtained with qTvC and MCS) are measured for three observers. The TvC functions (at 65%, 79%, and 92%) collectedwith the MCS, presented as circles, are repeated in the top and bottom rows, with error bars reflecting variability estimates (1 SD). Presented as shadedregions (±1 SD) are TvC functions estimated with the qTvC, with 240 trials (a–c) or 480 trials (d–f). There is excellent agreement between the TvC

estimates obtained with both methods. The variability of MCS estimates is less than that exhibited by qTvC estimates, but qTvC estimates used only 12–25% of the trials.

Fig. 10. Comparing TvC estimates obtained with qTvC and the method of constant stimuli (MCS). Presented are TvC estimates provided by four qTvC

runs, plotted against an estimate obtained with constant stimuli. A weighted Deming regression, used for method comparison (Martin, 2000), provides theleast squares line fit and weighted correlation coefficient, between the mean qTvC and MCS threshold estimates.


a = �0.175 (ra = 0.052), and r = .987. Furthermore, fol-lowing Martin (2000), we computed the systematic bias,BðcÞ, between threshold estimates at a given contrastthreshold c, obtained with the qTvC and the MCS, as:

BðcÞ ¼ aþ cðb� 1Þ ð14Þand the standard deviation of estimated bias as:

rBðcÞ ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffir2

a þ c� r2b � ðc� 2� logðsMCSÞÞ

q; ð15Þ

where ra is the standard deviation of the intercept estimatea, rb is the standard deviation of the slope estimate b, andlogðsMCSÞ is the weighted mean of threshold estimates

obtained with the method of constant stimuli. We comput-ed systematic bias, BðcÞ, for three contrast levels, c = 3.6%,16.4%, and 72.7%, representing the lowest, middle, andhighest thresholds measured in the present experiment.Respectively, systematic bias estimates at these contrastlevels were, for 240 trials: BðcÞ ¼ 0:134;�0:002;and � 0:037, and rBðcÞ ¼ :084; :061; and :054; and for480 trials, BðcÞ ¼ 0:129; 0:003; and � 0:030, andrBðcÞ ¼ 0:074; 0:053; and 0:046. These results suggest thatover the range of threshold measurements observed in thepresent experiment, there was no significant systematic biasbetween qTvC and MCS threshold estimates.


4.3. Summary

The results of simulations and psychophysical studiesshow that the qTvC method needs less than 300 trials tosuccessfully estimate TvC functions at three performancecriteria, with acceptable precision (< 1.5 dB). Further, bothstudies showed that the qTvC method provides accurateestimates of TvC functions at three performance criteria(e.g., 65%, 79%, and 92% correct) with only 240 trials ofdata collection.

With this number of trials, the precision of these esti-mates is well within that expected for clinical or practicalapplications. As the psychophysical results demonstrate,more trials (�480) may be necessary for the precisiondesired in experimental applications. With 480 trials, theprecision of the qTvC is comparable with the method ofconstant stimuli with 1920 trials. We believe that the smallprecision loss is compensated by: (1) reducing data collec-tion by 75%; and (2) not worrying about stimulus place-ment. Without ideal stimulus placement, the estimatesprovided by constant stimulus methods rapidly lose preci-sion. The qTvC possesses the advantage of placing stimuliwithout any input from the experimenter.

The results of both the Monte-Carlo simulations and thepsychophysical experiment strongly suggest that the qTvC

method can measure external noise functions rapidly withreasonable accuracy and precision.

5. Discussion

In addition to using the external noise method to investi-gate mechanisms underlying observer state changes (e.g.,perceptual learning, memory decay, attention), there hasbeen great interest in applying the method to characterizevisual deficits in human observers. The goal of the currentresearch is to alleviate the burden of data collection withoutcompromising the rich data provided by external noisemethods. Built on an adaptive method developed to estimatepsychometric functions (Cobo-Lewis, 1996; Kontsevich &Tyler, 1999), we developed a Bayesian adaptive procedurethat exploits the regularities of TvC functions to efficientlysample the stimulus space of signal and noise contrast.Recently, Jeon et al. (in preparation) have developed a max-imum-likelihood procedure to estimate the parameters ofthe perceptual template model using the data obtained fromthe qTvC method. The procedure provides a good avenue fortheoretical modeling of the outcomes of the qTvC method.

By exploiting the regularities of the TvC functions, theqTvC method is much more efficient than the method ofconstant stimuli and other adaptive procedures that inde-pendently estimate thresholds in each external noise condi-tion. The efficiency of the qTvC emerges from itsparameterization of external noise functions (suggestedby the empirical TvC’s regularities) and its stimulus place-ment algorithm. Therefore, the qTvC method holds greatadvantages in domains in which these regularities hold.On the other hand, it would be prudent to use classical

methods to validate the assumed regularities in each newdomain of application. The experiment in this study pro-vides an example procedure for validating the qTvC

method. We believe the qTvC method will greatly facilitatethe application of external noise methods to examine per-ceptual deficits accompanying dyslexia, amblyopia, orsenescence.

There are several avenues for improving the applicationof the qTvC method. Successful applications of adaptivemethods depend on the assumption that subjects do notthemselves adapt their behavior to the stimulus placementalgorithm. Although the agreement between TvC estimatesobtained with the MCS and the qTvC suggests that thisassumption was not violated in our study, future develop-ment could constrain stimulus placement. For example, thestimulus placement algorithm could be computed over par-titions of the stimulus space, which can be alternatedbetween trials (as with interleaving staircases). Thoughsuch constraints might make stimulus placement slightlyless than optimal, it may improve the practical applicationof the qTvC method.

Two other aspects of the qTvC application can also begreatly improved: (1) In the current method, entropy calcu-lations for the posterior probability density (defined overthe three-dimensional parameter space) weigh all pointsin the parameter space equally. Future development couldapply differential weights on regions of the parameterspace, as Tanner and colleagues have suggested for estimat-ing psychometric functions (Tanner, Hill, Rasmussen, &Wichmann, 2005). (2) The current method determines themost informative stimulus for the next trial using a one-step ahead exhaustive search over a discretely sampledstimulus space. Although adequate in the current study,more efficient optimization of stimulus placement mayresult from using smooth functions to approximate theexpected entropy function (Cobo-Lewis, 1997) or the mar-ginal probability distributions in the parameter space.Searches for optimal stimuli could then be conducted overcontinuous stimulus dimensions. Additionally, in additionto entropy, other information-theoretic measures, such asFisher information (Cobo-Lewis, 1999), or mutual infor-mation (Cobo-Lewis, 1996; Paninski, 2005), can be usedto optimize stimulus sampling. Currently, such measurescan be computationally intensive, but this approach couldbe desirable for future development.

The adaptive Bayesian framework, originally developedto estimate psychometric threshold (Watson & Pelli, 1983),has been since refined by others to estimate the slope, low-er, and upper asymptotes of psychometric functions(Cobo-Lewis, 1996, 1999; King-Smith & Rose, 1997; Kon-tsevich & Tyler, 1999; Tanner et al., 2005). In this paper,we elaborated the framework to estimate TvC functionsat multiple performance criteria. The framework can beapplied to measure any behavioral function with a relative-ly simple and well-established functional form.

Building on the example of the qTvC method, wehave developed other adaptive Bayesian procedures


(‘‘quick Methods’’ or ‘‘qMethods’’) that search one-stepahead of the current trial (over a multi-dimensional stimu-lus space), and use the minimum expected entropy criterionto choose the stimulus maximizing the information (definedover a multi-dimensional parameter space) on the next tri-al. For example, the quick CSF (qCSF) method maximizesthe information gained about four parameters describingthe contrast sensitivity function (Campbell & Robson,1968; Watson & Ahumada, 2005), by choosing stimuli ofthe appropriate spatial frequency and contrast on each tri-al. Other procedures we have developed measure psycho-metric functions in Yes/No tasks (Green & Swets, 1966),temporal contrast sensitivity functions (de Lange, 1958;Kelly, 1984), elliptical equi-discrimination contours (Mac-Adam, 1942; MacLeod & Boynton, 1979), visual perimetricfields (Bengtsson, Heijl, & Rootzen, 1997), and sensorymemory decay functions (Sperling, 1960). These classicalfunctions, and their corresponding psychophysical models,have previously provided great insight into perceptualmechanisms, but required much data collection. Develop-ing rapid and accurate methods for measuring these behav-ioral functions will allow the application of richpsychophysical methods without the burden of data collec-tion previously needed in the laboratory setting.

Acknowledgments

This research was supported by US Air Force Office ofScientific Research, Visual Information Processing Pro-gram, NSF grants BCS-9911801 and BCS-9910678, andNIMHgrant 1 R01 MH61834-01. We would like to thankAlan Cobo-Lewis and Mark Steyvers for usefulconversations.

Appendix A. Supplementary data

Supplementary data associated with this article can befound, in the online version, at doi:10.1016/j.visres.2006.04.022.

References

Ahumada, A. J. (1987). Putting the visual system noise back in the picture.Journal of the Optical Society of America A, 4, 2372–2378.

Ahumada, A. J., & Watson, A. B. (1985). Equivalent-noise model forcontrast detection and discrimination. Journal of the Optical Society of

America A, 2(7), 1133–1139.Alcala-Quintana, R., & Garcia-Perez, M. A. (2004). The role of

parametric assumptions in adaptive bayesian estimation. Psychological

Methods, 9(2), 250.Barlow, H. B. (1956). Retinal noise and absolute threshold. Journal of the

Optical Society of America, 46, 634–639.Bengtsson, B. O. J., Heijl, A., & Rootzen, H. (1997). A new generation of

algorithms for computerized threshold perimetry, SITA. Acta Oph-

thalmologica Scandinavica, 75, 368–375.Bennett, P. J., Sekuler, A. B., & Ozin, L. (1999). Effects of ageing on

calculation efficiency and equivalent noise. Journal of the Optical

Society of America A, 16(3), 654.Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10(4),

433–436.

Burgess, A. E., & Colborne, B. (1988). Visual signal detection: IV.Observer inconsistency. Journal of the Optical Society of America A, 2,617–627.

Burgess, A. E., Humphrey, K., & Wagner, R. F. (1979). Detection of barsand discs in quantum noise. Proceedings of the Society of Photo-optical

Instrumentation Engineers, 173, 34–40.Burgess, A. E., Wagner, R. F., Jennings, R. J., & Barlow, H. B. (1981).

Efficiency of human visual signal discrimination. Science, 214(4516),93–94.

Campbell, F. W., & Robson, J. G. (1968). Application of Fourier analysis tothe visibility of gratings. Journal of Physiology (London), 197, 551–566.

Chung, S. T. L., Levi, D. M., & Tjan, B. S. (2005). Learning letteridentification in peripheral vision. Vision Research, 45(11), 1399–1412.

Cobo-Lewis, A. B. (1996). An adaptive method for estimating multipleparameters of a psychometric function. Journal of Mathematical

Psychology, 40, 353–354.Cobo-Lewis, A. B. (1997). An adaptive psychophysical method for subject

classification. Perception & Psychophysics, 59, 989–1003.Cobo-Lewis, A. B. (1999). Parameterization-invariant adaptive methods

for psychophysics. Journal of Mathematical Psychology, 43, 580–588.Cohn, T. E. (1976). Detectability of a luminance increment—Effect of

superimposed random luminance fluctuation. Journal of the Optical

Society of America, 66(12), 1426–1428.Dao, D. Y., Lu, Z. -L., & Dosher, B. A. (2006). Adaptation to sine-wave

gratings selectively reduces the sensory gain of the adapted stimuli.Journal of Vision, in press.

de Lange, H. (1958). Research into the dynamic nature of the humanfovea-cortex systems with intermittent and modulated light. II. Phaseshift in brightness and delay in color perception. Journal of the Optical

Society of America, 48, 784–789.Dosher, B. A., & Lu, Z.-L. (1998). Perceptual learning reflects external

noise filtering and internal noise reduction through channel reweigh-ting. Proceedings of the National Academy of Sciences of the United

States of America, 95(23), 13988–13993.Dosher, B., & Lu, Z.-L. (1999). Mechanisms of perceptual learning. Vision

Research, 39, 3197–3221.Dosher, B. A., & Lu, Z.-L. (2000). Noise exclusion in spatial attention.

Psychological Science, 11(2), 139–146.Eckstein, M. P., Ahumada, A. J., Jr., & Watson, A. B. (1997). Visual

signal detection in structured backgrounds: II. Effects of contrast gaincontrol, background variations, and white noise. Journal of the Optical

Society of America A, 14(9), 2406–2419.Eckstein, M. P., Pham, B. T., & Shimozaki, S. S. (2004). The footprints of

visual attention during search with 100% valid and 100% invalid cues.Vision Research, 44(12), 1193–1207.

Gold, J. (2001). Signal and noise in perceptual learning. Ph.D. disserta-tion. Toronto, Canada: University of Toronto.

Gold, J., Bennett, P. J., & Sekuler, A. B. (1999). Signal but not noisechanges with perceptual learning. Nature, 402(6758), 176.

Green, D. M. (1990). Stimulus selection in adaptive psychophysicalprocedures. The Journal of the Acoustical Society of America, 87(6),2662–2674.

Green, D. M. (1995). Maximum-likelihood procedures and the inattentiveobserver. Journal of the Acoustical Society of America, 97(6),3749–3760.

Green, D. M., & Swets, J. A. (1966). Signal detection theory and

psychophysics. New York: John Wiley.Griffiths, J. W. R., & Nagaraja, N. S. (1963). Visual detection in intensity-

modulated displays. Radio and Electronic Engineer, 25, 225–240.Huang, C. B., Tao, L. M., Zhou, Y. F., & Lu, Z.-L. (submitted for

publication). Treated amblyopes remain deficient in spatial vision: Acontrast sensitivity and external noise study. Investigative Ophthalmol-

ogy & Visual Science.Jeon, S. T., Lu, Z. L., Lesmes, L. A., & Dosher, B. A. (in preparation).

Fitting the perceptual template model to adaptive external noisefunctions.

Kelly, D. H. (1984). Retinal inhomogeneity: I. Spatiotemporal contrastsensitivity. Journal of the Optical Society of America A, 1(1), 107–113.




King-Smith, P. E., & Rose, D. (1997). Principles of an adaptive methodfor measuring the slope of the psychometric function. Vision Research,

37(12), 1595.Kiorpes, L., & Movshon, J. A. (1998). Peripheral and central factors

limiting the development of contrast sensitivity in macaque monkeys.Vision Research, 38(1), 61–70.

Klein, S. A. (2001). Measuring, estimating, and understanding thepsychometric function: A commentary. Perception & Psychophysics,

63(8), 1421–1455.Kontsevich, L. L., & Tyler, C. W. (1999). Bayesian adaptive estimation of

psychometric slope and threshold. Vision Research, 39(16), 2729–2737.Leek, M. R. (2001). Adaptive procedures in psychophysical research.

Perception & Psychophysics, 63(8), 1279–1292.Legge, G. E., & Foley, J. M. (1980). Contrast masking in human vision.

Journal of the Optical Society of America, 70(12), 1458–1471.Legge, G. E., Kersten, D., & Burgess, A. E. (1987). Contrast discrimination

in noise. Journal of the Optical Society of America A, 4(2), 391–404.Legge, G. E., Rubin, G. S., Pelli, D. G., & Schleske, M. M. (1988).

Understanding low vision reading. Journal of Visual Impairment &

Blindness, 82(2), 54–59.Levi, D. M., & Klein, S. A. (2003). Noise provides some new signals about

the spatial vision of amblyopes. Journal of Neuroscience, 23(7),2522–2526.

Li, X., Lu, Z.-L., Xu, P., Jin, J., & Zhou, Y. (2003). Generating high gray-level resolution monochrome displays with conventional computergraphics cards and color monitors. Journal of Neuroscience Methods,

130, 9–18.Linnet, K. (1998). Performance of Deming regression analysis in case of

misspecified analytical error ratio in method comparison studies.Clinical Chemistry, 44(5), 1024–1031.

Lu, Z.-L., & Dosher, B. A. (1998). External noise distinguishes attentionmechanisms. Vision Research, 38(9), 1183–1198.

Lu, Z.-L., & Dosher, B. A. (1999). Characterizing human perceptualinefficiencies with equivalent internal noise. Journal of the Optical

Society of America A, 16(3), 764–778.Lu, Z.-L., & Dosher, B. A. (2000). Spatial attention: Different mechanisms

for central and peripheral cues? Journal of Experimental Psychology:

Human Perception and Performance, 26, 1534–1548.Lu, Z.-L., & Dosher, B. A. (2002). External noise methods and observer

models. Investigative Ophthalmology & Visual Science, 43, U821.Lu, Z.-L., & Dosher, B. A. (2004a). Perceptual learning retunes the

perceptual template in foveal orientation identification. Journal of

Vision, 4, 44–56.Lu, Z.-L., & Dosher, B. A. (2004b). Spatial attention excludes external

noise without changing the spatial frequency tuning of the perceptualtemplate. Journal of Vision, 4(10), 955–966.

Lu, Z.-L., & Dosher, B. A. (in preparation). Using external noise methodsand observer models to characterize observer states.

Lu, Z.-L., Jeon, S. T., & Dosher, B. A. (2004). Temporal tuningcharacteristics of the perceptual template and endogenous cuing ofspatial attention. Vision Research, 44(12), 1333–1350.

MacAdam, D. L. (1942). Visual sensitivities to color differences indaylight. Journal of the Optical Society of America, 32, 247–273.

MacLeod, D., & Boynton, R. M. (1979). Chromaticity diagram showingcone excitation by stimuli of equal luminance. Journal of the Optical

Society of America, 69, 1183–1186.Mangini, M. C., & Biederman, I. (2002). Prosopagnosics have low internal

noise? Journal of Vision, 2(7), 609a.Martin, R. F. (2000). General deming regression for estimating systematic

bias and its confidence interval in method-comparison studies. Clinical

Chemistry, 46(1), 100–104.Mortensen, U. (2002). Additive noise, Weibull functions and the approx-

imation of psychometric functions. Vision Research, 42(20),2371–2393.

Nagaraja, N. S. (1964). Effect of luminance noise on contrast thresholds.Journal of the Optical Society of America, 54(7), 950–955.

Paninski, L. (2005). Asymptotic theory of information—Theoretic exper-imental design. Neural Computation, 17, 1480–1507.

Pardhan, S. (2004). Contrast sensitivity loss with aging: Samplingefficiency and equivalent noise at different spatial frequencies. Journal

of the Optical Society of America. A, 21(2), 169–175.Pelli, D. G. (1981). Effects of visual noise. Ph.D. dissertation, Cambridge,

England: University of Cambridge.Pelli, D. G. (1985). Uncertainty explains many aspects of visual contrast

detection and discrimination. Journal of the Optical Society of America

A, 2, 1508–1532.Pelli, D. G. (1987). On the relation between summation and facilitation.

Vision Research, 27(1), 119–123.Pelli, D. G. (1990). The quantum efficiency of vision. In C. Blakemore

(Ed.), Vision: Coding and efficiency (pp. 3–24). Cambridge, UK:Cambridge University Press.

Pelli, D. G. (1997). The videotoolbox software for visual psychophysics:Transforming numbers into movies. Spatial Vision, 10(4), 437–442.

Pelli, D., & Farell, B. (1999). Why use noise? Journal of the Optical Society

of America A, 16(3), 647–653.Pelli, D. G., Levi, D. M., & Chung, S. T. L. (2004). Using visual noise to

characterize amblyopic letter identification. Journal of Vision, 4(10), 904–920.Skoczenski, A. M., & Norcia, A. M. (1998). Neural noise limitations on

infant visual sensitivity. Nature, 391, 697–700.Sperling, G. (1960). The information available in brief visual presentation.

Psychological Monographs, 74(11, Whole No. 498), 29.Sperling, A. J., Lu, Z.-L., Manis, F. R., & Seidenberg, M. S. (2005).

Deficits in perceptual noise exclusion in developmental dyslexia.Nature Neuroscience, 8(7), 862–863.

Strasburger, H. (2001a). Converting between measures of slope of thepsychometric function. Perception & Psychophysics, 63(8), 1348–1355.

Strasburger, H. (2001b). Invariance of the psychometric function forcharacter recognition across the visual field. Perception & Psychophys-

ics, 63(8), 1356–1376.Stromeyer, C. F., & Julesz, B. (1972). Spatial frequency masking in vision:

Critical bands and spread of masking. Journal of the Optical Society of

America, 62, 1221–1232.Tanner, T. G., Hill, N. J., Rasmussen, C. E., & Wichmann, F. A. (2005).

Efficient adaptive sampling of the psychometric function by maximiz-ing information gain. In H. H. Bulthoff, A. Mallot, R. Ulrich, & F. A.Wichmann (Eds.). Proceedings of the 8th Tubinger Perception

Conference (Vol. 106). Kirchentellisfurt: Knirsch Verlag.Tjan, B. S., Braje, W. L., Legge, G. E., & Kersten, D. (1995). Human

efficiency for recognizing 3-D objects in luminance noise. Vision

Research, 35(21), 3053–3069.Tjan, B. S., Chung, S., & Levi, D. (2002). Limitation of ideal-observer

analysis in understanding perceptual learning. In Twenty-seventh

annual interdisciplinary conference, Jackson Hole, Wyoming.Treutwein, B. (1995). Adaptive psychophysical procedures. Vision

Research, 35(17), 2503–2522.Treutwein, B., & Strasburger, H. (1999). Fitting the psychometric

function. Perception & Psychophysics, 61(1), 87–106.Tyler, C. W., & Chen, C.-C. (2000). Signal detection theory in the 2AFC

paradigm: Attention, channel uncertainty and probability summation.Vision Research, 40(22), 3121–3144.

van Meeteren, A., & Boogaard, J. (1973). Visual contrast sensitivity withideal image intensifiers. Optik, 37, 179–191.

Watson, A. B., & Ahumada, A. J. (2005). A standard model for fovealdetection of spatial contrast. Journal of Vision, 5(9), 717–740, http://journalofvision.org/5/9/6/.

Watson, A. B., & Pelli, D. G. (1983). QUEST: A Bayesian adaptivepsychometric method. Perception & Psychophysics, 33(2), 113–120.

Wichmann, F. A., & Hill, N. J. (2001a). The psychometric function I:Fitting, sampling and goodness-of-fit. Perception & Psychophysics,

63(8), 1293–1313.Wichmann, F. A., & Hill, N. J. (2001b). The psychometric function II:

Bootstrap based confidence intervals and sampling. Perception &

Psychophysics, 63(8), 1314–1329.Xu, P. J., Lu, Z.-L., Qiu, Z. P., & Zhou, Y. -F. (under review). Identifying

mechanisms of amblyopia in gabor orientation identification withexternal noise. Vision Research.

http://journalofvision.org/5/9/6/

http://journalofvision.org/5/9/6/

bayesian adaptive estimation of threshold versus … adaptive estimation of threshold versus...

Documents