non-bayesian quickest change detection with stochastic sample

5090 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 61, NO. 20, OCTOBER 15, 2013

Non-Bayesian Quickest Change Detection WithStochastic Sample Right Constraints

Jun Geng, Student Member, IEEE, and Lifeng Lai, Member, IEEE

Abstract—In this paper, we study the design and analysis of op-timal detection scheme for sensors that are deployed to monitorthe change in the environment and are powered by energy har-vested from the environment. In this type of applications, detec-tion delay is of paramount importance. We model this problemas quickest change detection problem with stochastic energy con-straints. In particular, a wireless sensor powered by renewable en-ergy takes observations from a random sequence, whose distribu-tion will change at an unknown time. Such a change implies eventsof interest. The energy in the sensor is consumed by taking obser-vations and is replenished randomly. The sensor cannot take ob-servations if there is no energy left in the battery. Our goal is to de-sign optimal power allocation and detection schemes to minimizethe worst case detection delay, which is the difference between thetime when the change occurs and the time when an alarm is raised.Two types of average run length (ARL) constraints, namely an al-gorithm level ARL constraint and a system level ARL constraint,are considered. We propose a low complexity scheme in which theenergy allocation rule is to spend energy to take observations aslong as the battery is not empty and the detection scheme is theCumulative Sum test. We show that this scheme is optimal for theformulation with the algorithm level ARL constraint and is asymp-totically optimal for the formulations with the system level ARLconstraint.

Index Terms—Cumulative Sum test, energy harvesting sensor,non-Bayesian quickest change detection, sequential detection.

I. INTRODUCTION

R ECENTLY, the study of sensor networks powered byrenewable energy harvested from the environment

has attracted considerable attention [1]–[8]. Compared withthe sensor networks powered by batteries, the sensor networkspowered by renewable energy have several unique features suchas unlimited life span and high dependence on the environment,etc. Optimal power management schemes for each individualsensor and scheduling protocols for the whole network havebeen developed to optimize utility functions of communicationrelated metrics such as channel capacity [8], transmission delay

Manuscript received November 30, 2012; revised April 08, 2013 and July03, 2013; accepted July 04, 2013. Date of publication July 16, 2013; date ofcurrent version September 12, 2013. The associate editor coordinating the re-view of this manuscript and approving it for publication was Prof. Pascal Larz-abal. This work was supported by the National Science Foundation CAREERaward by Grant CCF-13-18980 and by the National Science Foundation byGrant DMS-12-65663. The results in this paper were presented in part at theIEEE International Conference on Acoustics, Speech and Signal Processing,Vancouver, Canada, May 2013.The authors are with the Department of Electrical and Computer Engi-

neering, Worcester Polytechnic Institute, Worcester, MA 01609 USA (e-mail:[email protected]; [email protected]).Color versions of one or more of the figures in this paper are available online

at http://ieeexplore.ieee.org.Digital Object Identifier 10.1109/TSP.2013.2273442

[4], [5], [7], transmission rate or network throughput [1]–[3],[6], [7]. However, besides these communication related metrics,there are other signal processing related performance metricsthat are also important for sensor networks targeted for certainapplications. For example, if a sensor network is deployed tomonitor the health of a bridge, then the detection delay betweenthe time when a structural problem occurs and the time whenan alarm is raised is of interest. As another example, if a sensornetwork is deployed for intrusion detection, then the detectiondelay and the false alarm probability are of interest. Until now,these alternative but important performance metrics have notbeen investigated for sensors powered by renewable energy.Power management schemes are of great importance for

both renewable energy powered and battery powered wirelesssensor networks [9]. In this paper, we focus on the design ofoptimal power management schemes for renewable energypowered wireless sensor networks when the detection delay isof interest. In such applications, wireless sensors are deployedto monitor the change in the environment. Such changes typ-ically imply certain activities of interest. For example, in thebridge monitoring, a change may imply that a certain structuralproblem has occurred in the bridge. As the result, it is ofparamount importance to minimize the detection delay afterthe presence of such a change, hence we formulate our problemas a quickest change detection problem, see a recent book [10]and recent surveys on this topic [11], [12]. Besides monitoringthe health of a bridge, quickest change detection also has manyother potential applications, such as the quality control [13],network intrusion detection [14], cognitive radio [15], etc. Wenote that the detection delay in the quickest change detectionproblem refers to the delay between the time when a changeoccurs and the time when an alarm is raised. It is not the delayfrom time zero to the time when an alarm is raised, since weare interested in the change.Non-Bayesian quickest change detection is one of the most

important formulations, which was first studied by G. Lordon[16] and M. Pollak [17]. In the standard non-Bayesian formu-lation, there is a sequence of observationswith a fixed but unknown change point . Before the changepoint , the sequence are independent and iden-tically distributed (i.i.d.) with probability density function (pdf), and after , the sequence are i.i.d. with pdf . Under an

average run length (ARL) to false alarm constraint, namelythe expected duration to a false alarm is at least , Lorden’ssetup is to minimize the “worst-worst case” detection delay

, where is thestopping time at which an alarm is raised, while Pollak’s setupis to minimize the “worst case” conditional average detection

1053-587X © 2013 IEEE

GENG AND LAI: NON-BAYESIAN QUICKEST CHANGE DETECTION 5091

delay . It has been shown that the Cumu-lative Sum (CUSUM) procedure is optimal for Lordon’s setup[18] and that the CUSUM procedure [19] and several variationsof the Shiryaev-Roberts (SR) procedure are asymptoticallyoptimal for Pollark’s setup [17], [20], [21]. Since no prior infor-mation about the change point is required, these non-Bayesiansetups are very attractive for practical applications.In the above mentioned classic setups, there is no energy con-

straint and the observations can be taken at every time slot. Inthis paper, we extend Lorden’s and Pollak’s setups to sensorsthat are powered by renewable energy. In this case, the energystored in the sensor is replenished by a random process and con-sumed by taking observations. The sensor cannot take observa-tions if there is no energy left. Hence, the sensor cannot takeobservations at every time instant anymore. The sensor needsto plan its use of power carefully. Moreover, the stochastic na-ture of the energy replenishing process will certainly affect theperformance of change detection schemes. Since the energy col-lected by the harvester in each time instant is not a constant buta random variable, this brings new optimization challenges.We first consider the scenario in which a unit of energy ar-

rives with probability at each time instant. For Lorden’s setup,two types of ARL constraints are considered in this paper. Thefirst type is an algorithm level ARL constraint, which puts alower bound on the expected number of observations taken bythe sensor before it runs a false alarm. With zero initial energy,we show that the optimal detection procedure is the well knownCUSUM procedure, and the optimal power allocation scheme isto allocate the energy as soon as it is harvested. The second typeARL constraint is on a system level, which puts a lower boundon the expected duration to a false alarm. In this case, for anyinitial energy level, we show that the CUSUMprocedure and theimmediate power allocation strategy is asymptotically optimalwhen the system ARL goes to infinity. For Pollak’s setup, wediscuss the problem only with the system level ARL in detail.As we can see later, the immediate power allocation coupledwith the CUSUM detection is actually asymptotically optimalfor both the system level ARL and the algorithm level ARL.We then consider a more general energy arriving model, in

which more than one unit of energy can arrive at each time in-stant. In this scenario, we show that a simple energy allocationpolicy, in which the sensor takes samples as long as there is en-ergy left in the battery, coupled with the CUSUM test is asymp-totically optimal for both Lorden’s and Pollak’s setups when thesystem level ARL goes to infinity.There have been some existing works on the quickest change

detection problem that take the sample cost into consideration.The first main line of existing works considers the problemunder a Bayesian setup. The main difference between theBayesian setup and the non-Bayesian setup is that in theBayesian setup, the change point is modeled as a randomvariable with a known distribution. No such assumption ismade in the non-Bayesian setup. [14] considers the design ofdetection strategy that strikes a balance between the detectiondelay, false alarm probability and the number of sensors beingactive. In particular, [14] considers a wireless network withmultiple sensors monitoring the Bayesian change in the envi-ronment. Based on the observations from sensors at each time

slot, the fusion center decides how many sensors should beactive in the next time slot to save energy. [22] discusses theBayesian quickest change detection problem with a constraintthat only a finite number of observations can be taken. [23]takes the average number of observations into consideration,and provides the optimal solution along with low-complexitybut asymptotically optimal rules. In [24], the authors propose aDE-CUSUM scheme for the non-Bayesian setup and show thatit is asymptotically optimal.The remainder of this paper is organized as follows. The

mathematical model is given in Section II. Section III presentsthe optimal solution for Lorden’s problem under the algorithmlevel ARL constraint and the performance analysis for the op-timal solution. In Section IV, we present asymptotically optimalsolutions for Lorden’s and Pollak’s problems under the systemlevel ARL constraint. Section V presents our results for a moregeneral energy arriving model. Numerical examples are givenin Section VI to illustrate the results obtained in this work. Fi-nally, Section VII offers concluding remarks.

II. PROBLEM FORMULATION

Let be a sequence of random variableswhose distribution changes at a fixed but unknown time . Be-fore , the ’s are i.i.d. with pdf ; after , they are i.i.d.with pdf . The pre-change pdf and post-change pdf areperfectly known by the sensor. We use and to denote theprobability measure and the expectation with the change hap-pening at , respectively, and use and to denote the case

.We assume that the energy arrives randomly at each time slot.

To facilitate the presentation and set up notation, we presentthe model for the case when the energy arriving process is aBernoulli process with parameter in this section. A more gen-eral model will be considered in Section V. Specifically, we use

to denote the energy arriving processwith , in which indicates that a unit of energyis collected by the energy harvester at time slot andmeans that no energy is harvested. is i.i.d. over . More-over, we use to denote its probability measure (correspond-ingly, we use to denote the expectation with respect to themeasure ), and we have .The sensor can decide how to allocate these collected ener-

gies. Let be the power allocationstrategy with , in which means that thewireless sensor spends a unit of energy on taking an observa-tion at time slot , while means that no energy is spentat time and hence no observation is taken.The sensor’s battery has a finite capacity . The energy ar-

riving process and the energy utilizing process will affect theamount of energy in the battery. We use to denote the energyleft in the battery at the end of time slot . evolves accordingto:

Denote as the initial energy stored in the sensor, we have. The energy allocation policy must obey the


causality constraint, namely the energy cannot be used before itis harvested. The energy causality constraint can be written as

(1)

We use to denote the set of all ’s that satisfy (1).The sensor spends energy to take observation. The observa-

tion sequence is denoted as , where

(2)

We call an observation a non-trivial observation if ,i.e., if the observation is taken from the environment. Denote

as the non-trivial observation sequence,

which is the subsequence of with all its non-trivialelements.

’s are not necessarily conditionally (conditioned on thechange point) i.i.d. due to the existence of . The distribu-tion of is related to both and . Therefore, we useand to denote the probability measure and the expectation ofthe observation sequence with the change happening at ,

respectively. However, we notice that is a conditionally

i.i.d. sequence, since is generated by either or , de-pending on whether this observation is taken before the changepoint or after .In this paper, we want to find a stopping time , at which the

sensor will declare that a change has occurred, and a power allo-cation rule that jointly minimize the detection delay. We con-sider three problem setups. The first one is Lorden’s quickestchange detection problem with an algorithm level ARL con-straint, which is formulated as

(3)

where is the set of all stopping time with , isthe total number of non-trivial observations taken by the sensorbefore it claims that the change has happened and

(4)

where . Unlike the standard Lorden’ssetup, here the worst case delay is a function of ob-servations controlled by ; hence the expecta-tion used in (4) is rather than . The algorithm level ARLconstraint uses expectation rather than because all thenon-trivial observations are i.i.d. with pdf under probabilitymeasure . Hence, the distribution law of is independentof the power allocation scheme and the energy arriving se-quence . As the result, this problem setup is robust against thevariation of the ambient environment.

The second problem considered in this paper is Lorden’squickest change detection problem with a system level ARLconstraint, which is formulated as

(5)

(P2) and (P1) have the same objective function, but their con-straints are quite different. For the system level constraint, alower bound is set on the expected duration to a false alarm. Thestopping time not only depends on the number of non-trivialobservations, but also relies on the time interval between eachtwo successive observations, hence the system level ARL con-straint depends on the power allocation and uses expecta-tion . Since the power allocation is further related to theenergy arriving process , this setup is more sensitive to theenvironment.In some applications, Pollak’s formulation is of interest since

its delay metric is less conservative than that of Lorden’s for-mulation. In our context, Pollak’s formulation can be written as

(6)

Even without the additional energy casuality constraint, the op-timal solution for Pollak’s formulation is still open [11], [12].Therefore, in this paper, we discuss only the asymptotic solutionfor Pollak’s formulation. In the sequel, we will see that the pro-posed asymptotically optimal solution under the system levelARL constraint is also asymptotically optimal under the algo-rithm level ARL constraint. Hence, in the paper, we discuss onlythe system level ARL constraint for Pollak’s formulation in de-tail.

III. OPTIMAL SOLUTION FOR LORDEN’S FORMULATION WITHTHE ALGORITHM LEVEL ARL CONSTRAINT

In this section, we study the optimal solution for (P1) underthe binary energy arriving model with . We useto denote the likelihood ratio (LR), and useto denote the log likelihood ratio (LLR). For the observationsequence , LR is defined as

(7)

The CUSUM statistic and Page’s stopping time can be writtenas [16]

and

for some constant threshold , respectively.In order to characterize the energy arriving and spending

time, for an arbitrary realization of the power allocation scheme


and energy arriving process , we use the following notationsthroughout of this section:1) to denote the time instants at whichthe sensor harvests a unit of energy, i.e., ;

2) to denote the time instants at whichthe sensor takes observations, i.e., .

Since , using above notations, the energy causalityconstraint indicates the following inequality:

(8)

In this section we also use to denote

the non-trivial observation sequence. Specifically, andare used interchangeably, but will be used

when we want to emphasize the sampling time. In particular,is the non-trivial observation taken by the sensor

at time using the energy collected at time .We emphasize that the above notations are related to the real-

izations of the stochastic processes and . Specifically,depends on the realization of , depends on the realization

of , and depends on both of them.

Generally, for a given detection strategy , the detectiondelay in (4) varies from different change point , hencethe worst case delay takes the supreme over . If there is anequalizer strategy which makes be a constant over ,it might be a good candidate for the optimal strategy for theminmax problem. Similar to the conclusion that Page’s stoppingtime is an equalizer rule for the classic Lorden’s problem [10],we have the following proposition:Proposition 3.1: The power allocation scheme and

Page’s stopping time together achieve an equalizer rule, i.e.,.

Proof: Since indicates that ’s are i.i.d. over, ’s are conditionally i.i.d. given the change point .Let . On the event , is a

non-increasing function of . Since and event, the worst case of happens at

, that is

(9)

Since ’s are conditionally i.i.d. under , is a homo-geneous Markov chain, then, .Remark 3.2: indicates that (or )

for every , that is, the sensor spends the energy on taking ob-servation immediately when it obtains an energy from the envi-ronment. Therefore, we call the immediate power allocationscheme in the sequel.Remark 3.3: The equalizer property plays a critical role in

the proof of (asymptotic) optimality and the performance anal-ysis in the sequel. From this property, we have

, which can greatly simplify the analysis.Since the proof of Proposition 3.1 holds regardless of the ARLconstraint, we can conclude that is also an equalizerrule for (P2).

The optimal solution of (P1) is described in the followinglemma.Lemma 3.4: With zero initial energy, the optimal power allo-

cation strategy for (P1) is , and the optimal stopping time iswith the threshold being a constant such that .Proof: The proof consists of two steps. The first step is to

show that for an arbitrary but given power allocation , is theoptimal stopping time. The second step is to show that under ,is the optimal power allocation scheme. A detailed proof is

provided in Appendix A.Remark 3.5: We emphasize that is an important as-

sumption for proving the optimality. Technically, the optimalityof relies on the inequality for every , which isonly true under . If , the optimal power alloca-tion is difficult to find, but is still a good strategy since it isasymptotically optimal as . As will be shown in Proposi-tion 3.7, the detection delay scales linearly with ;hence, the contribution of a finite initial energy is negligiblewhen .In the following, we analyze the performance of by

determining the detection delay and the algorithm level ARL.We notice that the strategy is independent of ;hence the following propositions hold for any initial energylevel. Since is a conditionally i.i.d. sequence under ,we can apply Wald’s identity in our analysis. We first have thefollowing proposition:Proposition 3.6: Suppose , then for any initial energy, we have

(10)

(11)

where is the stopping time

and denotes the event

Proof: The proof follows closely that of Theorem 6.2 in[10]. A detailed proof is given in Appendix B.In Proposition 3.6, ARL and are given as func-

tions of and , whose precise values are diffi-cult to evaluate. The following result, which is an extension ofLorden’s asymptotical result [16], shows scales lin-early with when .Proposition 3.7: As , then for any initial energy ,

we have

(12)

in which is the KL divergence of and .Proof: This statement can be shown by discussing the rela-

tionship between the one-sided sequential probability ratio test


(SPRT) and the CUSUM test. The discussion is similar to theproof of Lemma 4.2, therefore, we omit the proof for brevity.

IV. ASYMPTOTICALLY OPTIMAL SOLUTION UNDER THESYSTEM LEVEL ARL CONSTRAINT

In this section, we consider (P2) and (P3) for any value ofunder the binary energy arriving model. Inspired by the pre-

vious section, we propose to use the simple detection strategy. We will show that this simple strategy is asymptoti-

cally optimal for (P2) and (P3) as .The asymptotic optimality of in the rare false alarm

region can be shown by two steps. In the first step,we derive a lower bound on the detection delay for any powerallocation and detection scheme. In the second step, we showthat achieves this lower bound, which then implies that

is asymptotically optimal.The following lemma presents our lower bound on the detec-

tion delay.Lemma 4.1: For any initial energy , as ,

(13)

Proof: Please see Appendix C.This lower bound can be obtained

by for both (P2) and (P3), which is specified in Lemma4.2 and Lemma 4.4.Lemma 4.2: is asymptotically optimal for (P2) as

. Specifically, for any initial energy ,

(14)

Proof: As discussed in Remark 3.3, is an equalizerrule for (P2), i.e., .The statement can be shown by discussing the relationship

between the CUSUMand the one-sided SPRT. Denote the SPRTstatistic as

(15)

and define the stopping time

Since

we always have

Let , by the performance of the SPRT (Proposition 4.11in [10]), we have

Moreover, by (10) in Theorem 2 of [16], the thresholdwill guarantee

Then, by Lemma 4.1 we have

Remark 4.3: Although is shown to be asymptoti-cally optimal for (P2), we were not able to show the optimalityof . For a general power allocation , the ob-servation sequence is not necessarily conditionally i.i.d.anymore. This is one of the main challenges. In addition, thetechnique used in the proof of Lemma 3.4 cannot be appliedhere. Although the non-trivial observation sequence isrelatively easy to handle, it is difficult to evaluate the detectiondelay from since the detection delay is also related to thetime intervals between two successive non-trivial observations,which are not necessarily i.i.d. under a general power allocationstrategy .Lemma 4.4: is asymptotically optimal for (P3) as

. Specifically, for any initial energy ,

(16)

Proof: We consider the one-sided SPRT with threshold, which will guarantee Let denote

the stopping time of the SPRT starting at time instant , i.e.,

then Page’s stopping time can be written as

(17)

Note that

therefore, . Then, for an arbitrary ,

Here, (a) is due to (17), (b) is due to the fact that is indepen-dent of , and (c) is true because ’s are conditionallyi.i.d. under , hence has the same distribution under


as does under . Since , com-bining this with Lemma 4.1, we have

As we mentioned in Section II, although we consider Pollak’sformulation only under the system level ARL constraint in detailin this paper, the proposed strategy is also asymptoti-cally optimal for the formulation under the algorithm level ARLconstraint, which is stated in the following proposition:Proposition 4.5: For any initial energy , is

asymptotically optimal for Pollak’s formulation under thealgorithm level ARL constraint as , and we have

(18)

Proof: Following the similar argument used in (27), wehave

That is, under the immediate power allocation , the algorithmlevel ARL constraint can be equivalently convertedinto a system level ARL constraint . Settingfor a given , is equivalent to . By Lemma4.4, is asymptotically optimal under the system levelARL constraint, hence it is asymptotically optimal under thealgorithm level ARL constraint.

V. EXTENSION

In this section, we extend the original problem setup by as-suming that the energy harvester can receive more than one unitenergy at each time slot. Specifically, we assume that the en-ergy arriving sequence is i.i.d. over .

, in which means that the en-ergy harvester collects nothing at time and meansthat the energy harvester collects units of energy at time . Weuse to denote its probability mass function(pmf). Then the energy left in the battery at the end of time slotis updated by

The sensor has an initial energy level , and the energycausality constraint indicates that for .Under this setup, we consider (P2) and (P3). We propose to

use a generalized immediate power allocation strategy:

That is, the sensor keeps taking observations as long as the bat-tery is not empty.In the following, we show that this generalized immediate

power allocation combined with Page’s stopping time isasymptotically optimal for (P2) and (P3) in this random energy

arriving case. Corresponding to Lemma 4.1, Lemma 4.2 andLemma 4.4, we have Lemma 5.1 and Lemma 5.2.Lemma 5.1: For any initial energy , as ,

(19)

where .Proof: We first show that exists, and

.We show that is a regular and finite Markov chain. At

each time , has only possible states. If , thetransition probability is given as

If with , the transition probability is given as

The above transition probabilities indicate that is a regularMarkov chain. We denote the stationary distribution as

, where is the stationary probability forthe state . Since only happens whenand , then we have

Hence, exists, and .We denote . The rest of the proof follows the one

in Appendix C by replacing with .Lemma 5.2: is asymptotically optimal for (P2)

and (P3) as . Specifically, for any initial energy ,

(20)

and

(21)

Proof: Please see Appendix D.Remark 5.3: The above lemmas indicate that does not

affect the asymptotic optimality. Since the detection delay goesto infinity as , a finite initial energy , which only


contributes a finite number of observations, does not decreasethe detection delay significantly. However, the battery capacitywould affect the detection delay since the parameter is a

function of and .

VI. NUMERICAL SIMULATION

In this section, we give some numerical examples to il-lustrate the analytical results obtained in this paper. In thesenumerical examples, we assume that the pre-change dis-tribution is and the post-change distributionis . In this case, the KL divergence is

, and the signal-to-noise ratio

is defined as .In the first example, we illustrate the equalizer property

of under Lorden’s formulation. As we mentioned,the equalizer property plays a critical role in the performanceanalysis, since it allows us to study through a rela-tively simple expression . In this example, we compareour optimal strategy with a seemingly reasonable strategy: asave-test power allocation scheme combined with the CUSUMtest. The save-test power allocation is described as follows:

That is, the is a two-threshold strategy: 1) The sensor savesthe collected energy for future use if the energy stored in thebattery is less than a threshold and the CUSUM statistic isless than a threshold ; and 2) the sensor takes an observationwhen either of these two thresholds is exceeded. This rule saysthat if the CUSUM statistic is low (suggesting that a change hasnot happened yet) and the energy stored in the sensor is low, thesensor saves its energy. On the other hand, if either the sensorhas enough energy, or the CUSUM statistic is high, the sensorshould take an observation. In this simulation, we set ,

, , and . The simulationresult is shown in Fig. 1. In the figure, the blue line with circles isthe performance of , the green dash line with stars is theperformance of . This simulation confirms our analysisthat is an equalizer rule, i.e., .However, is not an equalizer rule. Actually, in thesave-test power allocation scheme, is larger thanothers. This is due to the fact that in the first time slot, both theCUSUM statistic and the energy stored in the sensor are zero,hence the sensor chooses to store its energy. The sensor will nottake observations until the stored energy exceeds . The dura-tion of this energy collection period is independent of the changepoint. Then, the worst case happens at , and the detectiondelay caused by the energy collection period is larger than thatcaused by the immediate power allocation. Since Lorden’s per-formance metric focuses on the worst case, the save-test powerallocation is not as good as the immediate power allocation.In the second example, we illustrate the relationship between

the detection delay and the expected number of observationsto false alarm with respect to the energy arriving probabilityunder setup (P1). In this simulation, we set ,

. The simulation result is shown in Fig. 2. In this figure, theblue line with circles is the simulation result for , thegreen line with stars and the red line with squares are results for

Fig. 1. The change point vs .

Fig. 2. Detection delay v.s. the algorithm level ARL.

Fig. 3. Detection delay v.s. the system level ARL.

and , respectively. The black dash line is theperformance of the classical Lorden’s problem, which serves asa lower bound since in this case the sensor can take observationsat every time slot. As we can see, for a given , the detectiondelay is in inverse proportion to the energy arriving probability. The larger is, the closer the performance is to the lowerbound.In the third scenario, we examine the asymptotic optimality

of for (P2) and (P3). In this simulation, we set ,and . In this case, we have. The simulation result is shown in Fig. 3. In this figure,

the blue line with circles is the performance of (P2). The red


Fig. 4. Detection delay v.s. the system level ARL.

line with squares is the performance of (P3), and the black dashis calculated by . Along all the scales, the red curveis below the blue one, which indicates that Pollak’s detectiondelay is smaller than Lorden’s detection delay. We also noticethat these three curves are parallel to each other, which confirmsthat the proposed strategy, , is asymptotically optimalsince the difference between them is negligible as .In the fourth scenario, we examine the asymptotic optimality

of for (P2) and (P3) in the extension case that the en-ergy arrives randomly both in amount and in time. In the simu-lation, we use , and we assume that the amount of energyis valued in the set . In this case, the proba-bility transition matrix is given as

(22)

In the simulation, we set , , ,, , then the stationary distribution is

and .In this simulation, we set and . The

simulation result is shown in Fig. 4. In this figure the blue linewith circles is the performance of (P2). The red line with squaresis the performance of (P3), and the black dash is calculated by

. Similar to the results obtained in the third simula-tion, along all the scales, Pollak’s detection delay is smaller thanLorden’s detection delay, and these three curves are parallel toeach other, which confirms that the proposed strategy, ,is asymptotically optimal as .In the last scenario, we compare our proposed strategy

with the seemingly reasonable strategydiscussed in the first simulation. In this simulation, the energyarriving process is the same as that in the fourth simulation.Moreover, we set , , . For , we set

and . In the simulation, we consider Lorden’sdetection delay, and we adjust the SNR from to bykeeping the system level ARL around 1100. The simulationresult is shown in Fig. 5. In this figure, the blue line with circlesis the performance of our proposed strategy , the redline with squares is the performance of . From the

Fig. 5. Performance of and under same system ARL.

TABLE IPERFORMANCE OF AND UNDER SAME SNR.

figure, we can see our proposed strategy has a smaller detectiondelay than in all parameter range.Another similar simulation is also conducted under a fixed

with varying system level ARL. By keeping therest of simulation parameters same as before, the simulation re-sult is listed in Table I. This simulation result also shows that

outperforms .

VII. CONCLUSION

In this paper, we have studied the non-Bayesian quickestchange detection problem using a sensor powered by energyharvested from the environment. Since the energy harvester col-lects energy randomly, the quickest change detection problemis subjected to a casual energy constraint. Three non-Bayesianquickest change detection problem setups, namely Lorden’sproblem under the algorithm level ARL, Lorden’s problemunder the system level ARL and Pollak’s problem under thesystem level ARL, have been considered. For the binary energyarriving model, we have shown that the immediate powerallocation scheme coupled with the CUSUM procedure isoptimal for the first setup when the initial energy level is zero,and is asymptotically optimal for the second and the third setupas ARL goes to infinity for any initial energy level. For themore general energy arriving model, we have shown that theproposed generalized immediate power allocation coupled withthe CUSUM procedure is still asymptotically optimal for thesecond and the third setups.In terms of future work, similar to [23], [24], it will be inter-

esting to include the cost of using energy in the problem formu-lation. It will also be interesting to consider the Bayesian versionof the problem.We will also consider the problem with multipledistributed sensors.


APPENDIX APROOF OF LEMMA 3.4

We first show that is optimal for any . For any path of anypower utility process , the quasi change point of the non-trivialobservation sequence is defined as

(23)

This implies that can be viewed as the change point happening

in the non-trivial observation sequence . Moreover,can be viewed as a stopping time on the non-trivial observa-

tion sequence. Therefore, a rule minimizing the detection delayamong is the same as the one minimizing

among . Specifically, the stopping rule is de-cided by

Since is a conditionally i.i.d. (conditioned on )sequence with pre-change distribution and post-change dis-tribution under any path of the power utility process , theabove problem is the classical Lorden’s quickest change de-tection problem [16], and the CUSUM test with threshold ,which is a constant such that , is optimal. Since theCUSUM test is path-wise optimal, it is optimal for any powerutility .To prove the optimality of under , we examine the fol-

lowing problem:

(24)

Notice that the objective function is the same as . Since

in which inequality (a) is due to (8), and equality (b) is truebecause under . Therefore, is optimal forthe problem (24).Since

in which the last equality is due to Proposition 3.1, we have

Combining this with the fact that

we know that is the optimal solution for (P1).

APPENDIX BPROOF OF PROPOSITION 3.6

We first examine the quantity . Since is i.i.d.under , can be viewed as a renewal process, with renewalsoccurring whenever the sum of LLR is less than or equal to zero,and with a termination when the sum is larger than or equal tothe upper threshold, that is,

where are i.i.d. repetitions of , and is the numberof repetitions before the termination. Let denote the in-dicator of the event that the repetition exits at the upperboundary. That is if the repetition exits at the upperboundary, and if the repetition exits at the lowerboundary, then . Hence, under , is ageometric random variable with

Then, we have

(25)

Since , and is i.i.d., we can apply Wald’sidentity:

(26)

Substituting (25) into (26), we have (10).Following the similar argument as above, we get

Denote as the time interval between two succes-sive observations, the pmf of is

and the average of the time interval between two successiveobservations is

For the average detection delay, we have

(27)


Here, (a) is due to equalizer property, (b) is the Wald’s identity.Then (11) follows.

APPENDIX CPROOF OF LEMMA 4.1

This proof relies on several supporting propositions and The-orem 1 of [19].

Proposition C.1: For an arbitrary but given power alloca-tion , let , we have

(28)

Proof: We first show that the inequality

(29)

holds almost surely under for any .To show this, we first consider the immediate power alloca-

tion , by the strong law of large numbers, we have

where is the amount of energy collected from to .In the immediate power allocation , is equal to the numberof nonzero elements in . We also have

in which is the quasi change point defined in (23). Therefore,under , as , we have

(30)

For an arbitrary power allocation with, the amount of energy allocated from to is boundedby the amount of energy collected in this period plus the amountof energy stored in the battery at time . That is,

, where denotes the number of nonzero elements in. Therefore, as ,

For the power allocation scheme with ,we have

Therefore, inequality (29) holds for any arbitrary . Notice thati) (29) holds in the almost sure sense, since (30) converges inthe almost sure sense; and ii) (29) holds for any realization of

For any , define

Due to (29), we have

which indicates that

Note that Proposition C.1 holds for every , therefore

(31)

To prove Lemma 4.1, we need Theorem 1 in [19], which isrestated as follows:

Theorem C.2: ([19]) Let be a random variable se-quence with a deterministic but unknown change point . Underprobability measure , the conditional distribution of is

for and is for . Let

If the condition

(32)

holds for some constant . Then, as ,

Proof: Please refer to [19].In our case, for any arbitrary but given power allocation ,

the pre-change conditional density of is given as


where is the Dirac delta function. Similarly, thepost-change conditional density is

Therefore, the log likelihood ratio in Theorem C.2 can bewritten as

which is consistent with the definition in (7). Moreover, (31)indicates that, for any arbitrary power allocation, (32) holds forthe constant . Therefore, the conclusion in Theorem C.2indicates the result for our case:

APPENDIX DPROOF OF LEMMA 5.2

We first prove the asymptotic optimality of forproblem (P2). The proof relies on some supporting propositionsand Theorem 4 of [19].

Proposition D.1: For the power allocation , we have

(33)

Proof: As we have shown in Proposition C.1, for any real-ization of , and , under the power allocationscheme , we have

Then for all ,

Therefore

(34)

The proposition follows from the fact that (34) holds for all.Proposition D.2: Under the power allocation scheme ,

Page’s stopping time satisfies

(35)

where

but

Proof: For any ,

(36)

Here, (a) is true because the likelihood ratio of and that of

are the same. Then we substitute with , and

change the probabilitymeasure correspondingly. and arethe new indices in corresponding to the original , and

in . (b) holds because under , are i.i.d., then wereverse the sequence. (c) is due to Doob’s martingale inequality,since is a martingale under with expectation 1.

By (36), we can simply choose ,and choose , the threshold of the CUSUM test, such that

.To prove Lemma 5.2, we need Theorem 4 ii) of [19], which

is restated as follows:Theorem D.3: ([19]) Let be a random variables se-

quence with a deterministic but unknown change point . Underprobability measure , the conditional distribution of is

for and is for . Denoteas


Denote as the threshold used in Page’s stopping time. Then

Denote as Lorden’s detection delay, i.e.,

If , the condition

holds for some constant , and as , there exists somewhich dependents only on such that

where

but,

Then,

Proof: Please refer to [19].By Proportion D.1 and D.2, is a strategy that satisfies

the conditions in Theorem D.3. Hence, if we chooseand in the theorem, it is easy to verify that

. Therefore,is asymptotically optimal for (P2).In the rest of this appendix, we show the asymptotic opti-

mality of for problem (P3).Lemma D.4:

(37)

Proof: Follow the similar argument in the proof of Lemma4.4, we have

(38)

We claim that

that is, at the change point , if there is energy left in the battery,the average detection delay tends to be smaller than that of thecase with an empty battery. Hence we have

. Then we have

Let , we have

We define a sequence of stopping timesin the following manner:1) Set . Define

2) Set . Define

That is, at change point , we discard all the energy left inthe battery and then start a SPRT under the power allocation .When the previous SPRT stops, we empty the battery again, andstart a new SPRT immediately. Then, this sequence of stoppingtime are independent with the same dis-tribution of under . Therefore, by the strong LLN,for an that large enough, we have

where . Since we have

as , , then

that is

If we ignore the overshoot, we will have

Then, we have

REFERENCES

[1] Z. Mao, C. E. Koksal, and N. B. Shroff, “Near optimal power and ratecontrol of mlti-hop sensor network with energy replenishment: Basiclimitation with finite energy and data strorage,” IEEE Trans. Autom.Control, vol. 57, pp. 815–829, Apr. 2012.


[2] C. K. Ho and R. Zhang, “Optimal energy allocation for wireless com-munication with energy harvesting constraints,” IEEE Trans. SignalProcess., vol. 60, pp. 4808–4818, Sep. 2012.

[3] C. Huang, R. Zhang, and S. Cui, “Throughput maximization for theGaussian relay channel with energy harvesting constraints,” IEEE J.Sel. Areas Commun., pp. 1–11, Aug. 2013.

[4] J. Yang and S. Ulukus, “Trading rate for balanced queue lengths fornetwork delay minimization,” IEEE J. Sel. Areas Commun., vol. 29,pp. 988–996, May 2011.

[5] J. Yang and S. Ulukus, “Optimal packet scheduling in a multiple ac-cess channel with energy harvesting transmitters,” J. Commun. Netw.,Special Issue on Energy Harvest. Wireless Netw., vol. 14, pp. 140–150,Apr. 2012.

[6] M. Gatzianas, L. Georgiadis, and L. Tassiulas, “Control of wirelessnetworks with rechargeable batteries,” IEEE Trans. Wireless Commun.,vol. 9, pp. 581–593, Feb. 2010.

[7] O. Ozel, K. Tutuncuoglu, J. Yang, S. Ulukus, and A. Yener, “Transmis-sion with energy harvesting nodes in fading wireless channels: Optimalpolicies,” IEEE J. Sel. Areas Commun., vol. 29, pp. 1732–1743, Sep.2011.

[8] R. Rajesh, V. Sharma, and P. Viswanath, “Capacity of fading Gaussianchannel with an energy harvesting sensor node,” in Proc. IEEE GlobalTelecommun. Conf., Houston, TX, USA, Dec. 2011, pp. 1–6.

[9] D. Tse and P. Viswanath, Fundamentals ofWireless Communication.Cambridge, U.K., Cambridge Univ. Press, May 2005.

[10] H. V. Poor and O. Hadjiliadis, Quickest Detection. Cambridge, U.K.:Cambridge Univ. Press, 2008.

[11] V. V. Veeravalli and T. Banerjee, “Quickest Change Detection,” inE-Reference Signal Processing, Rama Chellappa, Ed. et al. Am-sterdam, The Netherlands: Elsevier, 2013 [Online]. Available:[arXiv:1210.5552]

[12] A. Polunchenko and A. G. Tarkakovsky, “State-of-the-art in sequentialchange-point detection,” Methodol. Comput. Appl. Probabil., vol. 14,no. 3, pp. 649–684, 2012.

[13] S.W. Roberts, “A comparison of some control chart procedures,” Tech-nometrics, vol. 8, pp. 411–430, Aug. 1966.

[14] K. Premkumar and A. Kumar, “Optimal sleep-wake scheduling forquickest intrusion detection using sensor network,” in Proc. IEEEConf. Computer Commun. (Infocom), Phoenix, AZ, USA, Apr. 2008,pp. 1400–1408.

[15] L. Lai, Y. Fan, and H. V. Poor, “Quickest detection in cognitive radio:A sequential change detection framework,” in Proc. IEEE GlobalTelecommun. Conf., New Orleans, LA, USA, Nov. 2008, pp. 1–5.

[16] G. Lorden, “Procedures for reacting to a change in distribution,” Ann.Math. Statist., vol. 42, no. 6, pp. 1897–1908, 1971.

[17] M. Pollak, “Optimal detection of a change in distribution,” Ann.Statist., vol. 13, pp. 206–227, Mar. 1985.

[18] G. V. Moustakides, “Optimal stopping times for detecting changes indistribution,” Ann. Statist., vol. 14, no. 4, pp. 1379–1387, 1986.

[19] T. L. Lai, “Information bounds and quickest detection of parameterchanges in stochastic systems,” IEEE Trans. Inf. Theory, vol. 44, no.7, pp. 2917–2929, 1998.

[20] G. V. Moustakides, A. S. Polunchenko, and A. G. Tartakovsky, “A nu-merical approach to performance analysis of quickest change-point de-tection procedures,” Statistica Sinica, vol. 21, no. 2, pp. 571–596, 2011.

[21] A. G. Tartakovsky, M. Pollak, and A. Polunchenko, “Third-orderasymptotic optimality of the generalized Shiryaev-Roberts change-point detection procedures,” Theory Probabil. Appl., vol. 58, no. 3,pp. 534–565, 2011.

[22] J. Geng, L. Lai, and E. Bayraktar, “Quickest change point detectionwith sampling right constraints,” in Proc. Allerton Conf. Circuits Syst.Theory, Montecello, IL, USA, Sep. 2012.

[23] T. Banerjee and V. V. Veeravalli, “Data-efficient quickest changedetection with on-off observation control,” Sequential Anal., DesignMethods Appl., vol. 31, no. 1, pp. 40–77, 2012.

[24] T. Banerjee and V. V. Veeravalli, “Data-efficient quickest changedetection in minimax settings,” IEEE Trans. Inf. Theory, 2012, doi:10.1109/TIT.2013.2272313, preprint.

Jun Geng (S’13) received the B.E. and M.E. degreesfrom Harbin Institute of Technology, Harbin, China,in 2007 and 2009, respectively.He is currently working towards the Ph.D. degree

in the Department of Electrical and ComputerEngineering, Worcester Polytechnic Institute. Hisresearch interests are in stochastic signal processingand wireless communications.

Lifeng Lai (M’07) received the B.E. and M. E. de-grees from Zhejiang University, Hangzhou, China, in2001 and 2004, respectively, and the Ph.D. degreefrom The Ohio State University, Columbus, OH, in2007.He was a Postdoctoral Research Associate at

Princeton University from 2007 to 2009, and was anAssistant Professor at the University of Arkansas,Little Rock, from 2009 to 2012. Since August 2012,he has been an Assistant Professor at WorcesterPolytechnic Institute. His research interests include

information theory, stochastic signal processing and their applications inwireless communications, security, and other related areas.Dr. Lai was a Distinguished University Fellow of The Ohio State Univer-

sity from 2004 to 2007. He is a corecipient of the Best Paper Award fromIEEE Global Communications Conference (Globecom) in 2008, the Best PaperAward from IEEE Conference on Communications (ICC) in 2011 and the BestPaper Award from IEEE Smart Grid Communications (SmartGridComm) in2012. He received the National Science Foundation CAREER Award in 2011,and Northrop Young Researcher Award in 2012. He served as a Guest Ed-itor for IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, SpecialIssue on Signal Processing Techniques for Wireless Physical Layer Security.He is currently serving as an Editor for IEEE TRANSACTIONS ON WIRELESSCOMMUNICATIONS.

non-bayesian quickest change detection with stochastic sample

Documents