detection estimation lecture 7 -...
TRANSCRIPT
11/21/2015
2
Digital Communication
Problems
• Make a decision based on the received continuous‐time waveform
• after sampling continuous‐time waveform becomes discrete data set: 0 ,… , 1
• Mathematically, we form a function of the data: 0 ,… , 1 and then make decision
based on the value of the function• Detection theory : determine the function and the rule for making the decision
• Statistical hypothesis testing theory applies here
11/21/2015
3
Detection Problem
• RADAR• binary hypothesis testing: yes or no
• ,
• Digital Communication• 16QAM multiple hypothesis testing
• , , , , … ,
Detection of DC Level
0 ; , 0 ;
11/21/2015
4
Detection of DC Level
• For ON‐OFF Keying digital communication• “0” : send nothing
• “1”: send a pulse with amplitude 1
• two hypotheses are of equal likelihood
• notations for the pdfs become: 0 , 0
Asymptotics
• In detection, we are interested in detecting weak signal buried in noise with a big data record length
• in practice, the SNR is low
• Asymptotic analysis ( → ∞) is appropriate and useful
11/21/2015
5
Bayesian Binary Hypothesis Testing• The goal is to decide between two hypotheses H0 and H1 based on the observation of a random vector Y
• Select a decision function 0, 1• Effectively, the decision function partitions the observation domain into two disjoint sets:
Bayesian BHP
Source Decision⋅
Set of all possible observations
11/21/2015
6
Bayesian BHP
• # of decision functions = # possible partitions of the domain
• We seek the “optimal” one!
• Bayesian: all uncertainties are quantifiable• a‐priori probabilities:
• a cost function : the cost of deciding Hi when Hjhappens
Bayes Risk
Bayes Risk under :
Average cost:
Optimal decision rule is obtained by minimizing the above risk!
11/21/2015
7
Optimal Decision Rule
Risk is minimized when we include all the ’s into if the summand is negative!
Optimal Decision Rule
Reasonable assumption:
11/21/2015
8
Minimum P(error)
• Cost function:
Bayes Risk becomes:
LRT reduces to MAP detector:
Example
• Observe IID Poisson random variables
:!
:!
11/21/2015
9
Example
• Observe IID Exponential RVs
:
:
∼ , exponential distribution with mean 1.0
All in , ∞ or O.W.
Sufficient Statistics
• In general, we say S(Y) is a sufficient statistic for the binary hypothesis testing problem if the conditional density | | , is independent of
• Factorization criterion:
• then is a sufficient statistic for the binary hypothesis problem.
11/21/2015
10
Sufficient Statistics
cancels out when S is sufficient
Neyman‐Pearson Test
• NP test: maximize the detection probability while ensuring the false alarm rate is less than or equal to a threshold
0
11/21/2015
11
NP Test
• For a fixed , to maximize , , we need to design the decision rule such that:
• Complementary slackness requires the should be
chosen such that:
2 extreme tests
,
1 0 0
,
1 1 0
NP Test
Case 1: 1
Case 2: 1∃ such that
11/21/2015
12
NP Test
Case 3: 1
1 |
random test:
Receiver Operating Characteristic (ROC)• Performance of a test can be characterized by
• 1. Probability of detection
• 2. Probability of miss
• 3. Probability of false alarm
11/21/2015
13
ROC
• Conditional Bayes Risk:
• Overall Bayes Risk:
Bayes risk is entirely parametrized by the pair ( , )
ROC• Ideally, we want 1, 0. But this is not that feasible in practice
• Set of achievable pairs can be plotted into the square of [0,1]x[0,1]
11/21/2015
14
ROC Properties
• ROC is the curve of vs
• Let the likelihood ratio admits the pdf: | , we have:
Property 1: The points (0,0) and (1,1) belong to the ROC
ROC Properties
Property 2:
Slop of the ROC at , is equal to the threshold of
the corresponding LRT.
Why?
11/21/2015
15
ROC Properties
Property 3:
The domain of achievable pairs , is convex,
which implies the ROC curve is concave.
Property 4: All points on the ROC curve satisfy .
Example
Optimal Test:
11/21/2015
16
Example
Optimal Test:
randomization
MiniMax Hypothesis Testing
• Bayesian formulation:• assume the availability of the priors ,• this is rather optimistic
• for digital communication, ok
• for other applications, NG!
• Instead of guessing, we want to design the test conservatively by assuming the least‐favorable choice of priors and select the test minimizing the risk for this choice of priors.
11/21/2015
17
Incorrect Prior
• Bayes Risk with priors , :
, 1 1 1
If all the costs and a priori probabilities are known, we can find the optimal test!
Another situation: we design the test for the prior ∗, then we apply this test to other priors!
∗ , is simply a linear function of each test!
∗ ,
Incorrect Prior Effect
Fixed decision rule only optimal for prior ∗
Fact: is a concave function!
11/21/2015
18
MiniMax
• Imagine you and your adversary are playing a game:• You pick a test to minimize the risk function ,• You enemy tries to find the worst to maximize
,
• Question:• how would you choose the prior ?
• Safest approach is: • take the approach that minimizes the risk of the worst possible prior!
Minimax Test:minmax ,
MiniMax
• When the prior is not available, we want to design our test by assuming a particular prior ∗ such that
• the maximum risk among all prior is minimized
∗ 0 ∗ 1
11/21/2015
19
MiniMax
• When the prior is not available, we want to design our test by assuming a particular prior ∗ such that
• the maximum risk among all prior is minimized
MiniMax
• Some observations• 1. the minimax test must be an optimal Bayes test for some ∗
Minimax Test: minmax ,
minmax ,
orminmax ,
Otherwise, we can find an optimal Bayes test for a particular prior which exhibits lower max risk!
11/21/2015
20
MiniMax
• Some observations• 2. If there exists a prior ∗ such that ∗, 0
∗, 1 , the corresponding minimax test will be an equalizer test.
∗
∗ ∗ 0
∗ ∗
0, ,
Proof of Minimax Criterion
• Minimax Criterion: we should design the test assuming the following prior:
• ∗ argmax
min ,Proof:
∗ maxmin ,
∗ max ∗, minmax ,
maxmin , minmax ,
max ′, min , , ∀ ,
minmax , maxmin , [Always true]
∗ minmax ,
[Optimality of Bayes test]
[Definition of ∗]
[Property of ∗]
QED
11/21/2015
21
Minimax Criterion
• When ∗ is an interior maximum, we can simply find it by the equalizer rule:
∗, 0 ∗, 1
Example
Levy‐2.6: Binary non‐symmetric channel with , ∈ 0,1 ,1
1
11/21/2015
22
Example
Levy‐2.6: Binary non‐symmetric channel with , ∈ 0,1 ,1
1
1
0 1
Example
Levy‐2.6: Binary non‐symmetric channel with , ∈ 0,1 ,1
∗
1
1
0 1
11/21/2015
23
Example
• There does not exist an equalizer! What to do?• Randomize the Bayes tests: ∗ and ∗
M‐ary Hypothesis Testing
• M hypotheses: , 0,1, … , 1
• a‐priori probabilities: Pr
• The cost of deciding when holds:
• Problem: find the optimal decision rule which specifies an M‐fold partition of the whole observation space:
11/21/2015
24
M‐ary Hypothesis Testing
• For continuous‐valued observation, the Bayes risk for a decision is:
M‐ary Hypothesis Testing
• Optimal Bayes rule is given by:
• Special Case [min ProbError]: 1