Exploring the complexity limits of joint data detection and channel

estimation

Achilleas AnastasopoulosEECS Department, University of Michigan, Ann Arbor,

MI

University of Parma, ItalyMay 3, 2004

2

Overview

• Motivation• Theory

– Exact detection/estimation in less than exponential complexity for a class of problems

– Specific example: sequence and symbol-by-symbol detection in highly correlated fading

• Application– the family of “ultra-fast” decoders

• Extensions: arbitrary correlation, space-time codes, etc.

• Conclusions

3

Motivation: a simple problem

• Given complex numbers find that maximize the quantity

3 j1- ,4 j2- ,j1-4,, 321 zzz

}3) j(-14) j(-2j1)-4{(

}{

321

332211

ssse

szszszeM

}1,1{,, 321 sss

)1,1,1(),,()1(1)1(2-)1(4

12-4

321

321

sss

sssM

• Solution:

4

Motivation: a harder problem

• Given complex numbers find that maximize the quantity

3 j1- ,4 j2- ,j1-4,, 321 zzz

|3) j(-14) j(-2j1)-4(|

||

321

332211

sss

szszszM

}1,1{,, 321 sss

5

Motivation: a harder problem

• Solution: more difficult because we cannot decompose it into 3 smaller problems

6

Motivation: a communication problem

• Data detection in correlated fading (unknown to the receiver)

• Maximum Likelihood Sequence Detection (MLSqD):

Nknscz kkkk ,...,2,1

11

11

111

1111

)(

)|(),|()|(

kk

kk

kkkkk

kk

MM

fzff

s

szszsz

~}{ kc Complex Gaussian random process (fading),

~}{ kn AWGN

~}{ ks Sequence of M-PSK symbols,

7

Motivation: a communication problem

• Since the transition metric depends on the entire sequence, no dynamic programming (e.g., V.A.) solution is available complexity of optimal solution is exponential in N (i.e., test every possible sequence of length N)

• However if the channel coherence time is approximately L

• Conclusion: Complexity of approximate algorithms is

roughly exponential in L (counterintuitive: the slower the channel, the more complex the decoding !?!)

• Why is this problem relevant today?

)()(

),|()|(

1

11

11

kLkk

kk

kLk

kLkk

kkk zfzf

ss

szsz

8

Coding in channels with memory

…Coded bits

Channel Constraints

• According to the traditional belief, generation of the exact messages for decoding has exponential complexity w.r.t. channel coherence time

Code Constraintse.g., parity-check equations

9

Questions

• How accurate is the conventional wisdom that exact joint detection and estimation requires exponential complexity with respect to the channel coherence time?

• What is the connection with the problem of decoding turbo-like codes at low-SNR?

• What is the impact of the above question on the design of near-optimal approximate algorithms suited for ultra-fast integrated circuit implementation?

10

The basic problem

• In order to present all the ideas, let’s look at the simple problem of MAPSqD of an uncoded sequence in highly correlated fading

• All results generalize to the case of symbol-by-symbol soft metric generation (MAPSbSD).

• A concrete example will be used throughout the talk.

11

Working example• Uncoded M-PSK data sequence in complex

Gaussian fading (fading affects both amplitude and phase).

• Fading remains constant over N symbols (time selective fading with long memory)

),0(CN~ 0 NN N In)1,0CN(~c

nsz

2

1

2

1

2

1

c

n

n

n

s

s

s

c

z

z

z

NNN

N

kkk sp

1

)(~s

12

Perfect CSI case

)(log|cs-z| minargˆ

)(log minarg

)();;(CN maxarg

)(),|(maxargˆ

2kk

2

0

CSI

kks

k

NN

sps

pc

pNc

pcf

k

ssz

sIsz

sszs

s

s

s

which can be decomposed into N simple, symbol-by-symbol minimum distance problems

13

MAPSqD Solution (no CSI)

)(log)

maxarg

)(log)( minarg

)();;(CN maxarg

)()|(maxargˆ

00

2

10

0

MAPSqD

ssz

szIssz

sIss0z

sszs

s

s

s

s

pNE(NN

pN

pN

pf

s

H

NHH

NH

N

• Complexity of maximizing seems exponential w.r.t. N (metric cannot be decomposed)

• For M-PSK, each of the MN sequences needs to be tested explicitly.

2|| szH

14

Approximations

• Approximate solutions (developed over the last 15 years):

– Memory truncation:

Linear predictive receiver [LoMo90], [YuPa95], etc.

– Non-exhaustive search: PSP, M-algorithm, T-algorithm [RaPoTz95], [SeFi95], etc.

– Expectation-Maximization [GeHa97]

• They are all effective (especially for small channel memory)

)()( 1kLkk

kk ss

15

Basic contribution of this work

• The exact MAPSqD solution for this problem (and other problems of interest in communications) can be obtained with only polynomial complexity w.r.t. N

• Contrary to traditional belief, the slower the channel, the smaller the complexity

• The proof of this statement hints at approximate solutions with linear (and very small) complexity w.r.t. N

16

Sketch of proof

• First, transform the MAPSqD problem to a more complicated double-maximization problem

• This is an exact equality• Average likelihood generalized likelihood

N

kkk

C

Cs

H

λsλ

pN

pNE(NN

1

MAPSqD

2

0

2

00

2

),(M),M(

),M( max max argˆ

)(log||max)(log)

s

ss

ssz

ssz

s

17

More definitions… • Sequence-conditioned parameter estimate (Least Squares solution)

• Parameter-conditioned sequence estimate (linear complexity w.r.t. N )

• Order of maximization: two possible approaches

s

H

λ NENλλ

0

),M( maxarg)(ˆzs

ss

)](ˆ,),(ˆ),(ˆ[

)],(M maxarg,),,(M max[arg

),M( maxarg)(ˆ

21

1111

λsλsλs

λsλs

λλ

N

NNss N

sss

18

Approach A: Estimator-correlator

Parameter

estimator

Parameter

estimator

Parameter

estimator

1ˆ sλ

2ˆ sλ

NMλ sˆEstimator

),M(maxarg)(ˆ λλλ

ss

Metric

Function

Metric

Function

Metric

Function

Correlator

arg

max MAPSqDs

))(ˆ,M( maxargˆMAPSqD ssss

λ

Obviously exponential complexity w.r.t. N

1s

2s

NMs

19

Approach B: Parameter space scan

3λ 2λ1λ

λ

C

),M( maxarg)(ˆ λλ sss

Known Parameter

Detector

Known Parameter

Detector

Known Parameter

Detector

Known Parameter

Detector

1ˆ λs

2ˆ λs

3ˆ λs

λs

))),(ˆM( max(argˆˆMAPSqD λλλ

sss

arg

m

ax MAPSqDs

Metric

Function

Metric

Function

Metric

Function

Metric

Function

Unfortunately, this method has infinite complexity

20

Key idea

CλλT |)(ˆ s

Sufficient setfor detection

• Function remains constant for a subset of (this implies an optimal partition of )

)(ˆ λs

Known Parameter

Detector

Known Parameter

Detector

Known Parameter

Detector

C

3λ

2λ1λ

1ˆ λs

2ˆ λs

3ˆ λs

CC

21

Key idea

Parameter

estimator

Metric

Function )(ˆˆ

1sλ

Parameter

estimator

Metric

Function )(ˆˆ

2sλ

Parameter

estimator

Metric

Function )(ˆˆ

3sλ

arg

max

MAPSqDs

• Combine this with the Estimator-Correlator structure (Approach A)

• Function remains constant for a subset of (this implies an optimal partition of )

)(ˆ λs

Known Parameter

Detector

Known Parameter

Detector

Known Parameter

Detector

C

3λ

2λ1λ

1ˆ λs

2ˆ λs

3ˆ λs

CC

22

Almost there…

• Sufficient set size |T|~ N2 AND• There is a recursive algorithm to find T

with complexity ~N2 THUS• can be found with polynomial

complexity w.r.t. N MAPSqDs

QED

23

Parameter space partitioning example

)1(

)1(log

4 )( )(

)1(log)1(

)1(log)1(

),1(M),1(M

0

0

2

0

2

k

kikrk

kk

kk

kk

p

pNzz

pN

zp

N

z

λλ

• For each a parameter space boundary is defined (for BPSK) by the equation

which represents a line in the complex plain

Nk ,,2,1

24

-5 0 5-10

-8

-6

-4

-2

0

2

4

6

8

10

s(3-2j)=[+--++--+++]

s(-4-6j)=[+++++-----]

Complex plain

N=10, antipodal signaling

Es/N

0=3 dB

2 38 9

10

Parameter space partitioning example

^

^

Complex plane

25

Connection with Sphere Decoding

• No connection whatsoever with sphere decoding– Sphere decoding: worst case complexity is

exponential, but average complexity (at high SNR) is polynomial (for sufficiently small N)

– This approach: proves (worst-case/average-case) polynomial complexity irrespective of SNR

– However, sphere decoding is applicable to a much wider class of problems

• Possible research direction: combine the two approaches

26

Symbol-by-Symbol Detection

• What if we need to generate SbS reliability information (e.g., for turbo detection) ?

• Define a suitable metric:– Need to marginalize the sequence metric

over nuisance parameters.– If you choose max as the marginalization

operator (over sequences)

the problem becomes very similar to

MAPSqD, and can be solved with polynomial complexity.

),M( maxmax)()|(max)(::

λpfaSbSλasas

kkk

ssszss

27

Symbol-by-Symbol Detection

K

k

N

i

M

m

kN

kim

ki

k

K

ssassT

T

1 1 1111

21

]},,,,,,{[~

},,,{

sss

• Set T is no longer sufficient• Sufficient set can be found by expanding T

i.e., flip one bit at a time in each sequence in T

• Exact version of the bit flipping (or toggle/swap) approximate algorithms

28

Practical Implications: the ultra-fast receiver

• Previous results have mostly conceptual value

• However, the optimal algorithm hints at some ultra-fast approximate solutions– Instead of finding the optimal partition, use an

arbitrary partition of the parameter space– This implies an approximate set T’– The rest of the decoder remains as in the exact

case (i.e., expansion of T’ by bit flipping, etc)– No multiplication operations required

29

Example

Length 4000 uniform LDPC code with variable and check node degrees 3 and 6, respectively.

Block-independent, flat, complex fading channel (L=11)

1.0E-05

1.0E-04

1.0E-03

1.0E-02

1.0E-01

1.0E+00

3 3.5 4 4.5 5 5.5 6 6.5 7 7.5

Eb/No (dB)

BE

R

perfect CSISum-ProductFastQuantized-ParameterPilot-Only

Relative software complexity:perfect CSI 1Sum-Product 482Ultra-Fast 1.7Quantized-par. 5.5Pilot-Only 1

30

Generalization: arbitrary correlation

• Memoryless fading linear complexity• Constant fading polynomial complexity• What happens in the general case ?

• The answer depends on both the rank of the covariance matrix and its shape in a straightforward way

),(CN~

21 ,

cK0c

nScz

N

kkkk ,N,,knscz

cK

31

Extension: multiple antennae

• Can this receiver principle be extended to MIMO channels, i.e., space-time codes?

• Extension to multiple Rx antennae is straightforward

• Extension to multiple Tx antennae is trickier. Possible for:– Alamouti-type space-time codes– other orthogonal space-time codes (ongoing

research)

32

Example: 1Tx 2Rx

1.0E-05

1.0E-04

1.0E-03

1.0E-02

1.0E-01

1.0E+00

-1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4

Eb/No (dB)

BE

R

perfect CSISum-ProductFastQuantized-ParameterPilot-Only

Relative software complexity:perfect CSI 1 Sum-Product 519Fast 3.3Quantized-par. 38Pilot-Only 1

33

Other applications

• Joint data detection and forward phase and frequency acquisition/tracking

• The parameter space is and is partitioned by straight lines (simple

polygon processing algorithm: complexity~N3)

• Algorithm remains exact for small and large frequency offsets

• Also: 2-state trellis codes…

]5.0,5.0( ),[0,2

N,1,2,k ,2

f

nesz kjfkj

kk

]5.0,5.0()[0,2

34

Conclusions• Exact MAP Sq or SbS detection in channels

with memory is not necessarily an NP-hard problem.

• The proof of the above statement leads to new receiver structures (ultra-fast)

• Performance has been verified for several applications

• Extensions to certain classes of space-time codes is possible

• Several other joint data detection/estimation problems can be put under the same framework