Itetative AOA Estimation of Coherent SignalsKazumasa Kaneko∗, Akira Sano∗

∗Keio University/Dept. System Design Engineering, Yokohama, Japan

Abstract— The angles-of-arrival (AOA) estimation and beam-forming by an array antenna plays important roles in MIMOcommunication systems. In multipath environment, almost con-ventional subspace approaches with or without the eigendecom-position adopt the spatial smoothing (SS) using multiple subarrays.This paper proposes a new iterative AOA estimation of partiallyor fully coherent signals without adopting spatial smoothing usingsubarrays. The algorithm is derived by eliminating the coherentcomponents of the reconstructed signal covariance matrix. Oneof the notable advantages of the proposed method is to improvethe angle resolution due to larger aperture compared to ordinarySS-based methods using subarrays with small size. Furthermore,the proposed algorithm can directly be applied to non-uniformlyspaced linear array and circular array without using the arraytransformation matrix. The effectiveness is validated in numericalsimulation compared to the FBSS-based MUSIC.

I. INTRODUCTION

The angles-of-arrival (AOA) estimation by an array of sen-sors is a fundamental problem in various fields such as radar,sonar, communications, seismic data processing, and so on. Forthe AOA estimation, the maximum likelihood (ML) methodand variety of subspace-based approaches have been studiedextensively [1]. The ML method gives the optimal solutions,but the computational burden is high due to the necessity ofnonlinear and multidimensional optimization procedure [2][3].The advantage of most subspace-based methods (e.g., MUSIC,MODE, etc. [4][5]) over the ML method is their relatively com-putational simplicity, where the AOAs are estimated through thesearch of an one-dimensional spectrum or the calculation of theroots of a certain polynomial based on either the eigenvaluedecomposition (EVD) of an array covariance matrix or thesingular value decomposition (SVD) of a matrix of array data.

In the last decade, computationally simple subspace-basedAOA estimation have been developed in which EVD or SVDcomputation is not needed. In linear operation based meth-ods such as the BEWE (bearing estimation without eigende-composition) [6], OPM (orthonormal propagator method) [7],SWEDE (subspace methods without eigendecomposition) [8],and SUMWE (subspace-based method without eigendecompso-tion) [9], the signal or noise (null) subspace is easily obtainedfrom the array data relying on a partition of array responsematrix, and then the directions are estimated in a manner similarto that of the MUSIC. However, the subspace approacheswith and without eigendecomposition suffer serious degradationwhen the incident signals are coherent (i.e. fully correlated) insome practical scenarios due to multipath propagation, wherethe rank of the source signal covariance matrix becomes lessthan the number of incident signals. Although the WSF-E,a variant of BEWE, and SUMWE can resolve the coherent

signals, they use spatial smoothing using subarrays.The purpose of this paper is to give a new iterative AOA es-

timation of partially or fully coherent signals without adoptingspatial smoothing using subarrays. The algorithm is derivedby eliminating the coherent components of the reconstructedsignal covariance matrix. One of the notable advantages ofthe proposed method is to improve the angle resolution due tolarger aperture compared to ordinary SS-based methods usingsubarrays with smaller size [1][10]. Furthermore, the proposedalgorithm can directly be applied to non-uniformly spacedlinear array and circular array without using the interpolationor array transformation matrix [12]-[14].

II. ARRAY SIGNAL MODEL

Consider a linear array of M sensor with spacing d, andsuppose that p narrowband signals {sk(n)} with the centerfrequency f0 are in the field far from the array and impinge onthe array from distinct directions {θk}. Under the narrowbandassumption, the received noisy signal yi(n) at the ith sensorcan be expressed as

yi(n) = xi(n) + wi(n), i = 1, · · · ,M (1)

xi(n) =p∑

k=1

sk(n)e−jω0(i−1)τ(θk) (2)

where xi(n) is the noiseless received signal, wi(n) is theadditive noise, ω0 ≡ 2πf0, τ(θk) ≡ (d/c) sin θk, c is thepropagation speed, and {θk} are measured relative to thenormal of array. (1) and (2) are described more compactly as

y(n) = As(n) + w(n) (3)

where y(n), s(n), and w(n) are the vectors of the re-ceived signals, the incident signals and the additive noise,which are defined by y(n) = (y1(n), y2(n), · · · , yM (n))T ,s(n) = (s1(n), s2(n), · · · , sp(n))T , and w(n) = (w1(n),w2(n), · · · , wM (n))T . The array response matrix isdefined by A ≡ (a(θ1),a(θ2), · · · ,a(θp))T , and thearray response vector is also denoted by a(θk) =(1, e−jω0τ(θk), · · · , e−jω0(M−1)τ(θk))T in a case of uniformlyspaced linear array. The following assumptions are made:(A1) Without loss of generality, the signals {sk(n)} are allcoherent so that they are all some complex multiples of acommon signal s1(n); then, under the flat-fading multipathpropagation, they can be expressed as

sk(n) = βks1(n), for k = 1, 2, · · · , p

where βk is a complex attenuation constant with βk �= 0 andβ1 = 1.

(A2) The additive noise {wi(n)} is a temporally and spatiallycomplex white Gaussian random process with zero-mean andthe covariance matrix:

E{w(n)wH(t)} = σ2IMδn,t, E{w(n)wT (t)} = OM×M

(A3) The number of incident correlated signals p is known.Several methods are available for detection of the number ofsignals [11].

III. PROPOSED ITERATIVE ALGORITHM WITHOUT

SPATIAL SMOOTHING

A. Property of Covariance Matrix

We consider a case in which coherent signals impinge withtheir angle of arrival. The covariance matrix of the receivedsignal is given by

R = E{y(n)yH(n)} = ASAH + σ2I (4)

where S is the signal covariance matrix defined by S =E{s(n)sH(n)}. In a case in which the signals are coherentand correlated each other, the covariance can be expressed inthe decomposed form as

S = Sincoh + Scoh (5)

where

Sincoh =

rs,11 0 . . . 0

0 rs,22. . .

......

. . .. . . 0

0 . . . 0 rs,pp

Scoh =

0 rs,12 . . . rs,1p

r∗s,12 0. . .

......

. . .. . . rs,p−1,p

r∗s,1p . . . r∗s,p−1,p 0

The each diagonal element in Sincoh gives the autocorrela-tion of the signals, while Scoh consists of the non-diagonalcorrelated components of S. Since Sincoh includes the signalinformation without influence of correlation, we can constructthe covariance matrix by removing the influence of coherencyas

Rincoh = R − AScohAH (6)

Then we can apply the subspace approach such as MUSIC toRincoh to obtain the AOA estimates.

B. Proposed Iterative Algorithms

To realize the above idea, we propose an iterative procedureto update the AOA estimates θ

(i)1 , · · · , θ

(i)p at ith iteration.

According to the ith iteration, we define A(i), S(i), S(i)coh

and S(i)incoh. Then we premultiply and postmultiply the pseudo

inverse matrix for A(i) on the relation of (4) as

S(i) = (A(i)HA(i))−1A(i)H(R − σ2I)A(i)(A(i)HA(i))−1

where R is a sample covariance with N snapshots, and σ2 isthe estimated noise variance, which are given by

R =1N

N∑n=1

y(n)yH(n), σ2 =1

M − 1

M∑m=2

λm(R) (7)

where λm(R) is an eigenvalue of R. Then we decompose S(i)

into the diagonal and non-diagonal parts, like (5), as

S(i) = S(i)incoh + S

(i)coh (8)

Then, by removing the coherent part from R, we can recon-struct R

(i)incoh as

R(i)incoh = R − A(i)S

(i)cohA(i)H (9)

Then we apply the eigenvalue decomposition to R(i)incoh to

calculate the noise subspace E(i)n . If the signals are incoherent,

it follows that R(i)incoh = R and the noise subspace does not

depend on the AOA estimates, then MUSIC can be straight-forwardly used to obtain the AOA estimates. However, in thecoherent case, a(θ(i)

n ) is not orthogonal to the noise subspacewhich depends on the AOA estimates θ

(i)n due to (9). Therefore

we adopt the next performance index and minimize it to updateθ(i)n :

J =p∑

k=1

aH(θ(i)k )E(i)

n E(i)Hn a(θ(i)

k ) (10)

The following iterative procedure gives the updated AOAestimates as:

Step 1: Setup of Initial Conditions.Let i = 0, and set appropriate initial AOA estimate, θ

(0)k ,

k = 1, 2, · · · , p, which can be randomly chosen.

Step 2: Update of the AOA Estimates.

θ(i+1) = θ(i) − µ ∂J/∂θ|θ=θ(i) (11)

where µ is an optimal step size so that J(θ(i) − µ∂J/∂θ) isminimized.

C. Calculation of Gradient Vector

We show a closed form of the gradient vector as follows:Since the steering matrix A consisting of {a(θk)} is a functionof the AOAs {θk}, we can analytically calculate the variationsδA and δS of A and S for the perturbations δθ1, · · · , δθp, byneglecting the second and higher order terms, as

δS ∼= (AHA)−1(δAH(R − σ2I)A + AH(R − σ2I)δA)·(AHA)−1 − (AHA)−1(δAHA + AHδA)S−S(δAA + AHδA)(AHA)−1 (12)

By setting the diagonal elements of δS to zero to obtain δScoh,we can calculate δRincoh via the similar approximation as

δRincoh∼= −(AδScohAH + δAScohAH + AScohδA) (13)

Next we calculate the variations δEn of the noise subspace forthe perturbations of δRincoh by using the Stewart’s formula as

δEn∼= [δep+1, δep+2, · · · , δeM ] (14)

where

δep+1∼= U−(p+1)Λ−1

−(p+1)UH−(p+1)δRincohep+1

δep+2∼= U−(p+2)Λ−1

−(p+2)UH−(p+2)δRincohep+2

...

δeM∼= U−MΛ−1

−MUH−MδRincoheM

where U−(k) and Λ−(k) can be easily obtained from the EVDof Rincoh. By using the perturbations we finally can give anapproximate expression of the k-th element of the gradientvector ∇θJ = (∂J/∂θ1, · · · , ∂J/∂θp)T as

∂J

∂θk

∼= 2aH(θk)En(EHn δak + δEH

n a(θk))δθk

(15)

where ak ≡ a(θk + δθk) − a(θk).Remark 1: In order to avoid traps in any local minimum,

we have applicability of some intelligent approaches, i.e. theparticle swarm optimization, to obtain rough AOA estimatesand then we can use the estimates as the initial values for theproposed gradient method.

Remark 2: To improve the computational load for the gradi-ent vector, we can adopt a simpler iterative peak search methodin which the covariance matrix R

(i+1)incoh is updated, instead of

(9), as

R(i+1)incoh = R − A(i)S

(i)cohA(i)H (16)

Then we directly apply MUSIC to R(i+1)incoh to update the AOA

estimates and repeat the iteration. The comparison will bediscussed in the simulation study.

IV. SIMULATION RESULTS

The proposed algorithm is compared with the FBSS-basedMUSIC in the performance of estimation of AOAs. The simu-lation setup is almost the same as in [9], which is given asfollows: Let the number of antenna elements be M = 10,the number of snapshots be N = 128, the carrier frequencyfc = 2 [GHz], the interspacing of sensor elements in a case ofa ULA be a half of wave length, si(n) be QPSK signal, andwm(n) be complex white Gaussian random noise. The proposedmethod can be directly applied to a non-uniform linear arrayand circular array in the coherent signal case.Example 1: Convergence BehaviorWe show the convergence of the estimated AOA of threecoherent signals, which angles are −40◦, 5◦ and 30◦. The SNRis set at 10 dB. Fig.1(a) shows comparison of the two proposedmethods, the gradient method and the peak search method givenin Remark 2. The both give similar convergence behavior. Theangle spectra are also plotted in Fig.1(b).Example 2: Performance versus SNRThe performance of the proposed scheme is examined againstthe SNR. The incident angles of two coherent signals are set

0 2 4 6 8 101 3 5 7 96

4

2

0

2

4

6

Iterations

DO

A E

rror

s (d

eg)

Peak search method signal 1Peak search method signal 2Peak search method signal 3Gradient method signal 1Gradient method signal 2Gradient method signal 3

(a) Comparison of the gradient method and the peak search method(both proposed) in convergence behavior of AOA estimation errors.

80 60 40 20 0 20 40 60 8010

5

0

5

10

15

20

25

30

Angle (deg)

MU

SIC

spe

ctru

m [

dB]

Iteration 01210

(b) Angle spectra obtained in several iterations.

Fig. 1. Convergence profiles and estimation results by the proposed method.

0 5 10 15 20 2510

2

10 1

100

SNR (dB)

RSM

E (

deg)

FBSS based MUSIC subarray size 7ProposedCRB

0 5 10 15 20 2510

2

10 1

100

SNR (dB)

RSM

E (

deg)

FBSS based MUSIC subarray size 7ProposedCRB

Fig. 2. Comparison of RMSE of AOA estimates of two coherent signalsbetween the proposed method and the FBSS-based MUSIC.

at θ1 = 5◦ and θ2 = 12◦. The RMSEs of the estimates ofθ1 and θ2 are shown in Fig.2. The performance is comparedwith the FBSS-based MUSIC adopting subarrays with the sizem = 0.6(M+1) = 7. The performance of the proposed schemeis improved compared with the FBSS-based MUSIC, since theproposed method does not use subarrays with smaller size andcan gain larger aperture.

Example 3: Performance for Non-uniformly Spaced LinearArray AntennaThe proposed method can be directly applied to non-uniformlyspaced linear array antennas (see Fig.3) without using anytransformation matrices. On the same signal setup, the RMSEperformace is plotted in Fig.4 which shows that the proposedmethod give better performance than the conventional methodusing the transformation matrix. The performance of the pro-posed scheme is also studied in terms of the angular separationbetween two coherent signals. Two coherent signals impingeon the array along θ1 = 5◦ and θ2 = θ1 + ∆θ, where ∆θ is

varied from ∆θ = 0◦ to ∆θ = 16◦, and the other simulationparameters are same as those in Example 2, except that the SNRis fixed at 10 dB. Fig.5 shows comparison of the empiricalRMSEs between the proposed method and the conventionalmethod using the transformation matrix and the SS using thesubarray size 9. The proposed method can also give betterangular resolution performance.

d

Array Element

(d=λ/2)1.4d1.2d1.1d 0.7d0.9d1.1d1.4d0.6d

Fig. 3. Structure of non-uniformly spaced linear array

0 5 10 15 20 2510

3

10 2

10 1

100

SNR (dB)

RSM

E (

deg)

CouventionalProposedCRB

0 5 10 15 20 2510

3

10 2

10 1

100

SNR (dB)

RSM

E (

deg)

CouventionalProposedCRB

Fig. 4. Comparison of RMSE of AOA estimates of two coherent signalsbetween the proposed method and the conventional method.

4 6 8 10 12 14 16 18 2010

2

10 1

100

Angular Separation (deg)

RSM

E (

deg)

CouventionalProposedCRBCRB(incoherent)

4 6 8 10 12 14 16 18 2010

2

10 1

100

Angular Separation (deg)

RSM

E (

deg)

CouventionalProposedCRBCRB(incoherent)

Fig. 5. Comparison of angular separation performance between the proposedmethod and the conventional method.

Example 4: Performance for Circular Array AntennaThe performance of the proposed scheme is examined againstthe SNR. The incident angles of two coherent signals are setat θ1 = 5◦ and θ2 = 25◦. The RMSEs of the estimates ofθ1 and θ2 are plotted in Fig.6 and Fig.7 which show that theproposed method can give better performances against SNRand angular resolution than the conventional method adoptingthe transformation matrix with subarray size 9.

V. CONCLUSION

The iterative algorithm for AOA estimation for coherentsignals has been proposed. One of the advantages is that theproposed method does not depend on the antenna geometryand so it can be directly applied to a linear array with non-equally spacing or circular antenna. Furthermore, since thespatial smoothing using subarrays with small size is not needed,the aperture becomes larger and then the RMSE performancecan also be improved compared with the FBSS-based MUSIC.

0 5 10 15 20 2510

2

10 1

100

101

SNR (dB)

RSM

E (

deg)

CouventionalProposedCRB

0 5 10 15 20 2510

2

10 1

100

101

SNR (dB)

RSM

E (

deg)

CouventionalProposedCRB

Fig. 6. RMSE performance versus SNR in application of the proposed methodto circular arrays with ten antenna elements (M = 10) with comparison withthe conventional method.

10 15 20 25 3010

2

10 1

100

101

Angular Separation (deg)

RSM

E (

deg)

CouventionalProposedCRB

10 15 20 25 3010

2

10 1

100

101

Angular Separation (deg)

RSM

E (

deg)

CouventionalProposedCRB

Fig. 7. Angular resolution performance in application to circular arrays withcomparison with conventional method.

REFERENCES

[1] H. L. Van Trees, Optimum Array Processing, Part IV of Detection,Estimation, and Modulation Theory, John Wiley & Sons, 2002.

[2] P. Stoica and A. Nehorai, ”MUSIC, maximum likelihood, and Cramer-Rao bound,” IEEE Trans. Acoust., Speech, Signal Processing, vol.37,no.5, pp.720–741, 1989.

[3] B. Ottersten, M. Viberg, P. Stoica, and A. Nehorai, ”Exact and largesample maximum likelihood techniques for parameter estimation anddetection in array processing,” in Radar Array Processing, S. Haykin,J. Litva, and T.J. Shepherd, Eds., pp.99–151, Springer-Verlag, 1993.

[4] P. Stoica and K. C. Sharman, ”Maximum likelihood methods fordirection-of-arrival estimation,” IEEE Trans. Acoust., Speech, SignalProcessing, vol.38, no.7, pp.1132–1143, 1990.

[5] M. Viberg and B. Ottersten, ”Sensor array processing based on subspacefitting,” IEEE Trans. Signal Processing, vol.39, no.5, pp.1110–1121,1991.

[6] C. Yeh, ”Simple computation of projection matrix for bearing estima-tions,” IEE Proc., Pt. F, vol.134, no.2, pp. 146–150, 1987.

[7] S. Marcos, A. Marsal and M. Benidir, ”The propagator method for sourcebearing estimation,” Signal Processing, vol.42, no.2, pp.121–138, 1995.

[8] A. Eriksson, P. Stoica, and T. Soderstrom, ”On-line subspace algorithmsfor tracking moving sources,” IEEE Trans. Signal Processing, vol.42,no.9, pp.2319–2330, 1994.

[9] J. Xin and A. Sano, ”Computationally efficient subspace-based method fordirection-of-arrival estimation without eigendecomposition”, IEEE Trans.Signal Processing, vol.52, no.4, pp.876–893, 2004.

[10] S. U. Pillai and B. H. Kwon, ”Forward/backward spatial smoothingtechniques for coherent signals identification,” IEEE Trans. Acoust.,Speech, Signal Processing, vol.37, no.1, pp.8–15, 1989.

[11] J. Xin, N. Zheng and A. Sano, ”Simple and efficient nonparametricmethod for estimating the number of signals without eigendecomposi-tion”, IEEE Trans. Signal Processing, vol.55, no.4, pp.1405–1420, 2007.

[12] Q. Rao, C. C. Ko and W. Zhi, ”DOA estimation under unknown mutualcoupling and multipath”, IEEE Trans. Aerospace and Electronic Systems,vol.41, no.2, pp.565–573, 2005.

[13] B. Friedlander and A. J. Weiss, ”Direction finding using spatial smoothingwith interpolated arrays”, IEEE Trans. Aerospace and Electronic Systems,vol.28, no.2, pp.574–587, 1992.

[14] B. K. Lau, Y. H. Leung, et al., ”Transformations for nonuniform circulararrays operating in correlated signal environments”, IEEE Trans. SignalProcessing, vol.54, no.1, pp.34–48, 2006.