manifold optimization methods for hybrid beamforming in

14
1 Manifold Optimization Methods for Hybrid beamforming in mmWave Dual-Function Radar-Communication System Bowen Wang, Student Member, IEEE, Ziyang Cheng, Member, IEEE, Zishu He, Member, IEEE Abstract—As a cost-effective alternative, hybrid analog and digital beamforming architecture is a promising scheme for mil- limeter wave (mmWave) system. This paper considers two hybrid beamforming architectures, i.e. the partially-connected and fully- connected structures, for mmWave dual-function radar commu- nication (DFRC) system, where the transmitter communicates with the downlink users and detects radar targets simultaneously. The optimization problems are formulated by minimizing a weighted summation of radar and communication performance, subject to constant modulus and power constraints. To tackle the non-convexities caused by the two resultant problems, effective Riemannian optimization algorithms are proposed. Specifically, for the fully-connected structure, a manifold algorithm based on the alternating direction method of multipliers (ADMM) is developed. While for the partially-connected structure, a low- complexity Riemannian product manifold trust region (RPM-TR) algorithm is proposed to approach the near-optional solution. Nu- merical simulations are provided to demonstrate the effectiveness of the proposed methods. Index Terms—Dual-function radar-communication(DFRC), mmWave, hybrid beamforming, product manifold, Riemannian optimization. I. I NTRODUCTION T HE increasing number of wireless devices leads to more and more serious spectrum congestion. As a conse- quence, there is a serious spectrum overlap between different devices, such as airbone radars and navigation systems are close to the 3.4GHz band, which partially overlap with the Long Term Evolution (LTE) systems. Another instance is that the communication satellite (CS) systems and Wi-Fi systems operate in the C-band (4-8 GHz), which is also utilized by the military radars. To address the spectrum congestion between radar and communication, the communication radar spectrum sharing (CRSS) technology is a promising way. The existing works on the CRSS mainly focus on the two aspects. 1) Co-Existence between radar and communication systems Co-existence between radar and communication systems sharing the same bandwidth has been a primary investigation field in recent years [1]–[8]. For instance, the authors in [1] design the spectrally compatible waveform with desired spec- tral nulls, considering maximization the signal-to-interference- plus-noise ratio (SINR) for the multiple-input multiple-output (MIMO) radar. This method provides a naive approach to achieve such co-existence. However, this method is limited to the applications with widely separated spectra. As another B. Wang, Z. Cheng and Z. He are with School of Information & Communication Engineering, University of Electronic Science and Tech- nology of China, Chengdu 611731, China (email: B W [email protected]; [email protected]; [email protected]). way to support the co-existence, the authors in [4] project the radar signals onto the null-space of the interference channels between radar and communication, thus the interference from the radar to communication link is zero. Nevertheless, this scheme can not guarantee a satisfactory performance for the radar system. More recently, some joint design schemes of radar and com- munication systems are proposed to improve the performance of both sides [7]–[11]. For example, the authors in [7] propose a co-design of radar sampling matrix and communication codebook for a co-existence between matrix completion based MIMO radar and MIMO communication. In addition, some co-design approaches centered at communication performance are suggested in recent studies, wherein anti-radar-interference strategies are taken either at the communication receiver [9] or, using some prior information, directly at the communication transmitter [10], [11]. Although these co-design methods can achieve satisfactory CRSS, the cooperation systems operating in two separated platforms need some strict requirements, such as the synchronizations of frequency and sampling times. More importantly, these methods will bring extra hardware cost and complexity. 2) Dual-function radar communication systems To handle the limitations of the co-existence with two plat- forms, the dual-function radar communication (DFRC) system, where radar and communication share in a common platform, has been regarded as a more favorable approach to realize the CRSS [12]–[20]. Pioneering effort [12] considers the single- antenna system, in which several integrated waveforms that can achieve both radar and communication functions are pre- sented. However, these methods will result in a performance loss for radar and communication because of lacking of degree of freedoms (DOFs). As a step further, more recent works focus on the DFRC based on the MIMO systems. For example, in [13], [14], beampattern modulation methods are used to de- sign the DFRC systems, where the communicaton information is embedded in the radar beampattern via changing the radar beampattern either in amplitude or phase. Unlike the above beampattern modulation methods, index modulation (IM) is proposed in [15], in which the communication symbols are embedded into the radar waveforms in emission-constrained scenarios. Additionally, a common waveform is devised to communicate with the downlink multi-user and detect radar targets in [18], in which the trade-off problem between radar and communication is established by minimize the downlink multi-user interference (MUI) under radar-specific constraints. It should be noted that the above methods adopt a conven- tional fully-digital beamforming schemes, where each antenna arXiv:2111.15102v1 [eess.SP] 30 Nov 2021

Upload: others

Post on 06-May-2022

34 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Manifold Optimization Methods for Hybrid beamforming in

1

Manifold Optimization Methods for Hybridbeamforming in mmWave Dual-Function

Radar-Communication SystemBowen Wang, Student Member, IEEE, Ziyang Cheng, Member, IEEE, Zishu He, Member, IEEE

Abstract—As a cost-effective alternative, hybrid analog anddigital beamforming architecture is a promising scheme for mil-limeter wave (mmWave) system. This paper considers two hybridbeamforming architectures, i.e. the partially-connected and fully-connected structures, for mmWave dual-function radar commu-nication (DFRC) system, where the transmitter communicateswith the downlink users and detects radar targets simultaneously.The optimization problems are formulated by minimizing aweighted summation of radar and communication performance,subject to constant modulus and power constraints. To tackle thenon-convexities caused by the two resultant problems, effectiveRiemannian optimization algorithms are proposed. Specifically,for the fully-connected structure, a manifold algorithm basedon the alternating direction method of multipliers (ADMM) isdeveloped. While for the partially-connected structure, a low-complexity Riemannian product manifold trust region (RPM-TR)algorithm is proposed to approach the near-optional solution. Nu-merical simulations are provided to demonstrate the effectivenessof the proposed methods.

Index Terms—Dual-function radar-communication(DFRC),mmWave, hybrid beamforming, product manifold, Riemannianoptimization.

I. INTRODUCTION

THE increasing number of wireless devices leads to moreand more serious spectrum congestion. As a conse-

quence, there is a serious spectrum overlap between differentdevices, such as airbone radars and navigation systems areclose to the 3.4GHz band, which partially overlap with theLong Term Evolution (LTE) systems. Another instance is thatthe communication satellite (CS) systems and Wi-Fi systemsoperate in the C-band (4-8 GHz), which is also utilized by themilitary radars. To address the spectrum congestion betweenradar and communication, the communication radar spectrumsharing (CRSS) technology is a promising way. The existingworks on the CRSS mainly focus on the two aspects.

1) Co-Existence between radar and communication systemsCo-existence between radar and communication systems

sharing the same bandwidth has been a primary investigationfield in recent years [1]–[8]. For instance, the authors in [1]design the spectrally compatible waveform with desired spec-tral nulls, considering maximization the signal-to-interference-plus-noise ratio (SINR) for the multiple-input multiple-output(MIMO) radar. This method provides a naive approach toachieve such co-existence. However, this method is limitedto the applications with widely separated spectra. As another

B. Wang, Z. Cheng and Z. He are with School of Information &Communication Engineering, University of Electronic Science and Tech-nology of China, Chengdu 611731, China (email: B W [email protected];[email protected]; [email protected]).

way to support the co-existence, the authors in [4] project theradar signals onto the null-space of the interference channelsbetween radar and communication, thus the interference fromthe radar to communication link is zero. Nevertheless, thisscheme can not guarantee a satisfactory performance for theradar system.

More recently, some joint design schemes of radar and com-munication systems are proposed to improve the performanceof both sides [7]–[11]. For example, the authors in [7] proposea co-design of radar sampling matrix and communicationcodebook for a co-existence between matrix completion basedMIMO radar and MIMO communication. In addition, someco-design approaches centered at communication performanceare suggested in recent studies, wherein anti-radar-interferencestrategies are taken either at the communication receiver [9] or,using some prior information, directly at the communicationtransmitter [10], [11]. Although these co-design methods canachieve satisfactory CRSS, the cooperation systems operatingin two separated platforms need some strict requirements, suchas the synchronizations of frequency and sampling times. Moreimportantly, these methods will bring extra hardware cost andcomplexity.

2) Dual-function radar communication systemsTo handle the limitations of the co-existence with two plat-

forms, the dual-function radar communication (DFRC) system,where radar and communication share in a common platform,has been regarded as a more favorable approach to realize theCRSS [12]–[20]. Pioneering effort [12] considers the single-antenna system, in which several integrated waveforms thatcan achieve both radar and communication functions are pre-sented. However, these methods will result in a performanceloss for radar and communication because of lacking of degreeof freedoms (DOFs). As a step further, more recent worksfocus on the DFRC based on the MIMO systems. For example,in [13], [14], beampattern modulation methods are used to de-sign the DFRC systems, where the communicaton informationis embedded in the radar beampattern via changing the radarbeampattern either in amplitude or phase. Unlike the abovebeampattern modulation methods, index modulation (IM) isproposed in [15], in which the communication symbols areembedded into the radar waveforms in emission-constrainedscenarios. Additionally, a common waveform is devised tocommunicate with the downlink multi-user and detect radartargets in [18], in which the trade-off problem between radarand communication is established by minimize the downlinkmulti-user interference (MUI) under radar-specific constraints.

It should be noted that the above methods adopt a conven-tional fully-digital beamforming schemes, where each antenna

arX

iv:2

111.

1510

2v1

[ee

ss.S

P] 3

0 N

ov 2

021

Page 2: Manifold Optimization Methods for Hybrid beamforming in

2

require a radio frequency (RF) chain (including Digital-to-Analog converter, up converter, etc.) leading to high hardwarecost and power consumption, particularly in large-scale arraysystems, such as millimeter wave (mmWave) systems. Toreduce the required RF chains, a hybrid beamforming (HBF)architecture consisting of both analog beamforming and digitalbeamforming are widely studied for mmWave massive MIMOsystems [21]–[31], in which a small number of RF chainis utilized to implement the digital beamformer and a largenumber of phase shifters to realize the analog beamformer.By adopting this structure, the number of required RF chainscan be reduced significantly. According to the mapping fromthe RF chain to antennas, the hybrid beamforming architec-tures can be categorized into fully-connected [22]–[25] andpartially-connected (sub-connected) structures [26]–[28], asshown in Fig.1a and Fig.1b, respectively. Compared with fully-connected structure, partially-connected structure, where eachRF chain is connected only a subset of antennas, is more low-cost and power efficient.

In view of this, the fully-connected structure is firstlyintroduced for mmWave large-scale antenna arrays system in[22], in which the spatially sparse precoder is designed byutilizing the orthogonal matching pursuit (OMP) algorithm.Besides, the authors in [25] propose a step-by-step methodbased on the alternating minimize algorithm to design thehybrid beamformer mmWave MIMO system. Further, theauthors in [23] propose to design the hybrid beamformingand codebooks for wideband MIMO mmWave system withlimited feedback channel, in which an iterative algorithmis presented to find the optimal hybrid beamforming matrixfor a given codebooks. For partially-connected structures, itis challenging to design the hybrid beamformer due to thespecial hardware constraints. In [27], a successive interferencecancelation (SIC)-based hybrid beamforming is designed forpartially-connected structure. In addition, a greedy algorithm isdeveloped in [28] to dynamically optimize the subarrays in thepartially-connected structure for performance improvement.

It is worth pointing out that all the aforementioned workson the hybrid beamforming design just focuses on the com-munication systems. Recently, many researchers have devotedthemselves to the hybrid beamforming design for mmWaveDFRC system [32]–[35]. The mmWave DFRC system adopt-ing a hybrid beamforming structure is first presented in[32], where a weighting problem is derived to facilitate thetrade-off between MIMO radar and mmWave communication.Moreover, the authors in [33] propose to jointly design theanalog and digital beamformer for mmWave wideband Or-thogonal Frequency Division Multiplexing (OFDM) DFRCsystem, in which a weighted summation of the achievablespectral efficiency of communication and squared error (SE)of space frequency spectrum of radar is selected as objectivefunction. However, these works focus on designing the analogbeamformer by exploting the block coordinate descent (BCD)method, which may lead to performance loss. More impor-tantly, the BCD method has a high computational complexity,this is not suitable for large scale array. Benefiting from thevigorous development of optimizations on matrix manifolds[36] during the past decade, the most challenging non-convex

(a) Fully-connected structure (b) Partially-connected structure

Fig. 1. Two structures of hybrid beamforming in mmWave MIMO systems.

problems can be tackled more efficiently by manifold opti-mization. This motivates us to carry out the study on thedesign of analog and digital beamformers in DFRC system viamanifold minimization. The contributions of this paper can besummarized as follows.

1) The problems of the hybrid beamforming design of theDFRC system with the fully-connected and partially-connected structures are formulated by considering anoptimization criterion of a weighted summation of thecommunication and radar beamforming errors.

2) For the fully-connected structure, we propose themanifold alternating direction method of multipliers(MADMM) framework to tackle the hybrid beamform-ing optimization problem. To be more specific, wedecouple the digital and analog beamformers by usingthe ADMM framework, such that we can optimize thedigital and analog beamformers independently. More-over, the subproblem of updating ananlog beamfomer issuboptimally solved by utilizing the Manifold optimiza-tion with a low-complexity.

3) For the partially-connected structure, we propose theRiemannian product manifold (RPM) optimizationframework for the joint design of analog and digitalbeamforming. Based on the manifold optimization, theconstrained optimization problem reformulate as a un-constrained optimization problem, via transforming theconstant modulus and transmit power constraints intofeasible search region. We derive the expressions of theRiemannian product gradient and Hessian informations,and present the trust region (TR) algorithm to obtain thenear-optimal solution.

The rest of the paper is organized as follows. In SectionII, we introduce the system model for the mmWave DFRCsystem with hybrid beamforming structure. In Section III, weformulate our problem by considering the trade-off designsbetween radar and communication. In Section IV, an efficientMADMM algorithm for the hybrid beamforming with fully-connected structure is presented. In Section V, we propose theRiemannian product manifold trust region algorithm (RPM-TR) to design the hybrid beamformer with partially-connectedstructure. In Section VI, we analyze the numerical perfor-

Page 3: Manifold Optimization Methods for Hybrid beamforming in

3

Fig. 2. Overview of a mmWave dual-functional radar-communication system with hybrid analog and digital beamforming architecture at the transmitter (basestation). The base station transmits the waveforms to detect the interested radar targets and communicate with the downlink users.

mance of the proposed algorithms and compare our approacheswith the existing methods. Finally, we draws several conclu-sions in Section VII.

Notions:Unless otherwise specified, matrices are denotedby blod uppercase letters (i.e.,H), blod lowercase letters areused for vectors (i.e.,α), scalars are denoted by normal font(i.e.,β). (·)T , (·)∗ and (·)H represent the transpose, complexconjugate and Hermitian transpose respectively. The set of n-dimensional complex-valued (real-valued) vector and N ×Ncomplex-valued (real valued) matrices are denoted by Cn(Rn)and CN×N

(RN×N

), respectively. ⊗ and ◦ denote the Kro-

necker and Hadamard products between two matrices. | · | and‖ · ‖F represent determinant and Frobenius norm of argument,respectively. vec(·) and tr(·) stand for the vectorization andtrace of the argument, respectively. <e(·) denotes the realpart of argument, and E represents expectation of a complexvariable.

II. SYSTEM MODEL

Consider a mmWave MIMO-DFRC system, which transmitsprobing signals to targets of interest and sends communicationsymbols to downlink users simultaneously. As shown in Fig.2,the joint transmit system is equipped with NRF RF chainsand Nt antennas. The transmit array is assumed to be half-wavelength spaced uniform linear array (ULA), and sends Nsdata streams to K mobile users, each user is equipped withNr receive antennas. In our consideration, we assume that thenumber of RF chains is smaller than that of antennas at thetransmit end.

A. Communication Model

In hybrid beamforming system, the data stream s is firstprocessed by a digital beamformer FBB ∈ CNRF×Ns , andthen up-converted to the carrier frequency by passing throughNRF radio frequency (RF) chains. After that, the DFRCsystem uses an FRF ∈ CNt×NRF RF beamformer, whichis implemented using analog phase shifters, to construct thefinal transmitted signal. Mathematically, the Nt×1 transmittedsignal can be written as

x = FRFFBBs (1)

where, s ∈ CNs×1 is the transmitted symbol vector in thebaseband. Furthermore, it is assumed that E

(ssH

)= 1

NsINs

[22]. The transmitter should satisfy the transmission powerconstraint, i.e., ‖FRFFBB‖2F = Ns.

The received signal at the downlink user can be expressedas

y =√ρHFRFFBBs + n (2)

where ρ denotes the average received power, H ∈ CNr×Nt

is the downlink channel matrix, and n denotes the whiteGaussian noise with zero mean and variance σ2

n.Furthermore, there are several mapping strategies form RF

chains to antennas, here we consider the fully-connectedand partially-connected structures, as illustrated in Fig.1aand Fig.1b. For the fully-connected structure, the RF chainsare connected to all antennas, i.e. |FRF (i, j)| = 1,∀i, j.While for the partially-connected structure, we assume eachRF chain is connected to Nt/NRF antennas, such that wehave FRF = diag

[fRF1 , fRF2 , ..., fRFNRF

], where fRFn is

an Nt/NRF dimensional vector whose each element has aconstant modulus.

To incorporate the high free-space path loss and limitedscattering characteristics of the mmWave channels, we adopta clustered channel model, i.e. Saleh-Valenzuela model [22]–[24]. Under this model, the channel matrix can be written as

H =

√NtNrNclNray

Ncl∑i=1

Nray∑l=1

αilar (φril) at(φtil)H

(3)

where Ncl and Nray represent the number of clusters andthe number of rays in each cluster, and αil denotes the gainof lth ray in the ith propagation cluster. In addition, ar (φril)and at (φtil) represent the receive and transmit array responsevectors, where φril and φtil stand for azimuth angles of arrivaland departure (AoAs and AoDs), respectively. For N -antennaULA, the array response vector has the following generalform:

a (φ) =1√N

[1, ejπ sinφ, . . . , ejπ(N−1) sinφ

]T. (4)

Page 4: Manifold Optimization Methods for Hybrid beamforming in

4

In this paper, we assume that perfect channel state infor-mation (CSI) is known at the transmitter. Then, the channel’smutual information can be presented as [23], [27], [28]

R = log2

(∣∣∣∣INs+

ρ

Nsσ2n

HFRFFBBFHBBFHRFHH

∣∣∣∣) (5)

B. Radar Model

In the proposed DFRC system, the transmitter emits theprobing signals to the targets of interest. The baseband signalradiated towards a target located at the direction θ can bedescribed as

z (θ) = aHt (θ)Fs (6)

where F = FRFFBB and at (θ) has the similar form as (4).According to (6), the transmit beampattern of radar can be

given as

P (θ) =1

Nsat(θ)

HFFHat (θ) =

1

NsfHQ (θ) f (7)

where f , vec(F) = vec(FRFFBB) and the matrix Q (θ) isgiven by

Q (θ) = INs ⊗(at (θ) at(θ)

H)∈ CNtNs×NtNs (8)

In order to improve the detection performance, it is desiredto focus on the transmit power at the locations of the targetsof interest and to minimize the power at the sidelobes. Forthis purpose, we use integrated sidelobe to mainlobe ratio(ISMR) [37], [38] as an indicator to measure the propertyof the transmit beampattern, that is

ISMR =

∫ΘsP (θ)dθ∫

ΘmP (θ)dθ

=fHLsf

fHLmf(9)

where Θs and Θm represent the sidelobe an mainlobe regions,respectively. Ls and Lm are defined as

Ls =

∫Θs

Q (θ)dθ and Lm =

∫Θm

Q (θ)dθ (10)

III. PROBLEM FORMULATION

In this section, we first present the hybrid beamformingdesign for communication-only case and radar-only case, re-spectively. Then, based on the two special cases, we formulatea weighted summation problem to achieve a good trade-offbetween radar and communication for the DFRC system.

A. Hybrid Beamforming for Communication-Only System

In general, communication system need to maximize themutual information to guarantee the quality of service (QoS).In this paper, we mainly focus on the beamformer designand assume the receiver equipping with an ideal (fully-digital)decoders. Following this strategy, the problem of the hybridbeamformer design for the communication-only system can beestablished as

maxFRF ,FBB

R = log2

(∣∣∣INt + ρNsσ2

nHFRFFBBFHBBFHRFHH

∣∣∣)s.t. ‖FRFFBB‖2F = Ns

|FRF (i, j)| = 1,∀ (i, j) ∈ A(11)

where A is the feasible set of the analog precoder. Due to thenonconvexity of the objective function in (11), the problemis difficult to solve numerically. Nevertheless, it has beenshown in [22], [25] that maximizing the objective function in(11) approximately leads to the minimization of the followingproblem:

minFRF ,FBB

‖FRFFBB − FCom‖2Fs.t. ‖FRFFBB‖2F = Ns

|FRF (i, j)| = 1,∀ (i, j) ∈ A(12)

where FCom denotes the optimal fully digital precoder, whichcan be obtained by various approaches [25], [26], [30]. Here,this paper adopts zero-forcing (ZF) method to calculated it, as

FCom =1√

tr(HHH)−1

HH(HHH

)−1(13)

B. Hybrid Beamforming for Radar-Only System

On the other hand, for the radar-only system, the trans-mit beamformer is desired to carefully design to attain theminimum ISMR, the corresponding optimization problem isformulated as

minf

ISMR = fHLsffHLmf

s.t. ‖f‖2F = Ns(14)

whose optimal solution can be achieved by taking the gener-alized eigenvalue decomposition of (Lm,Ls) [37], i.e.

fRad = P(L−1s Lm

)(15)

where the operator P(·) denotes the principal eigenvector. Inorder to satisfy the power constraint in (14), the fRad shouldbe scaled as

√Ns

fRad

‖fRad‖ −→ fRad.Similarly to the communication-only case, the hybrid beam-

former is designed by considering the following surrogatefunction:

minFRF ,FBB

‖FRFFBB − FRad‖2Fs.t. ‖FRFFBB‖2F = Ns

|FRF (i, j)| = 1,∀ (i, j) ∈ A(16)

where FRad is the matrix form of fRad.

C. Hybrid Beamforming for DRFC System

In DFRC system, our aim is to design the hybrid beam-former to ensure not only the high-quality for communication,but also the detection performance for radar. To achieve thuspurpose, we introduce a weighted summation of the commu-nication and radar beamforming errors as the optimizationobjective, and the resultant problem can be formulated as

minFRF ,FBB

ϕ ‖FRFFBB − FCom‖2F

+ (1− ϕ) ‖FRFFBB − FRad‖2F (17a)

s.t. ‖FRFFBB‖2F = Ns (17b)|FRF (i, j)| = 1,∀ (i, j) ∈ A (17c)

where ϕ ∈ [0, 1] is a weighting factor (trade-off parameter)that determines the weights for communication and radar

Page 5: Manifold Optimization Methods for Hybrid beamforming in

5

performance. In Sections IV and V, we will consider theRF beamformer designs for the fully-connected structure andpartially-connected structure, respectively. It is noticed thatthe problem (17) is difficult to tackle due to the nonconvexconstraints. Nevertheless, we note that the feasible region inproblem (17) satisfies the manifold constraints. Consequently,in the following, we propose to solve the nonconvex problem(17) with the aid of the Riemannian optimization algorithms[36], [39], [40], which finds a suboptimal solution with low-complexity.

IV. MADMM BASED HYBRID BEAMFORMING FOR THEFULLY-CONNECTED STRUCTURE

In this section, we consider the hybrid beamforming designfor the fully-connected structure. In order to obtain a near-optimal solution to the problem (17) with the constraint|FRF (i, j)| = 1,∀i, j, the manifold alternating directionmethod of multipliers (MADMM) is proposed.

A. MADMM algorithm

As discussed in [41] that solving problem (17) with theADMM framework needs to decouple FRF and FBB inconstraint (17b). To attain such purpose, we introduce anauxiliary variable F, and recast the problem (17) as thefollowing problem:

minFRF ,FBB

ϕ ‖F− FCom‖2F + (1− ϕ) ‖F− FRad‖2F

s.t. F = FRFFBB

‖F‖2F = NS

|FRF (i, j)| = 1,∀i, j

(18)

This will enable us to solve the problem (18) more easilyunder the ADMM framework. Specifically, placing the equal-ity constraint F = FRFFBB into the augmented Lagrangianfunction yields

L (F,FRF ,FBB ,Λ)

= ϕ ‖F− FCom‖2F + (1− ϕ) ‖F− FRad‖2F+tr(ΛH (F− FRFFBB)

)+α

2‖F− FRFFBB‖2F

(19)

where Λ ∈ CNt×NS and α > 0 denote dual variables andpenalty parameter, respectively. In order to simplify the aug-mented Lagrangian function, we further expand the functionof (19) as

L (F,FRF ,FBB ,Λ)

= ϕ ‖F− FCom‖2F + (1− ϕ) ‖F− FRad‖2F

2

∥∥∥∥F +Λ

α− FRFFBB

∥∥∥∥2

F

(20)

Based on the iterative procedure of the ADMM algorithm,we then determine {F,FRF ,FBB ,Λ} via the following steps:

F(n) = arg minFL(F,F

(n−1)RF ,F

(n−1)BB ,Λ(n−1)

)s.t. ‖F‖2F = NS

(21)

F(n)RF = arg min

FRF

L(F(n),FRF ,F

(n−1)BB ,Λ(n−1)

)s.t. |FRF (i, j)| = 1,∀i, j

(22)

F(n)BB = arg min

FBB

L(F(n),F

(n)RF ,FBB ,Λ

(n−1))

(23)

Λ(n) = Λ(n−1) + α(F(n) − F

(n)RFF

(n)BB

)(24)

In the following, the solutions to problems (21), (22) and(23) are presented.

1) Update of F: For the fixed{

F(n−1)RF ,F

(n−1)BB ,Λ(n−1)

},

the minimization of (21) with respect to F is given by

minF

ϕ ‖F− FCom‖2F + (1− ϕ) ‖F− FRad‖2F

2

∥∥∥∥F +Λ(n−1)

α− F

(n−1)RF F

(n−1)BB

∥∥∥∥2

F

s.t. ‖F‖2F = NS

(25)

We note that its closed form solution can be attained byusing the Karush-Kuhn-Tucker (KKT) conditions. Specifically,introducing a Lagrange multiplier Φ on the power constraint,we obtain the following Lagrange function:

K (F,Φ)

= ϕ ‖F− FCom‖2F + (1− ϕ) ‖F− FRad‖2F

2

∥∥∥∥F +Λ(n−1)

α− F

(n−1)RF F

(n−1)BB

∥∥∥∥2

F

+ Φ(‖F‖2F −NS

)(26)

Based on the Lagrangian in (26), the KKT conditions foroptimality are then obtained as

∂K∂F

= 2ϕ (F− FCom) + 2 (1− ϕ) (F− FRad)

(F +

Λ(n−1)

α− F

(n−1)RF F

(n−1)BB

)+ 2ΦF = 0 (27a)

Φ(‖F‖2F −NS

)= 0 (27b)

Φ > 0 (27c)

Hence, based on the KKT conditions, the closed-formsolution to F is given by

Fopt =√NS

F∥∥F∥∥F

(28)

whereF = 2ϕFCom + 2 (1− ϕ) FRad

− α(

Λ(n−1)

α− F

(n−1)RF F

(n−1)BB

)2) Update of FRF : For fixed

{F(n),F

(n−1)BB ,Λ(n−1)

}, the

solution to subproblem (22) can be simplified as

minFRF

∥∥∥∥F(n) +Λ(n−1)

α− FRFF

(n−1)BB

∥∥∥∥2

F

s.t. |FRF (i, j)| = 1,∀i, j(29)

Page 6: Manifold Optimization Methods for Hybrid beamforming in

6

Based on the manifold optimization in [36], [39], [40], prob-lem (29) can be converted into the unconstrained optimizationproblem over manifold, given by

minfRF∈MfRF

g (fRF ) =α

2

∥∥∥∥f (n) +Υ(n−1)

α− fBBfRF

∥∥∥∥2

F

(30)

where f (n) = vec(F(n)

), Υ(n−1) = vec

(Λ(n−1)

), fBB =(

F(n−1)BB

)T⊗ I. MfRF

denotes an NtNRF dimensional com-plex circle manifold with the geometric structure being givenby

MfRF={fRF ∈ CNtNRF : |fRF | = 1

}(31)

Following [36], [39], in order to utilize the Riemannianoptimization methods, we need to define a tangent space whichis a linear space around any point on a smooth manifold.More concretely, the tangent space TfRF

MfRFat the point

fRF ∈MfRFis defined as

TfRFMfRF

={υ ∈ CNRFNt : <e (υ ◦ f∗RF ) = 0

}(32)

where υ denotes the tangent vector at point fRF ∈MfRF.

According to the [39], the Riemannian gradient ofMfRFat

fRF is a tangent vector gradg(fRF ) given by the projection ofthe Euclidean gradient Gradg(fRF ) on the the tangent spaceTfRFMfRF

:

gradg(fRF ) =ProjfRF(Gradg(fRF ))

=Gradg(fRF )

−<(Gradg(fRF )

∗ ◦ fRF)◦ fRF

(33)

where the Euclidean gradient Gradg(fRF ) is expressed as

Gradg(fRF )

= −α(f

(n−1)BB

)H (f (n) +

Υ(n−1)

α+ f

(n−1)BB fRF

)(34)

Based on the above Riemannian gradients, in what fol-low, we propose an Riemannian Conjugate Gradient (RCG)algorithm, which obtains the near-optimal solution with muchlower complexity [39], to solve the problem (22). For RCGmethod, the beamforming matrix to be updated at each iter-ation is computed by the descent direction and stepsize. Inthe k-th iteration, the search direction Γ

(k)fRF

is determined bythe Riemannian gradient gradg(f

(k)RF ) and the (k − 1)-th search

direction Γ(k−1)fRF

, which is given by

Γ(k)fRF

= −gradg(f

(k)RF

)+ σ

(k−1)fRF

Projf(k)RF

(k−1)fRF

)(35)

where σfRFis the polak-Ribiere parameter [42], defined as

σ(k)fRF

=

⟨gradg(f

(k)RF ),χ

(k)fRF

⟩⟨

gradg(f(k−1)RF ), gradg(f

(k−1)RF )

⟩ (36)

with 〈·, ·〉 denoting the usual Euclidean inner product, i.e.,〈p,q〉 = Re

(pHq

), and χfRF

standing for the differencebetween the current and the previous gradients, and it isexpressed as

χ(k)fRF

= gradg1(f(k)RF )− Proj

f(k−1)RF

(gradg1(f

(k−1)RF )

)(37)

Then, the stepsize µ(k)fRF

is chosen by the Armijo backtrack-ing line search method [42], and the beamforming matrix f

(k)RF

to be updated at the step (k + 1) is calculated by the retractionon the manifold, which is

f(k+1)RF = R

f(k)RF

(k)fRF

Γ(k)fRF

)(38)

where R is a mapping operator, named as retraction, whichmaps a vector from the tangent space onto the manifold itself,given by

RfRF(µfRF

ΓfRF) = vec

[(fRF + µfRF

ΓfRF)i

|(fRF + µfRFΓfRF

)i|

](39)

Based on the above analysis, the RCG Algorithm for solving(22) is summarized by Algorithm 1.

Algorithm 1 RCG Algorithm for solving (22)

Input: Set the initial variables F(n), F(n−1)BB , Λ(n−1), FRad,

FCom, α and η and Kmax.Output: F

(opt)RF .

1: Initialization: Randomly set F(0) ∈MF

2: Set k = 1.3: while k 6 Kmax and

∥∥∥gradg(f

(k)RF

)∥∥∥F> η do

4: Compute the stepsize µ(k)fRF

using the Armijo Rule.5: Calculate the Polak-Ribiere parameter based on (36).6: Obtain the descent direction according to (35).7: Update the beamforming matrix f

(k+1)RF by (38).

8: k = k + 19: end while

10: F(opt)RF = unvec

(f

(k)RF

)3) Update of FBB: For fixed

{F(n),F

(n)RF ,Λ

(n−1)}

, thesolution to subproblem (23) can be simplified as

minFBB

f3 (FBB) =

∥∥∥∥F(n) +Λ(n−1)

α− F

(n)RFFBB

∥∥∥∥2

F

(40)

It is easy to observe that the problem (40) is convexQuadratic Program without constraint, and its closed-formsolution can be attained as

F(n)BB =

((F

(n)RF

)HF

(n)RF

)−1(F

(n)RF

)H (F(n) +

Λ(n−1)

α

)(41)

4) Update Penalty Parameter α: In conventional ADMMalgorithm [41], using a fixed penalty parameter may leadto poor convergence performance. In order to improve theconvergence and make performance less dependent on theinitial choice of the penalty parameter, we use the followingupdate scheme of α [41]:

α(n) =

βα(n−1), if

∥∥∥F(n)−F(n)RF F

(n)BB

∥∥∥2

F

‖Λ(n)−Λ(n−1)‖ > γ

α(n−1)/β, if

∥∥∥F(n)−F(n)RF F

(n)BB

∥∥∥2

F

‖Λ(n)−Λ(n−1)‖ < 1γ

α(n−1), otherwise

(42)

where β > 1 and γ > 1. In our case, we set β = 2 andγ = 10.

Page 7: Manifold Optimization Methods for Hybrid beamforming in

7

According to the above analysis, the proposed methodfor designing the hybrid beamforming in the case of fully-connected structure is summarized in Algorithm 2.

Algorithm 2 MADMM for fully-connected structure hybridbeamforming designInput: Set the initial variables F0, F0

RF , F0BB , Λ0, α0, FRad,

FCom and Nmax.Output: F

(opt)RF and F

(opt)BB .

1: Initialization: Set n = 1, β = 2 and γ = 10.2: while n 6 Nmax do3: For given F

(n−1)RF ,F

(n−1)BB ,Λ(n−1), obtain the solution

of F(n) by using (28)4: For given F(n),F

(n−1)BB ,Λ(n−1), obtain the solution of

F(n)RF according to Algorithm 1.

5: For given F(n),F(n)RF ,Λ

(n−1), update F(n)BB using (41).

6: For given F(n),F(n)RF ,F

(n)BB , update Λ(n) via (24).

7: Update α(n) according to (42).8: n = n+ 19: end while

10: F(opt)RF = F

(Nmax)BB and F

(opt)BB = F

(Nmax)BB

At the end of this section, we analyze the complexityof the proposed MADMM algorithm for the hybrid beam-forming in the case of fully-connected structure. It shouldbe noted that the main computational complexity of theMADMM method is caused by solving three subproblems,i.e. (21), (22) and (23). Solving (21) needs complexities ofO (NsNRFNt). Using the RCG method solves (22) with thecomplexities of O

(I1NsNRFN

2t

), where I1 are the number

of the iterations required in RCG methods. Solving (23)needs complexities of O

(N2RFNt +NsNRFNt

). To sum-

marize, the overall complexity of the MADMM method isO(I0(I1NsNRFN

2t +N2

RFNt +NsNRFNt))

), where I0 isthe number of MADMM iterations. For comparison, we nextfocus on the computational complexity of the BCD method,which using BCD method to solve subproblem (23) needscomplexities O

(NsN

3RFNt +NsN

2RFN

2t

). It is easy to con-

clude that comparing with RCG method for (23), the BCDalgorithm requires overwhelmingly high computational cost,which makes it inapplicable for the massive MIMO DFRCsystem.

V. RIEMANNIAN PRODUCT MANIFOLD BASED HYBRIDBEAMFORMING FOR THE PARTIALLY-CONNECTED

STRUCTURE

In contract to the fully-connected structure, the partially-connected structure adopts less phase shifters and has lowerpower and hardware cost in mmWave systems. However, dueto the fact that the analog beamfomer matrix is a blockdiagonal structure with the unit modulus, the correspondingoptimization problem is hard to tackle. Therefore, in thissection, we propose the Riemannian product manifold trustregion (RPM-TR) algorithm to design the hybrid beamfomingwith low-complexity.

A. Problem Reformulation

Due to the special structure of the FRF , we have thefollowing equality:

‖FRFFBB‖2F =NtNRF

‖FBB‖2F = NS (43)

We note that the relationship between the partially-connected structure and fully-connected can be established asfollows,

FRF = FPS ◦ FD (44)

where FPS ∈ CNt×NRF stands for the fully-connectedphase shifter matrix, i.e. |FPS (i, j)| = 1,∀i, j, and FD ∈CNt×NRF denotes 0-1 connection-state matrix. For thepartially-connected structure, the connection-state matrix isFD = diag (1z,1z, ...,1z) ∈ CNt×NRF , where z = Nt

NRF

Based on the above analysis, the problem (17) can be rewrittenas

minFPS ,FBB

ϕ ‖(FPS ◦ FD) FBB − FCom‖2F

+ (1− ϕ) ‖(FPS ◦ FD) FBB − FRad‖2F

s.t. ‖FBB‖2F =NSNRFNt

,

|FPS (i, j)| = 1,∀i, j

(45)

For the convenience of deriving the problem, we use theJ (FPS ,FBB) denoting the objective function in (45).

B. Geometric Structure of the Riemannian Product Manifold

In order to solve this problem effectively, we recast the con-strained optimization problem (45) as an standard optimizationform over a product manifold M, as

min(FPS ,FBB)∈M

J (FPS ,FBB) (46)

where M = MFPS×MFBB

denotes the product manifoldof MFPS

={FPS ∈ CNt×NRF : |FPS(i, j)| = 1,∀i, j

}and

MFBB={

FBB ∈ CNRF×NS : ‖FBB‖F =√

NSNRF

Nt

}. For

the RPM, the tangent space T(FPS ,FBB)M can be decomposedas a product of two tangent spaces

T(FPS ,FBB)M = TFPSMFPS

× TFBBMFBB

= {(ζFPS, ζBB) : ζFPS

∈ CNt×NRF , ζFBB∈ CNRF×NS ,

Re (ζFPS◦ F∗PS) = 0,Re

(tr(FHBBζFBB

))= 0}

(47)Following some results in [36], [39], [40], the Riemannian

gradient gradJ (FPS ,FBB) can be expressed as

gradJ (FPS ,FBB)

= (gradFPSJ (FPS ,FBB) , gradFBB

J (FPS ,FBB))(48)

Page 8: Manifold Optimization Methods for Hybrid beamforming in

8

where the Riemannian gradients gradFPSJ (FPS ,FBB) and

gradFBBJ (FPS ,FBB) are computed by orthogonal projec-

tion from Euclidean space to the tangent space [36] as

gradFPSJ (FPS ,FBB)

= ProjFPS(GradFPS

J (FPS ,FBB))

= GradFPSJ (FPS ,FBB)

−<{GradFPSJ (FPS ,FBB) ◦ F∗PS} ◦ FPS (49a)

gradFBBJ (FPS ,FBB)

= ProjFBB(GradFBB

J (FPS ,FBB))

= GradFBBJ (FPS ,FBB)

−<{

tr(FHBBGradFBB

g (FPS ,FBB))}

FBB (49b)

where GradFPSJ (FPS ,FBB) and GradFBB

J (FPS ,FBB) de-note, respectively, the Euclidean gradients with respect to FPSand FBB , and they are calculated as

GradFPSJ (FPS ,FBB)

= 2ϕ{

((FD ◦ FPS) FBB − FCom) FHBB}◦ FD

+ 2 (1− ϕ){

((FD ◦ FPS) FBB − FRad) FHBB}◦ FD

(50a)GradFBB

J (FPS ,FBB)

= 2ϕ(FD ◦ FPS)H {(FD ◦ FPS) FBB − FCom}

+ 2(1− ϕ)(FD ◦ FPS)H {(FD ◦ FPS) FBB − FRad}

(50b)

Unlike the Riemannian manifold first-order optimization al-gorithm, for instance, RCG method in section IV, the second-order method adopts the Riemannian Hessian for quadraticinformation about the objective function, which enjoys morerapid convergence. Thus, to utilize second-order algorithmsolving (45), we need derive the explicit expression of theRiemannian Hessian in RPM. The Riemannian Hessian ofJ (FPS ,FPS) at (FPS ,FPS) ∈ M is a linear operatorHess(FPS ,FBB)J : T(FPS ,FBB)M 7→ T(FPS ,FBB)M [36],[39], [40] defined by

Hess(FPS ,FBB)J (FPS ,FPS)[ζ(FPS ,FBB)

]= ∇ζ(FPS,FBB)

gradJ (FPS ,FPS)(51)

for the point ζ(FPS ,FBB) ∈ T(FPS ,FBB)M. ∇ denotes theLevi-Civita connection of M. Specifically, for the RPM, theRiemannian Hessian of J (FPS ,FPS) holds that [36]

Hess(FPS ,FBB)J (FPS ,FPS)[ζ(FPS ,FBB)

]=(HessFPS

J (FPS ,FPS)[ζ(FPS ,FBB)

],

HessFBBJ (FPS ,FPS)

[ζ(FPS ,FBB)

]) (52)

where ζ(FPS ,FBB) is tangent vector on the tangent space,i.e. ζ(FPS ,FBB) ∈ T(FPS ,FBB)M, HessFPS

J (FPS ,FPS) andHessFBB

J (FPS ,FPS) denote, respectively, the RiemannianHessian of J (FPS ,FPS) with respect to FPS and FPS .Based on the classical expression of the Levi-Civita connectionon a Riemannian submanifold of a Euclidean space [36], [39],[40], the Riemannian Hessian can be calculated via directionderivative in the embedding space followed by an orthogonal

projection to the tangent space. Thus, the Riemannian Hes-sian HessFPS

J (FPS ,FPS) and HessFBBJ (FPS ,FPS) are

computed as [43], [44]

HessFPSJ (FPS ,FPS)

[ζ(FPS ,FBB)

]= ∇ζFPS

gradJ (FPS ,FPS)

= ProjFPS

(DgradFPS

J (FPS ,FPS)[ζ(FPS ,FBB)

])= ProjFPS

(eHessFPS

J −Re(FPS ◦ (GradFPS

J)∗) ◦ ζFPS

)(53a)

HessFBBJ (FPS ,FPS)

[ζ(FPS ,FBB)

]= ∇ζFBB

gradJ (FPS ,FPS)

= ProjFBB

(DgradFBB

J (FPS ,FPS)[ζ(FPS ,FBB)

])= ProjFBB

(eHessFBBJ) + Re (F∗BBGradFBB

J) ζFBB

(53b)

where DgradFPSJ (FPS ,FPS)

[ζ(FPS ,FBB)

]and

DgradFBBJ (FPS ,FPS)

[ζ(FPS ,FBB)

]are the directional

derivative of the Riemannian gradient gradFPSJ (FPS ,FPS)

and gradFBBJ (FPS ,FPS), the Euclidean Hessian

eHessFPSJ and eHessFBB

J denote, respectively, thedirectional derivative of the Euclidean gradient (50a) and(50b), which are computed as

eHessfPSJ

= 2ϕ{

((FD ◦ FPS) FBB − FCom) ζHBB}◦ FD

+ 2 (1− ϕ){

((FD ◦ FPS) FBB − FRad) ζHBB

}◦ FD

+ 2{

((FD ◦ ζPS) FBB + (FD ◦ FPS) ζBB) FHBB}◦ FD

eHessFBBJ

= 2ϕ(FD ◦ ζPS)H {(FD ◦ FPS) FBB − FCom}

+ 2(1− ϕ)(FD ◦ ζPS)H {(FD ◦ FPS) FBB − FRad}

+ 2(FD ◦ FPS)H {(FD ◦ ζPS) FBB + (FD ◦ FPS) ζBB}

C. Riemannian Product Manifold Trust Region Algorithm

Based on the above concepts, below we develop the RPM-TR algorithm [36], [39], [43], [45] to solve the problem. Trustregion methods [45] can be understood as an enhancement ofNewton’s method. Based on [36], [39], the main idea of RPM-TR can be summarized in three steps: 1) Generate a model thatis a good approximation of the objective function in a specificregion, 2) find the step that minimizes the model function inthat specific region, determined by the trust region size, and3) compute a new iterate based on a retracting mapping.

Firstly, we use a quadratic function to approximate theobjective function (45) at the kth iteration with the followingform:

wk

(ζ(

F(k)PS ,F

(k)BB

))= J

(F

(k)PS ,F

(k)BB

)+

⟨gradJ

(F

(k)PS ,F

(k)BB

), ζ(

F(k)PS ,F

(k)BB

)⟩+

1

2

⟨HessJ

(F

(k)PS ,F

(k)BB

)[ζ(

F(k)PS ,F

(k)BB

)], ζ(F

(k)PS ,F

(k)BB

)⟩(55)

Furthermore, since the model is only a local approximation ofthe pullback, we only trust it in a ball around the origin in the

Page 9: Manifold Optimization Methods for Hybrid beamforming in

9

tangent space. Thus, in the RPM-TR framework, the updateof (fRF ,FBB) within the trust region radius ∆k is attainedby solving

min(FRF ,FBB)

wk(ζ(fRF ,FBB)

)s.t.

⟨ζ(FRF ,FBB), ζ(FRF ,FBB)

⟩6 ∆2

k

(56)

Next, an approximate solution ζ(F

(k)PS ,F

(k)BB

) of the trust

region subproblem (56) is computed, for example using ainterior point method. Notice that the approximate solutionis a local minimum point in tangent space, i.e. ζ(

F(k)PS ,F

(k)BB

) ∈T(

F(k)PS ,F

(k)BB

)M.

Therefore, in the last step, the candidate for the new iterateis obtained by retraction mapping the approximate solution tothe product manifold M, which is

(F

(k)♦PS ,F

(k)♦BB

)= R(

F(k)PS ,F

(k)BB

)(ζ(F

(k)PS ,F

(k)BB

)) (57)

whereR(FPS ,FBB)

(ζ(FPS ,FBB)

)= (RFPS

(ζFPS) ,RFBB

(ζFBB))

(58)

with RFPS(ζFPS

) and RFBB(ζFBB

) being retraction opera-tors on manifold MFPS

and MFBB, respectively, and being

defined as

RFPS(ζFPS

) = vec

[(FPS + ζFPS

)i,j|FPS + ζFPS

|i,j

](59a)

RFBB(ζFBB

) =

√NSNRFNt

(FBB + ζFBB)

‖FBB + ζFBB‖F

(59b)

Once obtaining the candidate form (57), the quality of themodel (55) is assessed by forming the quotient

ρk =J(F

(k)PS ,F

(k)BB

)− J

(F

(k)♦PS ,F

(k)♦BB

)wk

(0(

F(k)PS ,F

(k)BB

))− wk (ζ(F(k)PS ,F

(k)BB

)) (60)

Depending on the value of ρk, the candidate(F

(k)♦PS ,F

(k)♦BB

)will be accepted or discarded and the trust-region radius ∆k+1

will be updated [36], [39]. Specifically, a specific procedureis accepted or rejected the

(F

(k)♦PS ,F

(k)♦BB

)based on the rule

[36], [39]:

(F

(k+1)PS ,F

(k+1)BB

)=

(F

(k)♦PS ,F

(k)♦BB

)ifρk > ρ′(accept),(

F(k)PS ,F

(k)BB

)otherwise(reject).

(61)Finally, the trust region radius ∆k+1 needs to be updated in

each iteration. More concretely, the following update of ∆k+1

is used [36], [39], [43]:

∆k+1 =

1

4∆k ifρk <

1

4

min (2∆k,∆′) ifρk >

3

4and

∥∥∥∥ζ(F(k)PS ,F

(k)BB

)∥∥∥∥ = ∆k

∆k otherwise(62)

According to the above analysis, the proposed RPM-TRalogrithm for solving problem (45) is summarized in Algo-rithm 3.

Algorithm 3 RPM-TR alogrithm for partially-connected struc-ture hybrid beamforming design

Input: Initial iteration(F

(0)PS ,F

(0)BB

)∈ M, maximum radius

∆′ > 0, reference beamforming matrix FRad, FCom,maximum iterations Kmax, gradient threshold δ.

Output: F(opt)PS ,F

(opt)BB .

1: Initialization: Set k = 1, ∆0 ∈ (0,∆′) and ρ′ ∈[0, 1

4

).

2: for k 6 Kmax and ‖gradJ (FPS ,FBB)‖F > δ do3: Approximately solve the subproblem (56), obtaining

ζ(F

(k)PS ,F

(k)BB

) ∈ T(F

(k)PS ,F

(k)BB

)M;

4: Calculate the candidate for next iterate according to(57);

5: Compute the ratio of actual to model improvementusing (60);

6: Accept or reject the tentative next iterate depending onthe rule (61);

7: Update the trust region radius according to (62);8: k = k + 1;9: end for

10: F(opt)PS =

(F

(k)PS

)and F

(opt)BB = F

(k)BB

We end this section by analyzing the complexity of theproposed RPM-TR algorithm. For the RPM-TR algorithm,the computational complexity mainly consists of computationsof Riemannian gradient and Riemannian Hessian with com-plexities of O

(N2t NRFNs

)and O

(N2t N

2RF +N2

t NRFNs),

respectively. Hence, the complexity of the algorithm 3 isO(Iiter

(N2t N

2RF +N2

t NRFNs))

, where Iiter stands for thetotal iteration numbers of the algorithm.

VI. SIMULATION RESULTS AND DISCUSSION

In this section, several sets of numerical simulations are pre-sented to evaluate the performance of our proposed algorithms.Unless otherwise specified, in all simulation, we assume thatthe transmitter with Nt = 32 antennas send Ns = 6 datastreams to receiver with Nr = Ns = 6 antennas. The numberof RF chains in transmit DFRC system are NRF = 16, and thereceiver adopts the fully digital beamforming structure. Thenumber of clusters and rays are set as, respectively, Ncl = 10and Nray = 5, and the average power of each cluster isαij = 1. The AoAs/DoAs of the all clusters φtil, φ

ril are

assumed to be uniformly distributed in [0, 2π). The terminationcriterion of RPM-TR algorithm is the line-search returns adisplacement vector smaller than 10−6. As for the MADMMalgorithm, we assume the maximum number of iterationsis Nmax = 200. For all simulation results, we denote thebeamformers FCom and FRad as ‘ZF precoder’ and ‘Ref.beampattern’ , respectively.

A. Convergence Performance

Fig.3 analyzes the convergence performance of the proposedMADMM and RPM-TR for solving problems (18) and (45)

Page 10: Manifold Optimization Methods for Hybrid beamforming in

10

0 20 40 60 80 100 120 140 160 180 200

Iteration Number

2

4

6

8

10O

bje

ctive V

alu

e

0 20 40 60 80 100 120 140 160 180 200

Iteration Number

0

2

4

6

Prim

al R

esid

ual

10-8

0 50 1000

510

-9

0 50 1003

3.5

4

(a) The convergence performance of the MADMM algorithm

0 1 2 3 4 5 6 7 8 9 10

Iteration number

2

4

6

8

10

Ob

jective

Va

lue

0 1 2 3 4 5 6 7 8 9 10

Iteration number

10-5

100

No

rm o

f th

e g

rad

ien

t o

f f

(b) The convergence performance of the RPM-TR algorithm

Fig. 3. The convergence performance of the RPM-TR algorithm andMADMM algorithm. The top figure in (a) and (b) show the objective value,and the bottom shows the gradient normal or primal residual

when considering the trade-off parameter ϕ = 0.5. FromFig.3a, it can be seen from the top figure that the objectivevalue in (20) converges very fast. Furthermore, the primalresidual tend to zero as the iterative process goes on, whichverifies the effectiveness of the proposed MADMM algorithm.In Fig.3b, the top plot shows objective function in (45) versusthe number of the iterations, which illustrate that the objectivevalue decreases with the iteration number. In addition, thenorm of the Riemannian gradient displayed in Fig.3b bottomtends to 10−6 within 10 iterations, which indicate the goodconvergence performance of the RPM-TR algorithm.

B. Spectral Efficiency Performance

In this subsection, we investigate the achievable rateof downlink communication obtain by different algorithms.Fig.4a shows the achievable rate as a function of the SNRvalue and trade-off parameter ϕ for the fully-connected struc-ture. For comparison purpose, the BCD-based method pro-posed in [46], [47] is also considered. As expected, the optimaldigital precoder can attain the best achievable rate. In addition,

-35 -30 -25 -20 -15 -10 -5 0 5

SNR (dB)

0

2

4

6

8

10

12

14

16

18

20

Achie

vable

Rate

(bp

s/H

z)

(a) The achievable rate of the fully-connected structure obtained by MADMM

-35 -30 -25 -20 -15 -10 -5 0 5

SNR (dB)

0

2

4

6

8

10

12

14

16

18

20

Achie

vable

Rate

(bps/H

z)

(b) The achievable rate of the partially-connected structure obtained by RPM-TR

Fig. 4. The achievable rate values versus noise power for different trade-offparameter ϕ

as can be seen from the Fig.4a that, the achievable rates of theMADMM and BCD increase with the SNR value and trade-offparameter. We also find that the MADMM algorithm performsslightly better than the BCD-based method.

Fig.4b compares the achievable rate obtained by the RPM-TR method to that by the exiting Alternating Minimization(AltMin) algorithm [25] for the partially-connected structure.The result in this figure shows s similar phenomenon to that inFig.4a. Moreover, it can be observed that the proposed RPM-TR algorithm outperforms over the AltMin algorithm in termsof the achievable rate. Furthermore, the comparison betweentwo hybrid beamforming structures show that the partially-structure results in a non-negligible performance loss whencomparing with the fully-connected structure.

C. Beampattern Performance

Fig.5 shows the beampattern behavior of the proposed algo-rithms, herein we assume the targets locating at the angles of[−30◦, 30◦]. The results indicate that the smaller the trade-off

Page 11: Manifold Optimization Methods for Hybrid beamforming in

11

-80 -60 -40 -20 0 20 40 60 80

Angle (°)

-40

-35

-30

-25

-20

-15

-10

-5

0

5

10B

ea

mpa

ttern

(d

B)

(a) Radar beampatterns obtained by MADMM algorithm for fully-connectedstructure

-80 -60 -40 -20 0 20 40 60 80

Angle (°)

-40

-35

-30

-25

-20

-15

-10

-5

0

5

10

Beam

pattern

(dB

)

(b) Radar beampatterns obtained by RPM-TR algorithm for partially-connectedstructure

Fig. 5. Radar beampatterns obtained by RPM-TR and MADMM algorithm

parameter ϕ is, i.e. more important weighting on the radar, thesidelobe level of the designed beampattern becomes better, onthe contrary, the achievable rates of downlink communicationget the worse. As a result, a trade-off parameter should bedelicately selected to achieve the a good balance between thecommunication and radar.

Next, we consider the ISMR values of the obtained beam-patterns for the partially-connected and fully-connected struc-tures. Fig. 6 show the beampattern ISMR values attainedby the MADMM, BCD, RPM-TR and AltMin algorithms,respectively. Note that the MADMM and BCD are for thefully-connected structure, and the RPM-TR and AltMin arefor the partially-connected structure. It is seen from the figurethat as ϕ increases, the ISMR properties obtained by all al-gorithms become worse and worse. Moreover, it is interestingto observe that for ϕ ≤ 0.7, the algorithms of the partially-connected structure have ISMR behaviors than those of thefully-connected structure, while vice versa for ϕ ≥ 0.8.

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-5

0

5

10

15

20

25

Avera

ge IS

MR

(dB

)

Fig. 6. The ISMR of RPM-TR, AltMin, MADMM and BCD algorithmsversus trade-off parameter

D. Performance With Different RF Chain numbers

In this part, we will compare the performance of the twohybrid beamforming structures for different numbers of RFchains NRF . The simulation parameters are set as follows:data stream NS = 4, received antennas Nr = NS = 4.Fig.7a shows the spectral efficiency achieved by the MADMMmethod as a function of the number of the SNR and NRFfor the fully-connected structure. For comparison purpose, theBCD is selected as the benchmark. From the figure, we can seethat although the proposed MADMM algorithm is bad than theBCD method when NRF < 8, and the two methods providenearly the same result when NRF > 8, as mentioned earlier,the MADMM method has a lower computational complexity.Moreover, we also find that the achievable rates increase withthe number of RF chains. Interestingly, the achievable ratesare the same and equal to a upper bound when NRF ≥ 8.This indicates that it is possible to reduce the number of RFchains in the DFRC system to reduce the hardware cost. Inaddition, it is no surprise that the achievable rates increasewith the SNR value.

Fig.7b displays the achievable spectral efficiency of theproposed hybrid beamforming method for partially-connectedstructure with different SNR. As expected, for the same SNR,the achievable spectral efficiency increase along with theNRF . Besides, it is interesting to note that the NRF has agreater effect on the partially-connected structure than thefully-connected one, and that the fully-connected structureis always better than the partially-connected one. Therefore,in practice, we should make a suitable balance between theperformance and the cost. Furthermore, we also observe thatthe proposed RPM-TR method outperforms the Altmin interms of the achievable rate.

Next, we compare the ISMR behavior of the fully-connectedstructure to that of the partially-connected one for differentRF chains NRF . Fig.8a shows the ISMR of the proposedMADMM as a function of the RF chains NRF and trade-offparameters ϕ, as a comparator, the BCD is also considered.

Page 12: Manifold Optimization Methods for Hybrid beamforming in

12

2 4 8 16 32

Number of RF chains NRF

0

5

10

15

20

25

30A

chie

vable

Rate

(bps/H

z)

MADMM BCD

(a)

2 4 8 16 32

Number of RF chains NRF

0

5

10

15

20

25

30

Achie

vable

Rate

(bps/H

z)

RPM-TR

AltMin

(b)

Fig. 7. The spectral efficiency values versus RF chains for different SNRwith trade-off parameter ϕ = 0.4. (a) Fully-connected structure, (b) partially-connected structure.

The results in Fig.8a show that with the increasing of RFchains NRF , the ISMR value becomes more large whenNRF < 2NS , and then ISMR is almost unchanged whenNRF > 2NS . This means that adding the number of RF chainsleads to the bad ISMR value instead. Moreover, it is alsoobserved that the proposed MADMM method performs betterthan BCD method in terms of the radar ISMR performance.

As a further illustration, Fig.8b shows that, for the caseϕ > 0.4, even though the proposed RPM-TR method isworse than the AltMin method regarding the radar ISRM, asshown in Fig. 7b, the proposed RPM-TR method achievesthe better communication rate. Combining Fig.7a and Fig.8a,we find that there is no need to increase the number of RFchains when NRF ≥ 8 in our considered case for the fully-connected structure. Besides, comparing Fig.7b and Fig.8b, weconclude that the communication rate is improved obviouslyby adding the number of RF chains, but radar ISMR behaviorswill become worse. This indicates that a reasonable trade-offbetween communication and radar should be made by setting

2 4 8 16 32

Number of RF chains NRF

-3

-2

-1

0

1

2

3

4

5

6

7

ISM

R (

dB

)

MADMM BCD

(a)

2 4 8 16 32

Number of RF chains NRF

-3

-2

-1

0

1

2

3

4

5

6

7

ISM

R (

dB

)

RPM-TR

AltMin

(b)

Fig. 8. The ISMR values versus RF chains for different trade-off parameterϕ. (a) Fully-connected structure, (b) partially-connected structure.

ϕ and NRF .

VII. CONCLUSION

In this paper, we have addressed the problems of the hybridbeamforming designs for mmWave DFRC systems with twohybrid beamforming architectures, i.e. the partially-connectedand fully-connected structures. The corresponding problemsare formulated by the trade-off optimization criterion. To dealwith the non-convex formulated problems, we have proposedtwo low-complexity algorithms, concretely, the MADMM forfully-connected and the RPM-TR for the partially-connectedstructure, respectively. The performance of the proposed meth-ods in terms of the achievable rate, ISMR value and beampat-tern behavior are assessed by numerical simulations.

The simulation results reveals that the proposed methodscan achieve the satisfactory radar and communication perfor-mance for the DFRC with both fully-connected and partially-connected structures. Besides, the proposed beamforming de-sign provide a flexible trade-off between the radar and com-munication performance by adjusting a trade-off parameter.

Page 13: Manifold Optimization Methods for Hybrid beamforming in

13

It should be noted that the beampattern performance of thepartially-connected structure is better than that of the fully-connected structure, while the quality of the communicationmay endure a loss.

REFERENCES

[1] Z. Cheng, B. Liao, Z. He, Y. Li, and J. Li, “Spectrally compatiblewaveform design for MIMO radar in the presence of multiple targets,”IEEE Transactions on Signal Processing, vol. 66, no. 13, pp. 3543–3555,2018.

[2] A. Khawar, A. Abdelhadi, and C. Clancy, “Target detection performanceof spectrum sharing MIMO radars,” IEEE Sensors Journal, vol. 15,no. 9, pp. 4928–4940, 2015.

[3] J. A. Mahal, A. Khawar, A. Abdelhadi, and T. C. Clancy, “Spectral coex-istence of MIMO radar and MIMO cellular system,” IEEE Transactionson Aerospace and Electronic Systems, vol. 53, no. 2, pp. 655–668, 2017.

[4] S. Sodagari, A. Khawar, T. C. Clancy, and R. McGwier, “A projectionbased approach for radar and telecommunication systems coexistence,”in 2012 IEEE Global Communications Conference (GLOBECOM),2012, pp. 5010–5014.

[5] A. Aubry, A. De Maio, M. Piezzo, and A. Farina, “Radar waveformdesign in a spectrally crowded environment via nonconvex quadraticoptimization,” IEEE Transactions on Aerospace and Electronic Systems,vol. 50, no. 2, pp. 1138–1152, 2014.

[6] C. Nunn and L. R. Moyer, “Spectrally-compliant waveforms for wide-band radar,” IEEE Aerospace and Electronic Systems Magazine, vol. 27,no. 8, pp. 11–15, 2012.

[7] B. Li, A. P. Petropulu, and W. Trappe, “Optimum co-design for spectrumsharing between matrix completion based MIMO radars and a MIMOcommunication system,” IEEE Transactions on Signal Processing,vol. 64, no. 17, pp. 4562–4575, 2016.

[8] B. Li and A. P. Petropulu, “Joint transmit designs for coexistence ofMIMO wireless communications and sparse sensing radars in clutter,”IEEE Transactions on Aerospace and Electronic Systems, vol. 53, no. 6,pp. 2846–2864, 2017.

[9] L. Zheng, M. Lops, and X. Wang, “Adaptive interference removalfor uncoordinated radar/communication coexistence,” IEEE Journal ofSelected Topics in Signal Processing, vol. 12, no. 1, pp. 45–60, 2018.

[10] F. Liu, C. Masouros, A. Li, T. Ratnarajah, and J. Zhou, “MIMO radar andcellular coexistence: A power-efficient approach enabled by interferenceexploitation,” IEEE Transactions on Signal Processing, vol. 66, no. 14,pp. 3681–3695, 2018.

[11] N. Nartasilpa, A. Salim, D. Tuninetti, and N. Devroye, “Communicationssystem performance and design in the presence of radar interference,”IEEE Transactions on Communications, vol. 66, no. 9, pp. 4170–4185,2018.

[12] C. Sturm and W. Wiesbeck, “Waveform design and signal processingaspects for fusion of wireless communications and radar sensing,”Proceedings of the IEEE, vol. 99, no. 7, pp. 1236–1259, 2011.

[13] J. Euziere, R. Guinvarc’h, M. Lesturgie, B. Uguen, and R. Gillard,“Dual function radar communication time-modulated array,” in 2014International Radar Conference, 2014, pp. 1–4.

[14] A. Hassanien, M. G. Amin, Y. D. Zhang, and F. Ahmad, “A dual functionradar-communications system using sidelobe control and waveformdiversity,” in 2015 IEEE Radar Conference (RadarCon), 2015, pp. 1260–1263.

[15] S. D. Blunt, M. R. Cook, and J. Stiles, “Embedding information intoradar emissions via waveform implementation,” in 2010 InternationalWaveform Diversity and Design Conference, 2010, pp. 000 195–000 199.

[16] C. Sahin, J. Jakabosky, P. M. McCormick, J. G. Metcalf, and S. D. Blunt,“A novel approach for embedding communication symbols into physicalradar waveforms,” in 2017 IEEE Radar Conference (RadarConf), 2017,pp. 1498–1503.

[17] D. Gaglione, C. Clemente, C. V. Ilioudis, A. R. Persico, I. K. Proudler,J. J. Soraghan, and A. Farina, “Waveform design for communicatingradar systems using fractional fourier transform,” Digital SignalProcessing, vol. 80, pp. 57 – 69, 2018. [Online]. Available:http://www.sciencedirect.com/science/article/pii/S1051200418302070

[18] F. Liu, L. Zhou, C. Masouros, A. Li, W. Luo, and A. Petropulu, “To-ward dual-functional radar-communication systems: Optimal waveformdesign,” IEEE Transactions on Signal Processing, vol. 66, no. 16, pp.4264–4279, 2018.

[19] P. Kumari, J. Choi, N. Gonzalez-Prelcic, and R. W. Heath, “Ieee 802.11ad-based radar: An approach to joint vehicular communication-radarsystem,” IEEE Transactions on Vehicular Technology, vol. 67, no. 4,pp. 3012–3027, 2017.

[20] K. V. Mishra, M. B. Shankar, V. Koivunen, B. Ottersten, and S. A.Vorobyov, “Toward millimeter-wave joint radar communications: A sig-nal processing perspective,” IEEE Signal Processing Magazine, vol. 36,no. 5, pp. 100–114, 2019.

[21] Xinying Zhang, A. F. Molisch, and Sun-Yuan Kung, “Variable-phase-shift-based rf-baseband codesign for MIMO antenna selection,” IEEETransactions on Signal Processing, vol. 53, no. 11, pp. 4091–4103, 2005.

[22] O. E. Ayach, S. Rajagopal, S. Abu-Surra, Z. Pi, and R. W. Heath,“Spatially sparse precoding in millimeter wave MIMO systems,” IEEETransactions on Wireless Communications, vol. 13, no. 3, pp. 1499–1513, 2014.

[23] A. Alkhateeb and R. W. Heath, “Frequency selective hybrid precodingfor limited feedback millimeter wave systems,” IEEE Transactions onCommunications, vol. 64, no. 5, pp. 1801–1818, 2016.

[24] A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Hybridprecoding for millimeter wave cellular systems with partial channelknowledge,” in 2013 Information Theory and Applications Workshop(ITA), 2013, pp. 1–5.

[25] X. Yu, J. Shen, J. Zhang, and K. B. Letaief, “Alternating minimizationalgorithms for hybrid precoding in millimeter wave MIMO systems,”IEEE Journal of Selected Topics in Signal Processing, vol. 10, no. 3,pp. 485–500, 2016.

[26] S. Han, C. I, Z. Xu, and C. Rowell, “Large-scale antenna systems withhybrid analog and digital beamforming for millimeter wave 5g,” IEEECommunications Magazine, vol. 53, no. 1, pp. 186–194, 2015.

[27] X. Gao, L. Dai, S. Han, C. I, and R. W. Heath, “Energy-efficient hybridanalog and digital precoding for mmwave MIMO systems with largeantenna arrays,” IEEE Journal on Selected Areas in Communications,vol. 34, no. 4, pp. 998–1009, 2016.

[28] S. Park, A. Alkhateeb, and R. W. Heath, “Dynamic subarrays for hybridprecoding in wideband mmwave MIMO systems,” IEEE Transactionson Wireless Communications, vol. 16, no. 5, pp. 2907–2920, 2017.

[29] S. Han, C. I, Z. Xu, and C. Rowell, “Large-scale antenna systems withhybrid analog and digital beamforming for millimeter wave 5g,” IEEECommunications Magazine, vol. 53, no. 1, pp. 186–194, 2015.

[30] F. Rusek, D. Persson, B. K. Lau, E. G. Larsson, T. L. Marzetta,O. Edfors, and F. Tufvesson, “Scaling up MIMO: Opportunities andchallenges with very large arrays,” IEEE Signal Processing Magazine,vol. 30, no. 1, pp. 40–60, 2013.

[31] A. Li and C. Masouros, “Hybrid analog-digital millimeter-wave mu-mimo transmission with virtual path selection,” IEEE CommunicationsLetters, vol. 21, no. 2, pp. 438–441, 2016.

[32] F. Liu and C. Masouros, “Hybrid beamforming with sub-arrayed MIMOradar: Enabling joint sensing and communication at mmwave band,”in ICASSP 2019 - 2019 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP), 2019, pp. 7770–7774.

[33] Z. Cheng, J. He, S. Shi, Z. He, and B. Liao, “Hybrid beamforming forwideband ofdm dual function radar communications,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and SignalProcessing (ICASSP). IEEE, 2021, pp. 8238–8242.

[34] Z. Cheng, Z. He, and B. Liao, “Hybrid beamforming for multi-carrierdual-function radar-communication system,” IEEE Transactions on Cog-nitive Communications and Networking, pp. 1–1, 2021.

[35] F. Liu, C. Masouros, A. P. Petropulu, H. Griffiths, and L. Hanzo, “Jointradar and communication design: Applications, state-of-the-art, and theroad ahead,” IEEE Transactions on Communications, vol. 68, no. 6, pp.3834–3862, 2020.

[36] P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization algorithms onmatrix manifolds. Princeton University Press, 2009.

[37] Z. Cheng, B. Liao, Z. He, J. Li, and J. Xie, “Joint design of the transmitand receive beamforming in MIMO radar systems,” IEEE Transactionson Vehicular Technology, vol. 68, no. 8, pp. 7919–7930, 2019.

[38] H. Xu, R. S. Blum, J. Wang, and J. Yuan, “Colocated MIMO radar wave-form design for transmit beampattern formation,” IEEE Transactions onAerospace and Electronic Systems, vol. 51, no. 2, pp. 1558–1568, 2015.

[39] N. Boumal, “An introduction to optimization on smooth manifolds,”Available online, May, 2020.

[40] C. Udriste, Convex functions and optimization methods on Riemannianmanifolds. Springer Science & Business Media, 2013, vol. 297.

[41] S. Boyd, N. Parikh, and E. Chu, Distributed optimization and statisticallearning via the alternating direction method of multipliers. NowPublishers Inc, 2011.

[42] J. Nocedal and S. Wright, Numerical optimization. Springer Science& Business Media, 2006.

Page 14: Manifold Optimization Methods for Hybrid beamforming in

14

[43] J. Li, G. Liao, Y. Huang, Z. Zhang, and A. Nehorai, “Riemanniangeometric optimization methods for joint design of transmit sequenceand receive filter on MIMO radar,” IEEE Transactions on SignalProcessing, vol. 68, pp. 5602–5616, 2020.

[44] P.-A. Absil, R. Mahony, and J. Trumpf, “An extrinsic look at theriemannian hessian,” in International conference on geometric scienceof information. Springer, 2013, pp. 361–368.

[45] A. R. Conn, N. I. Gould, and P. L. Toint, Trust region methods. SIAM,

2000.[46] P. Richtarik and M. Takac, “Distributed coordinate descent method for

learning with big data,” The Journal of Machine Learning Research,vol. 17, no. 1, pp. 2657–2681, 2016.

[47] Z.-Q. Luo and P. Tseng, “On the convergence of the coordinate descentmethod for convex differentiable minimization,” Journal of OptimizationTheory and Applications, vol. 72, no. 1, pp. 7–35, 1992.