andrea goldsmith fundamental communication limits in non-asymptotic regimes thanks to collaborators...
TRANSCRIPT
Backing off from infinity:
Andrea Goldsmith
fundamental communication limits in non-asymptotic regimes
Thanks to collaborators Chen, Eldar, Grover, Mirghaderi, Weissman
Information Theory and AsymptopiaCapacity with asymptotically small error achieved by
asymptotically long codes.Defining capacity in terms of asymptotically small error
and infinite delay is brilliant!Has also been limitingCause of unconsummated union between networks and
information theory
Optimal compression based on properties of asymptotically long sequencesLeads to optimality of separation
Other forms of asymptopiaInfinite SNR, energy, sampling, precision, feedback, …
Why back off?
Theory not informing practice
Theory vs. practice
Theory PracticeInfinite blocklength codesInfinite SNRInfinite energyInfinite feedbackInfinite sampling rates
Infinite (free) processingInfinite precision ADCs
Uncoded to LDPC-7dB in LTEFinite battery life1 bit ARQ50-500 Msps 200 MFLOPs-1B FLOPs8-16 bits
What else lives in asymptopia?
Backing off from: infinite blocklengthRecent developments on finite blocklength
Channel codes (Capacity C for n)Source codes (entropy H or rate distortion R(D))
[Ingber, Kochman’11; Kostina, Verdu’11]
Separation not Optimal Separation not Optimal[Wang et. Al’11; Kostina, Verdu’12]
Grand Challenges Workshop: CTW MauiFrom the perspective of the cellular industry, the Shannon bounds
evaluated by Slepian are within .5 dB for a packet size of 30 bits or more for the real AWGN channel at 0.5 bits/sym, for BLER = 1e-4. In this perhaps narrow context there is not much uncertainty for performance evaluations.
For cellular and general wireless channels, finite blocklength bounds for practical fading models are needed and there is very little work along those lines.
Even for the AWGN channel the computational effort of evaluating the Shannon bounds is formidable.
This indicates a need for accurate approximations, such as those recently developed based on the idea of channel dispersion.
Diversity vs. Multiplexing TradeoffUse antennas for multiplexing or diversity
Diversity/Multiplexing tradeoffs (Zheng/Tse)Error Prone Low Pe
r)r)(N(N(r)d rt*
rSNRlog
R(SNR)lim
SNR
dSNRlog
P log e
)(lim
SNRSNR
Whatis
Infinite?
Backing off from: infinite SNRHigh SNR Myth: Use some spatial dimensions for
multiplexing and others for diversity
*Transmit Diversity vs. Spatial Multiplexing in Modern MIMO Systems”, Lozano/Jindal
Reality: Use all spatial dimensions for one or the other*Diversity is wasteful of spatial dimensions with HARQAdapt modulation/coding to channel SNR
Diversity-Multiplexing-ARQ TradeoffSuppose we allow ARQ with incremental
redundancy
ARQ is a form of diversity [Caire/El Gamal 2005]
0
2
4
6
8
10
12
14
16
0 1 2 3 4
ARQ Window
Size L=1
L=2 L=3
L=4
d
r
Joint Source/Channel CodingUse antennas for multiplexing:
Use antennas for diversity High-RateQuantizer
ST CodeHigh Rate Decoder
Error Prone
Low Pe
Low-RateQuantizer
ST CodeHigh
DiversityDecoder
How should antennas be used: Depends on end-to-end metric
Joint Source-Channel coding w/MIMO
kRu Index
Assignment
s bits
p(i)Channel Encoder
s bits
i
MIMO Channel
Channel Decoder
Inverse Index Assignment p(j)
s bits
j
s bits
Increased rate heredecreases source
distortionBut permits
less diversity here
Resulting in more errors
SourceEncoder
SourceDecoder
And maybe higher total distortion
A joint design is needed
vj
Antenna Assignment vs. SNR
Relaying in wireless networks
Intermediate nodes (relays) in a route help to forward the packet to its final destination.
Decode-and-forward (store-and-forward) most common:Packet decoded, then re-encoded for transmissionRemoves noise at the expense of complexity
Amplify-and-forward: relay just amplifies received packetAlso amplifies noise: works poorly for long routes; low SNR.
Compress-and-forward: relay compresses received packetUsed when Source-relay link good, relay-destination link weak
SourceRelay Destination
Capacity of the relay channel unknown: only have bounds
Cooperation in Wireless Networks
Relaying is a simple form of cooperationMany more complex ways to cooperate:
Virtual MIMO , generalized relaying, interference forwarding, and one-shot/iterative conferencing
Many theoretical and practice issues: Overhead, forming groups, dynamics, full-duplex,
synch, …
Generalized Relaying and Interference Forwarding
Can forward message and/or interference Relay can forward all or part of the messages
Much room for innovation Relay can forward interference
To help subtract it out
TX1
TX2
relay
RX2
RX1X1
X2
Y3=X1+X2+Z3
Y4=X1+X2+X3+Z4
Y5=X1+X2+X3+Z5
X3= f(Y3) Analog network coding
Beneficial to forward bothinterference and message
In fact, it can achieve capacity
S DPs
P1
P2
P3
P4
• For large powers Ps, P1, P2, …, analog network coding (AF) approaches capacity
: Asymptopia?
Maric/Goldsmith’12
Interference AlignmentAddresses the number of interference-free signaling
dimensions in an interference channel
Based on our orthogonal analysis earlier, it would appear that resources need to be divided evenly, so only 2BT/N dimensions available
Jafar and Cadambe showed that by aligning interference, 2BT/2 dimensions are available
Everyone gets half the cake!
Except at finite SNRs
Backing off from: infinite SNRHigh SNR Myth: Decode-and-forward equivalent to amplify-
forward, which is optimal at high SNR*Noise amplification drawback of AF diminishes at high SNRAmplify-forward achieves full degrees of freedom in MIMO systems
(Borade/Zheng/Gallager’07)At high-SNR, Amplify-forward is within a constant gap from the capacity upper
bound as the received powers increase (Maric/Goldsmith’07)
Reality: optimal relaying unknown at most SNRs:Amplify-forward highly suboptimal outside high SNR per-node regime, which is
not always the high power or high channel gain regimeAmplify-forward has unbounded gap from capacity in the high channel gain
regime (Avestimehr/Diggavi/Tse’11)
Relay strategy should depend on the worst link
Decode-forward used in practice
Capacity and FeedbackCapacity under feedback largely unknown
Channels with memoryFinite rate and/or noisy feedbackMultiuser channels Multihop networks
ARQ is ubiquitious in practiceWorks well on finite-rate noisy feedback channelsReduces end-to-end delay
Why hasn’t theory met practice when it comes to feedback?
PtP Memoryless Channels: Perfect Feedback
• Shannon• Feedback does not increase capacity of DMCs
• Schalkwijk-Kailath Scheme for AWGN channels–Low-complexity linear recursive scheme –Achieves capacity–Double exponential decay in error probability
Encoder DecoderWW WW
+
Backing off from: Perfect Feedback
+Channel Encoder Decoder
Feedback Module
• [Shannon 59]: No Feedback
• [Pinsker, Gallager et al.]: Perfect feedback • Infinite rate/no noise
• [Kim et. al. 07/10]: Feedback with AWGN
• [Polyaskiy et. al. 10]: Noiseless feedback reduces the minimum energy per bit when nR is fixed and n
• Objective:
Choose and to maximize the decay rate of error probability
Gaussian Channel with Rate-Limited Feedback
+Channel Encoder Decoder
Feedback Module
• Constraints
Feedback is rate-limited ; no noise
A super-exponential error probability is achievable if and only if
• : The error exponent is finite but higher than no-feedback error exponent
• : Double exponential error probability
• : L-fold exponential error probability
m-bit Encoder
m-bit Decoder
m-bit Encoder
m-bit Decoder
Forward Channel
Feedback
Channel
If , sendTermination Alarm
Otherwise, resend with energy
Send back with energy
If Termination Alarm is received, report as the decoded message
Feedback under Energy/Delay Constraint
• Constraints Objective: Choose to
minimize the overall probability of error
Depends on the error probability model ε()
• Exponential Error Model: ε(x)=βe-αx
Applicable when Tx energy dominatesFeedback gain is high if total energy is large
enoughNo feedback gain for energy budgets below
a threshold
Feedback Gain under Energy/Delay Constraint
• Super-Exponential Error Model: ε(x)=βe-αx2
- Applicable when Tx and coding energy are comparable- No feedback gain for energy budgets above a threshold
Backing off from: perfect feedback • Memoryless point-to-point channels:
• Capacity unchanged with perfect feedback• Simple linear scheme reduces error exponent
(Schalkwijk-Kailath: double exponential)• Feedback reduces energy consumption
• Capacity of feedback channels largely unknown• Unknown for general channels with memory and perfect feedback• Unknown under finite rate and/or noisy feedback• Unknown in general for multiuser channels • Unknown in general for multihop networks
• ARQ is ubiquitious in practice• Assumes channel errors• Works well on finite-rate noisy feedback channels• Reduces end-to-end delay
No feedback
Feedback
Output feedbackChannel information (CSI)AcknowledgementsSomething else?
Noisy/Compressed
How to use feedback in wireless networks?
Interesting applications to neuroscience
For a given sampling mechanism (i.e. a “new” channel)What is the optimal input signal?What is the tradeoff between capacity and sampling rate?What known sampling methods lead to highest capacity?
What is the optimal sampling mechanism? Among all possible (known and unknown) sampling schemes
h(t)
SamplingMechanism
(rate fs)
New Channel
Backing off from: infinite sampling
Capacity under Sampling w/Prefilter
Theorem: Channel capacity
h(t)
)(th
)(t
)(tx )(ts
snTt
][ny
“Folded” SNR filtered by S(f) Determined by
waterfilling:suppresses aliasing
Capacity not monotonic in fsConsider a “sparse” channel
Capacity not monotonic in fs!
Single-branchsampling fails to exploit channel structure
Filter Bank Sampling
Theorem: Capacity of the sampled channel using a bank of m filters with aggregate rate fs
h(t)
)(th
)(t
)(tx
)(1 ts
)(tsi
)(tsm
)( smTnt
)( smTnt
)( smTnt
][1 ny
][nyi
][nym
Similar to MIMO; no combining!
Equivalent MIMO Channel Model
h(t)
)(th
)(t
)(tx
)(1 ts
)(tsi
)(tsm
)( smTnt
)( smTnt
)( smTnt
][1 ny
][nyi
][nym
( fX
( skffX
( skffX
)( skffH
)( fH
)( skffH
)( skffN
)( fN
)( skffN
( fYi
( fY1
( fYm
( fS1
( sm kffS
( fSi
( si kffS
( skffS 1
( fSm
( sm kffS
( si kffS
( skffS 1
Theorem 3: The channel capacity of the sampled channel using a bank of m filters with aggregate rate is
For each f
Water-filling over singularvalues
MIMO – DecouplingPre-whitening
Selects the m branches with m highest SNRExample (Bank of 2 branches)
highest SNR
2nd highest SNR
low SNR
( skffX 2
( fX
( skffX
( skffX
)( skffH
)( fH
)( skffH
)( skffN
)( fN
)( skffN
)( skffS
)( fS
)( skffS
)2( skffH
)2( skffN )2( skffS
Joint Optimization of Input and Filter Bank
low SNR
( fY1
( fY2
Capacity monotonic in fs
Can we do better?
Sampling with Modulator+Filter (1 or more)
h(t)
)(th
)(t
)(ts ][nyTheorem:
Bank of Modulator+FilterSingle Branch Filter Bank
Theorem
Optimal among all time-preserving nonuniform sampling techniques of rate fs
zzzzzzzzzz
)(ts ][nyzzzzzzzzzz
)(1 ts
)(tsi
)(tsm
)( smTnt
)( smTnt
)( smTnt
][1 ny
][nyi
][nym
equals
Backing off from: Infinite processing power
Is Shannon-capacity still a good metric for system design?
Our approach
Power consumption via a network graphpower consumed in nodes and wires
Extends early work of El Gamal et. al.’84 and Thompson’80
Fundamental area-time-performance tradeoffs
For encoding/decoding “good” codes,
Stay away from capacity!Close to capacity we have
Large chip-areaMore timeMore power
Area occupied by wires Encoding/decoding clock cycles
Total power diverges to infinity!
Regular LDPCs closer to bound than capacity-approaching LDPCs!Need novel code designs with short wires, good performance
ConclusionsInformation theory asympotia has provided much insight and
decades of sublime delight to researchers
Backing off from infinity required for some problems to gain insight and fundamental bounds
New mathematical tools and new ways of applying conventional tools needed for these problems
Many interesting applications in finance, biology, neuroscience, …