an fpga based phased array processor for the sub ...jweintro/vlbi/forbob/vinayaks graphics... · an...
TRANSCRIPT
An FPGA based Phased ArrayProcessor for the Sub Millimeter
ArrayVinayak Nagpal
Chalmers University of Technology, Sweden
advised by
Jonathan Weintroub
Smithsonian Astrophysical Observatory
Vinayak Nagpal, CTH – p. 1/28
Outline
Introduction: What we plan to build and why?
Vinayak Nagpal, CTH – p. 2/28
Outline
Introduction: What we plan to build and why?
Infrastructure: What did we have in hand?
Vinayak Nagpal, CTH – p. 2/28
Outline
Introduction: What we plan to build and why?
Infrastructure: What did we have in hand?
Design: How we went around building it?
Vinayak Nagpal, CTH – p. 2/28
Outline
Introduction: What we plan to build and why?
Infrastructure: What did we have in hand?
Design: How we went around building it?
Results: Does it work?
Vinayak Nagpal, CTH – p. 2/28
Outline
Introduction: What we plan to build and why?
Infrastructure: What did we have in hand?
Design: How we went around building it?
Results: Does it work?
Future: Where to go from here?
Vinayak Nagpal, CTH – p. 2/28
Outline
Introduction: What we plan to build and why?
Infrastructure: What did we have in hand?
Design: How we went around building it?
Results: Does it work?
Future: Where to go from here?
Demonstration in M247 for those who want tosee it work and discuss in more detail.
Vinayak Nagpal, CTH – p. 2/28
Motivation
(VLBI) at 0.8 mm with sufficiently long baselines →
≈ 20 µas resolution.
In sub millimeter electron scattering is reduced.
Imaging observation of the event horizon in a blackhole e.g. SgrA∗, M87.
Sub-millimeter telescopes: JCMT, CSO, HHT, (ALMA)etc and SMA.
SMA full collecting area + JCMT + CSO → we need aphased array with interface to standard VLBIrecorders, i.e. Mark V.
Vinayak Nagpal, CTH – p. 3/28
Project Objectives
Proof of Concept
Development time → 10 months.
INPUT: 8 Antennas (SMA or JCMT/CSO),Single Polarization, 500 MHz Bandwidth.
OUTPUT: Real Time Phased Sum of 8antennas spooled to Mark Vb VLBI datastorage unit.
Build a scalable system using state of artsampling and FPGA technology.
Vinayak Nagpal, CTH – p. 4/28
Why FPGAs?
Field Programmable Gate Arrays → Like aPLA but can do much more!
FPGA ASICShort development cy-cle.
Long cycles
High cost. Low cost for large vol-umes.
Reconfigurable Use or throw.Design can be up-graded
Designs carved in sili-con.
Vinayak Nagpal, CTH – p. 5/28
Partners
(1) CASPER team at UC Berkeley led by DanWerthimer building FPGA based radioastronomy signal processing technology.
Vinayak Nagpal, CTH – p. 6/28
Partners
(1) CASPER team at UC Berkeley led by DanWerthimer building FPGA based radioastronomy signal processing technology.
(2) MIT/Haystack using (1) to build DigitalBack End (DBE) for Mark Vb VLBI storageequipment.
Vinayak Nagpal, CTH – p. 6/28
Partners
(1) CASPER team at UC Berkeley led by DanWerthimer building FPGA based radioastronomy signal processing technology.
(2) MIT/Haystack using (1) to build DigitalBack End (DBE) for Mark Vb VLBI storageequipment.
Let us use (1) combine with (2), customizeand build the SMA beam former.
Vinayak Nagpal, CTH – p. 6/28
CASPER Technology
iBOB Board based onXilinx Virtex II Pro.Heart of SMA PhasedArray Processor
Vinayak Nagpal, CTH – p. 7/28
CASPER Technology
iBOB Board based onXilinx Virtex II Pro.Heart of SMA PhasedArray Processor
iADC boards plugdirectly into iBOBs andprovide high speedsampling.
Vinayak Nagpal, CTH – p. 7/28
CASPER Technology
iBOB Board based onXilinx Virtex II Pro.Heart of SMA PhasedArray Processor
iADC boards plugdirectly into iBOBs andprovide high speedsampling.
Other boards which wedidn’t need → BEE2CASPER flagship.
Vinayak Nagpal, CTH – p. 7/28
iBOB-iADC
Atmel ADC
(AT84AD001),
2Gsamples/sec or
1Gsample/sec
Vinayak Nagpal, CTH – p. 8/28
iBOB-iADC
Atmel ADC
(AT84AD001),
2Gsamples/sec or
1Gsample/sec
Xilinx Virtex II
Pro. 5MB SRAM,
≈ 50K logic cells,
2 PPC, Rocket
I/O
Vinayak Nagpal, CTH – p. 8/28
iBOB-iADC
Infiniband, VSI,
RS-232, 100BaseT
Ethernet, External
SRAM.
Vinayak Nagpal, CTH – p. 8/28
iBOB-iADC
Infiniband, VSI,
RS-232,
100BaseT
Ethernet, External
SRAM.
Symbolic Repre-
sentation.
Vinayak Nagpal, CTH – p. 8/28
BEE Design Flow
VHDL → Low
Level
Vinayak Nagpal, CTH – p. 9/28
BEE Design Flow
VHDL → Low
Level
Simulink → Higher
Level
Vinayak Nagpal, CTH – p. 9/28
BEE Design Flow
VHDL → Low
Level
Simulink → Higher
Level
Fixed Platform →
Details hidden in
design flow
Vinayak Nagpal, CTH – p. 9/28
BEE Design Flow
VHDL → Low
Level
Simulink → Higher
Level
Fixed Platform →
Details hidden in
design flow
Berkeley Libraries:
Interface, Radio
Astronomy
Vinayak Nagpal, CTH – p. 9/28
Mark Vb DBE
ADC 4x8bits
1024Mhz
8bits
256MHz
FourierTransform32 point
Gain Adjustper bin
VSIBusInterface
Mark Vb Recorder
Infiniband Link
iADC iBOB
Time Domain Frequency Domain
Analog InFromDownconverter
iBOB Based Mark Vb Re cording Interface
VLBI Station 1
VSI
MarkVb
Vinayak Nagpal, CTH – p. 10/28
Single Baseline Phased Array
τ
τ
τ
φ
f
Delay adjust.(Geo,Atm,Inst)
Vinayak Nagpal, CTH – p. 11/28
Single Baseline Phased Array
τ
τ
τ
φ
φ
f
f
MixMixLO LO
φ1φ2
φ1 − φ2
Delay adjust.(Geo,Atm,Inst)
Delay andphase adjust.
Vinayak Nagpal, CTH – p. 11/28
Single Baseline Phased Array
τ
τ
τ
φ
φ
f
f
MixMixLO LO
φ1φ2
φ1 − φ2
Delay adjust.(Geo,Atm,Inst)
Delay andphase adjust.
Delay andphase adjustwith fringerotation.
Vinayak Nagpal, CTH – p. 11/28
IF Subsystem
0.5 GHz 1.5 GHz1 GHz
SMA 1st DCV Block Filters (MHz)
528768 1008
10401280 1520
1024 MHz1024 MHzMM
MM
ff
ff
Vinayak Nagpal, CTH – p. 12/28
Phased Array Processor
M5 Recorder
DBE
VSI
VLBI IniBOB-1
iBOB-2
Ant 1
Ant 2
Ant 3
Ant 4
Ant 5
Ant 6
Ant 7
Ant 8
≈ 8 Gbps per antenna
Vinayak Nagpal, CTH – p. 13/28
Phased Array Processor
M5 Recorder
DBE
VSI
VLBI IniBOB-1
iBOB-2
Ant 1
Ant 2
Ant 3
Ant 4
Ant 5
Ant 6
Ant 7
Ant 8
8 Gbps
8 Gbps
≈ 8 Gbps per antenna
Vinayak Nagpal, CTH – p. 13/28
Time Domain Approach
Delay τ1
Delay τ2
Delay τ3
Delay τn
Simple
Accuracy:τmin <<Tsample
Cannot adjustphase
Vinayak Nagpal, CTH – p. 14/28
Frequency Domain Approach
FFT N
FFT N
FFT N
FFT N
φ adjust
φ adjust
φ adjust
φ adjust
Complex
Accuracy:
N → Large
Can adjust phase,
hence LO phase
compensate and
fringe rotation can
be done digitally.
Vinayak Nagpal, CTH – p. 15/28
Phased Array Processor
4x
Average
Average
Average
Average
Chunk FilterBW=480MHzCenter=760MHz
or Center=1280MHz
Chunk FilterBW=480MHzCenter=760MHz
or Center=1280MHz
Chunk FilterBW=480MHzCenter=760MHz
or Center=1280MHz
Chunk FilterBW=480MHzCenter=760MHz
or Center=1280MHz
Chunk FilterBW=480MHzCenter=760MHz
or Center=1280MHz
4x
4x
4x
4x1024Mhz
8bits
1024Mhz
8bits
1024Mhz
8bits
1024Mhz
8bits Digital DelayLineAccuracy 0.1ns
XAUI
Xilinx Virtex II Pro vp50iADC 2
iADC 1
Antenna 1Polarization P
Polarization P
Polarization P
Polarization P
Antenna 2
Antenna 3
Antenna 4
BPF
BPF
BPF
BPF
ADC
ADC
ADC
ADC
Demux by 4
Demux by 4
Demux by 4
Demux by 4
8bits
256MHz
8bits
256MHz
8bits
256MHz
8bits
256MHz
Digital DelayLineAccuracy 0.1ns
Digital DelayLineAccuracy 0.1ns
Digital DelayLineAccuracy 0.1ns
PPC
Delay Control
Delay Control
Delay Control
Delay Control
8bits
8bits
8bits
8bits
8bits
8bits
8bits
8bits
256MHz
256MHz
256MHz
256MHz
256MHz
256MHz
256MHz
256MHz
10bits
10bits
10bits
10bits
8bits
1024Mhz
To Infinibandconnector
Ts = 11024MHz
≈ 0.99ns
τmin = Ts
10
≈ 0.1ns
Vinayak Nagpal, CTH – p. 16/28
Coarse Delay
8bits
CONCAT32bits
FIFO
RD_PTR
WR_PTR
LOGICDELAY
DATA
Max Delay4000 ns
Delay Precision4 ns
Control via PPC.
Delay ↑. 4samples repeat.
Delay ↓. 4 sam-ples skipped.
Vinayak Nagpal, CTH – p. 17/28
Fine Delay
9101112
32bits
1234
5678
CONCAT
64bit
12345678
1234
2345
3456
4567
01
23DATA
SELECT
z−1
z−1
Barrel SelectorArrangement
Delay Precision1 ns
Vinayak Nagpal, CTH – p. 18/28
Super Fine Delay
1 2 3 4 5 6 7 8 9 10−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Sample
Mag
nitu
de
Digital Filter for Delay
y(n) = x(n − D)
Hi(z) = z( − D)
hD(n) =sin π(n−D)
π(n−D)
If D is fractional hD(n) becomes afractional delay filter.
FIR approximations of Fractional Delayfilters are not symmetrical.
Precompute Coefficients for fractionaldelays from 0.1 ns to 0.9 ns in steps of0.1 ns.
Load coefficients on demand and imple-ment real time FIR filter using 10 taps.
Vinayak Nagpal, CTH – p. 19/28
Super Fine Delay
1 2 3 4 5 6 7 8 9 10−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
Sample
Mag
nitu
de
Digital Filter for Delay
y(n) = x(n − D)
Hi(z) = z( − D)
hD(n) =sin π(n−D)
π(n−D)
If D is fractional hD(n) becomes afractional delay filter.
FIR approximations of Fractional Delayfilters are not symmetrical.
Precompute Coefficients for fractionaldelays from 0.1 ns to 0.9 ns in steps of0.1 ns.
Load coefficients on demand and imple-ment real time FIR filter using 10 taps.
Vinayak Nagpal, CTH – p. 19/28
Super Fine Delay
DATA IN
DATA OUT
C1 C2 C3 C4 C5
z−1z−1z−1z−1z−1
Complex designof demux-by-4FIR filter
4 Multiplicationsand 4 partialsums computedin every stage
No. of stages =
No. of Taps
Vinayak Nagpal, CTH – p. 20/28
Super Fine Delay
C1
C1
C1
C1
C1
C2
C2
C2
C2
C2
C3
C3
C3
C3
C3
C4
C4
C4
C4
C4
C5
C5
C5
C5
C5
s1s1
s2s2s2
s3 s3s3s3
s4s4 s4s4s4
s5s5s5s5s5
s6s6 s6s6
s7s7s7
s8s8
s9
s10
s11
s12
Complex designof demux-by-4FIR filter
4 Multiplicationsand 4 partialsums computedin every stage
No. of stages =
No. of Taps
Vinayak Nagpal, CTH – p. 20/28
It Works too!
0 10 20 30 40 50 60 70 80 90 100
−20
−10
0
10
20
Sample
Mag
Channel 0
0 10 20 30 40 50 60 70 80 90 100−30
−20
−10
0
10
20
Sample
Mag
Channel 1
0 10 20 30 40 50 60 70 80 90 100−30
−20
−10
0
10
20
Mag
Sam
ple
Average
Vinayak Nagpal, CTH – p. 21/28
It Works too!
−50 −40 −30 −20 −10 0 10 20 30 40 50
−1
−0.5
0
0.5
1
x 106
Lag
Cor
rela
tion
Correlation
Vinayak Nagpal, CTH – p. 21/28
It Works too!
0 10 20 30 40 50 60 70 80 90 100
−20
0
20
Channel 0
Sample
Mag
0 10 20 30 40 50 60 70 80 90 100
−20
0
20
Channel 1
Sample
Mag
0 10 20 30 40 50 60 70 80 90 100
−20
0
20
Average
Sample
Mag
Vinayak Nagpal, CTH – p. 21/28
It Works too!
−140 −120 −100 −80 −60 −40 −20 0 20
−1
−0.8
−0.6
−0.4
−0.2
0
0.2
0.4
0.6
0.8
1
x 106
Lags
Cor
rela
tion
Correlation
Vinayak Nagpal, CTH – p. 21/28
It Works too!
−60 −58 −56 −54 −52 −50
5
6
7
8
9
10
11
x 105
Lags
Cor
rela
tion
Correlation
Vinayak Nagpal, CTH – p. 21/28
How to get delays?
Geometry → knownwell enough.
Vinayak Nagpal, CTH – p. 22/28
How to get delays?
Geometry → knownwell enough.
Atmosphere → needto track
Vinayak Nagpal, CTH – p. 22/28
How to get delays?
Geometry → knownwell enough.
Atmosphere → needto track
Noisy Data.
Vinayak Nagpal, CTH – p. 22/28
How to get delays?
Geometry → knownwell enough.
Atmosphere → needto track
Noisy Data.
SMA Correlator?
Vinayak Nagpal, CTH – p. 22/28
How to get delays?
SMACORRELATOR
SMAPHASEDARRAY PROCESSOR
ANT 1ANT 2ANT 3ANT 4
τ1τ2τ3τ4
τ5τ6τ7τ8
Geometry → knownwell enough.
Atmosphere → needto track
Noisy Data.
SMA Correlator?
Instrumental →different paths.
Vinayak Nagpal, CTH – p. 22/28
How to get delays?
SMACORRELATOR
SMAPHASEDARRAY PROCESSOR
ANT 1ANT 2ANT 3ANT 4
τ1τ2τ3τ4
τ5τ6τ7τ8
Geometry → knownwell enough.
Atmosphere → needto track
Noisy Data.
SMA Correlator?
Instrumental →different paths.
Another Correlator?
Vinayak Nagpal, CTH – p. 22/28
What sort of Correlator?
7 Delay measurements.
Vinayak Nagpal, CTH – p. 23/28
What sort of Correlator?
7 Delay measurements.
δτ is small! Order of few minutes.
Vinayak Nagpal, CTH – p. 23/28
What sort of Correlator?
7 Delay measurements.
δτ is small! Order of few minutes.
Time Multiplex 7 Measurements.
Vinayak Nagpal, CTH – p. 23/28
What sort of Correlator?
7 Delay measurements.
δτ is small! Order of few minutes.
Time Multiplex 7 Measurements.
Single Baseline Correlator!
Vinayak Nagpal, CTH – p. 23/28
What sort of Correlator?
7 Delay measurements.
δτ is small! Order of few minutes.
Time Multiplex 7 Measurements.
Single Baseline Correlator!
Berkeley Library.
Vinayak Nagpal, CTH – p. 23/28
Correlator
PFB
PFB FFT
FFT
CONJ
X Avg
DATA_IN
DATA_IN
RS_232
DATA_OUT
FX.
Vinayak Nagpal, CTH – p. 24/28
Correlator
PFB
PFB FFT
FFT
CONJ
X Avg
DATA_IN
DATA_IN
RS_232
DATA_OUT
FX.
64 Complex/32Real Channel
Vinayak Nagpal, CTH – p. 24/28
Correlator
PFB
PFB FFT
FFT
CONJ
X Avg
DATA_IN
DATA_IN
RS_232
DATA_OUT
FX.
64 Complex/32Real Channel
Use PFB-FFTfrom CASPER.
Vinayak Nagpal, CTH – p. 24/28
Correlator
PFB
PFB FFT
FFT
CONJ
X Avg
DATA_IN
DATA_IN
RS_232
DATA_OUT
FX.
64 Complex/32Real Channel
Use PFB-FFTfrom CASPER.
Lots of challenges→ lots of time.
Vinayak Nagpal, CTH – p. 24/28
Correlator
PFB
PFB FFT
FFT
CONJ
X Avg
DATA_IN
DATA_IN
RS_232
DATA_OUT
FX.
64 Complex/32Real Channel
Use PFB-FFTfrom CASPER.
Lots of challenges→ lots of time.
Full frequencyoperation.
Vinayak Nagpal, CTH – p. 24/28
Correlator
PFB
PFB FFT
FFT
CONJ
X Avg
DATA_IN
DATA_IN
RS_232
DATA_OUT
FX.
64 Complex/32Real Channel
Use PFB-FFTfrom CASPER.
Lots of challenges→ lots of time.
Full frequencyoperation.
Dynamic Range.
Vinayak Nagpal, CTH – p. 24/28
Correlator
PFB
PFB FFT
FFT
CONJ
X Avg
DATA_IN
DATA_IN
RS_232
DATA_OUT
FX.
64 Complex/32Real Channel
Use PFB-FFTfrom CASPER.
Lots of challenges→ lots of time.
Full frequencyoperation.
Dynamic Range.
Sensitivity
Vinayak Nagpal, CTH – p. 24/28
Does it work?
Figure 1: Autocorrelation Vinayak Nagpal, CTH – p. 25/28
Does it work?
Figure 2: SNR = − 2dB Vinayak Nagpal, CTH – p. 25/28
Does it work?
Figure 3: SNR = − 9dB
Vinayak Nagpal, CTH – p. 25/28
Does it work?
Figure 4: SNR = − 12dB
Vinayak Nagpal, CTH – p. 25/28
Does it work?
Figure 5: SNR = − 15dB
Vinayak Nagpal, CTH – p. 25/28
Does it work?
−500 −400 −300 −200 −100 0 100 200 300 400 5000
10
20
30
40
50
60
70
MHz
dB
−500 −400 −300 −200 −100 0 100 200 300 400 500−3
−2
−1
0
1
2
3
MHz
Pha
se
Figure 6: No Delay Vinayak Nagpal, CTH – p. 25/28
Complete Picture
M5 Recorder
DBE
VSI
VLBI IniBOB-1
iBOB-2
Ant 1
Ant 2
Ant 3
Ant 4
Ant 5
Ant 6
Ant 8
8 Gbps
8 Gbps
≈ 8 Gbps per antenna
Correlator
Correlator
More work needed!
XAUI Linkintegration
DBE interface
CorrelatorSensitivity
Automatic delayextraction
Fringe Rotation
Bandwidth
Vinayak Nagpal, CTH – p. 26/28
Conclusion
The SMA Phased Array Processordevelopment has come a long way.
Most major blocks are ready and working.
Still some work is needed before the systemcan be ready for a sky observation.
Latest trends in FPGA technology and theCASPER paradigm are effective in makingdevelopment cycle times shorter!
Vinayak Nagpal, CTH – p. 27/28
Acknowledgements
SMA Team: Jonathan Weintroub, Bob Wilson,John Test, Taco, Jim Moran, Ray Blundell,Lincoln Greenhill.
CASPER Team: Dan Werthimer, Melvyn Wright,Aaron Parsons, Pierre Droz, Henry Chen,Patrick Crescini.
MIT/Haystack: Shep Doeleman, Brian Fanous,Alan Rogers, Alan Whitney.
Jon Conway: Onsala Space Observatory, Sweden.
Xilinx Inc.
Synopsys Inc.Vinayak Nagpal, CTH – p. 28/28