cross correlators & new correlators

Eleventh Synthesis Imaging WorkshopSocorro, June 10-17, 2008

Cross Correlators & New CorrelatorsMichael P. RupenNRAO/Socorro

2

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

What is a correlator?

• In an optical telescope…– a lens or a mirror collects the light & brings it to a focus

– a spectrograph separates the different frequencies

3

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

• In an interferometer, the correlator performs both these tasks, by correlating the signals from each telescope (antenna) pair:

4

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

•The basic observables are the complex visibilities:amplitude & phase

as functions ofbaseline, time, and frequency.

•The correlator takes in the signals from the individual telescopes, and writes out these visibilities.

5

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

The cross-correlation of two real signals and is

Correlator Basics

A simple (real) correlator.

6

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Antenna 1:

7

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Antenna 2:

8

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

=0:

9

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

=0.5:

10

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

=1:

11

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

=1.5:

12

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

=2:

13

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Correlation:

14

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Correlation of a Single Frequency

For a monochromatic signal:

and the correlation function is

So we need only measure with

15

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Correlation:

xI

xR

16

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

At a given frequency, all we can know about the signal is contained in two numbers: the real and the imaginary part, or the amplitude and the phase.

A complex correlator.

17

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

1.The simple approach:• use a filterbank to split the signal up into quasi-

monochromatic signals at frequencies • hook each of these up to a different complex correlator,

with the appropriate (different) delay:• add up all the outputs

2.The clever approach:instead of sticking in a delay, put in a filter that shifts the phase for all frequencies by /2

Broad-band Continuum Correlators

19

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Spectral Line Correlators

1. The simple approach:• use a filterbank to split the signal up into quasi-

monochromatic signals at frequencies• hook each of these up to a different complex correlator,

with the appropriate (different) delay:• record all the outputs:

20

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Fourier Transforms: a motivational exercise

Short lags (small delays) high frequencies

Long lags (large delays) low frequencies

Measuring a range oflags corresponds to measuring a range of frequenciesThe frequency spectrum is the Fourier transform of

the cross-correlation (lag) function.

21

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Spectral Line Correlators (cont’d)

2.Clever approach #1: the FX correlator• F: replace the filterbank with a Fourier transform• X: use the simple (complex) correlator above to measure the cross-

correlation at each frequency• average over time• record the results• Examples: NRO, VLBA, DiFX, ACA

3. Clever approach #2: the XF (lag) correlator• X: measure the correlation function at a bunch of different lags

(delays)• average over time• F: Fourier transform the resulting time (lag) series to obtain spectra• record the results• Examples: VLA, IRAM; preferred for >20 antennas

22

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

FX vs. XF

F

Fourier transform

t

t

v1

v2

S1()

S2()

S()

F

Fourier transform

X multiply Xmultiply

24

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Spectral Line Correlators (cont’d)

4. Clever approach #3: the FXF correlator• F: bring back the filter bank! (but digital: polyphase FIR filters,

implemented in field programmable gate arrays)- splits a big problem into lots of small problems (sub-bands)- digital filters allow recovery of full bandwidth (“baseband”) through

sub-band stitching• X: measure the correlation function at a bunch of different lags

(delays)• average over time• F: Fourier transform the resulting time (lag) series to obtain spectra• stich together sub-bands• record the results• Examples: EVLA/eMERLIN (WIDAR), ALMA (TFB+ALMA-B);

preferred for large bandwidths

25

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

FXF Output

16 sub-bands

26

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Implementation & choice of architecture

• Correlators are huge– Size roughly goes as NblBW Nchan= Nant

2BW Nchan

– Nant driven up by…• sensitivity (collecting area)• cost (small is cheap)• imaging (more visibilities)• field-of-view (smaller dishes ==> larger potential FoV)

– BW driven up by…• continuum sensitivity

– Nchan driven up by…• spectral lines (spectral resolution, searches, surveys)• Radio frequency interference (RFI) from large BW• field-of-view (fringe washing = beam smearing = chromatic aberration)

27

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Implementation & choice of architecture

• Example: EVLA’s WIDAR correlator (Brent Carlson & Peter Dewdney, DRAO)– 2 x 4 x 2= 16 GHz, 32 antennas– 128 sub-band pairs– Spectral resolution down to below a Hz– Up to 4 million spectral channels per baseline– Input: 3.8 Tbit/sec ~ 160 DVDs/sec (120 million people in

continuous phone conversation)– 40e15 operations per second (petaflops)– Output (max): 30 Gbytes/sec ~ 7.5 DVDs/sec

• N.B. SKA: ~100x larger: 4000 petaflops! (xNTD approach)

28

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

WIDAR today

2 of 256 Boards…

1 of 16 racks…

1.5 hours ago…

29

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

ALMA

1 of 4 quadrants

30

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Implementation & choice of architecture• Huge & expensive ==> relies on cutting-edge technology, with

trade-offs which change frequently (cf. Romney 1999)– Silicon vs. copper– Capability vs. power usage

• Example: fundamental hardware: speed & power usage vs. flexibility and “non-recoverable engineering” expense (NRE)– Application Specific Integrated Circuit (ASIC) (e.g., GBT, VLA,

EVLA, ALMA)– Field Programmable Gate Array (FPGA) (e.g., VLBA, EVLA, ALMA)– Graphics cards– Software (PCs; supercomputers) (e.g., DiFX, LOFAR)

• So big and so painful they tend to be used forever (exceptions: small arrays, VLA, maybe ALMA)

• Trade-offs are so specific they are never re-used (exception: WIDAR)

31

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Details, Details

• Why digital?– precise & repeatable– “embarassingly parallel” operations– piggy-back on industry (Moore’s law et al.)

• …but there are some complications as well…

32

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

1. Sampling: v(t) v(tk), with tk=(0,1,2,…)t– For signal v(t) limited to 0<≤∆, this is lossless if

done at the Nyquist rate: ∆t ≤1/(2∆)

– n.b. wider bandwidth finer time samples!– limits accuracy of delays/lags

2. Quantization: v(t) v(t) + – quantization noise– quantized signal is not band-limited oversampling

helps• N.B. FXF correlators quantize twice, ruling out

most analytic work…

Digitization

33

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Quantization & Quantization Losses

34

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

• We measure the cross-correlation of the digitized (rather than the original) signals.

• digitized CC is monotonic function of original CC• 1-bit (2-level) quantization:

– is average signal power level – NOT kept for 2-level quantization!–roughly linear for correlation coefficient

• For high correlation coefficients, requires non-linear correction: the Van Vleck correction

Cross-Correlating a Digital Signal

35

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Van Vleck Correction

Digital correlation coefficient

True

cor

r’n c

oeff

36

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Correlation Coefficient & Tsys

• Correlation coefficients are unitless– 1.0 ==> signals are identical

• More noise means lower corr’n coeff, even if signal is identical at two antennas

• Must scale corr’n coeff by noise level (Tsys) as first step in calibration

37

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Spectral Response: XF Correlator

Observed = true x truncation at max

Observed = true * spectral response

convolved with (sin max)/(max)

(sin max)(max)

X(

)

max

max

X(

)

max

S

S

38

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

• XF correlator: limited number of lags N ‘uniform’ coverage to max. lag Nt Fourier transform gives spectral response

- 22% sidelobes!- Hanning smoothing

• FX correlator: as XF, but Fourier transform before multiplication spectral response is

- 5% sidelobes

Spectral Response; Gibbs Ringing

39

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

sinc( ) vs. sinc2( )

XFFX

Channel

Spe

ctra

l Res

pons

e

41

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Michael’s Miniature Correlator

0.3

V1

V2

Signals come in… sampled… quantized…delayed… multiplied…integrated & normalized

42

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

FXF Output: sub-band alignment & aliasing

16 sub-bands

43

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

FXF Output: sub-band alignment & aliasing

16 sub-bands

44

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

•The size of a correlator (number of chips, speed, etc.) is generally set by the number of baselines and the maximum total bandwidth. [note also copper/connectivity costs…]

• Subarrays… trade antennas for channels

• Bandwidth -- cut : same number of lags/spectral points across a smaller : Nchan= constant narrower channels: …limited by filters

How to Obtain Finer Frequency Resolution

45

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

-- recirculation:• chips are generally running flat-out for max. (e.g.

EVLA/WIDAR uses a 256 MHz clock with = 128 MHz/sub-band)

• For smaller , chips are sitting idle most of the time: e.g., pass 32 MHz to a chip capable of doing 128 M multiplies per second

add some memory, and send two copies of the data with different delays

Nchan 1/ 2

…limited by memory & data output rates

46

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

VLA Correlator: Bandwidths and Numbers of Channels

50

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

New Mexico Correlators

VLAEVLA

(WIDAR) VLBAArchitecture XF FXF FX

Quantization 3-level 16/256-level 2- or 4-level

Nant 27 40 20

Max. 0.2 GHz 16 GHz 0.256 GHz

Nchan 1 - 512 16,384 - 262,144 256 - 2048

Min. 381 Hz 0.12 Hz 61.0 Hz

dtmin 1.7 s 0.01 s 0.13 s

Power req’t. 50 kW 135 kW 10-15 kW

Data rate 3.3 x 103 vis/sec 2.6 x 107 vis/sec 3.3 x 106 vis/sec

51

Eleventh Synthesis Imaging Workshop, June 10-17, 2008

Current VLA EVLA/WIDAR