casper instrumentation at green bank

April 8/9, 2003 Green BankGBT PTCS Conceptual Design Review

John Ford

September 28, 2009

Casper Instrumentation at Green Bank

The NRAO is operated for the National Science Foundation (NSF)

by Associated Universities, Inc. (AUI), under a cooperative

agreement.

2

Outline

• NRAO

• Configurable Instrument Collection for Agile Data Acquisition (CICADA)

– Green Bank Ultimate Pulsar Processing Instrument (GUPPI)

– West Virginia University Pulsar Processing Instrument (WUPPI)

– GUPPI - 2– Future instruments

3

NRAO Mission

The (US) National Radio Astronomy Observatory enables forefront research into the Universe at radio wavelengths.

• In partnership with the scientific community, we:– provide world leading telescopes,

instrumentation and expertise,– train the next generation of scientists and

engineers, and– promote astronomy to foster a more scientifically

literate society.

4

Aperture Synthesis Telescopes

• Very Long Baseline Array (VLBA)– Network of 10 telescopes spread over

United States– Resolution of a 5000 mile diameter

telescope• Highest Resolution of any telescope on

earth or in space– Sometimes the GBT becomes part of this

array

• (E) Very Large Array (VLA)

5

Green Bank Facilities

• 2700 Acre site in Pocahontas County– 8 Radio Telescopes on site– National Radio Quiet Zone

• 150 mile rectangle centered around Green Bank and Sugar Grove

• Provides protection for the sensitive receivers on the telescopes

– Green Bank Science Center

6

Panoramic View of Green Bank Telescopes

7

CICADA

• Configurable Instrument Collection for Agile Data Acquisition– FPGA based data acquisition and

processing– Uses CASPER tools and hardware

• Umbrella program for organizing FPGA projects– Purchase/obtain boards, software, development systems– 3 ROACH, 2 BEE2, 5 iBOB, 6 ADC, 3 ADC-2, 10 GbE

switches, servers, etc.

CICADA People

• GUPPI– Randy McCullogh, Jason Ray , Patrick Brandt, Ron Duplain,

Paul Demorest, Scott Ransom

• WUPPI/43 meter stuff– Shilpa Bollineni, Glen Langston, Paul Demorest– WVU Folks

9

CICADA Projects

• Spectrometer Proposals– KFPA backend– Replacement for GBT Spectrometer

• Event capture machines– Used for transient events

• Pulsar machines– Incoherent machine (GUPPI, WUPPI)– Coherent dedispersion machine GUPPI-2

GUPPI

11

Sampler( iBOB)

GUPPI BEE2 XAUI alignment

GUPPI BEE2 XAUI alignment

14

GUPPI BEE2 signal processing design

15

GUPPI BEE2 signal combining FPGA

beef.gb.nrao.edu

*.nrao.edu

GUPPI command prompt

Python SOAP client

GUPPI GBT M&C Manager

C++ SOAP client

GUPPI Software Overview

100 MbE 10 GbE

Ethernet

XAUIout-of-bandcontrol bits

equipment

BEE2

runs guppi controller on Debian + BORPH O/S

IBOBs

no custom controller software

ADCs

Clock Synthesizer

Power Strip

clock input

power

XAUIGBT IF

telnet over Ethernet

telnet over Ethernet

systems control signals data streams notes/comments .

guppi_daq

threads:• net• disk• fold (for folding mode)will also run on GPU nodes in GUPPI Phase II

guppi controller

modules:• DAQ• POWER• SYNTH• BEE2 guppi_statusd

gets GBTStatus

shared memory

Unix socket

shared memory

GUPPI Status

• Currently available GUPPI Modes– 128 to 4096 channels– 2 polarizations– 100 to 800 MHz bandwidth– Full Stokes or Total Intensity Only– Decimation in frequency and time in CPU– Records up to ~250 MB/S continuously– Disk space is a considerable problem

• Fully integrated into the GBT Observing System

GUPPI's Advantages

Searching: 800MHz of BW, 4096 chan, and RFI-resistance (from polyphase filterbank) will make GUPPI a “Super-Spigot” that will be unbeatable for searches from 1-5 GHz (previously the BCPM and Spigot)

High Dynamic-Range Studies: 8-bit sampling, high spectral resolution and Full Stokes will make GUPPI perfect for scintillation studies, HI absorption, Faraday rotation measurements, polarization studies, and singlepulse studies (previously the Spectral Processor)

Ultra-High Precision Timing: 8-bit sampling, 800MHz of BW, and RFI resistance will allow unprecedented timing precision from 1-3 GHz for millisecond pulsars (i.e. the NANOGrav project and the search for nHz gravitational waves; previously GASP/CGSR2)

GUPPI-2

• Uses GUPPI to feed a 9-machine cluster of GPU’s via 4 10 GbE ports.

• 128, 256, or 512 coarse channels will be supported• 100 MHz processed per GPU at “reasonable” DM’s.• Only modification to GUPPI is a new output FPGA

personality and software to control the nodes

GUPPI-2 experiments

GUPPI-2 Experiments

WUPPI Crab Pulsar Data

Future Projects

• Not funded, but…– K band Array receiver spectrometer– GBT Spectrometer Replacement– Transient searches on old telescopes (like fly-eye, with bigger

and fewer dishes)– Lunar Neutrino project

• Several 43m projects:– WUPPI– Receiver project

24

K band Feedhorn Array Spectrometer

• 7 pixels, dual polarization– 21 ROACH boards, 14 ADC's – Needs to be in receiver room

• 3 GHz bandwidth• Cross correlation between polarizations on a single

pixel• “Zoom” mode for multiple narrow windows in a

bandpass• Fast dump mode for pulsar use on a few polarizations• Normal dump times of 1 ms

Array Spectrometer design

Reconfigurable Open

ArchitectureComputingHardware

Reconfigurable Open

ArchitectureComputingHardware

Front-endElectronics

Reconfigurable Open

ArchitectureComputingHardware

IF 0

Pixel 0

Pixel 1

Pixel 7Pixel 6

Pixel 5Pixel 4

Pixel 3Pixel 2

Front-endElectronics

IF 1

10 Gb Switch

10 GB X 210 GB X 2

10 Gb Switch

32 Core Pipeline Processor

Disk Controller

Disk Controller

Dual10 GbLinks

Fibers FromGBT

40 TB

40 TB

Up to 500 MB/sec to disks,

sustained

Fibers To Jansky Lab Equipment

Room

Signal Processors

Signal Combiner/Accumulator

To existing GB Network infrastructure

43m Receiver and Spectrometer

The End

• Summary:– GUPPI is working well

• Designs available for distribution in unadorned form now (version 7.1 right now)

• Plan to submit the FPGA designs, software, and hardware designs to the CASPER wiki