1446 ieee transactions on nuclear science, vol. 63, no....

9
1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. 3, JUNE 2016 First Results From High-Resolution Front End Electronics for Water Cherenkov Air Shower Detectors Equipped With Cyclone ® V FPGA Zbigniew Szadkowski, Member, IEEE Abstract—The paper presents first results from the Front- End Board (FEB) with the biggest Cyclone ® V E FPGA 5CEFA9F31I7N, supporting 8 channels sampled up to 250 MSps @ 14-bit resolution. Considered sampling for the planned upgrade of the Pierre Auger surface detector array is 120 MSps, however, the FEB has been developed with external anti-aliasing filters to keep a maximal flexibility. Six channels are targeted to the SD, two the rest for other experiments like: Auger Engineer- ing Radio Array and additional muon counters. More channels and higher sampling generate larger size of registered events. We used the standard radio channel for a radio transmission from the detectors to the Central Data Acquisition Station (CDAS) to avoid at present a significant modification of a software in both sides: the detector and the CDAS (planned in a future for a final design). Several variants of the FPGA code were tested for 120, 160, 200 and even 240 MSps DAQ. Tests confirmed a stability and reliability of the FEB design in real pampas conditions with more than 40 C daily temperature variation and a strong sun exposition with a limited power budget only from a single solar panel. Seven FEBs have been deployed in a hexagon of test detectors on a dedicated Engineering Array. Index Terms— Cyclone ® V, FPGA, front-end, Pierre Auger Observatory, surface detector, trigger. I. I NTRODUCTION T HE Pierre Auger Observatory is the world’s largest ground based detector of the ultra high-energy cosmic rays (UHECR) [1]. It uses three types of detectors: surface, fluorescence and radio. The surface detector (SD) array con- taining 1680 water Cherenkov detectors spread over an area of 3000 km 2 started to operate since 2004. The currently used Front-End Boards are equipped with no-more produced ACEX ® and obsolete Cyclone ® FPGA (40 MSps/15-bit of dynamic range). New challenges from physics [2]–[7] require a significant upgrade of the SD electronics to improve the quality of measurements and pick-up from a background of extremely rare events. Huge progress in electronics allows a reasonable Manuscript received June 21, 2015; revised March 29, 2016; accepted May 10, 2016. Date of publication May 16, 2016; date of cur- rent version June 21, 2016. This work was supported by the Pol- ish National Center for Research and Development under NCBiR Grant ERA/NET/ASPERA/02/11 and by the National Science Centre (Poland) under NCN Grant 2013/08/M/ST9/00322. The author is with the Department of Physics and Applied Informatics, Faculty of High-Energy Astrophysics, University of ód´ z, 90-236 ód´ z, Poland (e-mail: [email protected]). Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TNS.2016.2567320 Fig. 1. On Chip Termination of LVDS lines. SD electronics improvement in a realistic budget. The Pierre Auger Observatory is planned to operate until 2023. II. ELECTRONICS FOR THE SURFACE DETECTORS The Cherenkov light is detected in the surface detectors by three 9-inch photomultiplier tubes (PMTs) from which the signals of the anode and last dynode are digitized by 10-bit ADCs. The currently used Front-End Boards equipped with the ACEX ® [8] and Cyclone ® [9] FPGA are sampled at 40 MHz. However, both FPGA families are already obsolete. Data readout of the enhanced surface detector stations will be facilitated by replacing the current readout electronics by modern state-of-the-art electronics providing three-four times faster sampling, a significant enhanced dynamic range. The electronics, currently used in surface detectors, were designed 10 or more years ago. 10 years in electronics development is an epoch. Several components are not pro- duced anymore, the replacement of the failed components becomes a significant factor. On the other hand, better and better understanding of fundamental processes imposes on experiments new challenges requiring higher resolution, faster measurements with higher accuracy, with more sophisticated algorithms, etc. A significant part of challenges can be accom- plished with a new much powerful, energy-efficient electronics with dedicated, embedded signal processing blocks allowing an implementation of much more complicated, mathematical algorithms in real time. In the prototype we decided to implement the biggest chip from the Altera ® low-cost Cyclone ® V E family – 5CEFA9F31I7 (industrial version with a temperature range 40 C– +85 C – because of the large daily variation of tem- perature 40 C) to retain maximal flexibility in the development of sophisticated, resource consuming algorithms (e.g., discrete cosine transform for recognition of “old” inclined show- ers [10]–[12] or artificial neural networks for recognition of 0018-9499 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Upload: others

Post on 09-Oct-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. 3, JUNE 2016

First Results From High-Resolution Front EndElectronics for Water Cherenkov Air ShowerDetectors Equipped With Cyclone® V FPGA

Zbigniew Szadkowski, Member, IEEE

Abstract— The paper presents first results from the Front-End Board (FEB) with the biggest Cyclone® V E FPGA5CEFA9F31I7N, supporting 8 channels sampled up to 250 MSps@ 14-bit resolution. Considered sampling for the plannedupgrade of the Pierre Auger surface detector array is 120 MSps,however, the FEB has been developed with external anti-aliasingfilters to keep a maximal flexibility. Six channels are targeted tothe SD, two the rest for other experiments like: Auger Engineer-ing Radio Array and additional muon counters. More channelsand higher sampling generate larger size of registered events.We used the standard radio channel for a radio transmission fromthe detectors to the Central Data Acquisition Station (CDAS) toavoid at present a significant modification of a software in bothsides: the detector and the CDAS (planned in a future for a finaldesign). Several variants of the FPGA code were tested for 120,160, 200 and even 240 MSps DAQ. Tests confirmed a stabilityand reliability of the FEB design in real pampas conditions withmore than 40◦C daily temperature variation and a strong sunexposition with a limited power budget only from a single solarpanel. Seven FEBs have been deployed in a hexagon of testdetectors on a dedicated Engineering Array.

Index Terms— Cyclone® V, FPGA, front-end, Pierre AugerObservatory, surface detector, trigger.

I. INTRODUCTION

THE Pierre Auger Observatory is the world’s largestground based detector of the ultra high-energy cosmic

rays (UHECR) [1]. It uses three types of detectors: surface,fluorescence and radio. The surface detector (SD) array con-taining 1680 water Cherenkov detectors spread over an areaof 3000 km2 started to operate since 2004. The currentlyused Front-End Boards are equipped with no-more producedACEX® and obsolete Cyclone® FPGA (40 MSps/15-bit ofdynamic range).

New challenges from physics [2]–[7] require a significantupgrade of the SD electronics to improve the quality ofmeasurements and pick-up from a background of extremelyrare events. Huge progress in electronics allows a reasonable

Manuscript received June 21, 2015; revised March 29, 2016; acceptedMay 10, 2016. Date of publication May 16, 2016; date of cur-rent version June 21, 2016. This work was supported by the Pol-ish National Center for Research and Development under NCBiR GrantERA/NET/ASPERA/02/11 and by the National Science Centre (Poland) underNCN Grant 2013/08/M/ST9/00322.

The author is with the Department of Physics and Applied Informatics,Faculty of High-Energy Astrophysics, University of ódz, 90-236 ódz, Poland(e-mail: [email protected]).

Color versions of one or more of the figures in this paper are availableonline at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TNS.2016.2567320

Fig. 1. On Chip Termination of LVDS lines.

SD electronics improvement in a realistic budget. The PierreAuger Observatory is planned to operate until 2023.

II. ELECTRONICS FOR THE SURFACE DETECTORS

The Cherenkov light is detected in the surface detectorsby three 9-inch photomultiplier tubes (PMTs) from whichthe signals of the anode and last dynode are digitized by10-bit ADCs. The currently used Front-End Boards equippedwith the ACEX® [8] and Cyclone® [9] FPGA are sampledat 40 MHz. However, both FPGA families are already obsolete.Data readout of the enhanced surface detector stations willbe facilitated by replacing the current readout electronics bymodern state-of-the-art electronics providing three-four timesfaster sampling, a significant enhanced dynamic range.

The electronics, currently used in surface detectors, weredesigned 10 or more years ago. 10 years in electronicsdevelopment is an epoch. Several components are not pro-duced anymore, the replacement of the failed componentsbecomes a significant factor. On the other hand, better andbetter understanding of fundamental processes imposes onexperiments new challenges requiring higher resolution, fastermeasurements with higher accuracy, with more sophisticatedalgorithms, etc. A significant part of challenges can be accom-plished with a new much powerful, energy-efficient electronicswith dedicated, embedded signal processing blocks allowingan implementation of much more complicated, mathematicalalgorithms in real time.

In the prototype we decided to implement the biggestchip from the Altera® low-cost Cyclone® V E family –5CEFA9F31I7 (industrial version with a temperature range−40◦C – +85◦C – because of the large daily variation of tem-perature 40◦C) to retain maximal flexibility in the developmentof sophisticated, resource consuming algorithms (e.g., discretecosine transform for recognition of “old” inclined show-ers [10]–[12] or artificial neural networks for recognition of

0018-9499 © 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

Page 2: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

SZADKOWSKI: FIRST RESULTS FROM HIGH-RESOLUTION FRONT END ELECTRONICS 1447

Fig. 2. Front-End Board with Cyclone V E FPGA and 4 double 14-bit ADCs (ADS4249 – Texas Instr.) with maximal 250 MHz sampling frequency [16].

”young” inclined showers [13] and wavelet trigger [14], [15]).The FPGA receives digitized data from very fast (250 MSps)14-bit ADCs (ADS4249 – Texas Instr.).

The 40 MHz sampling ADCs currently in use (AD9203)provide data output in the CMOS unipolar format. ADCs,used in the FEB prototype are equipped with LVDS dataoutput. The differential data transmission method used inLVDS is less susceptible to common-mode noise than single-ended schemes like CMOS. Differential transmission usestwo wires with opposite current and voltage swings insteadof the one wire used in CMOS to convey the data. TheCyclone® V devices provide a 100 �, on-chip differentialtermination option on each differential receiver channel forLVDS standards. On-chip termination saves board space byeliminating the need to add external resistors on the board.All I/O pins and dedicated clock input pins support on-chipdifferential termination (Fig. 1).

Differential LVDS signals have to be converted inside theFPGA into single-ended ones (diff:$00006 block for ADCclock Clk_ABP/M_ADC_120 MHz (Fig. 3). Decoded data(after the splitter block) are latched in a cascade regis-ters clocked by the dedicated clock providing by the ADC.ADC data are next synchronized in the next stage which isalready clocked by the global clock (120 MHz). The clockClk_ABP/M_ADC_120 MHz is generated in a dedicated PLLcircuit driven from the ADC by the clock with rising edgesoptimized to a maximal range of stable latched data.

When the data is captured without the sufficient setupand hold time in the LVDS receiver, then it is called

marginal capturing. The ADS4249 provides a center-alignedsource-synchronous data and clock. For a source-synchronousinterface, the goal of the FPGA is to keep the clock-to-dataphase relationship seen at the input pins the same at theinternal I/O registers, to maintain the maximum timing margin.Therefore, the PLL is setup to simply recreate the input clockusing source-synchronous compensation mode and a 0◦ phaseshift. Source-synchronous compensation mode maintains thephase relationship between the clock and data coming fromthe ADS4249 inside of the FPGA by matching the delayfrom the clock input pin to the I/O registers and the delayfrom the data input pins to the I/O registers. A clock phaseshift is not needed inside of the FPGA because the ADS4249already provides a center-aligned data clock and the source-synchronous compensation of the FPGA’s PLL maintains thisrelationship. Unfortunately the source synchronous mode doesnot provide satisfactory results. We had to use the normalmode with a dynamical phase tuning.

III. DYNAMIC PHASE SHIFT WITH ALTERA PLL IP CORE

Fig. 3 shows a PLL circuitry providing necessaryclocks with a possibility of dynamical reconfiguration.For each ADC clock two FPGA IP core routines(e.g., pll_ADC_120 MHz_AB and pll_AB_reconfig – Fig. 3)are required. They are controled by the NIOS® (Fig. 5).

Fractional PLLs (fPLLs) in the Altera Cyclone® V familiesuse divide counters and different voltage-controlled oscilla-tor (VCO) taps to perform frequency synthesis and phaseshifts. Output clock in the PLLs can be reconfigured by the

Page 3: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

1448 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. 3, JUNE 2016

Fig. 3. The PLL circuitry controling the internal and external clocks. Ext_Clk (40 MHz) comes from the UB (as one of the output from the splitterCY2308SXI-1 (Cypress). The global_Clk_120 MHz is the global internal clock. (for other variants it corresponds to 160, 200 or 240 MHz, respectively). TheUB_Clk_40 MHz) drives all communication processes with the UB (e.q. DMA transfer). The pll_out_120 MHz_P/N is a pair of LVDS clock driving the clocksplitter ADCLK846 (Analog Devices) for 3 ADCs (channels 1–6). For higher sampling these outputs generate 160, 200 or 240 MHz clocks, respectively. Thepll_out_200 MHz_P/N is a pair of LVDS clock driving the 4th ADC chip for channels 7 and 8. The pll_ADC_120 MHz_AB together with the pll_AB_reconfigare the circuits providing the online reconfiguration of the clock frequency and the phase shift for each ADC chip (the firmware uses 4 such blocks for channels1–2, 3–4, 5–6 (the same frequency e.g., 120 MHz) and 7–8 (independent frequency e.g., 200 MHz). The upper part shows the LVDS input converter.

Fig. 4. A structure of the PLL circuit. All N, M C counters, charge pumpcurrent, loop-filter parameters as well as dynamic phase shifting can be setin Altera PLL and Altera PLL Reconfig IP Cores via mgmt ports.

counter settings and dynamically phase-shift in fPLLs. We canalso change the charge pump and loop filter components,which dynamically affect the fPLLs bandwidth. The clockfrequency, fPLL bandwidth and phase shift can be updatedin real time without reconfiguring the entire FPGA.

The following fPLL components are reconfigurable in realtime using the dynamic reconfiguration IP core (see Fig. 4 andexample of parameters in Table I):

• Post-scale output counter (C)• Feedback counter (M)• Prescale counter (N)• Charge pump current (ICP) and loop-filter components

(R, C)• Dynamic phase shifting of each counter• Fractional division (MFRAC) for Delta Sigma Modulator

Fig. 4 shows an internal structure of the PLL circuit withcorresponding N, M and C-counters. The programmable phaseshift feature allows the PLLs to generate output clocks with a

Fig. 5. A structure of the NIOS® I/Os.

fixed phase offset. The VCO frequency of the PLL determinesthe precision of the phase shift. The resolution of phase shiftis a function of VCO frequency, with the smallest incremental

Page 4: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

SZADKOWSKI: FIRST RESULTS FROM HIGH-RESOLUTION FRONT END ELECTRONICS 1449

TABLE I

A TABLE SHOWING PARAMETERS NEEDED FOR M AND C COUNTERS (N-COUNTER SET AS BYPASSED) TO DYNAMICALLY MODIFY A PHASE SHIFTFOR 120 MHZ SAMPLING FREQUENCY. THE ONLY RANGE OF 180◦ – 225◦ IS SHOWN. SYMMETRICAL DATA FOR A PHASE BELOW 180◦ LOOKS

SIMILAR. ALL EARLIER CALCULATED PARAMETERS ARE STORED IN THE ON-CHIP MEMORY SUPPORTED BY THE NIOS® PROCESSOR.

step equal to 1/8th of the VCO period. For the Cyclone® V themaximal legal reference clock frequency is 700.0 MHz. TheVCO frequency can reach 1.6 GHz. E.g. for fV C O = 960 MHzphase shift steps of 133 ps are possible.

IV. NIOS® SOFT-CORE PROCESSOR

The current Front-End Boards based on the ACEX® andCyclone® FPGAs can register events from channels with10-bit resolution in an interval of 19.2 μs, which correspondsto 768 time-bins. A 768·60·10-bit matrix is standard format forSD data. Higher resolution, higher sampling and longer tracesrequire a new format; however, this is a task for the updatedunified board (UUB) which is currently in the developmentphase. This intermediate FEB has to use the standard dataformat in order not to change any cell in a data transmissionchain.

For considered 120 MSps in the same interval-time of19.2 μs and for 12-bit resolution in 8 channels the size ofeach event is 2304 · 8 · 12-bit = 27 kB instead of 6 kB for theformat currently in use. For higher sampling frequencies (or aprobably variant: 120 MSps for the SD channels : 1st to 6th) +200 MSps for the supported experiments like AERA (channels7th + 8th) the event size is already 32 kB. The standard DMAchannel can transmit only 6 kB of data (Fig. 6). In order notto lose the rest of data they have to be temporarily stored andwait to be transmitted in a proper period of time.

A data management can be supported by the soft-coreNIOS® processor (Fig. 5). The 32-kB block of samples isfrozen temporarily in the fast dual-port RAM buffers and hasto be transferred from the FPGA to the NIOS® and nextwritten in the external SDRAM. High voltage on the PMTsis adjusted to get ∼ 100 Hz T1 trigger rate. The commu-nication time between the local SD station and the CDASdoes not exceed 10 seconds. When an event is registered,the local station sends a message to the CDAS informing onthe event. The CDAS analyses messages from neighboringdetectors to recognize a coincidence in several tanks whichmay correspond to an energetic shower. If this case occursthe CDAS sends a request to ”fired” detectors to send thefull data. For ∼ 100 Hz data rate and 10 seconds storage

time a memory size for temporarily data storage should beat least 100 s−1 · 10 s · 32 kB = 32 MB. The internal FPGAmemory is too small to store such amount of data. An externalmemory has to be used. The Cyclone® V Front-End Boardsare equipped with an external SDRAM (U24 and U25 onFig. 2 driven by sdram ports of NIOS (Fig. 5). Each eventis recorded with a corresponding time stamp, which allowslater, on request, an identification of address, where data werewritten in the SDRAM and an extraction of full event data fora transmission to the CDAS [17]. Additionally, two UARTsallow direct connections to the PC. Three ports (adc0_1,adc2_3 and adc4_5) support 2 × 14-bit ADCs, adc7_8 portsupports 2 × 12-bit ADC for additional detectors. The NIOS®

calculated the average values and RMS of ADC for each chan-nel. Based on the generated statistics, a necessary data for PLLregisters are sent via mgmt ports. Appropriate PLL circuits (fora dynamical phase shift) are driven and read via de-multiplexerand multiplexer fast blocks in the FPGA fabric, which arecontrolled by mgmt_ctrl_sel port. The NIOS® shifts the clockphase to provide the lowest level of the LVDS artifacts.

V. PRELIMINARY TESTS IN THE ”JAMIE” TEST DETECTOR

Eight Front-End Boards with 10 Power Supply Boards weredelivered to Argentina on March 6, 2015. All FEBs weresuccessfully tested in the laboratory in Malargüe and in the”Jamie” test surface detector close the Assembly Building inthe Pierre Auger Observatory.

Power Supply Boards (Fig. 8) are equipped with DC-DC converters providing analog and digital voltages to theFEBs. The production power of the solar panel is 10 Wfor a continuous work. The Cyclone® V FEBs consume16.5–19 W depending on activated resources and samplingfrequencies (Table II). Both DC-DC converters can be disabledby external signals. The MAX_V CPLD can control DC-DCconverters via Darlington opto-couplers 4N33M (Fig. 9). Ana-log and digital DC-DC converters can be controlled inde-pendently. The 6th and 7th columns in Table II show thecontinuous current consumptions (in Ampere). Two next pairsshow the current consumptions for disabled digital and analogDC-DC converters, respectively.

Page 5: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

1450 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. 3, JUNE 2016

Fig. 6. Structure of transfered data. The standard FEBs are equipped in 10-bit ADCs. For each PMT, 2 ADCs cover the voltage range of 61 μV – 2.0 Vas high-gain (HG_10) and low-gain (LG_10) channels with 5 bits of overlap. The DMA allows a transfer of packages of 10-bit only (the structure of datatransmission remains the same). 14-bit data in high-gain (HG_14) and low-gain (LG_14) channels are cropped to 12 MSBs in the HG_14 and only 8 MSBsin the LG_14. The new FEBs with the old DMA configuration cover the voltage range of 31 μV – 8.0 V. Graphs show an example of event registered in theDPRAM and transferred to the PC via NIOS® and the RS232 serial port (J11 or J12 connector on the Fig. 2). This is a platform, where LVDS artifacts areclearly visible and which supports to tune manually the phase in the PLL to avoid any LVDS spurious reading.

TABLE II

POWER CONSUMPTION FOR 2, 3, . . . , 9 FEBs AND VARIOUS SAMPLING FREQUENCIES. CURRENTS GIVEN FOR +24 V LABORATORY POWER SUPPLIES.SEE TEXT FOR A DESCRIPTION OF THE COLUMNS.

The average current consumption of the UB is 0.15 A,which corresponds to 3.6 W. The FEB is a daughter boardwith additional power supplies. Disabling the digital powersupply on the FEB (digital DC-DC converters – Fig. 8) doesnot decrease a total power consumption, but vice-versa thepower consumption increases, due to a leakage from the analogto digital section via semiconductors on the board. The leakageto an opposite direction is also visible, although on a muchlower scale. By disabling the analog DC-DC converters we canreduce the total power by ∼6−7 W. The ”idle” mode (withouta data acquisition) takes still ∼10.5 W, which value is slightlyabove a theoretical specification for the solar panels and doesnot allow a normal DAQ. Fig. 7 show a continuous work ofthe FEBs in real pampas conditions. Long breaks correspond

to a poor weather, when batteries cannot be charged above thehysteresis threshold.

Clocks driving the ADCs are generated in the FPGA PLLcircuits and taken out as LVDS pairs. One of them is nextsplit into 3 clock pair lines supplying ADCs digitizing PMTsignals. The double ADC for channels 7th and 8th is drivenby the 2nd PLL. In order to reduce the power consumption inthe idle mode outgoing clocks can be disabled in an additionalPLL circuits with enable control port (Fig. 3). Switching offthe clocks for the ADC reduce the power consumption onadditional 0.8 W for the +3.6 V power supply. This meansthe reduced power consumption for anticipated 120 MHzsampling frequency could be below 10 W – a continuouspower productivity of used solar panel (compare data for

Page 6: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

SZADKOWSKI: FIRST RESULTS FROM HIGH-RESOLUTION FRONT END ELECTRONICS 1451

Fig. 7. Battery voltages (monitoring files) in 6 surface detectors: Lucho (972), Naif (943), Anabel (944), Brisa (945), Cuixart (949) and Rosario (954).

FEB_2 and FEB_5 in Table II). Nevertheless, we observedpoor charging of battery during cloudy days. We decided toequip the Power Supply Boards with a relay that cut the power

only for the FEBs and controlled by a simple comparator witha hysteresis (23.0 – 25.0 V). The UB is directly controlledby the Tank Power Control Board (TPCB). This means the

Page 7: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

1452 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. 3, JUNE 2016

Fig. 8. Power board providing +3.3 V for the digital section and +/−3.6 Vfor the analog section. Both DC-DC converters can be independently turnedoff to reduce a power consumption or to reload the FPGA configuration filefrom the non-volatile memory EPCQ256.

Fig. 9. A daughter board plugged into J23 connector on the Front-End Boardto remotely control the analog and digital power supplies by short-cutting ofdedicated pins in DC-DC converters.

UB operates permanently till the battery voltage is above acritical level.

VI. TRIGGER RATE

The trigger for the SD array is hierarchical. Two levelsof trigger (called T1 and T2) are formed at each detector.T2 triggers are combined with those from other detectors andexamined for spatial and temporal correlations, leading to anarray trigger (T3). The T3 trigger initiates data acquisition.

Two independent trigger modes are implemented as T1.The first T1 mode is a simple threshold trigger (TH)which requires the coincidence of the three PMTs eachabove 1.75 I peak

V E M . The Vertical Equivalent Muon (VEM) is aunit used for a calibration corresponding to ∼ 50 ADC-unitsfor 40 MHz sampling frequency. The 2nd mode is designatedthe “Time-over-Threshold” trigger (ToT) and at least 13 binsin 120 time-bins of a sliding window of 3 μs are requiredto be above a threshold of 0.2 I peak

V E M in coincidence in 2 outof 3 PMTs.

A high voltage on the PMTs is calibrated to obtain anaverage trigger rate on a level of ∼ 100 Hz. Normally,thresholds are compensated vs. temperature by the UB every∼ 1 minute.

A temperature dependence of pedestals for PMTs usuallyis not so high (±1 ADC-units : PMT2,3 – Fig. 10). If higherpedestal variation occurs (e.g., PMT1) the trigger rate mayvary dramatically. Fig. 11(a) was created for artificial con-ditions to show how a relatively small pedestal fluctuationsdramatically change trigger rates. We moved the pedestal

Fig. 10. A dependence of the pedestals vs. time.

Fig. 11. A dependence of trigger rate vs. time for uncompensated (uppergraph – data recorded April 1–6, 2015) and compensated (lower graph – datarecorded May 17–18, 2015) pedestals. A dramatically higher trigger rate inafternoon periods corresponds to direct falling of the sun on the solar panelbut also on the tank. This causes significant increment of the temperatureinside the electronic box and simultaneously increment of pedestals. When atrigger threshold is not compensated, a difference between a trigger thresholdand a pedestal decreases and this effect corresponds to an efficient decreasingof the trigger threshold. In order to avoid abrupt changes of the trigger ratestrigger thresholds are tuned internally in the FPGA much more frequently(∼ 0.5 ms) than in a standard way externally from the UB (∼ 1 minute).

compensation task from the UB to the FPGA. Pedestals foreach PMT channel are averaged in 216 steps. The refreshmenttime is ∼ 0.5 ms. Fig. 11(b) shows a dependence of the THtrigger rate vs. time for 5 days continuous measurements withcompensated threshold levels. A sigma-delta compensation ofthe pedestal level allows keeping the trigger rate in an expected

Page 8: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

SZADKOWSKI: FIRST RESULTS FROM HIGH-RESOLUTION FRONT END ELECTRONICS 1453

Fig. 12. A dependence of the TH trigger rate for various TH and ToTthresholds (49 ≤ T hrT H ≤ 57, 6 ≤ T hrT oT ≤ 8) as well as for variousOccupancy (20 ≤ Occ ≤ 40) in the 3 μs sliding window.

Fig. 13. Trigger rates for the ToT trigger for various parameters:49 ≤ T hrT H ≤ 57, 6 ≤ T hrT oT ≤ 8, 20 ≤ Occ ≤ 40) in the 3 μs slidingwindow (above) with a zoom (below) focusing on suitable parameters for afuture operation. In the old design with 40 MHz sampling the Occupancyfor the ToT trigger was 13 (see Section VI). A simple scaling by a factor3 = 120 MHz/40 MHz does not provide satisfactory stable conditions.We tested several variants of the Occupancy (Occ) and ToT thresholds(T hrT oT ). Results from above graphs suggest rather more careful ToT triggeranalysis.

level (Fig. 11(b)). Nevertheless, we see still small periodic ratevariations.

The TH trigger rate in Fig. 11(b) is below the standard100 Hz rate obtained in detectors with 40 MHz samplingfrequency. The calibration procedure setting high voltageson the PMTs was developed for 40 MSps and it should be

Fig. 14. Histograms of noise for lab (upper graph) and field (lower graph)conditions. LVDS artifacts for PMT3 are visible as spurious values muchlower than a pedestal level.

optimized to narrower time-slots. At present, its reliabilityfor 120 MSps is a little bit problematic. If the 120 MSpscalibration was successful, we used these PMT high voltagesfor all next tests.

The trigger rates are very sensitive on thresholds. THand ToT trigger rates were measured for several vari-ants of thresholds (TH+ToT), Occupancy (Occ) and slid-ing window width (ToT) to optimize these parameters forhigher sampling DAQ. Fig. 12 shows a dependence of theTH trigger rate for several parameters. An expected 100Hz trigger rate corresponds to the TH-threshold of 54–55ADC-units (∼1.05 I peak

V E M ) above the pedestal independently ofT hrT oT and Occ. ToT rates strongly depend on the T hrT oT

and Occ. Fig. 13 shows that T hrT oT should be exactly7 ADC-units above the pedestal and the Occ could be between36 and 40 time-bins in order to keep ∼ 2 Hz ToT trigger ratesas for 40 MSps DAQ.

VII. NOISE AND LVDS ARTIFACTS

Noise in each channel of the FEBs in the laboratoryconditions is on a level of ∼ 2 ADC-units (Fig. 14(a)). LVDSartifacts are eliminated by a adjustment of the phase in PLLcircuits. However, during tests in real surface detector wefound an unexpected feature. For some channels the optimalphase shifts were 185.6◦ or 186.4◦. They correspond to 960or 840 MHz of the VCO inside the FPGA. Unfortunately,by these conditions the CDAS does not recognize the UB

Page 9: 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. …kawe.wfis.uni.lodz.pl/kfd/pdf/ZS2016-TNS-SDE.pdf · 2016. 9. 6. · 1446 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63,

1454 IEEE TRANSACTIONS ON NUCLEAR SCIENCE, VOL. 63, NO. 3, JUNE 2016

and triggers. Only a simple change to 180◦ in all chan-nels ( fV C O = 360 MHz) allowed a stable daq. However,180◦ phase shift in all channels does not correspond tooptimal LVDS transfer. We observe LVDS artifacts in somechannels not the same in various FEBs. Noise from PMTsis also relatively high (much wider histograms in Fig. 14).Additionally, there are visible ranges in signal histograms farfrom expected averaged pedestal. E.g. PMT3 in Fig. 14(b)gives a significant contribution of ADC signals 128 or 256ADC-units below the pedestal. This means that 7th and 8thbit of the ADC in the 3rd channel sometimes delay.

VIII. CONCLUSION

All FEBs delivered to Argentina passed successfully all testsin the field conditions. We tested standard protocols (3-foldcoincidences for the threshold trigger, 768 time bins in high-gain and low-gain channels, the same DMA data structure)with higher sampling rates (120, 160, 200 and 240 MSps) and12-bit resolution in a narrower window (6.4, 4.8, 3.84 and3.2 μs instead of 19.2 μs due to higher sampling rate) andwith standard and optimized thresholds. On the AHDL level,ADC data were processed with 120/160/200 or 240 MHz clockand stored in the left port of the dual-port RAM. Data wereread from the right-port with 40 MHz clock to be transferredvia DMA in a standard way.

Cloudy days impaired a power budget and selected detectorshad to be remotely switched off at night preventing the batterynot to be discharged to a level menacing of a total damage.We had to add a remote temporary digital power disconnection(supplied from the same power line) to allow independentFPGA reprogramming from the standard configuration chip(a theoretically simultaneous turning on of the power for UBand FEB practically generates a race hazard which sometimeskeeps the FPGA not programmed).

Temporarily, we had to agree on the DAQ via CDAS withnon-optimized phase shifts in PLLs, which was a reason ofsome LVDS artifacts in transmitted data. Nevertheless, suchsingle time-bin spikes can be easily repaired offline.

We believe that data obtained from these test FEBs providevery useful information for the final design of the new elec-tronics. Furthermore, this data will be useful to develop andtest new triggers taking profit from the faster sampling rate ofthe signals than that of the current electronics.

ACKNOWLEDGMENT

The author would like to thank R. Sato for a help ininstallations of the FEB in the test Jamie tank and in tanks

on the Engineering Array, R. Squartini for an establishmentof the internet link allowing a remote control of the Jamietest detector from the University of ódz (Poland) as well asthe entire Pierre Auger Collaboration for providing the surfacedetector infrastructure for tests of new Front-End Boards andnew FPGA algorithms.

REFERENCES

[1] J. Abraham et al., “Properties and performance of the prototype instru-ment for the Pierre Auger Observatory,” Nucl. Instrum. Methods Phys.Res. A, vol. 523, pp. 50–95, 2004.

[2] J. Abraham et al., “Upper limit on the cosmic-ray photon flux above1019 eV using the surface detector of the Pierre Auger Observatory,”Astropart. Phys., vol. 29, p. 243, May 2008.

[3] J. Abraham et al., “Upper limit on the cosmic-ray photon fraction at EeVenergies from the Pierre Auger Observatory,” Astropart. Phys., vol. 31,p. 399, Jul. 2009.

[4] J. Abraham et al., “Upper limit on the diffuse flux of ultrahigh energy tauneutrinos from the Pierre Auger Observatory,” Phys. Rev. Lett., vol. 100,p. 211101, May 2008.

[5] J. Abreu et al., “Search for point-like sources of ultra-high energyneutrinos at the Pierre Auger Observatory and improved limit on thediffuse flux of tau neutrinos,” Astrophys. J. Lett., vol. 755, p. L4, 2012.

[6] J. Abreu et al., “Search for first harmonic modulation in the rightascension distribution of cosmic rays detected at the Pierre AugerObservatory,” Astropart. Phys., vol. 34, p. 627, Mar. 2011.

[7] J. Abraham et al., “Correlation of the highest-energy cosmic rays withnearby extragalactic objects,” Science, vol. 318, p. 938, 2007.

[8] Z. Szadkowski, “The concept of an ACEX® cost-effective first levelsurface detector trigger in the Pierre Auger Observatory,” Nucl. Instrum.Methods Phys. Res. A, vol. 551, pp. 477–486, 2005.

[9] Z. Szadkowski, K.-H. Becker, and K.-H. Kampert, “Development of anew first level trigger for surface array in the Pierre Auger Observatorybased on the CycloneTM Altera® FPGA,” Nucl. Instrum. Methods Phys.Res. A, vol. 545, pp. 793–802, 2005.

[10] Z. Szadkowski, “A spectral 1st level FPGA trigger for detection ofvery inclined showers based on a 16-point discrete cosine transformfor the Pierre Auger Observatory,” Nucl. Instrum. Methods Phys. Res.A, vol. 606, pp. 330–343, 2009.

[11] Z. Szadkowski, “Trigger board for the Auger surface detector with100 MHz sampling and discrete cosine transform,” IEEE Trans. Nucl.Sci., vol. 58, no. 4, pp. 1692–1700, Aug. 2011.

[12] Z. Szadkowski, “Optimization of the detection of very inclined showersusing a spectral DCT trigger in arrays of surface detectors,” IEEE Trans.Nucl. Sci., vol. 60, no. 5, pp. 3647–3653, Oct. 2013.

[13] Z. Szadkowski and K. Pytel, “Artificial neural network as a FPGA triggerfor a detection of very inclined air showers,” IEEE Trans. Nucl. Science,vol. 62, no. 3, pp. 1002–1009, Jun. 2015.

[14] Z. Szadkowski and A. Szadkowska, “FPGA based wavelet trigger inradio detection of cosmic rays,” Braz. J. Phys., vol. 44, pp. 805–810,Dec. 2014.

[15] Z. Szadkowski, “An optimization of the FPGA based wavelet trigger inradio detection of cosmic rays,” IEEE Trans. Nucl. Sci., vol. 62, no. 3,pp. 993–1001, Jun. 2015.

[16] Z. Szadkowski, “Front-end board with Cyclone V as a test high-resolution platform for the Auger_Beyond_2015 front end electronics,”IEEE Trans. Nucl. Sci., vol. 62, no. 3, pp. 985–992, Jun. 2015.

[17] Z. Szadkowski, “A prototype of readout electronics based on anunified Altera platform for underground Muon counters triggered bysurface detectors,” IEEE Trans. Nucl. Sci., vol. 61, no. 2, pp. 720–726,Apr. 2014.