1 atlas lehman review, silicon rod douglas ferguson, richard jared, john joseph and lukas tomasek...

1

ATLAS Lehman Review, Silicon ROD

Douglas Ferguson, Richard Jared,

John Joseph and Lukas Tomasek

Wisconsin

May 21 to May 23, 2003

2


Material Covered

ROD Schedule and High Level Summary

Scope

Major Events

ROD Overview

Current Status

3

ATLAS Lehman Review, Silicon RODROD Schedule and High Level Summary

ID WBS Task Name Start Finish

1 1.1.3 Wed 1/31/01 Wed 2/2/05

2 1.1.3.3 Design ROD Cards Sat 2/1/03 Fri 3/28/03

3 Pixel specific Formater VHDL Sat 2/1/03 Fri 3/28/03

4 1.1.3.4 ROD Test Stand Wed 1/31/01 Tue 3/26/02

5 1.1.3.4.3 SCT/Pixel Test Stand Software Wed 1/31/01 Tue 3/26/02

6 1.1.3.8 ROD Fabrication Fri 10/4/02 Thu 4/15/04

7 1.1.3.8.1 Production Model II ( 10% of SCT) Tue 10/22/02 Mon 6/16/03

8 Production Model II Fab/Parts Ordering Tue 10/22/02 Mon 1/13/03

9 Production Model II Debugging Wed 1/15/03 Tue 2/11/03

10 1.1.3.8.1.3 Pixel User Evaluation Mon 3/3/03 Thu 5/29/03

11 1.1.3.8.1.4 SCT User Evaluation Wed 1/1/03 Thu 4/3/03

12 1.1.3.8.1.5 Pixel ROD FDR ATLAS Fri 5/30/03 Fri 5/30/03

13 1.1.3.8.1.6 Pixel ROD PRR ATLAS Mon 6/16/03 Mon 6/16/03

14 1.1.3.8.1.7 SCT ROD, BOC, TIM FDR ATLAS Fri 4/4/03 Fri 4/4/03

15 1.1.3.8.1.8 SCT ROD, BOC, TIM PRR ATLAS Mon 4/21/03 Mon 4/21/03

16 1.1.3.8.2 ROD SCT Production Tue 4/22/03 Mon 10/20/03

17 1.1.3.8.2.1 ROD SCT Production Fabrication Tue 4/22/03 Mon 7/28/03

18 ROD SCT Prod Fab Mat'l/Labor Tue 4/22/03 Mon 7/28/03

19 ROD SCT Production Debugging Tue 7/1/03 Mon 10/20/03

20 ROD SCT Production complete Mon 10/20/03 Mon 10/20/03

5/30

6/16

4/4

4/21

10/20

O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A

2003 2004 2005

4

ATLAS Lehman Review, Silicon RODROD Schedule and High Level Summary

ID WBS Task Name Start Finish

21 1.1.3.8.3 ROD Pixel Production Tue 6/17/03 Mon 3/1/04

22 ROD Pixel Production Fabrication Tue 6/17/03 Mon 9/15/03

23 ROD Pixel Production debugging Tue 8/19/03 Mon 3/1/04

24 ROD Pixel Production Complete Mon 3/1/04 Mon 3/1/04

25 1.1.3.8.4 Purchase ROD Crates Fri 10/4/02 Thu 4/15/04

26 Purchase ROD SCT Crates Fri 10/4/02 Thu 4/15/04

27 Purchase ROD Pixel Crates Mon 9/1/03 Wed 4/14/04

28 1.1.3.9 ROD Shipping, Installation and Repair Wed 2/12/03 Wed 2/2/05

29 1.1.3.9.1 ROD Installation and Repair Wed 2/12/03 Wed 2/2/05

30 ROD Install & Repair FY03 Mat'l/Labor $sWed 2/12/03 Wed 2/11/04

31 ROD Install & Repair FY04 Mat'l/Labor $sWed 10/1/03 Thu 9/30/04

32 ROD Install & Repair FY05 Mat'l/Labor $sFri 10/1/04 Wed 2/2/05

33 ROD Installation/Final commissioning completeWed 2/2/05 Wed 2/2/05

34 1.1.3.9.2 ROD Shipping Thu 6/12/03 Thu 6/10/04

35 ROD Shipping FY02 Mat'l/Labor $s Thu 6/12/03 Thu 6/10/04

36 1.1.3.10 Project Management Wed 10/2/02 Tue 9/16/03

37 Proj Mgmt FY03 Mat'l/Labor $s Wed 10/2/02 Tue 9/16/03

3/1

2/2

O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M A

2003 2004 2005

5


High Level Summary

9% of SCT production RODs complete

Authorized Funds Paid to Date/Committed W Totals($) M&S($) Labor($) Travel($) Totals($) M&S($) Labor($) Travel($1.1.3RODs 1905.5 1486.0 407.5 12.0 167.7 51.5 112.2 4.0

Milestones ETC02 Actual Projected1.1.3.8.1.6 Pixel ROD PRR 6/03 9/031.1.3.8.1.8 SCT ROD PRR 4/03 6/031.1.3.8.2 SCT Prod Complete 10/03 11/03 1.1.3.8.3 Pixel Prod Complete 3/04 3/04 1.1.3.8.4 Pur ROD SCT Crates 3/03 9/02 Two crates have been ordered1.1.3.9.1 Installation complete 2/05 2/05

6


ROD Comparison of Costs

Small call on contingency

The ETC03 had a +48K difference from ETC02 (pixel included). This is in the noise. The pixel ROD have been added to the baseline.

7


Scope

Baseline Scope ATLAS ROD

Pixel test beam support hardware

Design to production models SCT and pixel RODs

Development of ROD test stand

Fabrication of SCT RODs

Fabrication of pixel RODs

Purchase of SCT and pixel ROD crates

Management Contingency

allocated

8

ATLAS Lehman Review, Silicon RODMajor Events 1. Fabrication and debugging of 9 RevC production model RODs has been performed.2. Successful testing of data flow through the RevC production model ROD at Cambridge.3. The test stand software has been upgraded to support pixel user evaluation. 4. DSP software has been coded to support user evaluation of SCT and pixel user evaluation. 5. The ROD passed the US ATLAS FDR. 6. SCT user evaluation has had much effort in the UK. They have configured modules, readout events and histogrammed the the data. 7. The ATLAS PRR was held on May 13, 2003. The chairman Philippe Farthouat has said that we passed (report on May 23). 8. Pixel user evaluation has started. Modules have been configured. Testing of events readout is currently being done . 9. The pixels will require a revised ROD (no changes to VHDL of C code. FPGAs will be changed to provide more derandomizing space and the DSPs will be changed to provide more memory space.

9


Pixel or Si Stripfront end electronics Clock, Trig. and control

Event data and register contents

Clock and trigger ATLAS

Clock, Trigger& event number

ATLAS Read-outBuffers

Formatted Event Data

Local processing

Event Data,Registers andStatus

Configurationand Control

ROD

Preprototype

Card

1.0

BUSY

XON/XOFF

Distribution

RCCTIM

Back of Crate Card

ROD Overview

10


1 of up to 16 shown

PreprototypeROD

1 of up to 16 shown

Tim

Distribution

RCC

Processor

Trans Data

Rec. Data

DC Power

R/W Bus

CLK,Trigger &Control

Event data &RegisterContents

ATLASEther net

VME

VME

VME,configuration,Control andMonitoring

1.11.2

ROBData

ROD data

ATLASClk &Trigger

XON/XOFF

XON/XOFF

Clock, Trigger,ids, resets, CAL

Rec. CLK

BUSY BUSY

1.0 ROD Card

Back of Crate

Tran CLK

Bias

Card

Optical Interface

Clock and Trigger

Crate

11

ROD Implementation: FPGA Front End / DSP Back End

96Data RCVR

ROD Controller: Operation,

Command, Trigger

43

FPGA Formatter and EFB96

Bac

k of

Cra

te C

ard

/ Fro

nt E

nd E

lect

roni

cs

Input Memory & XCVR Control

VME Slave Interface

Sli

nk o

n B

OC

VM

E B

us

32

Cont / Addr

Data Bus

Con

trol

, Sta

tus,

Mas

k

Trig Data

Tim

ing

Inte

rfac

e

Mod

ule

BOCSetup Bus

Serial Output Link

Serial Input Link

Xon/Xoff

ROD Busy

Dec

. Tri

g C

ount

Control & Status for Block Xfer/ DMA / S-Link

DSP Event Trapping,

Histograms (Four DSP Chips)

DM

A 1

Router FPGAS-Link & DSP DMA

Host Port Interface

Data Valid

S-link Data

DM

A 4

Router Halt Output

DMA Cont.

Eve

nt D

ata/

Tir

g T

ype

R/W Bus

R/W Bus

R/W Bus

...

Bus

y

Program ResetManager

Tok

en a

nd D

ynam

ic M

ask

S-link Data

Boot FPGAs

12


controllerFPGA

bootFPGA

program FLASHpower suppliesslave DSPs & memories

routerFPGAs

eventbuilderFPGA

debug &derandomizingmemories

ROD busbuffers

data formatter FPGAs real time data path

master DSP& memory

13


Current Status (ROD hardware)The data and control path in the ROD have been coded and tested. The SCT VHDL is in maintenance mode (changes implemented as needs are found). C code is mostly written for module calibration.

EFB FPGA VHDL in maintenance mode

Router FPGA VHDL in maintenance mode

Formatter FPGA SCT VHDL in maintenance mode The formatter for the pixel 40MHz links has been written. The formatter VHDL for the pixel 80Mhz links has to be written.

Controller FPGA VHDL in maintenance mode

Program Reset Manager VHDL in maintenance mode

14


Current Status (ROD software )The infrastructure software is written and tested. Many primitives have been written and tested. The code needed for module calibration has been written. The code is being upgraded as the user evaluation needs are defined from use.

The histogramming code will need (for speed) to be executed in machine language once the full functionality is tested by the users. Queuing of events in the FIFOs will also be needed to improve the speed.

Future needs:

histogram fitting,occupancy measurement,error detection, counting & processing.

15


Current Status (Test Stand Software )

The test stand software is complete and is in maintenance mode. Updates are made as needed.

Concerns:

The upgrade to the pixel ROD has some risk because the FPGA and DSP pins will change and the printed circuit card will be rerouted. Three boards will be initially fabricated to reduce risk.

The time to calibrate the SCT modules is too long. The C code will have to be changed to improve speed.

16


US ATLAS FDR (short version)Review Committee Report

for the SCT & Pixel ROD US-ATLAS FDR of 20-Aug-2002Introduction:A Final Design Review of the Read Out Driver (ROD) intended for use by the ATLAS SCT and Pixel subsystems was held at LBNL on 20-Aug-2002. The primary intent of this review was to evaluate the present status of the ROD design for use by the SCT and Pixels with a goal of approving the build of 9 pre-production RODs so that further user evaluation can be obtained prior to a Production Readiness Review (PRR) as early as possible. While the goal is for the ROD design to be compatible for both SCT and Pixels, it is recognized that there has been as yet no user evaluation of the design with Pixel readout. Given the schedule constraints of the SCT and the overall cost constraints of the project, the plan is to evaluate the appropriateness of the present ROD design for both SCT and Pixels and decide on the pre-production build knowing that further design modifications may be needed for Pixels after appropriate user evaluation is completed with that readout.

The review committee included: Christopher Bebek – LBNLKevin Einsweiler – LBNLAlex Grillo – UCSCHenrik von der Lippe – LBNLAbe Seiden - UCSC

17


US ATLAS FDR (continued)Recommendations:The committee was impressed with the amount of work accomplished since the last review. Basic functionality

of the SCT ROD has been shown along with the first examples of on-board histogramming. While more user evaluation is still needed to be sure that the ROD meets all requirements for SCT and Pixels, this is at least a good start. The major obstacle to such evaluation is software for a DAQ system and focused attention of users running the ROD through all its necessary operations. User evaluation of the Pixel version is awaiting a Pixel module to readout.

The committee makes the following recommendations to the ROD design group:1. It is clear that the ROD development is ready to proceed to implement all identified design changes into a

new PCB layout. Also, more ROD units will enhance the chances to obtain more, much needed, user evaluation. Therefore, the committee strongly recommends that the design group proceed as quickly as possible to complete layout of the next version of the ROD PCB and fabricate sufficient boards to eventually load 9 boards. Given the changes to be made to the layout, we recommend that initially only two boards be loaded and tested. The committee at first thought that only one should initially be loaded for testing but we have accepted the advice of the ROD design team that the initial debug of two boards facilitates diagnosis of problems by having two samples to compare. Once basic functionality is demonstrated, the remaining 7 boards can be loaded.

Note: 3 then 7 boards have been fabricated. These boards have been used in user evaluation. 2. The alternate footprint (BGA456 in addition to BGA676) for the Formatter on pixel boards should not be

executed. We believe that this creates too big a risk in the layout for an option that may not be needed or used. If the planned Formatter FPGA proves to have insufficient performance for the Pixel readout, one of the higher performing FPGAs with the same footprint can be utilized (405/EM or 600/E). The added cost in that case is a worthwhile risk instead of the risk to the layout now.

Note: The change was not executed.

18

ATLAS Lehman Review, Silicon RODUS ATLAS FDR (continued) 3. The allowable power supply margins of the FPGAs and DSPs should be analyzed vs. the regulation spec

of the DC-DC converter over the temperature and supply range expected for the board. If it is not convincing that a single supply will work reliably for long-term use, the power plane should be split and two DC-DC converters should be employed. While splitting the power plane does imply some risk, care at this time to avoid errors may be a better risk than possible reliability problems over the lifetime of the ROD.

Note: An analysis was performed and no change was made. 4. Attempt to optimize the FPGA VHDL code as planned to reduce the utilization. At the time of the PRR,

the utilization must be reviewed again. If it is still uncomfortably high to accommodate possible needed enhancements during commissioning, a decision will have to be made then to upgrade to the high cost FPGA.

Note: FPGA utilization is now Formater 70%, EFB 56% Router 67%, Controller 70% and PRM 50%. 5. A discussion should be held with the person now re-designing the S-link board for ATLAS reviewing the

ROD’s S-link interface vs. those of the S-link to confirm that there are no compatibility issues. Note: The S-link was reviewed with CERN and has been tested as part of the user evaluation.6. There seem to be legitimate concerns about how the detailed phase of the data streams from the on-

detector electronics will be monitored and timed relative to the TIM-generated crossing clock. Since the BOC is the module responsible for this synchronization, it is suggested that this is the correct place to implement monitoring of the timing of the “raw” data streams. The SCT/Pixel off-detector design group should consider how this may best be implemented in the final system. However, we suggest that the pre-production RODs should simply provide a reasonable number of header-based test point, close to the FPGAs, for logic-analyzer analysis, but should not attempt to provide multiplexed front-panel access to the full set of input data streams.

Note: The BOC supports passing the raw data to the ROD. A multiplexer has been added to the Formater FPGA to allow selection of one stream at a time for monitoring on a front panel connector. Module timing is performed by varying the BOC timing and observing the results with the ROD.

19


US ATLAS FDR (continued) 7. Since the current ROD design does not meet the requirement to readout events via VME at 1kHz with a

full 16 RODs/crate and it would require a major redesign to try to accomplish this, but it appears that one ROD/crate would be sufficient for any planned ROD use, the committee recommends that this requirement be changed to “VME readout at 1kHz for 1 ROD/crate.”

Note: The requirement was changed to 100 Hz. Current test indicate that the ROD can be read out at full luminosity at 300 Hz.

8. The SCT community is strongly urged to devote more resources to write the necessary DAQ software so that more detailed user evaluation can be performed. To expedite this work, it is recommended that one of the existing RODs be sent immediately to Oxford to allow that group to begin learning to use the ROD.

Note: The SCT community now has a large effort devoted to user evaluation and preparation for the ROD use at the macro assembly sites.

The committee wishes to thank the ROD design group for their time preparing for this review and presenting a

clear status of the project. Those members of committee who are members of the ATLAS collaboration would also like to thank Christopher Bebek and Henrik von der Lippe for their contribution to ATLAS by spending the time to review this complex development.

1 atlas lehman review, silicon rod douglas ferguson, richard jared, john joseph and lukas tomasek...

Documents

silicon rodatlas lehman

pixel rod prr

sct rod prr

silicon rodcurrent status

revised rod

silicon rod high level

pixel user evaluation

pur rod sct crates