Proposed full upgrade of the L1CTT/DFE for Run2b

Meenakshi NarainBoston University

Outline:ProposalReviewers comments and responses

Schedule Cost

Groups

Simulation and Algorithm development: Kansas:

Graham Wilson, Carsten Hensel Manchester:

(Liang Han), Terry Wyatt Notre Dame:

Mike Hildredth

Hardware: Boston University:

Meenakshi Narain, Eric Hazen, Ulrich Heintz, Shouxiang Wu

FNAL: Marvin Johnson, Stefan Gruenendahl, Jamieson

Olsen

Physics Motivations

In Run IIa and RunIIb, the L1 Track Trigger Provides:

CFT tracks for L1Muon Seeds High pT isolated track trigger

capability CFT tracks +CPS clusters

embryonic electrons Found tracks for STT

As instantaneous luminosities hit 1e31, the rejection for L1CTT drop drastically.

Accomplish high eff with low fake rates using the full granularity of the CTT.

Run IIb

Run IIa

L1CTT architecture

A multistage system Analog Front End (AFE):

Signals from the tracker Mixer

Sort signals in trigger sector wedges

Digital Front End (DFEA): Track Trigger logic

Octant (CTOC) Combine track informationfrom several DFE boards

CTTT Construct track trigger terms

Trigger Manager Construct 32 AND/OR terms used by the L1 Trigger Framework in forming the trigger decision

L1CTT architecture

The approved upgrade: Replace Digital Front End daughter boards (DFEA):

Need to fit 3-5 times more equations

Rebuild these cards using larger FPGAs

Use XC2V6000 Chip with 6Msystem gates

Lessons from Run2a commissioning

Current “approved plan” upgrade of the DFEA daughter boards. Provide a 5x increase in available logic resources.

With the experiences over the last year while commissioning the present Run2a L1CTT system a few very important issues have surfaced:

Lack of extensive testability of the input and output information from the DFEA

Excessive Firmware download times Clock and SCL signal distribution (dependent on

upstream info) A shortage of spare backplanes (there are none)

The Proposal

In order to add testability to the system, one needs: A new motherboard (daughterboard – to a less extent)

design. Improved diagnostics: input and output buffers, and L3

capability.

To alleviate colossal download times we need: A new DFE crate controller with a

faster connection to the D0 online computers. A new DFE backplane design

Utilize this chance to put the cables to the back. No transition boards.

Also an independent SCL/clock distribution This decouples us from upstream downtime issues

Detailed specifications of the boards are being developed – follow links through the Run2 trigger web page

Installation & Commissioning

The proposed scheme is driven by the desire to enhance testability and reliability of firmware downloads.

Eases Installation and commissioning

Plan to run on a partial crate of the new system on the platform in parallel with the existing DFEA using LVDS splitters

Key to successful commissioning of the new system

Advantages: Can assemble the whole crate outside of the collision hall Extensive testing of the entire chain possible before putting in

collision hall Enhanced testing capability Use Run2a Data derived test vectors to verify

Take out the old crates (2 of them) and replace them with new ones

Use the existing Run2a tools used for commissioning Low level changes will be transparent at user level

Rescope Review

Review held on April 5th, 2004 Reviewers: Darin Acosta (UF), John Anderson, Bill Freeman (Fermilab)

Charge to the Committee: Total cost of the new CTT project.

Is the total cost well estimated and does it include all necessary cost for materials and labor?

Contingency: What is the uncertainty of the cost estimate (i.e. how much

contingency should be assigned to the total cost of the project?) Schedule of the new CTT project:

Is the schedule reasonable and attainable? Milestones and tracking.

Are there sensible milestones in the schedule that will allow management to track the progress of the project?

Technical Feasability: Is the project feasible as described and does it optimize

installation and commissioning time? This is not a detailed technical review – more of an overall “sanity check”.

Responses to Review Comments

DFEA chip decision, merging MB/DB Already received gift from Xilinx for all XC2V6000 Start with merged scheme (shorter prototype

cycle) and split later on if needed

48V power distribution system: Q: “The risk is that the system may induce

EMI effects to other sensitive D0 electronicsA: In the last 2 months done various tests – no affects

seen on the calorimeter. Add shielding as a precaution.

Responses to Review Comments

Crate Controller: Question design choice, suggest to use commercial

solutionsA: We had explored commercial off the shelf choices.

– These solutions will force use of either a VME or PCI interface, adding complexity to the DFE as well.

– Radiated EMI from the various busses on a commercial board presents a risk to cal electronics

Considered a hybrid solution for the CC using commercial VME module like L3 single board computer and a PMC for SCL.

Rejected:– as simple hardware transforms into a large software project

(including a new driver). – Software to implement SCL interrupts is a good bit of effort.– 16 MHz VME clock would require noise shielding.

Our design is based on the CMS/OSU design – a commercial solution– Send raw Ethernet frames over a gigabit linkSend raw Ethernet frames over a gigabit link– Reuire PC/Linux software effort (drivers available)Reuire PC/Linux software effort (drivers available)– Use OSU’s VirtexII Pro rocket I/O transceiver (dedicated EE at Use OSU’s VirtexII Pro rocket I/O transceiver (dedicated EE at

OSU)OSU)– Also add bi-directional parallel port interface to board.Also add bi-directional parallel port interface to board.

Responses to Review Comments

DSAT: DFEM/DFEA tester Question investment of resources in this

project, suggest this be merged into DFEM itself. And include SCL on the tester.A: We need a tester somewhere. In the two weeks after the review we invested in a

tester schematic. We are holding sending it for layout.

However, we do prefer a standalone tester – as something is needed to test/ power-up DFEA/DFEM prior to the setup at FNAL is put together.

We feel this is not a huge resource investment, while merging it with DFEM/DFEA will put unnecessary load on the one engineer doing the DFEA work and may delay the DFEA (which we want to avoid at any cost).

Responses to Review Comments

Test Setup: Q: A lot of value put it in the in-situ tests of the

upgraded L1CTT system during this year’s shutdown, need to develop contingency plans to test outside of the collision hall in case this is not met A: We do have a plan for test-setup outside the collision

hall For the Fall 2004 shutdown all we really need to achieve

is installation of a new crate with power supply and backplane, the LVDS cable extensions and the splitters. – Crate controller and DFEA prototypes can be added (and

replaced) during shorter accesses later. However, we still know from experience with the

current system that there is no replacement for testing in situ, especially with regard to timing.

COST & SCHEDULE

Schedule Aggressive but manageable schedule Needed to start the project at yesterday’s

timescale Timing of anticipated “Fall Shutdown” – and

“CTT slice test” a critical ingredient for ensuring success

Milestones

Milestone Date

Backplane and Crate Controller

Milestones

Milestone DateDFEA/DFEM/ FPGAs

Production

Fall 2004 Slice Test

Fall 2004 shutdown: Aug 23rd and lasts 13 weeks.

During this time plan to run a partial crate of the new system on the platform in parallel with the existing DFEA using LVDS splitters.

Plan to assemble the partial test crate prior to the shutdown. Extensive testing of the entire chain

possible before putting in collision hall

Readiness for Slice Test

Crate Controller:

Readiness for Slice Test

Backplane:

Readiness for Slice Test

Optical DCL/LVDS Splitters:

Readiness for Slice Test

DFEA/DFEM:

L1 CTT Cost Change

WBS # Description Original Cost Revised Cost M&S FNAL Labor

1.2.3 Level 1 Tracking $870,182.47 $1,131,251.00 $114,228.53 $146,840.00

1.2.3.1

Prototype L1 Central Track Trigger Algorithm Coded And Simulated With FPGA Simulation Tools $0.00 $0.00 $0.00 $0.00

1.2.3.2 Develop Target CTT Algorithm $85,272.00 $85,272.00 $0.00 $0.00

1.2.3.3

Target L1 Central Track Trigger Algorithm Firmware Coded And Simulated With FPGA Simulation Tools $0.00 $0.00 $0.00 $0.00

1.2.3.4 Develop Test Procedures $1,550.40 $1,550.40 $0.00 $0.00

1.2.3.5 DFEA Preproduction I $106,347.60 $106,347.60 $0.00 $0.00

1.2.3.6 DFEA Preproduction II $51,620.12 $49,433.00 ($2,187.12) $0.00

1.2.3.7 DFEA Production $625,392.35 $537,370.00 ($67,742.35) ($20,280.00)

1.2.3.8 DFEA Backplane (BP) N/A $33,617.00 $15,137.00 $18,480.00

1.2.3.9 DFEA Crate Controller (CC) N/A $96,656.00 $7,096.00 $89,560.00

1.2.3.10 optical DCL (host) N/A $41,480.00 $0.00 $41,480.00

1.2.3.13 LVDS Splitter N/A $20,438.00 $2,838.00 $17,600.00

1.2.3.11 DFE Motherboard (DFEM) N/A $140,449.00 $140,449.00 $0.00

1.2.3.16 DSAT - DFEM/DFEA Standalone Tester N/A $18,638.00 $18,638.00 $0.00

  All costs are shown in FY02 Dollars     $114,228.53 $146,840.00

L1 CTT Cost Change

Total Cost change to DOE MIE: (escalated, burdened, etc)

Original Revised ChangeCost Cont. Cost Cont. Cost Cont$361k $155k $717k $186k $356 $31k

Total difference in cost + new contingency estimate is: $387k

This cost will come from the management reserve.

Lowers the contingency on the total project cost from 33% to 30%.

Have not touched money earmarked for AFE II upgrade