The CCB, TTC and so The CCB, TTC and so forthforth

Paul Padley, Mike MatveevPaul Padley, Mike Matveev

Current CCB Status

5 CCBs were built in 2001 and distributed 15 more sets of boards have been

manufactured 10 of these boards have been stuffed and

tested Remaining boards will be built in April..May

CCB Plans

Current board is “Preproduction Prototype” However we will not consider it fully tested

until it has been used in a structured test beam with TTC system.

At this time we are still getting firmware change requests and we will for some time.(perhaps even in 2008?)

TTC Software Status

Karol Bunkowski of Warsaw visited and installed and wrote software

Windows GUI application Console application A Rice graduate student is learning the

code by porting to LinuxA point of discussion, should the TTC be in a

separate 6U crate, as this is how it will be in the end. (ie don’t port to Dynatem?)

CCB, Phos 4, TTCrx

CCB Irradiation Test

• Conducted at the UC Davis 63 MeV proton beam facility in January 2002 by J. RobertsJ. Roberts with Rice undergrad

• Altera EPF10K100ABC356 PLD (uses LUT for logic functions and SRAM configuration elements) and EPC2 EPROM under test

• Both devices were accessible remotely over long JTAG cable Mezzanine card with PLD and EPROM

Test Information

Neither TTC, nor VME systems were involved in a test

Design under test was running at 40 Mhz (dynamic mode)

16-bit Pseudo-Random Bit Stream (PRBS) generators were used to produce test patterns

Two streams of test patterns passed through a pipeline registers and were compared at the end

Error signal was transmitted to control room for counting

Test Results PLD

Altera PLD was irradiated with a dose up to 1 kRad

20 SEU were observed, with an average dose of 50 Rad to get an error

Cross Section of SEU ~2.5x10-9 cm2 (consistent with ALCT test results obtained in 2000)

No Single Event Latch-ups

Test Results EPROM

Altera EPROM was irradiated with a dose up to 26 kRad

No errors in read-back configuration up to 25.6 kRad

Read-back failed at 25.8 kRad

Cross Section of SEU < 5x10-12 cm2

Interpretation

SE upsets in SRAM configuration seem to be the dominant effect during PLD irradiation (consistent with other published results). All errors were recoverable by remote configuration from EPROM. No power cycling required.

For Altera EPF10K100ABC356 the worst case SEU rate at full LHC luminosity would be

2.5x10-9 cm2 x 1011 neutrons/cm-2 / 5x107 sec = 5x10-6 sec-1

(one SEU per Altera PLD in ~60 hours)

Report is available on the web at:http://bonner-ntserver.rice.edu/cms/projects.html

Solution 1

Reload Altera PLD from EPROM periodically (just like everything else in crate)

Altera can decode “Hard Reset” and issue to itself, after it has issued it to the crate

Configuration time ~120 milliseconds

Issues with Solution 1

CCB inactive during reconfiguration Some minor modifications needed on

CCB and mezzanine cards

Solution 2

Use existing UCLA-built mezzanine card based on Xilinx XCV600E

Faster reconfiguration (25 ms) SEU rate one order of magnitude

less? (based on OSU and UCLA tests)

Issues with Solution 2

Significant changes in CCB main board

Must convert firmware to Xilinx from Altera

Still need to reconfigure periodically

Solution 3

Build new mezzanine card based on one time programmable fpga (Actel A54SX32 for example)

Expected SEU rate is three orders of magnitude less than Altera (based on tests for Atlas)

One SEU per fpga in 5 years

Issues with Solution 3

Not reprogrammable so when do you port a design into antifuse version?

New mezzanine design (6 months engineering)

Suggestion from Gena• Use PLD for start of running at low luminosity.

Switch to non-reprogrammable logic when luminosity increases

(Costs ~$100,000 to make such a switch)

PHOS4 features

•CERN designed 4-channel delay ASIC with 1 ns delay precision•Radiation Hard•Programmable (write-only) over I2C bus

Issues with PHOS4

The PHOS4 is intended for use with non-interruptable clock source (since the chip utilized DLL circuitry)

If clock is interrupted, then the behavior is unpredictable wrong delays “dead-looking” channels no direct way to identify what is wrong with the chip Only power cycling can help

Lack of dedicated “reset” pin Delay settings are not readable back via I2C bus

We have suggested that the chip be redesigned

Clock Interruptions?

In our opinion, the clock should never be interrupted (except on power cycling)

TTC tree is designed so clock won’t be interrupted

A question to CMS/LHC is “How often can we expect the clock to be interrupted?”

Interruptions

What to do if interruptions are a problem?

PHOS4 can be simply removed from the CCB boards (all 4 socketed) and clocks wired directly to LVDS transmitters

no individual slot-to-slot and module-to-module clock adjustments

no extra effort to program I2C same PCB layout, minimal on-board

changes

Another Question

All clock lines in the custom peripheral backplane are point-to-point LVDS of the same length.

Do we really need to adjust the 40Mhz clock from slot to slot and from module to module?

Another Alternative

If fine clock adjustments on the backplane are still needed

PHOS4 can be replaced with another device, for example, 3D7408 proposed by OSU

1-channel commercial CMOS programmable delay chip

This device has 45/55 output duty cycle instead of 50/50 at 40Mhz (tested at Rice). Is it acceptable for DMB/TMB/ALCT?

layout changes on a CCB board required Radiation tolerance?

TTC IssuesTTC Issues

Trouble ReportedTrouble Reported

At the last CMS week (and At the last CMS week (and continuing) reports from many continuing) reports from many groups that they can not drive optical groups that they can not drive optical links with a TTC derived clocklinks with a TTC derived clock

(Sorry, but nobody [but us] is (Sorry, but nobody [but us] is documenting what they are doing, so documenting what they are doing, so I can’t point you to anything written I can’t point you to anything written on the topic)on the topic)

Our report is at Our report is at http://bonner-ntserver.rice.edu/cms/jitter.pdfhttp://bonner-ntserver.rice.edu/cms/jitter.pdf

The IssueThe Issue

TTC group has specified that they TTC group has specified that they will deliver clock good to 50ps rms.will deliver clock good to 50ps rms.

Groups using CERN GOL Groups using CERN GOL (not us)(not us) claim TTCrx can’t drive their linksclaim TTCrx can’t drive their links

Reports of jitter measurements Reports of jitter measurements ~500ps~500ps

At last CMS week microelectronics At last CMS week microelectronics group agreed to improve TTCrx group agreed to improve TTCrx ( time scale?)( time scale?)

Our MeasurementOur Measurement

Mezzanine Card ECP680-1102-630C ECP 680-1102-610B

TTCrx ASIC operating voltage +5.0V +3.3V +5.0V

Clock40Des1 jitter, ps (no BC, no L1A) 153 170 330

Clock40Des1 jitter, ps (BC commands + L1A) 183 215 360

New TTCrx Old TTCrx

Our TestCCB

TTCVI

BIT3

TTCVX

OPTO

OPTO

TTCrx

ERROR

1 m

100 m

100 m

VME 9U VME 6U

COPPER CABLE OPTICAL CABLE

• OLD AND NEW TTCrx BOARDS WERE TESTED WITH 40.00 Mhz CLOCK SOURCE FROM TTCvx MODULE• 40.00 Mhz CLOCK WAS MULTIPLIED BY 2 BY AV9170 CHIP• NO ERRORS OBSERVED IN PRBS TEST FROM ONE OPTOBOARD TO ANOTHER AT 80.00 Mhz (BER < 10-13 c-1)

40 Mhz

Clockmultiplier

PC•

•

•

•• •

•

••••••

Our ResultOur Result

Clock jitter is lower for the newest TTCrx Clock jitter is lower for the newest TTCrx ASIC (Version 3.1, 12/01)ASIC (Version 3.1, 12/01)

Jitter increases if broadcast commands and Jitter increases if broadcast commands and L1A are transmitted from the TTC systemL1A are transmitted from the TTC system

Jitter distribution for ASIC Ver.3.1 looks Jitter distribution for ASIC Ver.3.1 looks “normal” (unlike previous version)“normal” (unlike previous version)

Clock jitter is lower if the new ASIC is Clock jitter is lower if the new ASIC is powered from +5Vpowered from +5V

Jitter introduced by any of two TTCrx ASICs Jitter introduced by any of two TTCrx ASICs and other components in the clock and other components in the clock distribution circuitry at our testing setup is distribution circuitry at our testing setup is tolerable for TLK2501 transceivers operating tolerable for TLK2501 transceivers operating at 80.00 Mhz using prototype MPC-SR linkat 80.00 Mhz using prototype MPC-SR link

DAQMB testDAQMB test

A further test is interesting: A further test is interesting: driving EMU DAQ link with TTC driving EMU DAQ link with TTC derived clock.derived clock.

Requires TTCrx, TTCvi, TTCvx Requires TTCrx, TTCvi, TTCvx and softwareand software

TTCrx Mezzanine Card Status

• Only one new board (borrowed from Wisconsin) is available so far for EMU electronics. Being used for CCB testing at Rice

• Some TTCrx changes have been requested from CERN designers:

- low profile connectors (otherwise the board doesn’t fit the CCB)

- better ID/reset scheme (separate SubAd[7..0] and Data[7..0] output busses from ID input bus)

• Next versions of the TTCrx ASIC and mezzanine card are expected (later in 2002?- 2003?) with lower clock jitter and other improvements

TTCvi and TTCvx Status

• One set of TTCvi and TTCvx modules is being used for CCB testing at Rice

• Originally designed at CERN for Atlas. Apparently, CERN will build the next version of the TTCvi. Several CMS-specific requirements have been defined for the TTCvi upgrade. Also, the TTCvx should provide 80.16Mhz clock source rather than 80.00Mhz

ConclusionConclusion

There are many issues that EMU will need There are many issues that EMU will need to address as a group that impact to address as a group that impact

integration in the peripheral crate and with integration in the peripheral crate and with the TTC systemthe TTC system