17/02/06h-rorckip heidelbergtorsten alt the new h-rorc h-rorc
DESCRIPTION
17/02/06H-RORCKIP HeidelbergTorsten Alt H-RORC H-RORC : HLT Read-Out Receiver Card Tasks: –Receiving of the raw detector data –Injecting the data into the main memory of the hosts of the HLT framework –Online processing of the data in hardware –Sending processed data out of the HLT Implementation: –PCI card 64bit/66MHz with XILINX Virtex4 –2 (half) Common Mezzanine Connectors (CMC) –2 fast serial links : TAGNET –DDR-SDRAM / ETHERNET / FLASHTRANSCRIPT
17/02/06 H-RORC KIP HeidelbergTorsten Alt
The new H-RORC
H-RORC
17/02/06 H-RORC KIP HeidelbergTorsten Alt
H-RORC Prototype
• PCI 3.3 V form factor• 12 layers, 6 routing layers• 2 SIU/DIU connectors
17/02/06 H-RORC KIP HeidelbergTorsten Alt
H-RORC• H-RORC : HLT Read-Out Receiver Card
• Tasks:– Receiving of the raw detector data– Injecting the data into the main memory of the hosts of the HLT
framework– Online processing of the data in hardware– Sending processed data out of the HLT
• Implementation:– PCI card 64bit/66MHz with XILINX Virtex4– 2 (half) Common Mezzanine Connectors (CMC)– 2 fast serial links : TAGNET– DDR-SDRAM / ETHERNET / FLASH
17/02/06 H-RORC KIP HeidelbergTorsten Alt
H-RORC Details
• RORC functionality– Xilinx Virtex4 LX40 FPGA– “SecureConfiguration” with FLASH memory and CPLD– PCI interface with 64 bit / 66MHz – XILINX LogiCORE– 2 x half CMC interface to two SIU/DIU cards– fast serial links to connect multiple RORCs
• Additional functionality– 4 independent DDR-SDRAM modules with up to 1Gb– Optinal Xilinx Prom for standalone programming– “Ready for Linux” (Ethernet interface, RS232, FLASH memory)
17/02/06 H-RORC KIP HeidelbergTorsten Alt
Overview
XILINXVIRTEX4
LX40
DDR-SD
DDR-SD
DDR-SD
DDR-SD
TAGNET
USER-FLASH
CFG-FLASH
PLATFORMPROM
XC95144CPLD
CM
C-C
onnectorC
MC
-Connector
Power 1V2
Power 2V5
Power 1V8
TAGNET
RS-232
ETH-PHY
PCI-66/64 PCI-Power 3V3
Memory Configuration
Serial links
LVDS links
PO
WE
RC
MC
-J11/J22
OSC
OSC
17/02/06 H-RORC KIP HeidelbergTorsten Alt
Overview: RORC Functionality only
XILINXVIRTEX4
LX40
DDR-SD
DDR-SD
DDR-SD
DDR-SD
TAGNET
USER-FLASH
CFG-FLASH
PLATFORMPROM
XC95144CPLD
CM
C-C
onnectorC
MC
-Connector
Power 1V2
Power 2V5
Power 1V8
TAGNET
RS-232
ETH-PHY
PCI-66/64 PCI-Power 3V3
Memory Configuration
Serial links
LVDS links
PO
WE
RC
MC
-J11/J22
OSC
OSC
17/02/06 H-RORC KIP HeidelbergTorsten Alt
FPGA Details
• 41.472 Logic Cells • 288 kb distributed RAM ( 18.432 x 16bit)• 1728 kb dual-port block RAM (96 x 18 kBit)• 64 DSP slices:
– 18x18 two’s complement multiplier– 48bit accumulator & adder/subtracter
• 8 Digital Clock Manager (DCM)• 4 Phase-Matched Clock Dividers (PMCD)• 640 User I/Os• Flexible I/O technology : i.e. PCI, DDR, DDR2• partial/full reconfigurable while operating
17/02/06 H-RORC KIP HeidelbergTorsten Alt
PCI Test setup
ChipScope
Tracer-SW
Terminal
Internal Mem 4 KByte DMA controller
PCI Core 64/66
Virtex4
PCI-BUS
HOST MAIN MEMORY
H-RORC
Catalyst TA600 PCI TracerLINUX-KERNEL
HOST-PC
A HOST is equipped with a H-RORC and a PCI tracer. The transaction can be monitored by normal output to a terminal, a PCI tracer and inside the FPGA by using the ChipScope logic analyzer
17/02/06 H-RORC KIP HeidelbergTorsten Alt
64bit Master Read• Start of a 64bit PCI read
burst from address 0x01587000 initiated by the H-RORC
• PCI-bus activity monitored by external tracer
• 4 64bit words are transmitted by the target before disconnect wo. data
As seen in the 32bit master read, the chipset disconnects a memory read after 32 bytes of transfered data -> Internal buffer of the chipset
17/02/06 H-RORC KIP HeidelbergTorsten Alt
DMA Write PerformanceMeasured performance of DMA data cycles on the PCI bus.
Theoretical bandwidth of a 64bit/66MHz PCI bus:
528 MByte/s
0,00
50,00
100,00
150,00
200,00
250,00
300,00
350,00
400,00
450,001 2 4 8 16 32 64 128
256
512 1k 2k 4k 8k 16k
32k
64k
128k
256k
512k 1M 2M 4M 8M
DMA block length [64bit words]
data
rate
[MBy
te/s
]
For each DMA block length 10/100/1000/10000 transactions have been performed
The time for each transaction was measured by a counter with a 15ns granularity
The transmitted data was checked for errors: No errors occured
17/02/06 H-RORC KIP HeidelbergTorsten Alt
DIU-PCI Test Setup
SIUHLT-RORC
SENDER
DIU H-RORC
PCI-TRACER
RECEIVER
DDL
CHIPSCOPE
PCI-TRACER SW
MAIN-MEM
MAIN-MEM
Terminal Terminal
The sender simulates a data source and sends data via the SIU-DDL-DIU to the H-RORC which stores the data in the main memory via PCI DMA
17/02/06 H-RORC KIP HeidelbergTorsten Alt
DIU Test results
• The DIU was operated with 66MHz. A full synchronization scheme was used to synchronize with the PCI clock
• Command sequences were send to the DIU via PCI to open a link
• Data was send by the SIU to the DIU. Verification of the data was done with ChipScope, PCI tracer and by comparing the data automatically.
• Different test patterns were used: - ramp - walking one - A5A5A5A5