cots technology for high energy physics instrumentation · pdf filecommunication. modern...
TRANSCRIPT
![Page 1: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/1.jpg)
COTS Technology for High Energy
Physics Instrumentation
Kevin SchultzVice President, R&D
National Instruments
![Page 2: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/2.jpg)
![Page 3: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/3.jpg)
![Page 4: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/4.jpg)
“I invented nothing new. I simply assembled
into a car the discoveries of other men
behind whom were centuries of work. . . .
Had I worked fifty or ten or even five years
before, I would have failed. So it is with
every new thing.”
— Henry Ford
![Page 5: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/5.jpg)
Innovation often means moving existing ideas
from where they’re known to where they’re not–
often in new combinations.
bridging distant worlds
and
building new worlds
“… to do for scientists and engineers,
what the spreadsheet did for financial
analysts …”
![Page 6: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/6.jpg)
The National Instruments Vision
“To do for scientists and engineers
what the spreadsheet did for financial analysis.”
Virtual Instrumentation
![Page 7: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/7.jpg)
Traditional Instruments
Accura
cy (
Bits)
28
26
24
22
20
18
16
14
12
10
8
41 10 100 1K 10K 100K 1M 10M 100M 1G 10G 100G
Sampling Rate (S/s)
6
NI Products, 2004NI Products, 1995
NI Products, 2010
NI Products, 2005
Leveraging Semiconductor Technology
![Page 8: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/8.jpg)
LEGO®
MINDSTORMS® NXT“the smartest, coolest
toy of the year”
CERN Large
Hadron Collider“the most powerful
instrument on earth”
Graphical System Design
![Page 9: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/9.jpg)
Diversity of Applications
ElectronicsSemiconductors Computers
No Industry > 10% of Revenue
Advanced
Research &Big Physics
PetrochemicalFood
ProcessingTextiles
AutomotiveTelecom
ATE Military/Aerospace
![Page 10: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/10.jpg)
Today’s Control System Challenges
ITER - Plasma
Diagnostics & Control
ESO Extremely Large
Telescope - Mirror Control
Modern system complexity is increasing
![Page 11: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/11.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
![Page 12: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/12.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Programming/Processing
![Page 13: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/13.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Programming/Processing
Communication
![Page 14: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/14.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Programming/Processing
Communication
Timing/Sync
![Page 15: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/15.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Programming/Processing
Communication
Timing/Sync
Custom HW
![Page 16: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/16.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Programming/Processing
![Page 17: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/17.jpg)
Moore’s LawDriving the computer revolution over the last 20 years
1970 1975 1980 1985 1990 1995 2000 2005 2010
CP
U S
pe
ed
Clock Speed (kHz) Transistor Count
Multicore Processors
![Page 18: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/18.jpg)
![Page 19: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/19.jpg)
![Page 20: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/20.jpg)
Traditional Development Tool Mine Field
Thread Synchronization
Race Conditions
Deadlock/Livelock
Processor Cache Effects
Flow of Data
Load Balancing
Sequential Performance
Scalability to Multiple CPUs
Non-Determinism
Priority Inversion
Lock Contention
![Page 21: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/21.jpg)
“The concurrency revolution is likely to be
more disruptive than the OO revolution…”- Herb Sutter, CEO, Microsoft, The Free Lunch is Over
"Parallel programming is perhaps the largest
problem in computer science today”- Stanford CS Department chair Bill Dally
“Nobody knows how to program those things”– Steve Jobs talking about multicore processors
The Parallel Programming Challenge
![Page 22: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/22.jpg)
Increasing Levels of Software AbstractionA
bst
ract
ion
System complexity
Machine code
Assembly language
C
C++
C#
System design platform
![Page 23: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/23.jpg)
PXIPC/Mac/Linux FlexRIO
Data Flow C Code Textual Math Simulation Statechart
CompactRIO Custom
High-Level Design Models
Graphical System Design Platform
![Page 24: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/24.jpg)
LabVIEW Execution FundamentalsRun
Queue
Execution
ThreadsRun
Time
Matrix – Vector Multiply
Multiple
Cores
…
Compilation
![Page 25: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/25.jpg)
From 1 to 8 cores
![Page 26: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/26.jpg)
Tokamak – Shape Control
RjZRRR
R o
2
21
Shape
Reconstruction
Tomography
Soft X-Rays
Magnetic
Sensors
Bolometric
Sensors
Grad-Shafranov
Solver
ControllerPID, MIMO
Target Shape
40 GFLOPs
Double Precision
1 ms loop rate
![Page 27: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/27.jpg)
Extremely Large Telescope DesignESO-E Mirror 4 (M4) Wavefront Real-Time Computing
1.87 TFLOPs
[5k x 56k] x 56k every 300μs
64 compute nodes (512 cores)
14k samples
every 2 ms
Group of 4
compute nodes
5k samples
every 2 ms
3 additional
“stars”
![Page 28: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/28.jpg)
1
10
100
1000
10000
2006 2007 2008 2009 2010 2011 2012
GF
LO
Ps
Real-Time HPC Trend
Tokamak Plasma
Diagnostics
40 GFLOPs (DBL)
1 ms loop rate
1M Real FFT/s
1k samples
500k Complex FFT/s
1k samples
ESO E-ELT Mirror Control
M4: 2 TFLOPs, 2 ms loop rate
M1: 1 TFLOPs, 1 ms loop rate
Financial Application
(Proposed)
1.2 TFLOPs
European Options
Synchrotron
Tomography
(Proposed)
4 TFLOPs
3D 8k x8kx8k voxels
Tandem Mirror
Plasma Control
(Proposed)
1-2 TFLOPs (DBL)
1 ms loop rate
![Page 29: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/29.jpg)
Plasma Diagnostics & Control with NI LabVIEW RT
• Poisson PDE– Fourier/DST
– Hockney
– Cyclic reduction
– Combination method
• N-cores require specific parallelization strategies
• Goal: 1 ms sustained and in a highly deterministic way
![Page 30: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/30.jpg)
Plasma Diagnostics & Control with NI LabVIEW RT
• Plasma control in nuclear fusion Tokamak with LabVIEW
on an eight-core real-time system
“…with LabVIEW, we obtained a 20X processing speed-up on an
octal-core processor machine over a single-core processor…”
Louis Giannone
Lead Project Researcher
Max Planck Institute
![Page 31: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/31.jpg)
System
Controller
PXImc Processor
Modules
PXI Multi-Chassis (PXImc)
One Way Latency = 6 uS, Throughput = 670 MB/S
![Page 32: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/32.jpg)
LabVIEW’s GPU Computing Module
![Page 33: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/33.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Communication
Programming/Processing
![Page 34: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/34.jpg)
Open Architecture
• Controls standards
– EPICS, TANGO, CORBA,
TINE, C
• Connectivity to different
devices
– OPC, Modbus, TCP/IP, UDP,
EtherCAT, Serial
• Flexibility
– Windows, RTOS, Linux, FPGA
![Page 35: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/35.jpg)
EPICS Software Architecture
• Distributed Clients and Servers (IOC – I/O Controllers)
• Network protocol: Channel Access
• Each IOC holds a subset of EPICS database variables
Channel Access
Analog I/O, Digital I/O, Motion Control, Image Acquisition, etc.
IOC (I/O Controller)
I/O HW
IOC (I/O Controller)
I/O HW
IOC (I/O Controller)
I/O HW
IOC (I/O Controller)
I/O HW
EPICS Client EPICS Client
![Page 36: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/36.jpg)
EPICS in LabVIEW
– Support for Channel Access server
and client
• Windows
• RT – VxWorks & Pharlap (Server only)
– Prototype code to run full EPICS IOC
Server side by side with LabVIEW
Real-Time
• Custom option for CompactRIO
![Page 37: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/37.jpg)
Los Alamos LANSCE
• Ongoing migration to a cRIO system with embedded EPICS
– 12 binary outputs
– 36 binary inputs
– 12 analog inputs
– 5 stepper motor channels
• Full IOC functionality allows access to all
record fields and EPICS utilities
• Maximum flexibility for partitioning the
problem
– LabVIEW for beam diagnostic
– EPICS for industrial control
![Page 38: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/38.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Communication
Timing/Sync
Programming/Processing
![Page 39: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/39.jpg)
Generate SignalsGenerate Signals
Time-Based
Signal-Based
Share Physical Clocks / Triggers
Share Time
…
Ethernet (1588)
IRIG
GPS
Etc.
Signal vs. Time-Based Synchronization
![Page 40: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/40.jpg)
Czech Institute of Plasma Physics
• Thomson scattering system
• Synchronized high speed data
acquisition
– 92ch running at 1GS/s
– Tight synchronization over 3 PXI
chassis
– Skew < 500 ps
![Page 41: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/41.jpg)
M1 Control - Proposed System SetupNI cRIO Node for Local
Sensor / Actuator I/O
• 1 cRIO Node per MirrorSupervisor
…
Deterministic EtherCAT
Network Ring
• 25-30 cRIO Nodes per
eCAT Network Ring
NI PXI Distributed
Mirror Controller
• 6 eCAT Network Rings
per Distributed Mirror
Controller
Supervisory
Network
• 6 Distributed Mirror
Controllers per
Supervisor
Synchronization
via 1588
![Page 42: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/42.jpg)
Signal-based
Time-based
GPSIRIG-B
10-12 sec
Precision
sec
10-3 sec
10-6 sec
10-9 sec
10-2m 100m
Proximity
101m 102m 103m 104m 105m Global<10-4m
PXI Multichassis
Synchronization Technologies
![Page 43: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/43.jpg)
Timing Solutions
• Greenfield Technology
– 8 Channel PXI Digital Delay Generator
– 4 high precision delays• 1ps resolution, <50ps rms jitter
– 4 auxiliary delays• 5ns resolution, <100ps rms jitter
• Available on the front panel and on PXI Trig
• Micro-Research Finland
– PXI Event Generator/Event Receiver
– Working on a cRIO Event Receiver
![Page 44: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/44.jpg)
NI and CERN White Rabbit• Partnering with CERN in developing White Rabbit (WR)
• Performance– Distance: > 10 km
– Scale: > 2000 nodes
– Accuracy: < 1ns skew, < 100 ps jitter• Compensates for propagation delay (cable length, temperature variation, etc.)
• Leverage Industry standards (802.x, IEEE 1588, SyncE)– Gigabit Ethernet communication with deterministic capability
• Generally Applicable
• Leverage for future PXIe modules
![Page 45: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/45.jpg)
White Rabbit: Synchronization over Distance
Distance
10-12 sec
Precision
sec
10-3 sec
10-6 sec
10-9 sec
100m 101m 102m 103m 104m 105m GlobalDistance
Signal-Based
PXI-6682
Time Based
White Rabbit
![Page 46: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/46.jpg)
Modern System Complexity Increasing
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Communication
Timing/Sync
Custom HW
Programming/Processing
![Page 47: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/47.jpg)
Factors Driving Need for Custom Hardware
• Very tight control loops
• Onboard signal processing
• Specialized communication protocols
• Custom flexible timing
• Massively parallel processing
• REAL NEED: Combination of COTS and Custom!
![Page 48: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/48.jpg)
Field Programmable Gate Array (FPGA)
• What it is
– A silicon chip with unconnected gates/processing resources
• How it works
– Define behavior in software
– Compile and download to the hardware
• Advantages
– Reconfigurable
– Reliability
– Parallel execution
![Page 49: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/49.jpg)
FPGA Technology
I/O Blocks
Programmable
Interconnects
Logic
Blocks
![Page 50: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/50.jpg)
CPU
Performance
(GFLOPs)
FPGA
Performance
(GMACs)
1997 2001 2002 2004 2005 2006 20091999
5
50
500
5,000
5
50
500
5,000
FPGAs
CPUs
Parallel Architectures Drive Performance
![Page 51: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/51.jpg)
How to Program FPGAs?
• Schematic capture
– Graphical depiction
– Easy to understand
– Vendor specific
– Ex: ViewDraw, Ease
• Hardware Description languages (HDL)
– Text based – “Firmware”
– Generic or vendor specific
– Ex: VHDL, Verilog
![Page 52: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/52.jpg)
LabVIEW FPGA vs VHDL
Counter Analog I/O I/O with DMA
66 Pages ~4000 lines
![Page 53: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/53.jpg)
FPGA Programming: The Ultimate in
Multicore, Multiprocessor Development
![Page 54: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/54.jpg)
NI Labs | LabVIEW FPGA Direct Access to Preexisting Xilinx CORE Generator IP Libraries
![Page 55: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/55.jpg)
FPGAProcessor I/O Modules
Standard RIO Architecture
![Page 56: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/56.jpg)
NI CompactRIO
Real-Time Processor Reconfigurable FPGA
I/O Modules
•Reconfigurable FPGA for high-speed and custom I/O timing, triggering, control
•Real-Time Processor for deterministic, stand-alone operation, logging and analysis
•I/O Modules with built-in signal conditioning for connection to sensors/actuators
Extreme Ruggedness• -40 to 70 C temperature range• 50g shock, 5g vibration
Low Power Consumption• 9 to 35 VDC power, 7-10 W typical
![Page 57: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/57.jpg)
Over 60 NI and 3rd Party C Series Modules
• Analog Input
― Up to 1 M/s, simultaneous sampling
― 4, 8, 16, and 32-ch options
― Built-in signal condition for sensors
― Strain gages, accelerometers, thermocouples, RTDs
― Up to ± 60 V, ±20 mA
― 12, 16 and 24-bit resolution
― Available ch-to-ch isolation
• Analog Output
― Up to100 kS/s simultaneous updating
― Up to 16-ch per module
― ±10 V, ±20 mA
― Isolation
• Digital I/O
― Up to 10 MHz timing
― Counter/timer, PWM
― 8 and 32-channel options
― 5V/TTL, 12/24/48 V logic levels
• Specialty
― 2-port CAN modules
― Brushed DC servo motor drive
• Third Party Modules
― LIN, Profibus, WLAN 802.11, MIL-1553, ARINC-429, GPS, and more
![Page 58: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/58.jpg)
NI FlexRIO Adapter Module• Interchangeable I/O
• Digital or analog
• NI FlexRIO Adapter Module
Development Kit (MDK)
PXI/ PXIe
NI FlexRIO FPGA Module• Virtex-5 FPGA
• 132 digital I/O lines
• Up to 512 MB of DRAM
• Peer-to-peer data streaming
PXI Platform• Data transfer
• Synchronization
• Clocking/triggers
• Power/cooling
NI FlexRIO
![Page 59: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/59.jpg)
NI FlexRIO Partner Modules
100 MHz
PPMU
Camera Link
and GigE
Multi-gigabit
optical
Dual gigabit
Ethernet
Video and
AutomotiveTime to Digital
Convertor
![Page 60: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/60.jpg)
Xilinx Virtex 5 FPGA
Socketed CLIP Socketed CLIP
LabVIEW FPGA VI
DRAM Memory DRAM Memory
PXI Bus
Soc
kete
d C
LIP
CLIP CLIP CLIP
Custom Front-End
…
Custom Module Development
![Page 61: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/61.jpg)
NI Real-Time Hypervisor for Linux
Windows PC
Hypervisor System*
Supported RT
I/O
Supported
Linux I/O
*Must program
LabVIEW Real-Time
application from Windows
![Page 62: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/62.jpg)
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Communication
Timing/Sync
Custom HW
Programming/Processing
![Page 63: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/63.jpg)
CERN Collimator Alignment
• 550+ axes of motion
• Across 27 km distance
• The jaws have to be positioned with
an accuracy which is a fraction of the
beam size (200μm)
• Synchronized to
– < 5ms drift over 15 minutes
– Maximum jitter in μs
![Page 64: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/64.jpg)
Mechanical Design
Discrete and Sequential
Logic
Motion Control
Design
Logging, Database
HMI
Networking
Machine Condition
MonitoringMachine Vision
Motors and
Actuators
Sensors and Signal
Conditioning
Modern
Machine
Embedded
System Design
Communication
Timing/Sync
Custom HW
Programming/Processing
![Page 65: COTS Technology for High Energy Physics Instrumentation · PDF fileCommunication. Modern System ... •Network protocol: Channel Access ... optical Dual gigabit Ethernet Video and](https://reader031.vdocuments.site/reader031/viewer/2022030503/5aaf81257f8b9a25088d9250/html5/thumbnails/65.jpg)