irio technology developing applications for ... - indico
TRANSCRIPT
Page 1
IRIO TECHNOLOGY: DEVELOPING APPLICATIONS
FOR ADVANCED DAQ SYSTEMS USING FPGAS
Mariano Ruiz, S. Esquembri, A. Carpeño, J. Nieto, A. Bustos,
E. Bernal, D. Sanz, E. [email protected]
Universidad Politécnica de Madrid
Page 220th Real Time Conference 2016 June 9th
Outline
• Motivation
• RIO, FlexRIO and cRIO Devices
• Development cycle
• IRIO Project
– IRIO Library
– NIRIO-EPICS Device Support
– IRIO NDS C++ classes
• Design Methodology
• Applications
– ITER fission chamber
– Image acquisition system
– Advanced applications: using GPUs for ITER diagnostics
• Conclusions
Page 320th Real Time Conference 2016 June 9th
Motivation
• FPGAs provide reconfigurable
hardware with deterministic data
preprocessing capabilities
• Graphical tools such as LabVIEW
for FPGA reduces development and
integration time
• The combination of both
technologies with EPICS simplifies
the development of complex
control, data acquisition and
processing systems
PXIe ChassisPICMG CPU
RIO R
series
FPGA
MXI/PCIe
FlexRIO
Cards
FilteringFFT
.
.
.
.
CA
FPGA
MXI/PCIe
IRIO is a set of software tools
simplifying the integration of
RIO devices in EPICS
Page 420th Real Time Conference 2016 June 9th
RIO/FlexRIO Devices
Write Registers
Read registers
ADCs
DACs
Digital I/Os
CameraLink
DMA
Adapter module
FPGA
HOST
IOCIRIO
PVs
analog
binary
enum
waveform
image
FlexRIO
The developer defines the
functionality
programming the FPGA EPICS connects a user defined
device with PVs for
configuration and supervision
12
Page 520th Real Time Conference 2016 June 9th
Development cycle: LabVIEW for FPGA
LabVIEW VI VHDLBitfile
FPGA Target
User Generated
Auto Generated
LabVIEW FPGA
XILINX compiler
FPGA Target
2
4
5
1 3
FPGA project developer
Page 620th Real Time Conference 2016 June 9th
Reducing development time in FPGAs using templates
Acquisition
Control
AI0
DMA 1
Controller
DO0-DO7
AO Waveform
Generation
and Control
DMA Channel 0
Frame
Generation
DMA 0
Controller
Auxiliar
Variables
Processing
DMA CHANNEL 0
Transfer Control
DaqStartStop
DebugMode
SamplingRate0Overflow0
Siz
e:
un
sig
ne
d 6
4 b
its
De
pth
:
10
24
SamplingRate1
DMAEnable1
DMAEnable0
InsertedIOModDeviceTempModCorrectInitDone
ExpectedIOMod
FRef
SGFreq
SGSignalType
SGPhase
A0Enable
NChannels0
FrameType0
SampleSize0
BlockNWords0
Size: signed 16 bits
Depth: 1024
Device
Config &
Status
Signal
Processing
Algorithm
DMA Channel 1
Frame
Generation &
Transfer Control
Overflow1
FrameType1
SampleSize1
BlockNWords1
NChannels1
Pattern
Generation
14 8
DI0-DI7
14 14 14
Samples16 x NFrame64
Inte
rme
dia
te
FIF
OWrite
Po
rt
Re
ad
Po
rt Frame64
Siz
e:
un
sig
ne
d 6
4 b
its
De
pth
:
10
24
Frame64
DM
AT
tHO
ST
0
FIF
OWrite
Po
rt
Re
ad
Po
rt Frame64
16
16
2
3
CH0BUFFER
FIFO
Write
Po
rt
Re
ad
Po
rt
16
CH1BUFFER
FIFO
Write
Po
rt
Re
ad
Po
rt
CH2BUFFER
FIFO
Write
Po
rt
Re
ad
Po
rt
CH3BUFFER
FIFO
Write
Po
rt
Re
ad
Po
rt
16
16
1616
CH3 Sample
CH2 Sample
CH1 Sample
CH0 Sample
CH3 Sample
CH2 Sample
CH1 Sample
CH0 Sample
16
16
16
16 Processed Sample
16
2
3
125MHz Module Clock 0 Region 100MHz Base Clock Region
Siz
e:
un
sig
ne
d 6
4 b
its
De
pth
:
10
24
Frame
64
DM
AT
tHO
ST
1
FIF
OWrite
Po
rt
Re
ad
Po
rt
Frame64
16
A0Val
125MHz Module Clock 1 Region
8
16
16
16
16
16
2
SGAmp
PatternSelect
D0Enable
16
2
DOVal
40MHz Base Clock Region
200MHz Base Clock Region
9 AuxAO
8 AuxDO
8 AuxDI
11 AuxAI
DMA1 Data
DMA0 Data
SGFreq
SGSignalType
SGPhase
A0Enable
A0Val
SGAmp
PatternSelectD0Enable
DOVal
Overflow
FrameTypeSampleSize
BlockNWordsNChannels
DMAEnableSamplingRateDaqStartStopDebugMode to
/fro
m H
OS
T C
om
pu
ter – P
CI E
xp
ress In
terf
ace
AI1
AI2
AI3
LVDS
ADC
MultiChannel
ADC
MultiChannel
AI CH0
AI CH1
AI CH2
AI CH3SPI
DAC
AO CH0
SPI
D I/O 0-7
Page 720th Real Time Conference 2016 June 9th
Using RIO devices in Linux
C API
FPGA
NI RIO kernel
modules
NI RIO libraryLinux User
space
Linux Kernel
space
Hardware
RIO/FlexRIO
Device
1-5
C/C++
applicattion6
C API
LabVIEW VI VHDLBitfile
FPGA Target
User Generated
Auto Generated
LabVIEW FPGA
XILINX compiler
FPGA Target
2
4
5
1 3
FPGA project developer
Page 820th Real Time Conference 2016 June 9th
IRIO Project: IRIO Library
NI RIO Linux Device Driver (kernel)
NI RIO Linux Device Driver (kernel)
Hardware(I/O Module)
Hardware(I/O Module)
IRIO LibraryIRIO Library
EPICS Device SupportEPICS Device Support Application Program
Application Program
FPGA C API
SDD Toolkit
Application Configuration
User Space
Kernel
• Identification of the resources
implemented in the FPGA
– The Design Rules document
describes the rules for the FPGA
implementation
• Provides an API simplifying the
interface with the FPGA.
– Access to FPGA registers
– Analog input
– Digital I/O
– DMA acquisition
– Image acquisition using cameralink
• Serial line for camera
configuration
– Signal Generation (DDS)
Page 920th Real Time Conference 2016 June 9th
IRIO resources mapping
VHDL
XILINX compiler
FPGA bitfileFPGA header file
IRIO Library
NI RIO API Library
2
1
3
LabVIEW Code
Software-hardware interface
Resource mapping
FPGA
Adapter Module
Page 1020th Real Time Conference 2016 June 9th
IRIO Project: EPICS driver using asynDriver
EPICS device driver using asynDriver
implementation for RIO devices
(FlexRIO and cRIO) using IRIO library
Automatically connects the PVs with
FPGA resources using IRIO library
If the user changes the FPGA design
no compilation is needed
ITER SDD generates the complete
software unit
NI RIO Linux Device Driver (kernel)
NI RIO Linux Device Driver (kernel)
Hardware(I/O Module)
Hardware(I/O Module)
IRIO LibraryIRIO Library
EPICS Device SupportEPICS Device Support Application Program
Application Program
FPGA C APIFPGA C API
SDD Toolkit
Application Configuration
User Space
Kernel
Page 1120th Real Time Conference 2016 June 9th
IRIO Project: C++ classes for Nominal Device Support
NI RIO Linux Device Driver (kernel)
NI RIO Linux Device Driver (kernel)
Hardware(I/O Module)
Hardware(I/O Module)
IRIO Library
EPICS Device SupportApplication
Program
FPGA C API
User Space
Kernel
NDS-IRIO
• Nominal Device Support
approach defines a set of
classes and PVs to be used for
EPICS driver implementation.
• NDS-irio is the set of NDS
extended classes to use
FlexRIO devices
• Simplify the implementation
of EPICS device support for
FlexRIO using NDS
Page 1220th Real Time Conference 2016 June 9th
Design Methodology
LabVIEW VI VHDLBitfile
FPGA Target
User Generated
Auto Generated
LabVIEW FPGA
XILINX compiler
FPGA Target
2
4
5
1 3
FPGA project developer Windows
Host
Use IRIO Library
IRIO user
EPICS application
?
YES
NO
Use NDS-IRIO
EPICs device support implemented with
NDS
IRIO user application
Using NDS?
YES NO
NI-RIO EPICS Devicedriver
EPICs device support implemented with
ASYN
6
7 8
Linux Host with
CCS
FPGA
NI RIO kernel
modules
NI RIO library
NI-RIO
EPICS
IRIO
Library
Linux User
space
Linux Kernel
space
Hardware
RIO/FlexRIO
Device
EPICS core
IOC app
CA
NDS-IRIO
1-5
C/C++
applicattion
6
7
8Codac Core System
Page 1320th Real Time Conference 2016 June 9th
ITER Fission chamber diagnostic use case application based on FlexRIO technology
• Integrate deterministic diagnostic into the
FPGA (4 ADC sampling at 125MS/s).
Data processing to detect/count pulses,
RMS, and campbelling
FPGA Real Time
Preprocessing
NI PXIe-7966R + 5761
125MHz data
acquisition
Software
triggerHardware
trigger
AI[0-2] Low Pass Filter
ON/OFF
activationCutoff freq
selectable
1.25MHz
12.5MHz
Downsampling
N factor
configurable
N
Pulses
detection
Campbelling
Current
CH0
CH1
CH2Hardware
processing
DMA transfers to HOST
Measurement info.
Pulse info:
sample peak detection
Width & heigth
Raw data acquired
DMA 1
DMA 0
I/O Registers
Pulses detection
Campbelling
Current
PXI trigger line 2
Every 1ms for
TimeStamps generation
14 bit at
250MS/s per
channel
Page 1420th Real Time Conference 2016 June 9th
ITER PXIe Fast controller: Image acquisition (cameralink).
PCIe 1.1
x4 1GB/s
PICMG
Fast Controller
(fc18-3)
NI PXIe 7962R + NI
CameraLink adapter
module
NI PXIe Chassis 1065
EoSens_3CL_MC3010
EDT PCIe8 Dva camera link simulator
Device PVs
Image
Channel
PVs
Setup
Readback
Can we add more processing capabilities?
Is it possible to add a GPU?
Page 1520th Real Time Conference 2016 June 9th
FPGA+GPU processing
• NI-RIO Linux Device
Driver modified to
implement direct DMA
from FPGA to GPU
NVIDIA TESLA GPU
ITER FAST PLANT SYSTEM CONTROLLER
NI PXIe 1062Q
CAMERA
NI-1483+PXIe-7966R
PCIe(16x)
MXI Link(4x)
PXIe(4x)
CameraLink
SW
Codac Core System 5.0
EPICS NDS v2.3.7
NI FlexRIO-GPU bundle
CUDA 6.5 SDK
NI LabView for FPGA
CAMERA
LINK
DEVICE
NI FLEXRIO
DEVICENVIDIA
GPU
HA
RD
WA
RE
KE
RN
EL
SP
AC
EU
SE
R S
PA
CE
FLEXRIO
DRIVER
NIRIO C API
for FPGA
NDS
IOC DATABASE
NDS-GPU
APPLICATION
CUSTOM
GPU API
CUDA
API
IOC
FlexRIO-GPU
Bundle
GPU
DRIVER
Page 1620th Real Time Conference 2016 June 9th
Conclusions
• We have defined a design methodology for implementing advanced data and
image acquisition applications with RIO/FlexRIO devices, integrated with
EPICS using IRIO software
• We have developed different LabVIEW/FPGA patterns and libraries for RIO
devices
• It is not necessary to rewrite or even recompile the EPICS device support for
every cRIO/FlexRIO configuration
• IRIO tools integrated in ITER CODAC Core System V5.2 (February 2016)
• IRIO tools are GPLV2
• Current users of IRIO:
– ITER Diagnostics use cases, KSTAR project, Russian DA (cRIO)
– ESS Bilbao
Page 17
Thank you very much for your attention!!
questions?
Mariano Ruiz & Sergio Esquembri
Universidad Politécnica de Madrid, Spain
Technical University of Madrid
IRIO TECHNOLOGY: DEVELOPING APPLICATIONS FOR
ADVANCED DAQ SYSTEMS USING FPGAS