linux driver user guide · 2018-01-30 · amba dma controller (with axi memory interface) linux...

29
AMBA DMA Controller (with AXI memory interface) Linux Driver User Guide Version 1.1 - January 2015 1 Linux Driver User Guide 32-Bit/64-bit AXI modes

Upload: others

Post on 11-Apr-2020

32 views

Category:

Documents


1 download

TRANSCRIPT

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

1

Linux Driver User Guide

32-Bit/64-bit AXI modes

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

2

CONTENTS

1 INTRODUCTION ............................................................................................................................. 6

2 CONTEXT OF A TYPICAL SOFTWARE DRIVER & IP CORE ...................................................... 7

3 LINUX DEVICE DRIVER ............................................................................................................... 10

3.1 INTRODUCTION ........................................................................................................................ 10 3.2 DRIVER OPERATION & FUNCTIONAL DESCRIPTION .................................................................... 10 3.3 HIGH AND LOW PRIORITY CHANNELS ........................................................................................ 11 3.4 LINUX/UC-LINUX NETWORK INTERFACE EXPOSURE ................................................................... 11 3.5 DMA CONTROLLER DEVICE DRIVER NETWORK INTERFACE ROUTING & EXPOSURE .................... 12 3.6 LINUX RAW SOCKET & TIMESTAMP SUPPORT ............................................................................ 12 3.7 FILE AND DIRECTORY STRUCTURE ........................................................................................... 13 3.8 SOFTWARE DEVICE DRIVER STRUCTURE .................................................................................. 16 3.9 DESCRIPTORS ......................................................................................................................... 17

4 DRIVER INSTALLATION .............................................................................................................. 18

4.1 INTRODUCTION ........................................................................................................................ 18 4.1.1 Linux Commands .............................................................................................................. 18

4.2 MAKE FILES ............................................................................................................................ 19 4.3 INSTALLATION AND KERNEL OBJECT DRIVER REGISTRATION ...................................................... 19

4.3.1 Installation Instructions ...................................................................................................... 19 4.3.2 Linux-PC ............................................................................................................................ 20 4.3.3 Cross-compiled target embedded device ......................................................................... 20 4.3.4 Check the UIO ................................................................................................................... 20

4.4 CONFIGURATION BY PARAMETER PASSING ............................................................................... 21 4.4.1 DMA Controller Device Driver Parameter List .................................................................. 21 4.4.2 DMA Controller Device Driver Parameter Definition and Meanings ................................. 23 4.4.3 Bring Parameter changes into effect ................................................................................. 26 4.4.4 Controlling DMA using the <implementation.h> header file at compile time .................... 26

5 CONTACT ..................................................................................................................................... 28

6 DOCUMENT HISTORY ................................................................................................................. 29

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

3

About This Specification

This specification introduces the MorethanIP AMBA High-Speed DMA Controller accompanying Linux/uC-Linux device driver and how to make/compile and build.

Intended Audience

This document is fully intended to be viewed and reference by MorethanIP/Nine Ways customers using the technology for larger designs and projects.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

4

List of Figures

Figure 1 - Context of the Linux Driver ..................................................................................................... 7 Figure 2 – L2 PTP-1588 support and network interface exposure ......................................................... 8 Figure 3 - Different scenarios of DMA Controller usage ......................................................................... 9 Figure 4 - Device Driver Schematic Structure ....................................................................................... 16 Figure 5 - General Principle of a Descriptor Table ............................................................................... 17 Figure 6 – Typical layout of the <moduleparms> file in the base directory of the Device Driver .......... 23 Figure 7 – File contents of <implementation.h> .................................................................................... 26

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

5

List of Tables

Table 1: Design Kit Directories Description .......................................................................................... 13 Table 2: Typical and Relevant Linux BASH Commands ...................................................................... 18 Table 3: Default parameter values in C Header file for the DMA Device Driver ................................... 22 Table 4: Definition of <moduleparms> file for the DMA Device Driver ................................................. 23 Table 5: Contact Information ................................................................................................................. 28 Table 6: Document History Entry Log ................................................................................................... 29

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

6

1 Introduction

This document outlines and covers all aspects of the DMA Controller uC-Linux/Linux software device driver which is provided by Nine Ways R&D Ltd additional to the IP core deliverable. The emphasis is to allow Ethernet frames to/from a MAC/SWITCH into the CPU sub-system using hardware in the form of an AXI master that takes advantage of the popular and common OS platform Linux/uCLinux.

Application programmers particularly those who are increasingly benefitting from being able to port powerful data processing software into an embedded environment are increasingly looking for an ability to process huge exposure to network traffic on target platforms. This is in stark contrast to previously only having the processing ability to observe or manage data network traffic on behalf of other systems because of the bottleneck and restrictions of software layers associated with network hardware including master DMA blocks as part of a wider processor system.

Linux is provided openly by most vendors of FPGA who have gone down the route of hard coding an ARM processor into the device (ASIC area). The DMA package includes a specific device driver that fully compiles into most distributions of the operating system although the vendors will provide a cut-down small version of Linux known as uC-Linux. With this operating system running, all of the powerful toolsets, programs, utilities and many years of experience of Linux will be available to the application user with the wide range of support both from the vendor and the wider programming community.

As the software device driver is minimal in processor execution overhead, the constraints and responsibility are placed upon the application programs running on Linux in the CPU such as the ARM processor. The hardware is designed to perform as much processing of the data transfer as possible with only the setup and completion work to be carried out between the device driver in the ARM and the DMA hardware IP core.

In no way does it change or modify any of the Kernel, IP stack or the way the API sockets interface to the application. The device driver is modular to meet the flexible needs of different Linux kernel compilations, processor architecture and surrounding driver support.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

7

2 Context of a typical Software driver & IP core

The controlling software driver needed to initialize and handle the RX and TX Ethernet frames through the DMA Controller IP hardware, can be written in any language, in any operation system environment, or even bare-metal situations where the driver functionality controlling the DMA occupies near full execution of the CPU.

Figure 1 - Context of the Linux Driver

The Linux device driver and its associated hardware DMA Controller are highlighted in dashed red. They only form a small part of the overall infrastructure that will integrate to provide an overall solution for an application.

The hardware DMA Controller is represented by the thick red arrow and is only concerned with reading and writing Ethernet packet frames into a buffer in the processor subsystem memory. Multiple packets can be fetched and transmitted to/from the DMA controller to the MAC/SWTICH FIFO forming a transmit TX packet and data can be written to the RAM constituting a receive RX.

A black arrow is shown from the memory linking the IP stack with the application API. This shows how the memory allocation in RAM is constant for all of those. The DMA device driver, the IP stack and application software in the layers above only pass on and process the pointer location to the frame buffer in working RAM.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

8

In its most ideal operation, the only transfer of data is performed by the DMA Controller IP in and out of the DDR memory, and then the Linux kernel transfers a pointer around the entire software OSI layer. If the packet then replies from the application layer API with just modified data contents performed by the intended algorithm of its usage, then essentially the DMA Controller moves packets into RAM, they are processed by high-level software and then DMA transferred back out to the MAC forming a read-process-write mechanism. (Note: this is only achievable with careful management of memory buffers and full control of the IP stack with regards to zero or minimal CPU-copy).

Please Note: to achieve this minimal data transfer within the software, the application user must adhere to script programming method described in the DMA Controller Linux & Software Device Driver Document (separate manual PDF to this User Guide).

Linux Applications

eth1 UioX / Switch Access

uCLinux Kernel including TCP/UDP/IP stack with Raw-Sockets (but no

SO_TIMESTAMP compile support)

AMBA DMA uCLinux Device driver

SWITCH Core

User space

Kernel space

NetFusion/M2S-SOM-F484 PCBPHY/PCS

ARM processor

SmartFusion2 FPGA

Media layer

1588 Hardware timer

FPGA fabric

1588 TAG removal/insert

High-Speed DMA controller

eth0 eth2

HIGH Priority netif() LOW Priority netif()

L2 1588 Raw-Sock netif()

Direct physical memory Software register access via uCLinux UIO mechanism from user-space to hardware fabric

LOW Priority Traffic

HIGH Priority Traffic 1588 L2 Frame Pipe

IP frames & 1588 L2 Frames BOTH with time-stamp TAGS

PTP IP Raw-Sock netif()

eth3PTP Dedicated timestamp interface

Normal Apps no time-stamping requirements

Time-stamping

application

PTP Control Application with L2 1588 support & Hardware Timer control

PHY/PCS

Media layer

SWITCH AXI Management Port

DMA IRQ

TS IRQ DLR IRQ

TS SHARED IRQ DLR SHARED IRQ

Figure 2 – L2 PTP-1588 support and network interface exposure

Note: The (above) illustration is based on an example on a Smartfusion2 FPGA platform. However, this can be considered on any technology.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

9

Figure 3 - Different scenarios of DMA Controller usage

The Figure (above) illustrates the typical IP cores that may be requested by a system designer that is to be an integrated design inside a target device such as an FPGA. The Linux device driver that is provided only handles and interacts primarily with this DMA Controller which is common to all of the examples in the illustration.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

10

3 Linux Device Driver

3.1 Introduction

The driver for the DMA Controller fulfills the following tasks:

1. It exposes a regular network device in the Linux stream architecture (eth0, eth1, eth2, eth3 or mtip0, mtip1, mtip2, mtip3) usually with the pneumonic </dev/eth0 /dev/mtip0> or </dev/eth1 /dev/mtip1 … … …> that any application layer API program can access. This along with very importantly, the BASH Unix scripting that binds powerful high-level programs to work together.

(The configuration file for the driver source allows for the naming of the exposed network device).

Note: the network interface name “eth” or “mtip” are just ideal default names. They are configurable text wise in the software device driver.

Important Note: eth2 and eth3 are specific routable interfaces in the device driver exposed to handle the L2 PTP-1588 protocol support.

2. It manages the full control of the data throughput of the DMA master AXI controller invoking memory transfer to/from the DDR physical contiguous SDRAM.

3. It exposes a UIO interface for the API application programs (/dev/uio0) to be able to access all of the MorethanIP register mapping of all integrated hardware logic IP cores with-in the silicon device of the ARM processor.

4. Finally, the device driver initializes ALL hardware IP cores additional to the DMA Controller module such as any MAC, PHY, PCS and/or SWITCH cores (however the default deliverable driver usually will only handle DMA registers with basic minimal support for the additional MAC/SWITCH hardware). The aim is not to provide full operational control and management but purely for basic information gathering such as Link UP and DOWN and flow control error checking via IRQ Interrupt Service Routines (ISRs).

The driver is implemented in a kernel module named MTIP-DMA-SW.

3.2 Driver Operation & Functional Description

The Linux kernel device driver named mtipsw-dma-sw is an installable independent module written in the C-programming language. It is considered separate in meaning because it is not part of the compiled kernel or available in the mainline released kernel from <www.linux.org>. In fact this device driver that is bundled with the DMA Controller comes packaged as C source code with [make] file scripts for GCC Linux. It allows the Ethernet frame to/from the DMA hardware connected to a MAC/SWITCH to pass to a network interface in uC-Linux/Linux. From here, it then allows as standard, application programs to communicate with a network via standard sockets using IP and other allowed upper layer protocols. The application programs run as threads in Linux utilizing all the powerful features of such a well established OS.

The relevance of distributing as source is to allow as standard with in a Linux environment, for the compiler and linker to detect and adjust for the host architecture and surrounding kernel structure. As there is no specific kernel that is needed (although stay more recent than version 2.6.14) the compilation and build of the device driver source to a kernel object file will fit to its surrounds. In the event that a significant step in kernel produces a compilation error, then you will have to contact MorethanIP GmbH or Nine Ways R&D Ltd for support. (www.nineways.co.uk)

Commonly though, you will be using an ARM FPGA/ASIC sub-system that will already come bundled with a preferred uC-Linux kernel so the supplied kernel object file that accompanies the source (compiled for the uC-Linux ARM configuration) will all you need to consider for most situations.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

11

3.3 High and Low Priority Channels

As already discussed the DMA Controller concept is built on a dual RX and dual TX channel mechanism. Basically, from the FIFO interface on a MAC/SWITCH all the way to the application software programs in the processor sub-system, a dual channel pathway in each direction exists offering overtaking of packet frames.

This sounds unnecessary as the frame usually will stay in order along the route of a fiber router hop (unless the higher layer router functionality obeys IP fragmentation rules). However, the section of the pathway in between a given MAC and the software applications handling the data/information contained in the Ethernet packets is often bottlenecked and the slowest part of the whole process.

In order to allow for this section of restriction to be less hindering to chosen packets, two pathways for the TX system and two pathways for the RX system allow for priority to take place and thus, lower priority Ethernet frames are held up in their lanes whilst the higher priority are transacted through first.

The Linux software device driver shall provide for the two lanes per direction. Thus CH0, CH1 for RX-DMA. Also CH0, CH1 for TX-DMA.

No other assumptions or priority takes place with the device driver as the hardware reads the TX-DMA first and the IP stack upper layers in the OS can decide themselves the priority of the reading of the RX-DMA from the descriptor information.

The device driver just has to provide the parallel channels in both directions.

3.4 Linux/uC-Linux Network Interface Exposure

The device driver will cause the Linux/uC-Linux OS to expose two separate network interfaces to the application layers for the DMA Controller connected to a MAC/SWITCH.

Note: Whilst the name of the exposed network interfaces is configurable (described below) it will be illustrated and described as /dev/eth0 and /dev/eth1 here in this sub-section.

The CH0 for RX and TX together will be piped through to exposed interface /dev/eth0. This is easy to remember as eth0 is CH0 high priority.

Also, CH1 for RX and TX together will be piped through to exposed interface /dev/eth1. At this point, it is widely assumed that the software application programs that bind() to selected network interfaces will ultimately control how the TX-DMA effectively utilizes this mechanism. The RX-DMA will automatically tend to (according to the hardware specification in the MAC/SWITCH IP cores) prioritize and order the packets arriving. This means the upper layer software is not aware of the prioritization. However, it is for the system architect and designer to ensure that the different software using different Ethernet traffic types to ensure that the application programs are binding to the relevant exposed network interfaces.

Usually, bridge protocol packets are assigned higher priority and so they could be using /dev/eth0 and the RSTP demon can assign bridge <br0> to use this network interface. This allows for those BPDU packets to be processed much higher priority as the network topology has to be prioritized to a much more important extent that the normal Ethernet frame data traffic.This whole mechanism is to enforce packet type prioritization from end point software applications, through DMA, MAC interfaces, switches and routing. Not all of this can often be achieved but for this product, the DMA Controller is fully ready for the functionality requirement of channel priority.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

12

The network interfaces exposed in the /dev directory in Linux are fully enabled in the uCLinux we will use. The exposed network interfaces created by instantiating the DMA device driver are as follows;

/dev/eth0 Normal HIGH priority network interface

/dev/eth1 Normal LOW priority network interface

/dev/eth2 Dedicated Raw Socket for PTP-1588 specific applications

/dev/eth3 L2 1588 control packets pipe

The UIO is currently enabled in the Kernel and is controlled by the [.config] file when making and building the arm-uclinux image. UIO functions are available to the applications as well as the device drivers that partner the UIO pipe that cuts effectively through the uCLinux Kernel layers. The physical and virtual memory models are preserved but unified at the same time with the UIO mechanism.

Very Important Note: most uCLinux Kernels have no virtual memory management and so the UIO is transparent. Nevertheless, so the system is portable and compatible with more sophisticated Linux distributions and technology platforms, the UIO shall still be implemented.

3.5 DMA Controller Device Driver Network Interface Routing & Exposure

Three different incoming packets shall be handled according to their category. Obviously, the L2 PTP-1588 specific frames will be identified via the frame type field in the Ethernet specification.

If no match, then a customized user function is going to be made available for customers to use a function wrapper to allow for an identification algorithm to take place with user intended packets. In other words, for the PTP dependant application running on top of the uCLinux Kernel, the DMA device driver has to know what type of packet, port number, and content maybe also - to be passed to the PTP application needing the time-stamp information. An example of this maybe an AVB protocol for instance needing very precise arrival times and synchronization.

If no match still, then the incoming frame is stripped of the inserted TAG containing the time-stamp, and then passed to either the high or low priority existing normal Ethernet interfaces. These interfaces shall be able to handle TCP and UDP unlike the dedicated raw socket interfaces that will pass the entire arrival frame to the application layer.

3.6 Linux Raw Socket & Timestamp Support

The uCLinux used does support raw sockets (if enabled in the [.config] build file in the Kernel). The application is presented with the entire frame passed to the network interface in the DMA device driver module.

Timestamp specific functions that are meant to be support in most Linux Kernels since version 2.6.14 are not available in the Emcraft system. Apparently a lot of functionality has been cut out (along with pthreads) to keep the Kernel footprint compact. In order to get round this issue, the PTP handling client running on top of the Kernel will be implemented with functions that use the UIO mechanism to access the hardware timer in the 3-Port SWITCH directly.

The PTP client chosen that shall be ported as an application will initially contain the function calls to the non-existent uCLinux Kernel library will be modified to access the hardware timer directly also.

The hardware timer is access via physical memory location using the AHB-lite bus matrix. The 3-Port SWITCH has software-mapped registers that configure and control the PTP 1588 SWITCH functionality. This includes reading and adjusting the counter value.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

13

3.7 File and Directory Structure

The root directory of the DMA Controller package is called AMBA_DMA_Controller which contains a documentation directory (containing these documents), libero, quartus, simulation, source, synthesis, documentation, LinuxDriver which houses all of the IP core hardware source, simulation and test-bench facilities, and finally the Linux device driver.

Table 1: Design Kit Directories Description

Directory Description

libero

Contains the <MakeLibero> file script that copies the agnostic generic driver in this package into a SmartFusion MicroSemi project HDL directory.

Note: You will need to edit this file to select the target directory

quartus Altera Quartus project example that synthesizes the core (if available – and may be empty as a directory)

simulation Scripts to configure and execute simulation.

source Design and Testbench Source Files. The Testbench sub-directory contains the DDR-AXI Slave ARM model for verification of the ARM sub-system memory controller.

synthesis

Synopsys Synthesis Scripts.

An example implementation script is provided to show the list of files and their order. If you have the Synopsis/Synplify package installed on your computer, executing the <SynopsisLint.bat> file will perform a full scale synthesis check on the DMA package as stand-alone.

Note: You must have Synopsis tools installed on your PC running Windows XP or higher.

documentation For the customer package, this directory is at the same level as the above and contains all of the PDF documents for this product.

LinuxDriver Contains the entire sub-tree of all the C code, build files, make script and binaries of the device driver software.

It is the last directory in the table (above), LinuxDriver that contains all of the source code, build and make files, configuration files and kernel object compiled target. In a Microsoft Windows operating system environment, the scripts and GCC compilation process does not exist and so you can only view the files in the development tree. If using Linux/UC-Linux or a Linux platform running under Windows (Cygwin, Mingw etc) then the tree structure with in the LinuxDriver directory is the whole source code environment.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

14

The only other tools that must be globally present in the Linux system is GCC, make and BASH. Once in the <LinuxDriver> directory you see include <directory> object <directory>

Makefile, moduleparms, mtipDMAcontroller_main.c, mtipDMAcontroller_dmaCORE.c, mtipDMAcontroller_extraHW.c, mtipDMAcontroller_macFIFO.c, mtipDMAcontroller_switchFIFO.c, mtipDMAcontroller_UIO.c, mtipsw_dma_s.ko

include <directory> - All C header files for the DMA device driver

object <directory> - DMA device driver target object file compilation dump Makefile - The main build system Makefile script

moduleparms - DMA device driver settings file upon driver registration mtipDMAcontroller_main.c, - DMA device driver kernel object structured main source mtipDMAcontroller_dmaCORE.c, - Portable Send(), Receive(), Open(), Close() functions for DMA main functionality written to be generic. mtipDMAcontroller_extraHW.c, - Portable functions for extra device setup for specific custom hardware for instance bespoke target device initialization via IO Register access (initialization and shutdown only). mtipDMAcontroller_macFIFO.c, - Portable functions for the possibility of MAC IP core setup for the attached hardware to the DMA via IO Register access (initialization and shutdown only). mtipDMAcontroller_switchFIFO.c, - Portable functions for the possibility of SWITCH IP core setup for the attached hardware to the DMA via IO Register access (initialization and shutdown only). mtipDMAcontroller_UIO.c, - Specific Linux structured and bound source file to allow application software in Linux to send/receive sideband data to the RX and TX of the DMA device driver and ten through to/from the DMA hardware and MAC/SWITCH interconnection.

mtipsw_dma_s.ko, - If not cleaned, this file exists as the last (delivered default) kernel object of the DMA device driver

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

15

Within the include sub-directory: implementation.h – C header file – #define controls for the entire AMBA DMA

Controller device driver in the previous directory

mtipDMAController.h – C header file – Used as an include for DMA Controller mtipmacFIFO.h – C header file – MAC FIFO interface C definition for DMA Controller mtipUIO.h – C header file – UIO memory base access include file for the DMA Controller mtipextraHW.h – C header file – Specific hardware platform include file for the AMBA DMA Controller in the previous directory mtipswitchFIFO.h – C header file – SWITCH FIFO interface C definition for DMA Controller

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

16

3.8 Software Device Driver Structure

Figure 4 - Device Driver Schematic Structure

The only interfacing that is performed from the bottom of the Linux kernel IP stack to the network layer hardware is the above illustrated device driver that this document describes. Furthermore, it is not execution exhaustive. It is only the setup of the transmit buffers or the indication of received packets that is processed by the bottom of the Linux stack device driver as far as the ARM processor execution is concerned. The AXI bus is used but is driven by the DMA Controller hardware core in parallel to the ARM processor. In effect, the IO-Register Map and the IRQ interrupt service routine is the bottom of the visibility of the software environment. If any waiting has to occur, then operating system sleep commands and task relinquishing are invoked to allow much needed execution time to go to important processing such as the application layers.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

17

3.9 Descriptors

Figure 5 - General Principle of a Descriptor Table

The fundamental principle of the DMA device driver for the Linux operating system is to provide minimal processing overhead beneath the kernel IP stack so that the API application layers can obtain most of the processing time. In order for this to happen, the DMA hardware has to be efficient as possible with its transactions and especially the interaction with the device driver module. The more there are Ethernet frame packets that are handled combined for both TX and RX in the DMA hardware the less processing time used in software. The memory reference pointers in each entry of the table can be to anywhere in the physical address space that is allowed under the rules of the operating system. Using kmalloc() or new() will usually allocate a buffer that can be used. However (as described later in the manual) the driver always makes sure that the allocation is adjusted to be 32-bit/64-bit align depending on the architecture size of the bus that a user specifies. The buffers are allocated and initialized at device driver startup and stay allocated ready for use during operation until the driver is closed and de-registered. However, only the RX buffers are fixed and re-used. The TX Ethernet frame buffers are passed down from the IP stack as it is the upper stack or the application that is in control of the memory allocation for TX.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

18

4 Driver Installation

4.1 Introduction

With the device driver for Linux that is bundled with the DMA Controller package, it will handle the management of the DMA hardware in the host silicon device from the associated ARM processor also resident with the uC-Linux/Linux system.

4.1.1 Linux Commands

Command Description

lsmod list the loaded kernel modules

rmmod unload a loaded kernel modules

insmod load a kernel module (.ko file)

make recompile kernel modules

dmesg print kernel buffer (e.g. boot messages, driver debug output)

nano standard Gentoo text editor

emerge install new software (requires Internet connection)

man show man page

lspci show the available PCI/PCIe devices

ifconfig show/set network settings (option -a shows inactive interfaces)

route show/set routing table

ip advanced manipulation of routing tables (package: sys-apps/iproute2)

netio TCP/UDP benchmark

rc-update add/remove boot scripts

dhcpcd DHCP client

patch apply a patch (diff output)

Table 2: Typical and Relevant Linux BASH Commands

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

19

4.2 Make Files

The bundled source code for the DMA hardware package deliverables is a directory of source files; make files and a pre-compiled [.ko] Kernel object module already configured, compiled and built for the ARM-cortex processor running uC-Linux. This accepts no DDR management and a JFFS flash file system. It should just run when installed on the software bundle that ARM distributed. This is vendor independent in terms of the silicon device (whether FPGA, ASIC or different manufacturers).

The make file is automatically invoked when you type the command “make”. This automatically compiles all the associated source and header files into an object file then to a Kernel object module.

You do not have to clean the make targets first. It is a very simple process and utilizes the GCC toolset provided by most distributions of Linux. However if you do not have them installed prior to building with [make] then please refer to your Linux distribution guidelines on installing new components such as “emerge” with Gentoo or “apt-get” with Debian/Ubuntu.

For instance “emerge –av gcc” could be what you type to install the GCC toolset on a Gentoo Linux distribution. However, if you are not intending to make any deviation from the standard ARM uC-Linux distribution provided with development boards that you are wishing to use on your target hardware device, then you can just use the provided [.ko] already packaged and included with the deliverables.

4.3 Installation and Kernel Object Driver Registration

The installation of the device driver into the silicon device ARM processor’s Linux environment is reasonably straight forward:

4.3.1 Installation Instructions

1. Power UP the silicon device (FPGA/ASIC) with the ARM processor ready. If on a Linux PC, power-up, login and establish a terminal screen.

2. Configure the device through the JTAG connector. This loads the [.stp/.sof] IP hardware cores from the target device project IDE (such as Libero or Quartus). This is hosted in the Fabric logic next to the ARM processor. You may need to reboot the unit after. This could be an ARM based FPGA development board PCB or your custom in-house designed PCB solution. If you are using a Linux PC just ensure that the PCI express or PCMCIA attached hardware (that will host the DMA Controller and Ethernet hardware) is enabled, programmed and operational.

3. Compile the device driver if necessary. Change directory to the provided driver. If on a Linux-PC type "cd LinuxDriver". If you are using a target cross-compiled environment then ensure you change into the base directory of that package.

4. If on an ARM cross-compilation environment also type "source ACTIVATE.sh" then "./starttftp". The TFTP server is started here ready for kernel file uploads from the /home directory. Note: This does not have to be done if compiling for a Linux-PC.

5. Type “make" then press ENTER. If you are confident that the provided Linux kernel and distribution are sufficient for your product solution, then skip this instruction as you already have provided a Kernel object file [mtipsw_dma_sw.ko] in the directory.

# cd [BASE DIRECTORY]

# make

4. At this stage, you have (irrespective of environment and irrespective of whether you did type "make" - as there would already have been a kernel object in the directory by default) a compiled device driver [.ko] file in the directory now.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

20

4.3.2 Linux-PC

If you wish to try out the device driver immediately and get it running just temporarily until removed or the system is rebooted, type "make run". This installs then initializes and shows the status of the registration on screen. You should see the device drive and the [ifconfig] exposed /dev/eth0, /dev/eth1, /dev/eth2, /dev/eth3.

Note: The name “eth” is default only and shall be configurable.

Once you are happy, and you have tried out the device driver, and maybe gone through an iteration of editing code, recompiling and the running as above, you can do the following to install permanently into the Linux kernel:

Then type "make install". Once you are ready and happy, reboot the Linux-PC. You can check the device driver once re-powered by typing "lsmod".

4.3.3 Cross-compiled target embedded device

Having typed "make" in the base directory of the development tree, you are already in a position to download the object uC-Linux kernel file into the target FPGA. The significant difference here as opposed to a Linux-PC full Linux distribution is that this build process generates a full [vmlinuz] kernel package. This is because uC-Linux bundles all modules into the core kernel and has no libraries or attached modules.

Therefore, there is no device driver registration and/or installation. We just have to ensure that this full uC-Linux kernel package is transferred into the FLASH/ROM of the ARM processor sub-system.

Type "./savetftp". This copies the package into the TFTP home directory (/home) on the Linux PC you are cross-compiling on.

For MicroSemi SmartFusion2, for instance, in u-boot on the USB terminal type "run update" then "reset". This is very specific so you should refer to the Emcraft Software documentation from http://www.emcraft.com/

4.3.4 Check the UIO

Check if the UIO interface is available that should have been registered and setup upon device driver instantiation

# lsuio

-> should show "mtipsw-dma-sw"

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

21

4.4 Configuration by Parameter Passing

The device driver can be configured how to behave along with a host of other parameters. This is done upon driver installation. If no parameters are passed then default values are assumed (like the example of module installation in the previous section).

It is generally accepted that the decision on parameters is taken when a user designs the project and decides how the device driver should be setup to run. Therefore, being able to set the configuration at driver installation time allows the user in control of the API application programs to set the behavior of the performance of the device driver for the DMA controller hardware.

4.4.1 DMA Controller Device Driver Parameter List

The parameters for the Linux device driver are kept in <moduleparms> file in the driver base directory of the delivered package. This is a text file but is not a C header file or any other programming language format. In fact, it is a standard scripting format that is piped into <xargs> and <insmod> Linux commands upon device driver registration and instantiation. This is the same for both a Linux-PC build and a cross-compiled embedded target device with uC-Linux.

The parameters are listed below and categorized along with their default presumed value if not mentioned in the <moduleparms> file.

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

22

Table 3: Default parameter values in C Header file for the DMA Device Driver

Variable storage type Parameter Name Default software device driver value before if not assigned in configuration file

static unsigned short int mtip_rxfullthresh 63

static unsigned short int mtip_txfullthresh 63

static unsigned short int mtip_rxemptythresh 0

static unsigned short int mtip_txemptythresh 0

static unsigned int mtip_IObase 0xFF200000

static unsigned int mtip_fabricSDRAM_base 0

static unsigned int mtip_mssSDRAM_base 0

static unsigned int mtip_axiburst 1

static unsigned int mtip_axitout 1

static unsigned int mtip_axiholdoff 0

static int mtip_irq -1

static unsigned short int mtip_rx_xstatrange 4

static unsigned short int mtip_tx_xstatrange 4

static unsigned short int mtip_rx_ceoltout 0

static unsigned short int mtip_framebuffermax 1900

static char mtip_axiaccess normal

static char mtip_protocolchecksum off

static char mtip_axirmw off

static char mtip_netnameprefix mtip

static char mtip_FIFO mac

static char mtip_linkup up

static char mtip_extrahardware no

static char mtip_debug no

static char mtip_diagnostic no

static char mtip_rxdescfullpolicy group

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

23

Figure 6 – Typical layout of the <moduleparms> file in the base directory of the Device Driver

4.4.2 DMA Controller Device Driver Parameter Definition and Meanings

Table 4: Definition of <moduleparms> file for the DMA Device Driver

Parameter Name Description Value Range

mtip_rxfullthresh Amount of RX descriptor entries to activation RX FULLTHRESH IRQ

0 to 63

mtip_txfullthresh Amount of TX descriptor entries to activation TX FULLTHRESH IRQ

0 to 63

mtip_rxemptythresh Amount of RX descriptor entries to activation RX EMPTYTHRESH IRQ

0 to 63

mtip_txemptythresh Amount of TX descriptor entries to activation TX EMPTYTHRESH IRQ

0 to 63

mtip_IObase Physical Memory Address of start of hardware configuration registers

0 to FFFFFFFF

mtip_fabricSDRAM_base Physical Memory address of main SDRAM DDR device from the perspective of the CPU

0 to FFFFFFFF

mtip_mssSDRAM_base Physical Memory address of main SDRAM DDR device from the perspective of the AXI-Master DMA Controller in the FPGA Fabric/ASIC

0 to FFFFFFFF

mtip_axiburst How many maximum burst can be performed with a Read or Write AXI transaction

1 to 16

mtip_IObase=0xFF200000

mtip_fabricSDRAM_base=0x00000000

mtip_mssSDRAM_base=0x00000000

mtip_irq=-1

mtip_extrahardware=no

mtip_FIFO=mac

mtip_linkUP=up

mtip_debug=no

mtip_diagnostic=no

mtip_axiburst=1

mtip_axitout=2

mtip_axiholdoff=0

mtip_axiaccess=normal

mtip_axirmw=on

mtip_rxfullthresh=63

mtip_txfullthresh=63

mtip_rxemptythresh=0

mtip_txemptythresh=0

mtip_rx_xstatrange=4

mtip_tx_xstatrange=4

mtip_rx_coeltout=0

mtip_rxdescfullpolicy=group

mtip_framebuffermax=1900

mtip_protocolchecksum=off

mtip_netnameprefix=mtip

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

24

mtip_axitout How many blocks of 256 clock cycles of the memory bus can elapse without a response of either <data> or <address> signals before AXI Master hardware transaction aborted and error recorded in descriptor entry. So 2 represents 512 clock cycles etc

1 to 255

mtip_axiholdoff After an AXI RX/TX burst action, how many clocks cycles to wait before the next burst. This occurs even if burst=1. This allows in more complex processor sub-systems to event of allowing the AXI bus to not be constantly asserted by the DMA Controller.

Note: Zero disables

0 to 255

mtip_irq IRQ number in Linux/uC-Linux of the main DMA Controller hardware IP core connected to the processor sub-system

-1 (disabled)

1 to 255

mtip_rx_xstatrange How many of the 32-bit word in the RX descriptor entries are in active use

0 to 4

mtip_tx_xstatrange How many of the 32-bit word in the TX descriptor entries are in active use

0 to 4

mtip_rx_ceoltout How many milliseconds of no RX DMA received frames before we activate COEL IRQ

0

mtip_framebuffermax Maximum number of bytes (excluding preamble) permitted through RX/TX DMA control pathways. Additional data is truncated and error recorded in descriptor entry

0 to 16834

mtip_axiaccess What type of AXI Master transaction is carried out

normal / exclusive / locked / reserved

mtip_protocolchecksum Which TCP/UDP checksum function is undertaken? This is for efficiency of DMA process – IP stack can be slow. The device driver can perform this but ideally if the MAC/SWITCH supports, it is best if the hardware calculates and the IP socket checksum flag is disabled

off / driver / hardware

off = Leave checksum to IPstack

driver = Driver calculates checksum

hardware = MAC/SWITCH calculates

mtip_axirmw Enable the Read-Modify-Write AXI sideband feature. Used if target device processor sub-system has transaction control of whether to AXI write entire data width overwriting memory or read-modify then write back (useful for last partial word writing)

off / on

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

25

mtip_netnameprefix What network name shows up in Linux such as /dev/eth[X] or /dev/<yourname>[X]

<Any Text Name>

For instance: eth / mtip / net

mtip_FIFO What interface is the DMA connected to on the network side

none / mac / switch

mtip_linkup Mechanism to inform Linux kernel of the status the physical network link

down / up / bit(register, bit, polarity)

If option = bit then bracket parameters are:

register = MAC/SWITCH IO Register to read for linkup detection bit = 0 - based bit position within IO register polarity = 1 or 0 that is XORed with the linkUP bit Example: mtip_linkup=bit(0x30403FF0, 3, 1) If mtip_linkup just set to “up” or “down” then that remains permanently reported as that state

mtip_extrahardware Is there any specific target hardware to be initialized of bespoke requirements

no / yes

mtip_debug Switch to control the debug into the current /dev/tty console

no / yes

mtip_diagnostic Enables or disables the device driver diagnostics mode. If enabled, the driver upon initialization enters a forever iteration of test-bench combination testing of TX sizes, burst sizes and expects the Ethernet frames to be looped back in hardware to the RX DMA. The results are displayed on screen.

Note: You must have the hardware in loop-back mode or wired to be looped back from TX to RX on the MAC.

no / yes

mtip_rxdescfullpolicy Configures the RX-DMA channel selection on how to behave when required channel is approaching FULL

group / individual

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

26

4.4.3 Bring Parameter changes into effect

If/when the above parameter(s) have been changed to the desire settings, save the file <moduleparms> and follow the instructions earlier in the overall section to re-build the uCLinux kernel using “make”.

Program the target device with the new uCLinux compressed file, or if using a Linux-PC as your target with a full Linux system then install the driver.

4.4.4 Controlling DMA using the <implementation.h> header file at compile time

Figure 7 – File contents of <implementation.h>

/* ***************************************************************************** * [implementation.h] - Header file for the target device platform definitions * ******************************************************************************* * Copyright (C) 2013 Nine Ways Research & Development Ltd (UK) AND Morethanip GmbH. All Rights Reserved. * Address : iDCentre, rtc Business Park, London Road, DE24 8UP. (+44)-(0)1332 258847 * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * Author(s): Mr Paul. P. Bates of Nine Ways R&D Ltd: [[email protected]] [[email protected]] *- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * <TODO> list: * *- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * Revision: Date: Author: Change Description: * 0.10 02/01/2014 Paul Bates Initial file coding before release *- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - *$Id: [implementation.h] 001 2014-01-01 10:31:00Z tshu $ **************************************************************************************/ #ifndef IMPLEMENTATION_H_ #define IMPLEMENTATION_H_ /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -* /* CHANGE THE SETTINGS IN THIS SECTION */ /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -*/ /* Select from the following what technology and development platform you wish to compile */ //#define PLATFORM_ARCH_SMARTFUSION2 /* Enable extra specific call-back hardware routines for SmartFusion2 */ #define PLATFORM_ARCH_ALTERA_C5SOC /* Enable extra specific call-back hardware routines for Altera CycV-SOC */ /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ /* Select from the following what FIFO interface IP-core hardware support you wish to compile */ //#define SWITCH_MORETHANIP_5PORT /* Enable the detection and checking of the Industrial 5-Port Blackbox */ //#define SWITCH_MORETHANIP_5PORT_DIAGNOSTICS /* Use diagnostics and never exit for the 5-port SWITCH */ //#define SWITCH_MORETHANIP_3PORT /* Enable support for initializing the MTIP 3-Port SWITCH */ //#define SWITCH_MORETHANIP_3PORT_DIAGNOSTICS /* Use diagnostics and never exit for the 3-port SWITCH */ #define MAC_MORETHANIP_ANYSPEED /* Enable support for initializing the MTIP 3-Port SWITCH */ /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ /* Select from the following what PHY IC physical hardware support you wish to compile */ //#define PHY_NETFUSION /* Enable control and support for the Marvell Alaska-QuadPHY */ //#define PHY_NETFUSION_FIBER /* Enable control and support for Netfusion-SFP Fibers */ //#define PHY_M2SSOM /* Enable control and support for the [PHY-M2S-SOM] */ #define PHY_RENESAS_UPD606 /* Enable control and support for the Altera Renesas [uPD606 PHY] */ //#define SWITCH_MORETHANIP_MARVELL_QUADPHY_DIAGNOSTICS /* Status of 3 ports on NetFusion after auto-negotiation */ //#define SWITCH_MARVELL_QUADPHY_LIMIT_TO_100MBPS /* Stops Marvell QuadPHY ever negotiating 1000Mbps(1G) */ /* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - */ /* General Device Driver Global Definitions */

//#define SUPPORT_1G_MODE_GMII /* Enables local MAC interface (even those in a SWITCH) to run at 1Gbit/s */ #define MTIPSW_DMA_SW_MEMORY_ALIGN 8 /* Select the AXI architecture size 4 = 32-bit and 8 = 64-bit AXI width */ #define MTIPSW_DMA_SW_VIRTUAL_MMU_ENABLED /* Configures the DMA DEvice Driver to handle Linux Virtual pointers */

#endif /* IMPLEMENTATION_H_ */

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

27

The DMA Controller device driver is not just controlled by the moduleparms text file, it is also governed by this very important C header file in the include sub-directory. Anything run-time changeable generally resides in the moduleparms configuration text file as this is used to inject those parameters into the instantiation of the device driver at boot-up. Changes to this file do not require a re-compilation of the source code of the driver. However, for the more fundamental compilation settings and configuration of the main features, the implementation.h is used to allow the user to change and experiment with the DMA Controller functionality. The following lists the available #defines in the code:

(1) MTIPSW_DMA_SW_MEMORY_ALIGN (2) MTIPSW_DMA_SW_VIRTUAL_MMU_ENABLED

MTIPSW_DMA_SW_MEMORY_ALIGN

#define that MUST always be defined and indicates the byte alignment of the architecture of the AMBA DMA Controller IP core that is used with the 3-Port SWITCH. 4 = 32-bit and 8 = 64-bit

MTIPSW_DMA_SW_VIRTUAL_MMU_ENABLED #define that if desired, compiles the DMA device driver to handle the virtual/physical MMU platform on the Linux OS

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

28

5 Contact

MorethanIP GmbH

E-Mail : [email protected]

Internet : www.morethanip.com

Europe

Muenchner Str. 199

D-85757 Karlsfeld

Germany

Tel : +49 (0) 8131 333939 0

FAX : +49 (0) 8131 333939 1

Nine Ways Research & Development Ltd

E-Mail : [email protected]

Internet : www.nineways.co.uk

UK

Unit A.3, iDCentre, Lathkill House, rtc Business Park

London Road, Derby. DE24 8UP

United Kingdom

Tel : +44 (0) 1332 258847

FAX : +44 (0) 1332 258823

Table 5: Contact Information

AMBA DMA Controller (with AXI memory interface)

Linux Driver User Guide Version 1.1 - January 2015

29

6 Document History

Document Change Notices (DCO)

Version Description Created/Changed By Date

Version 1.0 Initial Release according to

Version 1.0 Paul Bates: Nine Ways 31st October 2013

Version 1.1 Updating document for new

C source files, scripts, PTP-

1588 support and build

system changes.

Paul Bates: Nine Ways 20th January 2015

Copyright © MorethanIP GmbH 2013. All Rights Reserved.

Table 6: Document History Entry Log