How to Get the Most Out of 2D: The Fujitsu “Iris” Graphics Engine

Fujitsu Technical Backgrounder 2

The importance of 2D graphics

Sophisticated 3D graphics capabilities are starting to appear in embedded devices such as cell phones and tablets. Still, 2D graphics are still needed in applications that either use 2D graphics only, or use both 2D and 3D capabilities. For example, POS terminals, industrial devices, and mid- to low-end automotive graphics applications only need 2D graphics. On the other hand, high-end automotive navigation systems may need both 2D and 3D for GUI implementation. This makes sense as the 3D engine might need to concentrate on the really graphics-intensive operations.

Given the importance of 2D in today’s embedded applications, this white paper will discuss two common approaches to 2D graphics in embedded applications: raster and vector graphics. The paper will then compare the two techniques and present a custom approach, the Fujitsu “Iris” 2D graphics engine, which combines their benefits. The subject will be approached only from the point of view of embedded applications, including automotive applications.

3D image visualization example

2D automotive display example

2D / 3D hybrid automotive display example

Fujitsu Technical Backgrounder 3

It is assumed that the reader is familiar with embedded systems and terms such as Flash ROM, CPU RAM, and VRAM as they are used in embedded world. The discussion is based on typical system configurations such as the two shown in Figure 1.

Figure 1: (a) Multi-chip Embedded Graphics Architecture

Figure 1: (b) Single-chip Embedded Architecture

Raster Graphics Engines

Raster graphics rely on pre-rendered images or bitmaps for operation. If it’s a multi-chip application, the bitmaps will typically be stored in the off-chip CPU Flash ROM. In a true single-chip solution, they will reside in a dedicated external Flash chip, which can be connected via a serial or parallel bus to the graphics chip. Raster engines use the Block Transfer (BLT) operation to transfer the bitmaps to the graphics engine. BLT operation is similar to memcopy operation.

There are two common architectures for raster graphics engines, as shown in Figure 2. One uses the line-buffer mechanism. This technique does not need a frame buffer for

pre-composing scenes. It fetches bitmaps (sometimes called sprites) one line at a time. The advantage is a smaller or no VRAM requirement, but the engine can quickly run out of steam if the application demands fairly dynamic graphics content.

The other architecture uses a frame buffer in VRAM. All bitmaps that are supposed to show up on the display are BLTed from Flash memory into VRAM ahead of time. The graphics engine will combine the bitmaps in the desired order to create the final scene, which will then be sent to the display. The choice of architecture depends on the application. This type of raster graphics architecture is useful if the application needs a really dynamic visual.

Figure 2: Sprite and BLT Engines

Vector Graphics Engines

Vector graphics engines are much more powerful than raster engines. In addition to handling pre-rendered bitmaps, vector engines can generate dynamic graphics elements on the fly, and can draw a 2D object using paths/primitives based on an input of mathematical equations. The engine can then give the object a real-life look using color fills and texture maps. Such engines can also translate, rotate, and scale the 2D objects using matrix multiplication.

The most popular vector graphics engine in embedded systems is OpenVG 1.1, which is based on an open standard defined by Khronos. Every semiconductor company will have its own implementation of this standard. The power, performance, and efficiency depend on the design. Figure 3 shows the OpenVG pipeline concept from Khronos.

Raster engines are simpler and less expensive to design and implement (than vector graphics engines).

Vector graphics engines are much more powerful than raster engines.

GPU, Multi-Chip

CPU

CAN MCU

Flash

Video In

RAM

Display

I2C, UART, etc.

GDC

Peripherals

Single Chip SOC

32-bit MCU

CAN, 12C, etc.buses

FlashVideo In

VRAM

Display

GDC

Peripherals

Display C

ontroller

Ext Flash Display

Sprite Engine

Line Buffet (e.g. 30K

B)

Display C

ontroller

Ext Flash Display

VR

AM

(e.g. 2MB

)

BLT Engine

Fujitsu Technical Backgrounder 4

Figure 3: OpenVG Pipeline Concept [Image based on graphic from http://www.khronos.org/openvg/]

Comparison of Raster and Vector Graphics Engines

Raster and vector engines have their pros and cons.

Raster engines are simpler and less expensive to design and implement. They rely on pre-rendered graphics and don’t have to do dynamic computations during operation. They can be designed without much difficulty using BLT-ers or memory-fetch units that transport bitmaps from permanent storage (Flash) to either a line buffer or VRAM. However, memory storage and bandwidth requirements for these engines can become excessive if the application requires long animated sequences or sophisticated effects, such as image scaling or rotation. Another drawback of raster engines is that image quality can be adversely affected if they need to scale up the image.

On the other hand, vector engines are excellent at handling dynamic graphics. They can create 2D objects from scratch using paths/primitives, color fills, and texture maps, based on a set of mathematical equations. Another benefit is the engine’s low memory size requirement. These engines do not need bitmap sequences to show animation. They can translate, scale, and rotate 2D objects without consuming much memory. They can also handle fonts well. However, this comes at the price of a complicated design. The vector engine or IP cab costs a lot as well, and these engines still need significant memory to show sophisticated graphics at a high frame rate.

Stage 1:Path, Transformation, Stroke and paint

Stage 2:Stroked Path Generation

Stage 3:Transformation

Stage 4:Rasterization

Stage 5:Clipping and Masking

Stage 6:Paint Generation

Stage 7:Image Interpolation

Stage 8:Blending and Antialiasing

TT*

Raster engines rely on pre-rendered graphics, while vector engines are excellent at handling dynamic graphics.

Fujitsu Technical Backgrounder 5

The Fujitsu “Iris” 2D Graphics Engine

Fujitsu has successfully balanced the two approaches in the “Iris” graphics engine. This IP combines the best of the two worlds: raster and vector graphics. Since the engine is based inherently on raster graphics, it is simpler than vector engines and has significantly less IP design and development cost. At the same time, through its bitmap-rotation and scaling capability, “Iris” requires much less memory and bandwidth than conventional 2D raster engines. The rotation and scaling capability allows it to show sophisticated and dynamic effects, such as needle draw or album cover turnstile, at a high frame rate.

The Fujitsu “Iris” 2D Graphics Engine

“Iris”, one of the key features of the Fujitsu FCR4 Cluster SoC family, is also a key technology included in the MB86R1x Emerald family of 2D/3D graphics SoCs. “Iris” is a modular system of building blocks, which can be combined to find the optimum scalable solution for customer requirements.

Key features of the “Iris” 2D graphics engine include:

Display controller / TCON• 4 display layers of overlay

• Video modes up to 40MHz pixel clock (e.g., QVGA, VGA, WVGA)

• TTL/RSDS interface with 12 freely programmable pulse generators

• Programmable phase shift for panel clock

• Horizontal/vertical flip

• 90/180/270 degree rotation

• LUT for non-linear correction

Signature unit (safety)• Per-pixel mask feature

Command sequencer• DMA operations

• Sub routines

• Synchronization to “Iris” events

• Unloads system CPU

• Memory mapped command FIFO

“Iris”pixel engine• Re-configurable concept (functionality can be selected

for blit or display)

• Optimized for memory saving

• Freely configurable pixel formats (packed 1, 2, 4, 8, 16, 24, 32 bpp)

• Color palettes

• Run-length decoding

• Single-pass rendering of blit sequence operations (no temporary buffers)

• Bitmap decompression with clipping on the fly (no need to store decompressed bitmaps)

• One-pass combinations of various operations (no need to store intermediate results)

• Multiple blend operations in one pass (no need to store intermediate results)

• Supported blend modes (constant + alpha, pre-multiplied)

The Iris graphics engine combines the best of the two worlds: raster and vector graphics.

Fujitsu Technical Backgrounder 6

“Iris” Architectural Details

In essence, “Iris” is a fast pixel engine that can rotate and scale pre-rendered graphics data, and handle compressed bitmaps. A block diagram is shown in Figure 4. All units in the pipeline work in synchronization with each other, courtesy of a signal that flows from display to start of the pipeline. Consequently, there is no need to flush the pipeline.

“Iris” is a fast pixel engine that can rotate and scale pre-rendered graphics data, and handle compressed bitmaps.

SPI I/OPAD

SPI PIX

CMD

IRIS

QUADSPI Core

AHBLogic

QBITSPI

AXI MasterInterface

AXI SlaveInterface

TCONDISP

Sig_err

AsyncFIFO.

TCONCore

4xRAM256

4xRAM256

MemoryInterface

Command Sequencer CoreDisplay Controller

Core

Pixel EngineCore

Config. WhiteAgent

ReadAgent

ConfigMemoryInterface

AXI64RW AXI64RW

ExternalDisplayInterrupt

Sub IP

MasterPort

SlavePort

Boxclock

Pixelclock

SPIclock

Global

DISPIOTCON_PixelBus_8bitSIG_PixelBus_10bit

Config.

SignatureCore

Config. Config.

64-bit Multi-Layer AXI Bus

32-bit AHB Bus

Fetch 0.7 Store

SIG

SignatureCore

Config.

GCTR

Protocol

Figure 4: “Iris” Block Diagram

Here is a brief description of the individual blocks and how they help “Iris” combine the best of the raster and vector-graphics techniques.

1. Pixel EngineAs Figure 4 shows, this block is the heart of the “Iris” engine. This flexible pixel-processing unit can have two instantiations

of pixel pipelines, one for blit and the other for display. The engine, which can be configured through an AHB interface, offers interrupt signals. Its main sub-units are its fetch, store, and core pixel-processing units. Their functions, in respective order, are to fetch pixel data from memory, store, and process them as needed.

Fujitsu Technical Backgrounder 7

2. Fetch UnitThis is a key part of the pixel engine. As the name indicates, the fetch unit gets pixel data from ROM or RAM storage units at high speed. It allows an image size as high as 1024x1024.

There are three different types of fetch units, depending on the action the unit performs on the graphics data while fetching.

• ROT: Rotates the incoming bitmap

• RLD: Decompresses the run-length encoded bitmap

• Light: Alpha blends the incoming pixel data

The fetch unit will start working after it receives a kick signal from the display side. It also performs color conversion on the graphics data. Optionally, it can also perform bilinear filtering, which is useful when the image is being rotated. Right before the graphics data is sent out of this block, the unit performs multiplications with constant color/alpha or an alpha pre-multiplication.

3. CLUTThe Color LookUp Table can be used to either look up or index colors. In the first role, the table can be used to neutralize non-linearity of color transmission or adapt the individual characteristics of the display panel. In the second role, the table contains reduced-length color code that points to the raw color values. This can help reduce memory consumption.

The block consists of a RAM with 256 entries. Each entry is 10-bits wide for each color component. CLUT is fully software programmable.

4. MatrixThis unit performs linear color correction or conversion (e.g., YUV to RGB conversion). The unit allows for full programming of the matrix and offset coefficients, and supports 3x3 matrix size. The unit can operate in one of the following three modes:

• Neutral mode: The unit does not perform any operation on the incoming data.

• Matrix mode: The incoming pixel data goes through a matrix conversion, as discussed previously.

• Pre-multiplication mode: The incoming pixel value (RGB) is pre-multiplied with the incoming alpha value.

5. ScalarAs mentioned earlier, a key functionality of “Iris” is the ability to upscale or downscale bitmaps through the scalar unit. This brings it closer to vector graphics engines. The unit is comprised of two components:

• for horizontal scaling

• for vertical scalingThe scalar can handle bitmap sizes from 1x1 up to 1024x1024. It also supports point and bilinear filtering to allow fine image quality.

6. ROPThe ROP unit performs the standard raster operations (e.g., AND, OR, NOR) on pixel values. This unit can take up to three input channels of pixel data.

7. BitBlendThis unit performs various blending operations on two input pixel sources. It performs both OpenVG 1.0 and OpenGL 2.0 blending modes.

8. LayerBlendThis unit is used to blend or merge two video layers. It can either perform an alpha-blending operation between the two layers or can just operate in a transparency mode with one layer getting higher priority over the other.

9. StoreThis unit stores pixel data received from the pixel engine into memory. Image sizes up to 1024x1024 are supported.

10. ExtDstThis is an output interface of the pixel engine for the pixel stream. It is used to pipe the stream to the display controller so that the content can be shown on the LCD screen.

11. Display ControllerThis unit generates the required control signals for operating a display in RGB mode using HSYNC and VSYNC signals as timing references. The pixel data from the pixel engine is transmitted to the display controller through a FIFO interface. The unit supports display clocks up to 40MHz.

The display controller has a dithering unit that can help smooth the appearance of image data on a low-resolution display panel. Both spatial and temporal dithering modes are supported.

12. Timing ControllerThis unit, usually referred to as TCON, generates control and data signals to directly communicate with the column and row drivers of a display panel. The freely programmable waveform of the generated timing control signals allows emulating almost every timing controller IC. The unit supports both single-ended TTL and differential RSDS signaling modes.

Reconfigurable Architecture“Iris” has a reconfigurable architecture. All the blocks can be flexibly organized to create a pipeline according to the application needs. For example, the number and type of fetch units can be changed. The pixel data can either be stored in memory through the store unit or sent to the display using extdst. In between the two operations, the pipeline can have many different combinations of pixel-processing units. Two possible configurations are shown in Figure 5.

Fujitsu Technical Backgrounder 8

Figure 5: Examples of Flexible Configurations of the Pixel Engine Pipeline

Conclusion

In short, “Iris” is a powerful 2D raster engine that combines the best qualities of raster and vector architectures in a flexible manner. Because of its raster architecture, the IP design and development cost is low. At the same time, its memory requirement is low because of the capability to support scaling and rotation functions, which typically have been a hallmark of vector engines.

The scalable engine can be used in either a 2D-only or a 2D/3D device that has an OpenGL-ES 2.0 type, high-end graphics engine. “Calypso” is an example of an “Iris” 2D only implementation. This single-chip SOC has an ARM® Cortex™ R4

CPU. “Iris” allows “Calypso” to sport powerful 2D graphics while keeping the graphics memory footprint low.

The MB86R1x “Emerald” family is an example of a 2D/3D configuration. This SOC has an ARM Cortex-A9 CPU and OpenGL-ES 2.0 engine with a fully programmable shader. The presence of “Iris” in the system allows the 3D engine to focus fully on the 3D aspects of the graphics while “Iris” takes care of the 2D side, such as HMI components.

The result is that organizations can more quickly and cost-effectively meet the ever-increasing need for 2D and 3D graphics in embedded applications.

Default display path

Default bit path

Switchable unit

Data path switch

Fetch #2(Rot)

Fetch #3(RLD)

Fetch #2(Rot)

Fetch #2(Rot)

Fetch #2(Rot)

Fetch #2(Rot)

Fetch #2(Rot)

Fetch #2(Rot)

Display Controller

Fetch #7(Light)

Fetch #1(Light)

Fetch #0(RLD)

Fetch #2(Rot)

LUT#0

Backgr. Mask

Foregr.

A S Rop#0

A S

Matrix#0

BltBlend#0

Scaler

Store#0

Memory

Bllt LayerBlock

Scaler

Backgr. Mask

FUjITSU SEMICONDUCTOR AMERICA, INC.Corporate Headquarters1250 E. Arques Avenue, M/S 333, Sunnyvale, CA 94085-5401Tel: (800) 866-8608 Fax: (408) 737-5999E-mail: FSA_inquiry@us.fujitsu.com | Website: http://us.fujitsu.com/semi

8

© 2012 Fujitsu Semiconductor America, Inc.All company and product names are trademarks or registered trademarks of their respective owners.

Printed in the U.S.A. GDC-TB-21418-03/2012

The FCR4 Cluster Family

The Fujitsu FCR4 Cluster family of devices offers innovative and scalable solutions for hybrid clusters that combine traditional meters and graphical displays. With their embedded Flash, the FCR4 MCUs and SoCs can serve as single-chip solutions or operate as companion chips for other devices to build high-performance systems for virtual or free programmable clusters.

FCR4 devices offer a powerful architecture based on the ARM Cortex R4 core and the Fujitsu “Iris” 2D graphics engine. The family includes the required safety and security features, meets today’s low power requirements, and fulfillsthe highest automotive quality standards.

The MB86R1x “Emerald” Family

The MB86R1x “Emerald”family is a 2D/3D graphics with integrated GDC and GPU. The series features an ARM Cortex-A9 processor, an enhanced “Ruby” graphics core and an “Iris” 2D graphics engine. The “Emerald” family is designed for high-end embedded graphical applications in the automotive market. The SoCs support Fujitsu’s new 360-degree Wrap-Around Video Imaging Technology.

The most recent member of the “Emerald family, MB86R12 “Emerald-P”, is the first SoC in the world with an integrated APIX®2 interface. The latest technology from Inova Semiconductor, APIX is a standard high-speed link for video and control in various automotive applications.

For more information about the latest graphics solutions from Fujitsu, please visit http://us.fujitsu.com/semi/gdc.

Related Technical Papers

To download the papers below as well as get more information about graphics display controller technology from Fujitsu, please visit http://us.fujitsu.com/semi/gdc/docs.

Selecting the Right Graphics Display Controller: Key Factors in Designing Embedded Graphics Architecture — provides the information needed to make an appropriate choice. The paper describes basic, mid-level and advanced graphics functionality; and analyzes factors such as display resolution, cost pressures, and special requirements. An overview of Fujitsu’s extensive GDC family and its 360o Wrap-Around Video Imaging Technology is also presented.

Unlocking the 3D Capabilities of Fujitsu Graphics Display Controllers — examines how 3D significantly improves 2D user interfaces and the end-user experience; provides an overview of Fujitsu’s 3D GDCs; and explains how to unlock the power of 3D with Fujitsu GDCs, including the MB86R0x “Jade” and MB86R1x “Emerald” series. Finally the paper describes how Fujitsu can help companies develop 3D user interfaces. Unlocking the 3D Capabilities of

Fujitsu Graphics Display Controllers

TB technical backgrounder

TB technical backgrounder

Selecting the Right Graphics Display ControllerKey Factors in Designing Embedded Graphics Architecture

Fujitsu_GDC_TechPaper_v4.indd 1 3/21/11 5:27 PM