nikolai daskalov n ivan pushkarov n angel gavrailov n atanas boev
TRANSCRIPT
Nikolai Daskalov n Ivan Pushkarov n Angel Gavrailov n Atanas Boev
First version of terminal device
MOBILE3DTV
Project No. 216503
First version of terminal device
Nikolai Daskalov, Ivan Pushkarov, Angel Gavrailov, Atanas Boev
Abstract: This report describes the design and implementation of the first version of the terminal device. While not having the final form factor of a mobile device, this version includes key components of the final system. Namely, the chosen processing platform has been coupled with two types of auto-stereoscopic LCD. Key SW components for the processing and playing stereo video have been targeted as well. From platform perspective the goal of this version is to help the team to assess the system architecture of the proposed device, the feasibility of implementation, the performance of targeted components and the interoperability between them from both HW and SW point of view. The second goal is to verify the system interfaces (again both HW and SW) and to establish some performance metrics so to quantify the performance of the integrated displays and developed player.
Keywords: OMAP, DVB-H, auto-stereoscopic LCD
MOBILE3DTV D6.2
2
Executive Summary
The first version of the terminal device (prototype) has been developed and implemented. At that stage no final form factor has been targeted. Instead, key HW components which will be used in the final version of the terminal device, such as the processing platform and the auto-stereoscopic LCD have been coupled together.
Key SW components and tools have been identified and used to support the first version of the mobile terminal device. Those are the platform-specific operating system, file system, tool-chain, and a H.264 decoder.
Two different auto-stereoscopic LCDs from two different vendors have been connected to the system. For the first LCD, parallel 24bit interface has been used. For the second LCD, standard DVI interface has been used.
New HW and SW components have been developed in order to accomplish the targeted version. These include a new LCD daughter-card for interfacing the selected LCD to the platform; EVM and corresponding SW support for the two particular displays, and stereo image renderers. The completed version is capable of decoding and rendering stereoscopic video content. The input is stereo video streams, encoded by H.264 simulcast, and stored in file and the output is rendered on any of the integrated auto-stereoscopic LCD.
Performance metrics for the used components and encoding methods are defined so quantify the performance of the integrated components. This information will be used in the next stages for testing and performance validation.
MOBILE3DTV D6.2
3
Table of Contents
Platform and tools selection .......................................................................................... 4
Operating system ................................................................................................... 4 Video decoder application ...................................................................................... 4 Video renderer ........................................................................................................ 5 Utilities .................................................................................................................... 6
Selection of autostereoscopic display ........................................................................... 8
Mobile 3D display technologies .............................................................................. 8 Selection of display model ...................................................................................... 9
Stereo-video encoding method and decoding tools modifications ................................ 10
Decoder modification ............................................................................................ 10 Additional software components ............................................................................ 11
1.1.1 Video renderer module .......................................................................... 11
Interfacing the MasterImage auto-stereoscopic LCD .................................................... 12
System requirements ............................................................................................ 12 1.1.2 General information ............................................................................... 12 1.1.3 Description of interface and data transfer ............................................. 13 1.1.4 Design of interface daughter-card .......................................................... 14
Bring-up SW Project ............................................................................................. 18 OS Dependant modifications and new components ............................................. 24 Results.................................................................................................................. 24
Interfacing the NEC auto-stereoscopic LCD ................................................................. 25
System requirements ............................................................................................ 25 Interface................................................................................................................ 25 OS Dependant modifications and new components ............................................. 25 Results.................................................................................................................. 25
Performance metrics definition and measurements ...................................................... 26
Optical parameters of the displays – a comparative analysis ............................... 26 Video decoding performance ................................................................................ 29
References ................................................................................................................... 30
MOBILE3DTV D6.2
4
Platform and tools selection
Operating system
Currently, two major high-level operating systems are dominating the smart-phones
market: Symbian and Windows mobile. However, they both imply high cost of starting the
development, especially for the kernel level components such as display driver, DVB-H
receiver driver, etc. At the same time, embedded Linux is getting more and more popular in
the smart-phones market. It is even more adopted in portable devices such as internet
tablets and MIDs. Vendors such as Nokia, SEMC, and Motorola have announced support
of or migration to Linux-based devices.
For the MOBILE3STV project Linux seems to be the obvious choice as operating system
for the mobile demonstrator based on the following benefits:
Fast ramp-up of the SW team;
Abundance of example software;
Availability of reference drivers for display, SPI, I2C and other peripheral interfaces
to be used in the project;
Interoperability between different project work packages and their outcomes;
Open source community support
Video decoder application
3430 SDP is bundled with video decoder test application, capable to decode MPEG4,
H.264, MPEG2 and WM9 video streams with no audio synchronisation. The platform,
denoted as OpenMAX (OMX) implements optimized decoder running on the DSP and
using some vendor-specific hardware accelerators. OpenMAX is standard for components
implementing different multimedia processing functions. It is managed by Khronos group,
which manages also several other standards such as OpenGL. OpenMAX is well accepted
by major multimedia platforms and frameworks vendors. The major purpose of it is to
create unified environment for creating multimedia frameworks. The block diagram of the
decoder application and its components is shown i Figure 1. It allows decoding video-
streams read from files and rendering on the LCD display. Typical example of the
command line to start the decoding is shown below:
./VideoDisplayTest 6 omx/patterns/*.h264 blank.vop 640 480 4 1 90 0 0 100 100 1 70 LCD
MOBILE3DTV D6.2
5
Figure. 1: Block diagram of video decode architecture in OMAP3430 SDP.
Video renderer
The video decoder application has no support for auto-stereoscopic LCD and the LCD
controller of OMAP3430 has no internal logic to interleave the data from the left and right
channels of the decoded sequence in order to be displayed on the selected LCD.
Therefore, an additional software video renderer has been developed. We have avoided
the straightforward implementation of this interleaving on the CPU, as such
implementation would result in high CPU load and subsequent system performance
degradation. Instead, we opted for a separate solution, as described in Chapter 3.3. The
integration of this interleaving implementation into the Linux frame buffer is in progress.
MOBILE3DTV D6.2
6
Utilities
The utilities include conversion software between different stereo video formats. The first
selected stereoscopic display supports and interleaved input frame while the second
selected display supports a side-by-side format. Therefore, the following conversion
formats are supported: from two-channel to side-by-side and back. From two-channel to
interleaved (intedigitized) frames and back. From side-by-side to interleaved frames and
back. A screen shot of the program interface is shown on Figure 2.
Figure 3 illustrates converted screen shot of the converted video for the NEC display
module while Figure 4 illustrates converted screen shot of the converted video for the
MasterImage display module.
The software takes the two videos and can create double non interlaced video (Figure 3),
or interlaced (Figure 4). See Figure 21 for an illustration of the interdigitization process.
Figure 2: Interface of the converter tool
MOBILE3DTV D6.2
7
Figure 3: Converted video screen shot for NEC module.
Figure 4: Converted video screen shot for MasterImage module.
MOBILE3DTV D6.2
8
Selection of autostereoscopic display
Mobile 3D display technologies
Currently, there is a wide range of 3D display technologies [1], [2], but not all of them are
appropriate for mobile use. For example, wearing glasses to aid the 3D perception of a
mobile device is highly inconvenient. The limitations of a mobile device, such as screen
size, CPU power and battery life limit the choice of a suitable 3D display technology.
Another important factor is backward compatibility – a mobile 3D display should have the
ability to be switched back to “2D mode” when 3D content is not available.
Autostereoscopic displays form a class of 3D displays which create 3D effect without
requiring the observer to wear special glasses. Such displays use dedicated optical
elements aligned on the surface of the screen so to ensure that the observer sees different
images with each eye. Typically, autostereoscopic displays are capable of presenting
multiple views to the observer, each one seen from a particular viewing angle along the
horizontal direction. However, the number of views comes at the expense of resolution and
brightness loss – and both are limited on a small screen, battery driven mobile device. As
mobile devices are normally watched by only one observer, two independent views are
sufficient for satisfactory 3D perception. At the moment, there are only a few vendors with
announced prototypes of 3D displays, targeted for mobile devices [3], [4], [5]. All of them
are two-view, TFT-based autostereoscopic displays [6].
The basic operational principle of an autostereoscopic display is to “cast” different images
towards each eye of the observer. This is done by a special optical layer, additionally
mounted on the screen surface which redirects the light passing through it. There are two
common types of optical filters – lenticular sheet [7] which works by refracting the light,
and parallax barrier [8] which works by blocking the light in certain directions. In both
cases, the intensity of the light rays passing through the filter changes as a function of the
angle, as if the light is directionally projected. These two technologies are shown in Figure
5.
MOBILE3DTV D6.2
9
a) b)
Figure 5: Technologies for redirecting the light in an autostereoscopic display. a) lenticular sheet, b) switchable parallax barrier
The main advantage of parallax barrier is the ability to switch it off, so the display works in
2D mode, thus providing backwards compatibility with 2D content. The main disadvantage
of the parallax barrier is that it blocks part of the light, resulting in a lowered brightness of
the display. In order to compensate for that, one needs an extra bright backlight, which
decreases the battery life. Comparing the cost of developing and manufacturing parallax
barrier and lenticular sheet, the former is much cheaper than the latter.
Selection of display model
Following the requirement for backwards compatibility, we initially focused on a parallax
barrier display. One of the few commercially available parallax barrier models was
Stereoscopic 3D LCD model MB403M0117135 produced by MasterImage, which is a
4.3”WVGA (800px x 480px) transmissive LCD display. Additional feature of this display is
the ability to switch the barrier between horizontal and vertical mode, which allows
landscape and portrait mode of 3D operation.
Due to the operation principle, a parallax barrier reduces the horizontal resolution twice
when operating in 3D mode. For the MasterImage display this results in the following set of
resolutions: 800x480 in 2D mode, 400x480 in landscape 3D mode, and 240x800 in portrait
3D mode.
In the mean time, we also considered another display – 3D HDDP LCD produced by NEC
[5]. It uses lenticular sheet, but allows switching of the display between 2D and 3D mode,
by simply sending the same 2D image to both channels. Also, the selection of size and
order of the subpixel components results in the same resolution both in 2D or 3D mode.
The colour components of a pixel are ordered vertically, instead of horizontally, which
suppresses angle-dependant colour artefacts.
R G B R G R G B
1 2 1 2 1 2 1R 1R
B R G B R G B R G
RR RR
LL LL
MOBILE3DTV D6.2
10
The pixels are twice more dense in horizontal than in vertical direction, thus called by the
developers from NEC Lab, Horizontal Double Density Pixel (HDDP) arrangement. It has
been described in [5]. Additionally, the display is able to work both in transmissive or
reflective mode, which ensures 3D operation in wide range or lighting condition. The
HDDP display is not yet commercially available. However, we received a pre-production
sample from NEC, which is 3.1 inch, with 427x240 resolution both in 2D and 3D mode.
As each 3D display model has advantages and disadvantages of their own, it was decided
to mount and interface both of them and to compare their optical quality as part of a
working system.
Stereo-video encoding method and decoding tools modifications
Decoder modification
Based on the results from D 2.2 report and considering capabilities of the chosen
developing platform and the existing H.264 decoder, at that stage we have focused on two
encoding approaches and respectively their decoding parts implementations, namely
H.264 Simulcast and H.264 SEI Message.
In the simulcast approach, we put left and right part of the stereo image side by side. The
result is H.264 stream with double frame size. Decoding this stream is equivalent to
decoding the stream with double frame size. If we are able to double the existing decoder
frame size, we can easily implement H.264 Simulcast.
When using H.264 SEI message, left and right views are interlaced, resulting in a stream
with double frame rate. From implementation point of view, decoding such stream is
equivalent to decoding mono video stream with double frame rate, and de-interlacing each
two frames in order to create stereoscopic view.
To implement H.264 Simulcast we need to have a decoder able to decode frames with
double frame size, preserving frame rate while implementing H.264 SEI message we need
a decoder able to decode with double frame rate, preserving frame size.
MOBILE3DTV D6.2
11
In order to implement H.264 SEI Message we need to modify these features of the existing
decoder:
decode SEI message
de-interlacing left and right views
constructing side-by-side frame for the display subsystem According to the comparative study of different encoding approaches described in
Deliverable 2.2, H.264 SEI Message gives about 35% bitrate saving vs. H.264 Simulcast
and is almost identical to MVC coding, which makes H.264 SEI message a strong favorite
for practical implementation of decoding stereo video streams.
For the first version of the terminal device, the H.264 simulcast approach has been
implemented, while the SEI message implementation is still under investigation. Though
inferior, the simulcast approach has been chosen due to the purpose of that version, i.e. to
be used in the subjective tests. Simulcast allows manipulating and experimenting with
videos of different quality (varying quality factors such as bitrate, framerate, transmission
modes, etc.) – simply code and decode the streams and put the left and right channels
side by side to be played further in the subjective tests.
Additional software components
1.1.1 Video renderer module
This component combines left and right views from the output of the H.264 decoder to
produce the format required for driving the respective auto-stereoscopic LCD display. In
the case of the MasterImage display this means converting the side by side video frames
to spatially interleaved (interdigitized) frames. In the case of NEC display driven by DVI
input the side by side frames are send directly with no format conversion.
MOBILE3DTV D6.2
12
Interfacing the MasterImage auto-stereoscopic LCD
System requirements
1.1.2 General information
One of the two interfaced displays is produced by the Korean company MasterImage. It
uses a parallax barrier to implement the auto-stereoscopic effect. The display consists of
two LCD modules, assembled together in one compound 3-D LCD module. The back
module (briefly “Main display”) is a common color LCD display of certain resolution. In front
of it, there is the second LCD serving as an optical filter (briefly “filtering display”). While
switched on, its role is to block the light to some directions and thus to cast two different
views to the eyes (left and right views) – hence the name „parallax barrier‟. When switched
off, it plays no role and the display module works in 2D mode. Figure 8 illustrates the
positioning of the two displays.
Figure 8: Arrangement of main and filtering displays
The main display characteristics are as follows:
Size 4.3 inches;
Resolution WVGA (800 x 480 pixels);
RGB Interface with 1 pixel / clock;
8-bit color depth, 16777216 colors;
10 LEDs back-light;
High luminance and contrast ratio, low reflection.
MOBILE3DTV D6.2
13
The filtering display features are as follows
Target display size 4.3 inch;
Target display resolution WVGA (800 x 480 pixels);
1.1.3 Description of interface and data transfer
The interface is 24-bit parallel data transfer interface. We connect the daughter board to
the OMAP3430 gpios trough the OPAM3430SDP.
Each pixel consists of three dots for each color - Red, Green and Blue. Each dot is
controlled by 8 bits (i.e. 8-bit color depth). This yields 16777216 different color states per
pixel. The data is transferred parallel trough 24 lines, controlled by the PCLK (pixel clock).
Figure 10 shows the time diagram, and the data transferred for one pixel clock pulse. For
each pixel clock pulse 24 bits are transferred.
Figure 10: Data transfer
MOBILE3DTV D6.2
14
The input data format of the 3D LCD is shown in Figure 11.
The main requirements concerning the timing of the 3-D LCD are shown in Table 1.
Table 1: Main Timing Requirements
Parameter Min Typical Max Unit
VSYNC - 60 75 Hz
HSYNC - 31,5 39,4 KHz
PCLK frequency 10 33,5 50 MHz
PCLK pulse width 8 - - ns
The targeted platform supports the upper values in order to be compatible with the 3-D
LCD. There are few more timings, but they are directly connected to the main listed above.
Figure 11: 3-D LCD input data format
1.1.4 Design of interface daughter-card
MOBILE3DTV D6.2
15
The daughter-card must provide the proper functionality of the 3-D LCD.
The main goals in the design of the daughter-card design are as follows
To connect the 5V supply voltage to the requested components;
To generate the supply voltage for the back-light of the 3-D LCD (30 – 34V);
To provide the proper power on/off sequence of the 3-D LCD;
To level-shift the signals from the platform to the Main display (1.8 to 3.3 V);
To generate the supply voltage of 3.3V (for the PIC controller, level-shifters,
power supply of the logic of the main display, etc);
To provide popper control of the PIC controller;
To fit the existing platform hardware.
A simple block diagram is depicted in Figure 12. The shown blocks are as follows
Board connector schematic – this is the connector to the platform interface.
Level shifter schematic – this block level-shifts the signals coming from the platform (the
platform display gpios) from 1.8 V to 3.3 V for logical high level.
LCD connector schematic – this block consists of the connector to the 3-D LCD and the
scheme that generates the power supply voltage of the back-light from 5 V input voltage.
3V3 power schematic – this block generates the 3.3 V voltage needed for the schematic.
LCD power sequence enabling schematic – this part provides the proper on/off power
sequence to prevent bad/faulty functionality or damage to 3-D LCD.
TN LCD control schematic – this part controls the filtering display, therefore controls the
mode of the 3D LCD.
MOBILE3DTV D6.2
16
Figure 12: Block diagram of the daughter-card
The board has been designed so to fit to the existing platform hardware. It has been
provided with stable fixation between the different connectors, and stable and safe
positioning of the 3-D LCD considering the manufacturer recommendation concerning the
exploitation.
The technology of the MasterImage display module is illustrated by the following photos.
Figure 13 shows a stereo frame to be displayed on the interfaced card. The picture is
interlaced so to create the desired 3D effect on the 3D module. Figure 14 and Figure 15
show the daughter-card hosting the 3D LCD as connected to the OMAP3430SDP.
MOBILE3DTV D6.2
17
Figure 13: Interlaced photo for the test.
Figure 14: The display working in 3-d mode
MOBILE3DTV D6.2
18
Figure 15: The display showing 3-d Picture with turned off 3-D mode
In Figure 14, the 3D effect has been activated (parallax barrier switched on) while in Figure
15 the barrier has been switched off.
Bring-up SW Project
The bring-up software project was developed in order to implement initial testing of the
display without all the complexity of Linux operating system. It could be divided into two
major parts:
Part A: Software, related to the TFT display
This software provides the following functions:
Basic system initialization (OMAP3430 SDP)
Display subsystem initialization
Create and display a test pattern
Picture loading
Picture display – Landscape and Portrait mode
MOBILE3DTV D6.2
19
Part B: Software, related to the stereoscopic overlay
This software is intended to provide the correct driving signals for the stereoscopic layer. It
is run on a separate embedded microcontroller in order to simplify the main software and
to provide independent and steady operation of the stereoscopic overlay.
Part A
The software for TFT display bring-up was created in the environment of CodeComposer
Studio. Although Linux would provide more flexibility in terms of interfaces, file loading,
etc., it is much easier to control certain subsystem (that is, Display Subsystem) parameters
without it. Additional benefit in this approach is that we can use JTAG emulator for
debugging.
There are two modules of the TFT bring-up software
Basic functional evaluation: Mainly for test of colours and display timings
“Normal use” functional evaluation: Display of (processed) stereoscopic images
The two block diagrams are shown in Figure 16 and 17 respectively
Figure 16: Block diagram of the Basic functional evaluation
Start
Initialize SDP system
Initialize Display Subsystem
Create & isplay test patterns
End
MOBILE3DTV D6.2
20
Figure 17: Block diagram of the “Normal use” functional evaluation
The blocks illustrate the following functions Initialize SDP subsystem
This functional block takes care of basic initialization steps, including but not limited to:
OMAP3430 low-level initialization: clocks, stacks, interrupts, etc.
OMAP3430 pads initialization
OMAP3430 SDP power supply initialization – i.e. Triton3 set up
Initialize Display subsystem
This functional block takes care of the OMAP3430 DSS initialization. Main functions
performed here are:
Display controller initialization: Type of display, Display size, Number of colors, Data
bus width, Pixel clock rate, VSYNC/HSYNC Timings
Graphics subsystem initialization: Number of layers, Transparency, Image
dimensions, Rotation, etc.
Start
Initialize SDP system
Initialize Display Subsystem
Load image
End
Process & Display Stereo image
MOBILE3DTV D6.2
21
Load image
In order to simplify the code, the image is hard-coded in the form of a pre-initialized C
array and is used as-is inside the test software. For the purpose of creating the C-array, a
small console application was created, which converts a JPG file into pre-initialized C-
array
Create and display test patterns
In order to strictly check any identify possible problems with the TFT Display, its settings or
settings of the OMAP3430 DSS, test patterns with the appropriate parameters are
generated (Figure 18):
Figure 18: Test patterns
Using these patterns, problem in routing of the TFT control signals (PCLK, VSYNC, DE)
and routing of colour driving signals were corrected.
Process and display of a Stereo Image
This functional block is responsible for processing the image data in a form, suitable for
displaying.
If the input is an interlaced Image, i.e. two images – for the left and the right eye
respectively, are already interlaced, there is no need of further processing. Example of
such an image is shown in Figure 19.
MOBILE3DTV D6.2
22
Figure 19: Interlaced left and right views to be displayed on the MasterImage display
If the input image is a side-by-side image of the left and right views, it has to be processed
and the two parts to be interlaced. Such image is shown in Figure 20.
Figure 20: Left and right views as put side-by-side
For the purpose of processing, two DMA channels were used in order to simultaneously
transfer the two halves of the original picture into a final interlaced image (see Figure 21).
MOBILE3DTV D6.2
23
Figure 21: DMA transfer of data from side-by-side format to interlaced frame
By using two DMA channels, the two halves are automatically transferred to the correct
places with no complex loops. The DMA channels use Double-indexing (i.e. indexing
within an Element Frame and an indexing of Frames), which is a key feature to automate
the entire process. The DMA indexing parameters are shown in Table 2:
Table 2: DMA indexing parameters
where:
X – Horizontal resolution of the output image (800 pixels in this particular case)
Y – Vertical resolution of the output image (480 pixels in this particular case)
DMA2 Channel Settings
Source Start Address Address of the (400, 0) pixel
Source Element indexing X
Source Frame indexing -(Y-1)*X+401
Destination Start Address Address of the (1, 0) pixel
Destination Element indexing X
Destination Frame indexing -(Y-1)*X+2
MOBILE3DTV D6.2
24
Part B
The stereoscopic overlay (the parallax barrier) is basically an LCD display, whose pixels
are driven in a specific way in order to achieve a row visibility blocking effect either in the
horizontal or in vertical directions.
OS Dependant modifications and new components
Regarding the Linux OS, changes on kernel display drivers and frame buffer have to be
accomplished. These changes have been implemented for the L12.20 baseline release of
the platform software.
Results
The MasterImage auto-stereoscopic display was successfully integrated to OMAP3430
SDP platform. A special interface daughter card, containing all needed logic and level
translation was developed. Stand-alone bring-up test project was developed in order to
verify the HW and SW components in the system. This stand-alone project allows loading
and display auto-stereoscopic images on the display, prepared by the interleaving tool,
developed for the project. The perceived quality of the stereo images using MasterImage
LCD is good. In addition, the team has been modifying the 3430 Linux frame buffer and
display driver to be included in the next release of the platform software.
MOBILE3DTV D6.2
25
Interfacing the NEC auto-stereoscopic LCD
System requirements
The NEC module is a lenticular-based auto-stereoscopic display, delivered as stand- alone
module, interfacing with the platform trough a DVI interface. The display consists of two
parts: ordinary LCD with certain resolution and micro-lenses filter positioned over the LCD.
That filter (i.e. the lenticular sheet) separates the left and right channels for each eye. 2D
mode is achieved by delivering the same content to the two channels. Otherwise, by
delivering stereo content, the display works in 3D mode. A sample video frame is shown in
Figure 25.
Figure 25: Side by side left and right frames for the NEC display
The input frame to be displayed is of resolution 852 x 240 pixels formed by two side-by-
side pictures with resolution 427 x 240 pixels each.
Interface
At that stage, we have access to the display module trough a DVI interface.
OS Dependant modifications and new components
In order to run NEC Module on SDP (OMAP3430 Zoom) a new kernel image was created
supporting the 852 x 240 pixels resolution. A Linux-based player application working on the
SDP deliver the video frames to the display.
Results
As the OMAP3430 Zoom platform supports the DVI interface to connect the NEC display
module no additional HW components were needed. After modification in the kernel driver
and frame buffer, stereoscopic images can be shown on the display. The perceived image
MOBILE3DTV D6.2
26
quality can be rated as excellent. In the next section, the two displays have been
compared for their 3D optical characteristics.
Performance metrics definition and measurements
Optical parameters of the displays – a comparative analysis
The measurement of crosstalk and angular luminance profile was based on the
methodology, proposed in [9]. Two displays were measured - MB403M0117135 from
MasterImage (designated here as “MasterImage”) and a 3.1” prototype of HDDP 3D LCD
produced by NEC (designated here as “NEC”). A number of test images were prepared,
visualized on each display, and photographed from different angles using digital camera.
The parameters of the camera were set in such a way, so the intensity of the captured
images does not cause saturation and is mostly in the linear range of the camera sensor,
as seen in Figure 31.
Two experiments were made, and a number of observation points were chosen for each
experiment. A set of test images were prepared. Each test image is a stereo-image where
each channel contains all pixels set at a certain brightness level. For example, in test
image “L0R64” all pixels from the left channel have value 0 and all pixels in the right
channel have value 64. Each test image was photographed from each point. The
observation points were restricted onto a plane perpendicular to the display surface as
shown in Figure 26. When using a 3D display, the eyes of an observer usually appear
close to that plane.
Figure 26: Selection of plane for measurements
At each observation point, all measured results were scaled from 0 to 1, where 0
corresponds to the value measured when the display was completely black, and 1
Plane of the measurements
MOBILE3DTV D6.2
27
corresponds to the value measured when all pixels from both channels had maximal
brightness. Additionally, the measurements were performed in a dark room.
In the first experiment we measured the luminance profile or each display. The observation
points were chosen along a line at typical observation distance from the display as shown
in Figure 27. Test images L0R0, L0R255, L255R0 and L200R255 were photographed.
Figure 27: Observation points for measuring angular dependant luminance profile
The results, shown in Figure 28, illustrate the main difference between the display
technologies used. The left and right channel of the NEC display have clearly separated
observation zones, where each channel is predominantly seen from a set of angles to the
left and to the right of the display respectively. The MasterImage display uses parallax
barrier, which creates a number of interleaved “left channel” and “right channel” visibility
zones.
a) b)
Figure 28: Luminance profiles: a) NEC, b) MasterImage
The approximate position of visibility zones of each channel is shown in Figure 29. For the
NEC display, stereoscopic image can be perceived from a set of angles, as long as each
eye of the observer appears in the designated zone. The observation zones of
MasterImage are much narrower, and as a result, 3D perception is possible from a number
of tight “sweet spots” in front of the display. The advantage of the second configuration is
that the MasterImage display can provide 3D image to a number of users simultaneously.
However, in order to have a satisfactory 3D perception, a user of MasterImage has to
Front of Display
Back of display
Observation distance(~30cm)
Observation points
-50 … 50 degrees
MOBILE3DTV D6.2
28
carefully choose a position in front of the display, and try to avoid positions where a
preudoscopic (reversed stereo) image will be seen.
a) b)
Figure 29: Vizibility zones left and right channels: a) NEC, b) MasterImage
The second experiment measured the crosstalk of each display. Two observation points,
mimicking the typical position of the eyes of an observer, as depicted in Figure 30. It is
assumed, that this is the “best case” scenario, which should result in the lowest crosstalk.
We prepared 34 test images for each display – L0R0, L0R16, L0R32…L0R240, L0R255,
L16R0, R32R0…L240R0, L255R0, L255R255.
Figure 30: Observation points for measuring the crosstalk
The results are shown in Figure 31. The crosstalk is nearly symmetrical across channels
for each display. The relative crosstalk of NEC is about 4%, while the crosstalk of
MasterImage is more that twice of that, 9%. It should be noted, however, that due to the
narrower observation zones of MasterImage, the measurement of crosstalk for that display
requires higher precision and might be less accurate. Still, the visual appearance of the
displays suggests lower crosstalk values for NEC.
RL
screen
right
eye
left
eye
R
L
screen
right
eye
left
eye
L LRL
R
R
Front of Display
Back of display
Observation distance (~30cm)
Interpupillary distance (~6.3cm)
Typical observationpoints
MOBILE3DTV D6.2
29
a) b)
c) d)
Figure 31: Crosstalk measurements for 3D displays: a) left channel crosstalk of NEC, b) right channel crosstalk of NEC, c) left channel crosstalk of MasterImage, d) right channel crosstalk of MasterImage
We summarize the results of the optical measurements in the following table:
Table 3: summary of the optical measurement results.
Parameter MasterImage NEC
3D crosstalk 9,00% 4,00%
Viewing distance ~10cm-90cm ~20-40cm, 30cm optimal
Viewing freedom ~7 degrees (for interpupilar
distance of 6.5cm)
1.2 degrees
Luminance profile See Figure 28b See Figure 28a
Video decoding performance
According to the conclusion of the video encoding method (chapter 2.1), the most
promising methods are H.264 simulcast or H.264 SEI method. For both methods, knowing
that for the moment the best performance we can get out of the decoder is VGA@30fps,
we were able to decode stereo sequences at HVGA@30fps, which is better than the
minimum requirements defined in the D6.1 report (QWVGA@25fps ), but less than the
target for the best decoding for the project (HWGA@30fps). Because the actual decoding
is happening on the DSP in OMAP3430, the CPU load for decoding is quite small –
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 32 64 96 128 160 192 224 255
L out
R out
Input value
Inte
nsi
ty (s
cale
d)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 32 64 96 128 160 192 224 255
L out
R out
Input value
Inte
nsi
ty (s
cale
d)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0
16
32
48
64
80
96
11
2
12
8
14
4
16
0
17
6
19
2
20
8
22
4
24
0
25
5L out
R out
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 32 64 96 128 160 192 224 255
L out
R out
MOBILE3DTV D6.2
30
average less than 25%. The re-formatting component is not consuming CPU resources
because the DMA based implementation.
References
[1] L. Onural, T. Sikora, J. Ostermann, A. Smolic, M. R. Civanlar and J. Watson: “An Assessment of 3DTV Technologies,” NAB Broadcast Engineering Conference Proceedings 2006, pp. 456-467, Las Vegas, USA, April 2006.
[2] P. Surman, I. Sexton, R. Bates, W. K. Lee, K. Hopf, and T. Koukoulas: “Latest Developments in a Multi-User 3D Display,” in Proc. SPIE Vol. 6016, Three-Dimensional TV, Video, and Display IV , 2005.
[3] Sharp Laboratories of Europe, website, http://www.sle.sharp.co.uk/research/optical_imaging/3d_research.php
[4] G. J. Woodgate, J. Harrold, “Autostereoscopic display technology for mobile 3DTV applications”, in Proc. SPIE Vol.6490A-19 (Stereoscopic Displays and Applications XVIII), 2007
[5] S.Uehara, T.Hiroya, H. Kusanagi; K. Shigemura, H.Asada, “1-inch diagonal transreflective 2D and 3D LCD with HDDP arrangement”, in Proc. SPIE-IS&T Electronic Imaging 2008, Stereoscopic Displays and Applications XIX, Vol. 6803, San Jose, USA, January 2008
[6] I. Sexton, P. Surman, ”Stereoscopic and autostereoscopic display systems.”, in IEEE Signal Processing Magazine, pp. 85-99, 1999
[7] C. Van Berkel and J. Clarke, “Characterisation and optimisation of 3D-LCD module design”, in Proc. SPIE Vol. 2653, Stereoscopic Displays and Virtual Reality Systems IV, (Fisher, Merritt, Bolas, edts.), p. 179-186, May 1997
[8] W. Izerman et al., “Design of 2d/3d switchable displays,” in Proc of the SID, volume 36, Issue 1, pp. 98-101, May 2005
[9] Boev, A., A. Gotchev and K. Egiazarian, “Crosstalk measurement methodology for auto-stereoscopic screens”, Proc. of 3DTV Con, Kos, Greece, 2007
Mobile 3DTV Content Delivery Optimization over DVB-H System
MOBILE3DTV - Mobile 3DTV Content Delivery Optimization over DVB-H System - is a three-yearproject which started in January 2008. The project is partly funded by the European Union 7th
RTD Framework Programme in the context of the Information & Communication Technology (ICT)Cooperation Theme.
The main objective of MOBILE3DTV is to demonstrate the viability of the new technology ofmobile 3DTV. The project develops a technology demonstration system for the creation andcoding of 3D video content, its delivery over DVB-H and display on a mobile device, equippedwith an auto-stereoscopic display.
The MOBILE3DTV consortium is formed by three universities, a public research institute and twoSMEs from Finland, Germany, Turkey, and Bulgaria. Partners span diverse yet complementaryexpertise in the areas of 3D content creation and coding, error resilient transmission, userstudies, visual quality enhancement and project management.
For further information about the project, please visit www.mobile3dtv.eu.
Tuotekehitys Oy TamlinkProject coordinator
FINLAND
Tampereen Teknillinen Yliopisto
Visual quality enhancement,
Scientific coordinator
FINLAND
Fraunhofer Gesellschaft zur Förderung der angewandten Forschung e.V
Middle East Technical UniversityError resilient transmission
TURKEY
Stereo video content creation and coding
GERMANY
Technische Universität IlmenauDesign and execution of subjective tests
GERMANY
MM Solutions Ltd. Design of prototype terminal device
BULGARIA
MOBILE3DTV project has received funding from the European Community’s ICT programme in the context of theSeventh Framework Programme (FP7/2007-2011) under grant agreement n° 216503. This document reflects onlythe authors’ views and the Community or other project partners are not liable for any use that may be made of theinformation contained therein.