How to use JSVM toolkit

orig. Feb. 9. 2007. – updated July 7. 2011.

Eun-Seok Ryuesryu@gatech.edu

Access the latest JSVM software

• Option 1. Use CVS– Use WinCVS or any other CVS client

User ID : "jvtuser" Password : "jvt.Amd.2".

– The path is "/cvs/jvt", and the module name is "jsvm“. For example, with a command line client, you can check out the software using the following commands.

• cvs –d :pserver:jvtuser:jvt.Amd.2@garcon.ient.rwth-aachen.de:/cvs/jvt login

• cvs –d :pserver:jvtuser@garcon.ient.rwth-aachen.de:/cvs/jvt checkout jsvm

(for further information, you can see the JSVM manual)

• Option 2. Use web browser (currently, it is not working)– Follow the link given below:

http://www.ient.rwth-aachen.de/cgi-bin/viewvcextUser ID : jvtuserPassword : jvt.Amd.2

Download test sequences

• URL : ftp://ftp.tnt.uni-hannover.de/pub/svc/testsequences/

• Contents :– City, Crew, Football, Foreman, Bus, Ice,

Harbour, Mobile, Soccer

• Resolution and frame rate :– 4CIF, CIF, QCIF : 60, 30, 15, 7.5 fps

Understand JSVM versioning

• In this slide, I used JSVM 8.1 (Feb. 07, 2007, CVS tag: JSVM_8_1)

• Current version (July 2011): JSVM 9.19.14• You can download any version using CVS tool.

• JSVM 9.8 was last version supporting error concealment (EC) options– As a decoding option, you could use “–ec 2 (frame

copy)”.

• JSVM has three contents– changes : bug fix histories– SW manual– source codes

Structure of the CVS repository for the JSVM (1)

folder content

bin location of binaries after building the softwareMore information about the binaries are given in section 1.4.

lib location of libraries after building the softwareMore information regarding the libraries are given in section 1.4.

JSVM source code and project files for the JSVM softwareAll files that are required for building and using the JSVM software

are contained in this folder.

JSVM/H264Extension/build workspaces and makesfilesWorkspaces are provided for Microsoft Visual Studio 6 and Microsoft

Visual Studio .NET. Makefiles are provided for Linux.

JSVM/H264Extension/data basic examples of encoder configuration filesThese examples are not guaranteed to be complete or to contain all

available configuration parameters. The examples only serve as a basis for writing encoder configuration files.

Structure of the CVS repository for the JSVM (2)

JSVM/H264Extension/include

includes files that are required by other libraries inside the JSVM projectOnly include files for classes that are required by other libraries or test projects should be placed into this folder. This folder contains a subfolder for the frequently used library H264AVCCommonLib. Include files of classes that are only required inside a library or test project are contained in the library or test project folder in JSVM/H264Extension/src.

JSVM/H264Extension/src include and source files for all libraries and test projectsThis folder contains all source files and the include files that are only required inside a library or test project. This folder is organized in appropriate sub-folders.

JSVM0-config-samples usage examplesThis folder contains usage examples with configuration files, scripts as well as some text descriptions. These usage examples are not updated with each change of the JSVM software, and hence they are not guaranteed to be up-to-date.

Structure of the CVS repository for the JSVM (3)

MVC-Configs configuration examples for Muliple View CodingThis folder contains examples including configuration files and

scripts for using the JSVM software for Multi View Coding (MVC).

Validation validation scripts for checking the JSVM softwarePerl scripts for validating the JSVM software are provided in this

folder. These scripts shall be run after each modification of the JSVM software. They have been designed to check the functionality of integrated tools. More information on how to run these scripts are given in 4.3.

changes.txt changes log fileThis file described the (main) changes from one CVS version to the

next. It starts with JSVM version 4.0 (CVS tag: JSVM_4_0).

Build JSVM Software (1)

• Build environment– Visual C++ 6.0

• Workspace file– H264AVCVideoEncDec.d

sw • Build→Batch Build :

Rebuild All– Linux (gcc ver. 4)

> cd JSVM/H264AVCExtension/build/linux

> make – .net

• Workspace file :– H264AVCVideoEncDec.sl

n (VC 7) H264AVCVideoEncDec_vc8.sln (VC 8)

• Build→Batch Build : Rebuild All

• The executable which has ‘d’ in their file name is for debugging (compiled with debug option).

===== binaries =====bin/BitStreamExtractorStatic.exebin/BitStreamExtractorStaticd.exebin/DownConvertStatic.exebin/DownConvertStaticd.exebin/FixedQPEncoderStatic.exebin/FixedQPEncoderStaticd.exebin/H264AVCDecoderLibTestStatic.exebin/H264AVCDecoderLibTestStaticd.exebin/H264AVCEncoderLibTestStatic.exebin/H264AVCEncoderLibTestStaticd.exebin/MCTFPreProcessorStatic.exebin/MCTFPreProcessorStaticd.exebin/PSNRStatic.exebin/PSNRStaticd.exebin/QualityLevelAssignerStatic.exebin/QualityLevelAssignerStaticd.exebin/SIPAnalyser.exebin/SIPAnalyserd.exe

===== libraries ===lib/H264AVCCommonLibStatic.liblib/H264AVCCommonLibStaticd.liblib/H264AVCDecoderLibStatic.liblib/H264AVCDecoderLibStaticd.liblib/H264AVCEncoderLibStatic.liblib/H264AVCEncoderLibStaticd.liblib/H264AVCVideoIoLibStatic.liblib/H264AVCVideoIoLibStaticd.lib

Build JSVM Software (2)

Recent version supports visual studio 2010 environment with “H264AVCVideoEncDec_vc10.sln”

JSVM Libraries (1)

library description

H264AVCCommonLibStatic common libThis library provides classes that are used by both the encoder and

decoder, as for example macroblock data structures, buffers for storing and accessing image data, or algorithms for deblocking.

H264AVCEncoderLibStatic encoder libThis library provides classes that are only used by the encoder. For

example, it includes classes for motion estimation, mode decision, and entropy encoding.

H264AVCDecoderLibStatic decoder libThis library provides classes that are only used by the decoder. For

example, it includes classes for entropy decoding and bit-stream parsing.

H264AVCVideoIoLibStatic io libThis library provides classes for reading and writing NAL units in

the byte-stream format as well as classes for reading and writing raw video data.

JSVM Libraries (2)

executable description

DownConvertStatic resamplerThe resampler can be used for spatial/temporal resampling (up-

sampling or down-sampling) of video sequences. More information on using the resampler are provided in section 2.1.

H264AVCEncoderLibTestStatic AVC/SVC encoderThe encoder can be used for generating single-layer (AVC) or

scalable (SVC) bit-streams. More information on using the encoder are provided in section 2.2.

H264AVCDecoderLibTestStatic SVC decoderThe decoder can be used for decoding AVC or SVC bit-streams and

reconstructing raw video sequences. More information on using the decoder are provided in section 2.3

BitStreamExtractorStatic bit-stream extractorThe bit-stream extractor can be used for extracting sub-bitstreams

with a lower spatio-temporal resolution and/or bit-rate from a global scalable (SVC) bit-stream. More information on using the bit-stream extractor are provided in section 2.4.

JSVM Libraries (3)

QualityLevelAssignerStatic quality level assignerThe quality level assigner can be used for generating a bit-stream with additional quality layer information given a scalable bit-stream. Beside the additional quality layer information input and output bit-stream are identical. More information on using the quality layer assigner are provided in section 2.5.

MCTFPreProcessor MCTF pre-processorThe MCTF pre-processing tool can be used for pre-filtering image sequences. More information on using the MCTF pre-processor are provided in section 2.6.

PSNRStatic PSNR toolThe PSNR tool can be used for measuring the PSNR between two raw video sequences. In addition it can be used for measuring the bit-rate of a given bit-stream. More information on using the PSNR tool are provided in section 2.6.

FixedQPEncoderStatic fixed QP encoderThis tool can be used for controlling the encoder and adjusting the bit-rate of the generated bit-stream. More information on using this tool are provided in section 2.8.

JSVM Libraries (4)

SIPAnalyser SIP AnalyserThe SIPAnalyser tool can be used to make the selective inter-layer

prediction decision. More information on using the PSNR tool are provided in section 2.9.

Test encoding and decoding

• Steps :– Download JSVM and test sequences– Build the JSVM– Modifying config files (*.cfg) based on JSVM manual

• You can use original config files in JSVM SW– (option) DownConvertStatic.exe

• If you want to do (1)downsampling, (2)upsampling, or (3) cropping, use this tool.

– H264AVCEncoderLibTestStatic.exe• There are many encoding options• This makes encoded SVC file such as ‘test.264’

– (option) packet loss testing with additional tools– BitStreamExtractorStatic.exe

• Extract specific layers what I want from ‘test.264’– H264AVCDecoderLibTestStatic.exe

• This makes reconstructed YUV file such as ‘test.yuv’– PSNRStatic.exe

• Measuring PSNR

Test en/decoding steps (Simple)

H264AVCEncoderLibTestStatic.exe

BitStreamExtractorStatic.exe

H264AVCDecoderLibTestStatic.exe

Test Sequence(.YUV)

Config fileConfig file

SVC-encoded file (.svc)

Extracted SVC-encoded file(e.g. Base layer only or Base+Enh1)

ReconstructedSequence (.YUV)

or or

layers

ReconstructedSequence (.YUV)Reconstructed

Sequence (.YUV)

oror

Test en/decoding stepsfor packet loss simulation (e.g. UEP)

With JSVM ver. 9.8 error concealment (EC) option (recent version does not support EC)

orOpen-source SVC decoder (this supports frame copy-based EC)

You can use packet loss simulator, but it drops NAL-based packet. You need to adjust each NAL size or use your own tool for accurate testing.

(jvt-q069 : available: http://wftp3.itu.int/av-arch/jvt-site/2005 10 Nice/JVT-Q069.zip)

CVS Usage (free tortoise CVS app) (1)

(1) input “:pserver:jvtuser:jvt.Amd.2@garcon.ient.rwth-aachen.de:/cvs/jvt”

(3) click “Fetch list”

(2) set “JSVM”

(3) click “OK”

Building Executable Files with VS 6.0

H264AVCVideoEncDec.dsw

Example of Test Contents

• Example of the Test Clip ‘Foreman’– FOREMAN_176x144_7.5_orig_01_yuv– FOREMAN_176x144_15_orig_01_yuv– FOREMAN_352x288_15_orig_01_yuv– FOREMAN_352x288_30_orig_01_yuv

Main Configure File – encoder.cfg

Main Configure File – encoder.cfg (2)

Layer Configure File - Layer 0.cfg

Layer Configure File – Layer1.cfg

Test Encoding

>H264AVCencoderLibTestStatic.exe –pf encoder.cfg

Bitstream Extract (1)

Bitstream Extract (2)

• BitstreamExtractorStatic.exe –pt trace3_sl0.txt test.264 test3_sl0.out –sl 0

• BitstreamExtractorStatic.exe –pt trace3_sl2.txt test.264 test3_sl2.out –sl 2

Example of Trace File

1. you can test packet dropping(loss) simulation by deleting some lines of text file. • Bitstream extractor’s option ‘-et trace’ uses a (modified) packet trace file for bit-

stream extraction.2. ‘Packet loss simulator(jvt-q069 : available: http://wftp3.itu.int/av-arch/jvt-site/2005 10

Nice/JVT-Q069.zip)’ omits those NAL data when it makes (received) bitstream with error pattern file (S. Wenger, “Error patterns for internet experiments,” ITU-T SG16 Doc. Q, 2002.).

• They are quite simple, but I don’t recommend those tools.

Test Decoding

• H264AVCDecoderLibTestStatic.exe test3_sl0.out test3_sl0.yuv 45 Frames Decoded

• H264AVCDecoderLibTestStatic.exe test3_sl2.out test3_sl2.yuv 90 Frames Decoded

Using a YUV Viewer

Checking PSNR Value

• PSNRStatic.exe 252 288 foreman_352x288_30_orig_01.yuv test3_sl2.yuv

Personal comments (1)

• My blog has more information on this : Multimedia Communications using Scalable Video Coding– http://r2d2n3po.tistory.com/

• Read JSVM software manual carefully, it explains all you want.

• AVC/SVC encoding time is quite long, and you may need to reduce encoding frame number.

• Bitstream extractor cannot extract enhan. layer only but extract base+enh layers. Thus, to extract a specific layer separately you need to write your own code (you can borrow my code from the blog mentioned above).

• For academic research you can use more options such as FMO and MCTF. You can also use ‘X264’, ‘ffmpeg’, ‘Yarm’, and ‘mp4box’ tools for your own testing.

Personal comments (2)

• Recent JSVM (after ver. 9.8) does not support EC(error concealment option). Even the old tool cannot decode damaged bitstream well (mostly died). You can use open SVC decoder from source forge website (http://sourceforge.net/projects/opensvcdecoder/) ; personally thanks Mederic Bleste

• and Mickael Raulet).

• you can use packet loss simulator because this tool just omits some NALs according to error pattern files. Even you can apply different packet loss rate to each layers with this tool.

These slides were originally prepared for internal lab seminar of operating system Lab in computer science and engineering department of Korea university, and I

updated some of them for general student and researcher. July 2011.

Eun-Seok Ryu(esryu@gatech.edu)