evalvid framework : scratch to whole

High Speed Networks Avish Shah - 12mict21

EvalvidIntroduction

EvalVid is a framework and tool-set for evaluation of the quality of video transmitted over a real or simulated communication network. It is targeted for researchers who want to evaluate their network designs or setups in terms of user perceived video quality. Besides measuring QoS parameters of the underlying network, like loss rates, delays, and jitter, a subjective video quality evaluation of the received video is provided. Currently H.263, MPEG-4 and H.264 are supported.

Framework and Design

In Figure ,the structure of the EvalVid framework is shown. The interactions between the implemented tools and data flows are also symbolized. In Section 3 it is explained what can be calculated and Section 4 shows how it is done and which results can be obtained.Also, in Figure , a complete transmission of a digital video is symbolized from the recording at the source over the encoding, packetization, transmission over the network, jitter reduction by the play-out buffer, decoding and display for the user. Furthermore the points, where data are tapped from the transmission flow are marked. This informa-tion is stored in various files. These files are used to gather the desired results, loss rates, jitter, and video quality. A lot of information is required to calculate these values.

The required data are (from the sender side):– raw uncompressed video– encoded video– time-stamp and type of every packet sentand from the receiver side:– time-stamp and type of every packet received– reassembled encoded video (possibly errorneous)– raw uncompressed video to be displayedSource: raw (uncoded) video files are usually stored in YUV format.

VS (Video Sender): generates two trace files necessary for the subsequent video quality evaluation- sender trace file- video trace file

Assignment – 1 : Evalvid


ET (Evaluate Trace): generates reconstructed erroneous video using also the receiver trace.

FV (Fix Video): is only needed if the used codec cannot provide lost frames (“empty” or the last decoded frames for lost frames)

PSNR (Peak Signal to Noise Ratio) is an objective quality measure.• It is a derivative of the well-known signal to noise ratio (SNR), which compares the signal energy to the error one.

Where

– MAXI is the maximum possible pixel value of the image

– MSE is the Mean Square Error: • I is the original image and K is the compressed one



• MxN is the dimension of both images

MOS (Mean Opinion Score) is a subjective quality measure.• MOS ranges from 1 (worst) and 5 (best):

• PSNR approximated to the MOS scale:

Evalvid and NS2• To experiment video transmission using the NS2 simulator

• Simulation produces a receiver trace file containing information necessary to reconstruct the possibly corrupted video at the receiver side.

• To decode the video files obtaining a YUV (raw) video:

- ffmpeg -i video.264 video_raw.yuv

• To create a compressed raw video: MPEG-4

- ffmpeg -s cif -r 30 -b 64000 -bt 3200 -g 30 -i video_raw.yuv –vcodec mpeg4 video_cod.m4v

• To create a MP4 files containing the video samples (frames) and a hint track which describes how to packetize the frames for the transport with RTP:

-./mp4box -hint -mtu 1024 -fps 30 -add video_cod.m4v video_encaps.mp4

• To obtain the YUV file created by decoding the coded video

- ffmpeg -i video_encaps.mp4 video_ref_raw.yuv

• To compute PSNR that shows the codec impact on video quality - ./psnr 352 288 420 video_raw.yuv video_ref_raw.yuv > psnr_prima.txt



Encoding the raw YUV video $mpeg4encoder.exe example.par


//example.par…………………………………Source.Width = 176Source.Height = 144// set the first encoding frame to 0Source.FirstFrame = 0// set the last encoding frame to 399Source.LastFrame = 399//set the prefix file nameSource.FilePrefix = "foreman_qcif“

Scalability.Spatial.PredictionType = "PBB" // One of "PPP", "PBB"


After encoding, a compressed video file “foreman_qcif.cmp” can be found

Get the Video Traffic Trace File

• $MP4.exe –send 224.1.2.3 5555 1000 foreman_cif.cmp > st

• This MP4.exe command will read the compressed file. Fragment each frame

• into small packets with the maximum size of 1000 bytes. Then send this packets

• to IP:224.1.2.3 Port:5555. Here, the IP and Port is not important. Because we

• just need the frame information (stored in st file).

Run the simulation scriptset ns [new Simulator]

set nd [open out.tr w]

$ns trace-all $nd

set max_fragmented_size 1000

#add udp header(8 bytes) and IP header (20bytes)

set packetSize 1028

set s1 [$ns node]

set r1 [$ns node]

set r2 [$ns node]

set d1 [$ns node]


0 H 29 1 segm at 33 ms 1 I 3036 4 segm at 66 ms 2 P 659 1 segm at 99 ms 3 B 357 1 segm at 132 ms 4 B 374 1 segm at 165 ms 5 P 693 1 segm at 198 ms 6 B 420 1 segm at 231 ms 7 B 460 1 segm at 264 ms


$ns duplex-link $s1 $r1 10Mb 1ms DropTail

$ns simplex-link $r1 $r2 0.18Mb 10ms DropTail

$ns simplex-link $r2 $r1 0.18Mb 10ms DropTail

$ns duplex-link $r2 $d1 10Mb 1ms DropTail

set qr1r2 [[$ns link $r1 $r2] queue]

$qr1r2 set limit_ 10

set udp1 [new Agent/myUDP]

$ns attach-agent $s1 $udp1

$udp1 set packetSize_ $packetSize

$udp1 set_filename sd_be

set null1 [new Agent/myUdpSink2]

$ns attach-agent $d1 $null1

$ns connect $udp1 $null1

$null1 set_trace_filename rd_be

set original_file_name st

set trace_file_name video1.dat

set original_file_id [open $original_file_name r]

set trace_file_id [open $trace_file_name w]

set frame_count 0

set last_time 0

while {[eof $original_file_id] == 0} { gets $original_file_id current_line

scan $current_line "%d%s%d%s%s%s%d%s" no_ frametype_ length_ tmp1_ tmp2_ tmp3_ tmp4_ tmp5_

# 30 frames/sec. (1 sec -> 1000000us)

set time [expr 1000 * 1000/30]



if { $frametype_ == "I" } set type_v 1

if { $frametype_ == "P" } set type_v 2

if { $frametype_ == "B" } set type_v 3

if { $frametype_ == "H" } set type_v 1

puts $trace_file_id "$time $length_ $type_v $max_fragmented_size"

incr frame_count }

close $original_file_id

close $trace_file_id

set end_sim_time [expr 1.0 * 1000/30 * ($frame_count + 1) / 1000]

puts "$end_sim_time"

set trace_file [new Tracefile]

$trace_file filename $trace_file_name

set video1 [new Application/Traffic/myTrace2]

$video1 attach-agent $udp1

$video1 attach-tracefile $trace_file

proc finish {} { global ns nd

$ns flush-trace

close $nd

exit 0 }

$ns at 0.0 "$video1 start"

$ns at $end_sim_time "$video1 stop"

$ns at [expr $end_sim_time + 1.0] "$null1 closefile"

$ns at [expr $end_sim_time + 1.0] "finish"

$ns run



sd_be

rd_be


$ns be.tcl

0.033333 id 0 udp 29 0.066666 id 1 udp 1000 0.066666 id 2 udp 1000 0.066666 id 3 udp 1000 0.066666 id 4 udp 36 0.099999 id 5 udp 659 0.133332 id 6 udp 357 0.166665 id 7 udp 374 0.199998 id 8 udp 693 0.233331 id 9 udp 420 0.266664 id 10 udp 460

0.047958 id 0 udp 29 0.126000 id 1 udp 1000 0.171689 id 2 udp 1000 0.217377 id 3 udp 1000 0.219451 id 4 udp 36 0.250482 id 5 udp 659 0.267352 id 6 udp 357 0.285232 id 7 udp 374 0.317532 id 8 udp 693 0.337225 id 9 udp 420 0.358945 id 10 udp 460


Video Quality Evaluation• execute et program

• $et.exe sd_be rd_be st foreman_cif.cmp err_be.cmp 0

Packet/Frame lost report

• The first line indicates that the total number of packets sent (p->nA) is 1022.

It includes 350 I frame packets (p->nI); 222 P frame packets (p->nP);

and 449 B frame packets (p->nB).

• The second line indicates that the total number of packets lost (p->lA) is 161.

It includes 109 I frame packets (p->lI); 22 P frame packets (p->lP);

and 30 B frame packets (p->lB).

• Decode the video

• $mpeg4decoder.exe err_be.cmp err_be 176 144 > df_be

• Error concealment

• $myfixyuv.exe df_be qcif 400 err_be.yuv myfix_be.yuv

• Calculate the average PSNR

• $avgpsnr.exe 176 144 420 foreman_qcif.yuv myfix_be.yuv

• avgerage psnr:24.363058


p->nA:1022, p->nI:350, p->nP:222, p->nB:449 p->lA:161, p->lI:109, p->lP:22, p->lB:30 f->nA:301, f->nI:34, f->nP:67, f->nB:199 f->lA:54, f->lI:20, f->lP:13, f->lB:21


Run the qos.tcl and compare the results from be.tcl


evalvid framework : scratch to whole

Documents