integrated error management in mod services

©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001

Integrated Error Management Integrated Error Management

in MoD Servicesin MoD ServicesPål Halvorsen, Thomas Plagemann, and Vera

GoebelUniversity of Oslo, UniK- Center for Technology at Kjeller

Norway


Overview

• Application Scenario

• Integrated Error ManagementIntegrated Error Management– Basic idea– Code requirements– Possible solutions– Evaluation

• Conclusions


Media-on-Demand server:Applicable in applications like News- or Video-on-Demand provided by city-wide cable or pay-per-view companies

Application Scenario

Network Network

Multimedia Storage Server

Project goals:Optimize performance within a single server:• Reduce resource requirements • Maximize number of clients

Retrieval is the bottleneck:Some important factors:• Memory management• Communication protocol processing• Error management


Traditional Error Management

FEC Encoder FEC Decoder


Integrated Error Management

FEC Encoder FEC Decoder


Correcting Code Requirements

• Erasure (known errors) and error (unknown errors) correcting

• Decoding throughput (recovery performance)

• Amount of redundant data

• Applicable within a single file

• Cost of decoding function dependent on the corruption rate


Possible Schemes• Error and erasure correcting codes

Bad performance (< 1.7 Mbps) Not dependent of corruption rate (< 3.5 Mbps without any corruption)

• Erasure correcting codes Adequate performance (> 6 Mbps) Corrects only erasures

• Combined schemes– Erasure correcting / Error correcting

Capable of recovering from most errors Even more redundant data Performance still a problem (< 1.7 Mbps)

– Erasure correcting / Error detection Adequate performance (> 6 Mbps) Capable of recovering from most errors (Almost) no additional redundant data


Prototype Design

• Combination of the traditional UDP checksum and an erasure correcting code (Cauchy-based Reed-Solomon Erasure)

• Disk array with 8 disks, i.e., 7 information disks and 1 parity disk:

Write operations a la RAID level 4/5

Read operations a la RAID level 0

One codeword, contained in one or more stripes, is read as one read operation

Each striping unit consists multiple symbols each transmitted as a UDP packet


Codeword Size and Amount of Redundancy

Requirements:• Codeword_size n stripe_size, n Z • Recover from one disk failure and (most) network errors

Our current approach:– Using a codeword of 256 symbols with 32 redundant symbols

corrects one out of 8 disks

corrects most errors in the network according to our error model, i.e., any 32 out of 256 packets


Packet (Symbol) Sizes - I

• Correcting codethroughput suggests a largesymbol size

Lowthroughput


• Correcting code (startup) delay suggests a small symbol size

Packet (Symbol) Sizes - II

Lowthroughput

Highstartup delay


Results and Observations – IExperimental Setup

• Assuming coding performance is only bottleneck

• Transmitted a 225 MB file(corresponding to a 5 minutes 6 Mbps playout)

• Used a worst-case loss scenario

• Used several different machines


Results and Observations – IIServer Side

• No extra disk space needed

• No additional time needed to retrieve parity data

• No overhead managing retransmissions

• Increased buffer space and bandwidth requirement of 12.5% storing and transmitting redundant data.

• Encoding throughput limitation of ~3 Mbps (Intel, 166 MHz) to ~25 Mbps (Intel, 933 MHz)

is eliminated by retrieving parity data from disk

The server can support more concurrent users


Results and Observations – IIIClient Side

• Increased buffer requirement for decoding the received data (2 MB, 3 MB, 5 MB, and 9 MB using 1 KB, 2 KB, 4 KB, and 8 KB packets, respectively)

• Additional CPU requirements recovering lost/corrupted data

• Additional (worst case) startup delay between ~0.1 s (Intel, 933 MHz) and ~4.5 s (Intel, 166 MHz)

decoding the first block can be experienced

• In-time decoding

hiccup free presentation


Conclusion

• The INSTANCE project aims at optimizing the data retrieval in an MoD server

• Integrated Error ManagementIntegrated Error Management

• Future work

• More information: www.unik.no/~paalh/instance

integrated error management in mod services

Documents