improved compression-latency trade-off via delayed-dictionary compression raanan refua tel aviv...
TRANSCRIPT
![Page 1: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/1.jpg)
Improved Compression-Latency Improved Compression-Latency Trade-Off viaTrade-Off via
Delayed-Dictionary CompressionDelayed-Dictionary Compression
Raanan RefuaRaanan RefuaTel Aviv UniversityTel Aviv University
Under the supervision of Prof. Yossi MatiasUnder the supervision of Prof. Yossi Matias
![Page 2: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/2.jpg)
Improved CompressionImproved Compression--Latency Latency TradeTrade--OffOff
Improving compression ratio for a given Improving compression ratio for a given latency constraintlatency constraint
Improving latency for a given compression Improving latency for a given compression ratioratio
![Page 3: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/3.jpg)
End-to-End CompressionEnd-to-End Compression
||
||,
X
Yr BA
EncoderX X
Decoder
Lenc LdecLtx
(EncodingLatency)
(TransmissionLatency)
(DecodingLatency)
User Latency
Net 1 Net 2Y
![Page 4: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/4.jpg)
PublicNetwork
PublicNetwork
Network ProcessorNetwork Processor
Network ProcessorNetwork Processor
The LAN-to-LANThe LAN-to-LAN Framework Framework
![Page 5: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/5.jpg)
PublicNetwork
PublicNetwork
IP Tunnel 1
IP Tunnel 2
Network Processor
Network ProcessorNetwork Processor
IP-TunnelIP-Tunnel
![Page 6: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/6.jpg)
PublicNetwork
PublicNetwork
Network ProcessorNetwork Processor
VPN over IPSecVPN over IPSec
![Page 7: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/7.jpg)
Packet Compression Algorithms TypesPacket Compression Algorithms Types
Stateless compression (Packet By Packet comp.)Stateless compression (Packet By Packet comp.) Streaming compression (Continuous comp.)Streaming compression (Continuous comp.)
– dependency among packetsdependency among packets
““Offline” compressionOffline” compression– compress & sendcompress & send
||
|)(|)(A BA, BA, BA,
F
FArrrr Fofflinestreamingstateless
offlineLstreamingLstatelessL BA,BA,BA,
![Page 8: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/8.jpg)
Compression Ratios vs Mode Compression Ratios vs Mode and Packet Sizesand Packet Sizes
0.00
0.20
0.40
0.60
0.80
1.00
ptt5
_fa
x/ptt5
/pic
blu
ediv
e.b
mp
Com
pute
r-ke
nnedy.
xls
rfc_
index.
txt
world
192.tx
tbib
le.tx
tlc
et1
0.tx
talic
e29.tx
tasy
oulik
.txt
plra
bn12.tx
tB
ooks
.htm
cp.h
tml
field
s.c
gra
mm
ar.ls
ppaper1
paper2
paper3
paper4
paper5
paper6
book1
book2
alle
zw.p
sE
.coli
new
sbib
xarg
s.1
obj1
obj2
sum
geo
wave
lets
.ppt
wap.p
df
the_fly
.wav
Bid
_Lig
ht.m
peg
mym
eagw
ave
.zip
sain
t_agnes.
mp3
Hortense
89a.g
ifC
heeta
h.jp
g
File Name
Co
mp
ressio
n R
atio
fp stateless ratio (|frag|=1500)fp stateless ratio (|frag|=125)fp streaming compression ratio (|frag|=1500)fp streaming compression ratio (|frag|=125)
Small Packets Improvement
![Page 9: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/9.jpg)
Potential ImprovementPotential Improvement
φ(Bstateless,Bstreaming)
0.50
1.00
1.50
2.00
2.50
ptt5
_fax
/ptt
5/pi
cbl
uedi
ve.b
mp
Com
pute
r-ke
nned
y.xl
srf
c_in
dex.
txt
wor
ld19
2.tx
tbi
ble.
txt
lcet
10.t
xtal
ice2
9.tx
tas
youl
ik.t
xtpl
rabn
12.t
xtB
ooks
.htm
cp.h
tml
field
s.c
gram
mar
.lsp
pape
r1pa
per2
pape
r3pa
per4
pape
r5pa
per6
book
1bo
ok2
alle
zw.p
sE
.col
ine
ws
bib
xarg
s.1
obj1
obj2
sum
geo
wav
elet
s.pp
tw
ap.p
dfth
e_fly
.wav
Bid
_Lig
ht.m
peg
mym
eagw
ave.
zip
sain
t_ag
nes.
mp3
Hor
tens
e89a
.gif
Che
etah
.jpg
![Page 10: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/10.jpg)
Streaming Compression Streaming Compression ImplementationImplementation
X Y X
Encoder
Packets
Dictionary
Rx1
Rx2Tx2
Tx1
[ACKsData]
Encoder
Decoder
Packets
Dictionary
Rx1
Rx2Tx2
Tx1Decoder
ACKs ACKs ACKs
![Page 11: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/11.jpg)
Latency in Streaming CompressionLatency in Streaming Compression
Large Decoding Latency due toLarge Decoding Latency due to Packet dropsPacket drops
– dominating latencydominating latency Packet reordering due toPacket reordering due to
– retransmissionsretransmissions– different pathsdifferent paths
X Y X
Encoder Decoder
Packets
Dictionary
Packets
Dictionary
![Page 12: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/12.jpg)
Decoding Latency in StreamingDecoding Latency in Streaming
1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 77, , 88, , 99, , 1010, , 66
Packet Packet 66 is dropped and retransmitted is dropped and retransmitted– LLdec dec for packet 7 = 4for packet 7 = 4
Packets Packets 7…107…10 cannot be decoded until cannot be decoded until 66 is is received.received.– The phrases of packets The phrases of packets 7…107…10 cannot be added cannot be added
to the dictionary until packet to the dictionary until packet 66 is received. is received.
![Page 13: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/13.jpg)
Actual Decoding Latency of StreamingActual Decoding Latency of Streaming
Average LAverage Ldecdec for each of 269 Planet-Lab nodes for each of 269 Planet-Lab nodes– 26012 packets per node26012 packets per node
Total Average LTotal Average Ldecdec = 62 packets = 62 packets LLdecdec=0 for Stateless=0 for Stateless
0
50
100
150
200
250
300
350
400
450
1 27 53 79 105 131 157 183 209 235 261
Decoding LatencyAverages
Total Average
![Page 14: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/14.jpg)
Improved Compression & Latency?Improved Compression & Latency?
We would like We would like
– Compression ratio of Compression ratio of Streaming Streaming
– Latency of StatelessLatency of Stateless
0
50
100
150
200
250
300
350
400
450
1 28 55 82 109 136 163 190 217 244
Streaming DecodingLatency Averages
Streaming TotalAverage
0.00
0.20
0.40
0.60
0.80
1.00
ptt5_fa
x/p
tt5/p
icblu
ediv
e.b
mp
Com
pute
r-kennedy.x
lsrf
c_in
dex.txt
world192.txt
bib
le.txt
lcet1
0.txt
alic
e29.txt
asyoulik
.txt
plrabn12.txt
Books.h
tmcp.h
tml
field
s.c
gra
mm
ar.
lsp
paper1
paper2
paper3
paper4
paper5
paper6
book1
book2
alle
zw
.ps
E.c
oli
new
sbib
xarg
s.1
obj1
obj2
sum
geo
wavele
ts.p
pt
wap.p
df
the_fly
.wav
Bid
_Lig
ht.m
peg
mym
eagw
ave.z
ipsain
t_agnes.m
p3
Hort
ense89a.g
ifC
heeta
h.jp
g
File Name
Co
mp
ressio
n R
ati
o
fp stateless ratio (|frag|=125)fp streaming compression ratio (|frag|=125)
![Page 15: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/15.jpg)
Delayed-Dictionary Compression Delayed-Dictionary Compression (DDC)(DDC)
Contains a sub-method called Basic-DDC Contains a sub-method called Basic-DDC (BDDC)(BDDC)
Streaming with Streaming with enforced delayenforced delay in the creation of in the creation of the dictionarythe dictionary
Dictionary does not use the most recent historyDictionary does not use the most recent history– Compression ratio is close to StreamingCompression ratio is close to Streaming– Decoding Latency is close to StatelessDecoding Latency is close to Stateless
![Page 16: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/16.jpg)
On-line Dictionary Compression On-line Dictionary Compression Algorithms ModelAlgorithms Model
Encoder - a model for incremental dictionary compression algorithm C. Themodel enables complete separation between the dictionary parser and the outputparser.
dictionary parserPd
dictionaryD
output parserPo
Input: T Output: C(T)
![Page 17: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/17.jpg)
On-line Dictionary Compression On-line Dictionary Compression Algorithms ModelAlgorithms Model
dictionary parserPd
dictionaryD
output decoder
Compressed text: C(T)Original text: T
Decoder - a model for incremental dictionary decompression C’. The modelenables complete separation between the dictionary parser and the output decoder.
![Page 18: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/18.jpg)
On-line Dictionary Compression On-line Dictionary Compression Algorithms Model GeneralizationAlgorithms Model Generalization
dictionary parserPd
dictionaryD(Δ)
output parserPo
Input: T Output: C(T)
D at point t is a function of text up to point t-Δ
![Page 19: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/19.jpg)
On-line Dictionary Compression On-line Dictionary Compression Algorithms Model GeneralizationAlgorithms Model Generalization
dictionary parserPd
dictionaryD(Δ)
output decoder
Compressed text: C(T) Original text: T
![Page 20: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/20.jpg)
Lempel-Ziv 77 with DelayLempel-Ziv 77 with Delay
ABABCAADAA... ABCBBBAC…
Coding positionPointer
Characters
|Window|=W Lookahead BufferDelay (Δ)
![Page 21: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/21.jpg)
LZ77 with Delay in Terms of PacketsLZ77 with Delay in Terms of Packets
ABC…
Coding positionPointer
|Window|=W CurrentlyEncodedPacket
Delay (Δ)
ABC…
![Page 22: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/22.jpg)
Basic Delayed-Dictionary Compression Basic Delayed-Dictionary Compression (BDDC) in Terms of Packets(BDDC) in Terms of Packets
Encoder Decoder
1.…………………1000…………1100..………………
Received by decoder Δ=99
Currently decoded packet
P1100=f(P1…P1000)P1101=f(P1…P1001)…
1100..……1000..………1
Encoding
Decoding
1100=f(1…1000)
![Page 23: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/23.jpg)
BDDC ImplementationBDDC Implementation
Header Information:Header Information:
S.N. of theS.N. of the current packetcurrent packet
S.N. of theS.N. of the last packet last packet
used used for encodingfor encoding
Packet is Packet is
CompressedCompressed
Y/NY/N
(a bit)(a bit)
![Page 24: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/24.jpg)
BDDC Implementation (cont.)BDDC Implementation (cont.)
X Y X
Encoder
Packets
Dictionary
Rx1
Rx2Tx2
Tx1
[ACKsData]
Encoder
Decoder
Packets
Dictionary
Rx1
Rx2Tx2
Tx1Decoder
ACKs ACKs ACKs
ΔQ
![Page 25: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/25.jpg)
Decoding Latency in BDDC (Decoding Latency in BDDC (ΔΔ=5)=5)
1, 2, 3, 4, 5, 1, 2, 3, 4, 5, 77, , 88, , 99, , 1010, , 66
Packet Packet 66 dropped and retransmitted dropped and retransmitted Now we can decode packets Now we can decode packets 7…107…10
– No decoding latencyNo decoding latency– The phrases of packets The phrases of packets 7…107…10 still can not be still can not be
added to the dictionary until packet added to the dictionary until packet 66 is received is received
![Page 26: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/26.jpg)
Effect of Encoding Delay on Effect of Encoding Delay on Compression QualityCompression Quality
Δ
r
![Page 27: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/27.jpg)
Encoding Delay Vs. Encoding Delay Vs. Compression QualityCompression Quality
ABABABADAA... ABCDBBBAC…
Coding positionPointer
Characters
|Window|=W Lookahead Buffer
ABABCDAA...
Without delay
![Page 28: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/28.jpg)
Encoding Delay Vs. Encoding Delay Vs. Compression Quality (cont.)Compression Quality (cont.)
ABABBADAA... ABCDBBBAC…
Coding positionPointer
Characters
|Window|=W Lookahead Buffer
ABABCDAA...
Delay (Δ)
With delay
![Page 29: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/29.jpg)
DDC vs Stateless vs StreamingDDC vs Stateless vs StreamingLarge PacketsLarge Packets
|header|=20 bytes, |payload|=1500 bytesConcatenation of 18 Calgary-Corpus files
0.40
0.45
0.50
0.55
0.60
0 5 10 15 20 25 30 35 40 50 60 70 80 90 100
200
300
400
500
600
700
800
900
1000
1500
2000
2500
3000
4000
5000
6000
Dictionary Delay Δ (packets)
Com
pres
sion
ratio
statelesscompressionratio
streamingcompressionratio
DDC-min
![Page 30: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/30.jpg)
DDC vs Stateless vs StreamingDDC vs Stateless vs StreamingSmall PacketsSmall Packets
|header|=20 bytes, |payload|=125 bytesConcatenation of 18 Calgary-Corpus files
0.500.550.600.650.700.750.800.85
0 510 15 20 50
100
200
300
400
500
750
1000
1100
1200
1500
2000
2500
3000
4000
5000
1000
015
000
2000
025
000
3000
035
000
4000
0
Dictionary Delay Δ (packets)
Comp
ress
ion ra
tio
statelesscompressionratio
streamingcompressionratio
DDC-min
![Page 31: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/31.jpg)
Effect of Encoding Delay on Effect of Encoding Delay on Decoding LatencyDecoding Latency
Δ
Ldec
![Page 32: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/32.jpg)
Effect of Encoding delay on Effect of Encoding delay on Decoding LatencyDecoding Latency
1………………m…………………………..i1=m+Δ
Received bythe decoder
Δ1=Δ
Currently decoded packet
1………………m……………i2=m+Δ/2
Δ2=Δ/2
1………………m……i3=m+Δ/4
Δ3=Δ/4
Δ1 > Δ2 > Δ3
Ldec1 < Ldec2 < Ldec3
![Page 33: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/33.jpg)
The Compression-Latency Trade-offThe Compression-Latency Trade-off
Ldec
Δ
r
![Page 34: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/34.jpg)
BDDC ConflictBDDC Conflict
1.0
0.5
0
BDDCAv
Ldec(packets)
DDCTraffic
CompressionRatio
020406080
1000 10 20 100
300
500
1000
1200
Dictionary Delay Δ (packets)
DDC Traffic Compression Ratio
BDDC Av Ldec (packets)
![Page 35: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/35.jpg)
Trade-off between the Decoding-Trade-off between the Decoding-Latency and the Compression RatioLatency and the Compression Ratio
Compression Ratio
Ldec
Stateless (Δ=∞)
Streaming (Δ=0)DDC (Δ)
Confirmed-Dictionary Compression (ΔConf)
![Page 36: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/36.jpg)
DDC algorithmsDDC algorithms
BDDC - Fixed Delay AlgorithmBDDC - Fixed Delay Algorithm BDDC - Adaptive Delay AlgorithmBDDC - Adaptive Delay Algorithm Confirmed-Dictionary Compression (CDC)Confirmed-Dictionary Compression (CDC) DDC-minDDC-min DDCDDC
![Page 37: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/37.jpg)
DDC Algorithms (cont.)DDC Algorithms (cont.)
Fixed Delay AlgorithmFixed Delay Algorithm ∆ ∆ is constantis constant ∆ ∆ is proportional to the network is proportional to the network
propagation delay (requires calibration)propagation delay (requires calibration)
![Page 38: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/38.jpg)
DDC Algorithms (cont.)DDC Algorithms (cont.)
Adaptive Delay AlgorithmAdaptive Delay Algorithm Maintains a Maintains a delay delay parameter, depending onparameter, depending on
– decoding successdecoding success– changes in changes in pp for packet drops for packet drops
![Page 39: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/39.jpg)
DDC Algorithms (cont.)DDC Algorithms (cont.)Confirmed-Dictionary Compression (CDC) AlgorithmConfirmed-Dictionary Compression (CDC) Algorithm The Encoder registers the ACKsThe Encoder registers the ACKs max max ΔΔ s.t. s.t.
ACK received for all packets prior to previous ACK received for all packets prior to previous ΔΔ packets packets
Ensures immediate decodingEnsures immediate decoding Tradeoff in the compression ratioTradeoff in the compression ratio Does not required a retransmissionDoes not required a retransmission
1…………………x……x……x……x……x…..
All packets receivedby the decoder
max Δ
Currently decoded packetACKs List
![Page 40: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/40.jpg)
DDC Algorithms (cont.)DDC Algorithms (cont.)CDC Variant – good for lossy trafficCDC Variant – good for lossy traffic Choose a large enough Choose a large enough ΔΔ If Packet If Packet ii was not ACKed was not ACKed
– Throw it from the queueThrow it from the queue
Does not required a retransmissionDoes not required a retransmission
![Page 41: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/41.jpg)
DDC vs Stateless, StreamingDDC vs Stateless, Streaming
DDC vs StatelessDDC vs Stateless Saves much larger amounts of BandwidthSaves much larger amounts of Bandwidth
– Especially for small packetsEspecially for small packets
DDC vs StreamingDDC vs Streaming Smaller latencySmaller latency DDC good for latency sensitive applicationsDDC good for latency sensitive applications
– Packet drops allowed (up to Packet drops allowed (up to ΔΔ) for decoding) for decoding– Packet reordering allowedPacket reordering allowed
![Page 42: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/42.jpg)
DDC-minDDC-min
Compress in parallel with Stateless and Compress in parallel with Stateless and BDDCBDDC
Transmit according to Transmit according to
min {Stateless,BDDC,Uncompressed}min {Stateless,BDDC,Uncompressed} Use a bit in the header to sign the Use a bit in the header to sign the
compression methodcompression method
![Page 43: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/43.jpg)
DDC = BDDC DDC = BDDC Stateless Stateless
DDBDDC BDDC i i == DD (1…i-(1…i-ΔΔ-1)-1)
DDDDC DDC i i = D= DBDDC BDDC ii D DStateless Stateless ii
DBDDC i = D(1…i-Δ-1) DBDDC i+1
DDDCi =
DBDDC i DStateless i
DDDC i+1DDDC i-1
Rollback
![Page 44: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/44.jpg)
DDC ImplementationDDC Implementation
DDBDDCBDDC i i DDStatelessStateless i i
DDDDC DDC ii
Last Phrase Number=j First Phrase Number=j+1
Dictionary Insertion:if (phrase xDBDDC i) & (phrase xDStateless i)
add x to DStateless i
![Page 45: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/45.jpg)
BDDC DDCBDDC
Graph Based DictionariesGraph Based Dictionaries
![Page 46: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/46.jpg)
Non Compact TriesNon Compact Tries
$
B G$
I
G L
L
BILL$
$
BIGGER$
BIG$
$
G
E
R
$GOOD$ GOSH$
O
O S
D H
$ $
O
W
L
I
N
G
BOWLING$$
A new phrase
![Page 47: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/47.jpg)
Eager DecodingEager Decoding
Each packet contains data about the Each packet contains data about the historyhistory
The Decoder may be able to decode The Decoder may be able to decode received packets, without all the received packets, without all the required historyrequired history
![Page 48: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/48.jpg)
When to Use DDC?When to Use DDC? Latency critical applicationsLatency critical applications Order is not required by application Order is not required by application
(preferable)(preferable) Packet drops are allowedPacket drops are allowed
– use retransmission for dictionary update use retransmission for dictionary update purposespurposes
![Page 49: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/49.jpg)
Example ApplicationsExample Applications
VoIP CompressionVoIP Compression Raw Video CompressionRaw Video Compression TFTP (over UDP)TFTP (over UDP)
– Fast transfer of filesFast transfer of files– Fast transfer of memory areasFast transfer of memory areas
![Page 50: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/50.jpg)
SMSC Encoder Decoder
PC1
PC2
PC4
PC3
TCP SS7
ICQClients
MobilePhoneUsers
User Latency
DecodingLatency
Example Applications (cont.)Example Applications (cont.) SMS Chat from a PC to a Mobile PhoneSMS Chat from a PC to a Mobile Phone
– SMS messages are transferred via the SS7 networkSMS messages are transferred via the SS7 network– SMS uses “Submit ACK”SMS uses “Submit ACK”
![Page 51: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/51.jpg)
Instant MessagingInstant Messaging– A packet should be enough for each messageA packet should be enough for each message
Network ManagementNetwork Management– Multicast/Broadcast of control messagesMulticast/Broadcast of control messages– Deploying information to remote nodes Deploying information to remote nodes – Sensor networksSensor networks
Example Applications (cont.)Example Applications (cont.)
![Page 52: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/52.jpg)
Experimentation on InternetExperimentation on Internet
Tx: planetlab1.tau.ac.ilRx: 269 Planet-Lab nodes
TAU
![Page 53: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/53.jpg)
Packet Drop ProbabilityPacket Drop ProbabilityExperiment Description:Experiment Description: Concatenation of 18 Calgary-Corpus filesConcatenation of 18 Calgary-Corpus files
– 3.25MB3.25MB Transmission with UDP Transmission with UDP
– Each packet contains 125B of dataEach packet contains 125B of data– Transmission to 269 nodesTransmission to 269 nodes
Results:Results:PPdropdrop=1.27%=1.27%PPsuccesssuccess=98.73%=98.73%Var=0.002Var=0.002
![Page 54: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/54.jpg)
Decoding Latency vs Encoding DelayDecoding Latency vs Encoding Delay
020406080
Dictionary Delay Δ (packets)
To
tal A
v L
dec (p
ackets
)
BDDC
Streaming
![Page 55: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/55.jpg)
LLdecdec Distribution of Streaming vs. Distribution of Streaming vs. BDDC with BDDC with ΔΔ=300=300
110
1001000
10000100000
100000010000000
1 225 449 673 897
Ldec (packets)
Nu
mb
er
of P
ackets
Ldec BDDCΔ=300
LdecStreaming
![Page 56: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/56.jpg)
A favorable trade-offA favorable trade-off
Compression Compression ratio close to ratio close to streamingstreaming
Decoding Decoding latency close to latency close to statelessstateless
020406080
Dictionary Delay Δ (packets)
To
tal A
v L
dec (p
ackets
)
BDDC
Streaming
0.500.550.600.650.700.750.800.85
0 510 15 20 50
100
200
300
400
500
750
1000
1100
1200
1500
2000
2500
3000
4000
5000
1000
015
000
2000
025
000
3000
035
000
4000
0
Dictionary Delay Δ (packets)Co
mpre
ssion
ratio
statelesscompressionratio
streamingcompressionratio
DDC-min
![Page 57: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/57.jpg)
Related WorkRelated Work
Header Compression:Header Compression: J. Lilley, J. Yang, H. Balakrishnan, and S.
Seshan, “A unified header compression framework for low-bandwidth links,” ACM/IEEE International Conference on Mobile Computing and Networking, pp. 131–142, 2000.
![Page 58: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/58.jpg)
Related Work (cont.)Related Work (cont.)
Stateless Payload Compression:Stateless Payload Compression: A. Shacham et al “IP payload compression
protocol (IPComp),” IETF, RFC 2393, 1998 R. Friend and R. Monsour, “IP payload
compression using LZS,” IETF, RFC 2395, 1998
R. Pereira, “IP payload compression using DEFLATE,” IETF, RFC 2394, 1998
![Page 59: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/59.jpg)
Related Work (cont.)Related Work (cont.)
Payload Compression: S. Dorward and S. Quinlan, “Robust data
compression of network packets,” Bell Labs, 2000, Unpublished manuscript
![Page 60: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/60.jpg)
Related Work (cont.)Related Work (cont.)S. Dorward and S. Quinlan - thwack
![Page 61: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/61.jpg)
Possible ExtensionsPossible Extensions
DDC can be used on any on-line DDC can be used on any on-line compression algorithm (e.g. Deflate)compression algorithm (e.g. Deflate)
Implementation of eager decodingImplementation of eager decoding Multi-Media applicationsMulti-Media applications
– voice compressionvoice compression– download Raw images to mobile phonesdownload Raw images to mobile phones
![Page 62: Improved Compression-Latency Trade-Off via Delayed-Dictionary Compression Raanan Refua Tel Aviv University Under the supervision of Prof. Yossi Matias](https://reader035.vdocuments.site/reader035/viewer/2022062515/56649cfa5503460f949cc0c0/html5/thumbnails/62.jpg)
ConclusionsConclusions
Improved Compression-Latency Trade-offImproved Compression-Latency Trade-off Delayed-Dictionary Compression AlgorithmDelayed-Dictionary Compression Algorithm Experimentation over the internet with Experimentation over the internet with
Planet-LabPlanet-Lab