multimedia storage issues
DESCRIPTION
Multimedia Storage Issues. Media vs. Documents. large file size write once, read many deadlines!. OS Review: Disk . Disk. head, spindle, track, sector, cylinder seek time, rotational latency. Disk Scheduler. read/write command. disk sched. I/O Request. Disk Scheduling Algorithm. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/1.jpg)
1
Multimedia Storage Issues
![Page 2: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/2.jpg)
NUS.SOC.CS5248OOI WEI TSANG
2
Media vs. Documents
large file sizewrite once, read manydeadlines!
![Page 3: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/3.jpg)
3
OS Review: Disk
![Page 4: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/4.jpg)
NUS.SOC.CS5248OOI WEI TSANG
4
Disk
head, spindle, track, sector, cylinderseek time, rotational latency
![Page 5: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/5.jpg)
NUS.SOC.CS5248OOI WEI TSANG
5
Disk Scheduler
disksched
I/O Request
read/write command
![Page 6: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/6.jpg)
6
Disk Scheduling Algorithm
![Page 7: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/7.jpg)
NUS.SOC.CS5248OOI WEI TSANG
7
Best Effort SchedulerFCFS: First come first serveSSTF: Shortest seek time firstSCAN: Serve all requests in one
direction, then in the other direction
C-SCAN: Serve all requests in one direction only, then “jump” back to the other end and repeat.
![Page 8: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/8.jpg)
NUS.SOC.CS5248OOI WEI TSANG
8
Other Schemes EDF: Earliest deadline first EDF-SCAN: Use SCAN to schedule
request with same deadline. FD-SCAN: SCAN towards direction
with earliest feasible deadline SSEDO: Rank request by deadline and
use formula (seektime*rank) SSEDV: Use formula (lifetime* +
seektime*(1- )
![Page 9: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/9.jpg)
9
Data Placement
![Page 10: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/10.jpg)
NUS.SOC.CS5248OOI WEI TSANG
10
Contiguous
![Page 11: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/11.jpg)
NUS.SOC.CS5248OOI WEI TSANG
11
Fragmented
![Page 12: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/12.jpg)
NUS.SOC.CS5248OOI WEI TSANG
12
Stripping (RAID-0)
![Page 13: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/13.jpg)
NUS.SOC.CS5248OOI WEI TSANG
13
Mirroring (RAID-1)
![Page 14: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/14.jpg)
NUS.SOC.CS5248OOI WEI TSANG
14
Parity (RAID-5)
![Page 15: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/15.jpg)
NUS.SOC.CS5248OOI WEI TSANG
15
Parity
![Page 16: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/16.jpg)
NUS.SOC.CS5248OOI WEI TSANG
16
Summary of Concerns
High ThroughputFault TolerantLoad balancing
![Page 17: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/17.jpg)
NUS.SOC.CS5248OOI WEI TSANG
17
Design Parameters
unit of striping (block size)degree of striping (num of disks)
![Page 18: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/18.jpg)
NUS.SOC.CS5248OOI WEI TSANG
18
Stripe Unit
![Page 19: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/19.jpg)
NUS.SOC.CS5248OOI WEI TSANG
19
Stripe Unit
![Page 20: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/20.jpg)
NUS.SOC.CS5248OOI WEI TSANG
20
Effects of Block Size
Block Size
servicetime
![Page 21: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/21.jpg)
NUS.SOC.CS5248OOI WEI TSANG
21
Problem
Find block size such that the service time for the most heavily loaded disk is minimize
![Page 22: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/22.jpg)
NUS.SOC.CS5248OOI WEI TSANG
22
Analysis Model: A Plan
service time of the busiest disk as a function of block size
expected num of blocks accessed on the busiest disk
expected num of blocks accessed on any disk
![Page 23: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/23.jpg)
NUS.SOC.CS5248OOI WEI TSANG
23
Assumptions
Non-redundant arrayAnalyze read operation only
![Page 24: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/24.jpg)
NUS.SOC.CS5248OOI WEI TSANG
24
Variables Declaration
Nd: Number of DisksNc: Number of ClientsB : Block sizeNb(i,j): Number of blocks client i
access from disk jNb(i,*): Total number of blocks
client i access
![Page 25: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/25.jpg)
NUS.SOC.CS5248OOI WEI TSANG
25
Analysis
Suppose i request m blocks
![Page 26: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/26.jpg)
NUS.SOC.CS5248OOI WEI TSANG
26
Reading 1 block
Prob(Nb(i,j)=1)
1
1
1
)),((
)),((
D
D
N
m D
DDb
N
m Db
NmNmNjiNP
NmmjiNP
![Page 27: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/27.jpg)
NUS.SOC.CS5248OOI WEI TSANG
27
Reading k block
Prob(Nb(i,j)=k)
1
1
1
)),((
))1(),((
D
D
N
m D
DDb
N
m DDb
NmNmkNjiNP
NmmNkjiNP
![Page 28: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/28.jpg)
NUS.SOC.CS5248OOI WEI TSANG
28
Total Num of Blocks from Disk jNb(*,j) =
CN
ib jiN
1
),(
![Page 29: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/29.jpg)
NUS.SOC.CS5248OOI WEI TSANG
29
Busiest Disk
Nmax = )(*,max..0
jNDNj
![Page 30: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/30.jpg)
NUS.SOC.CS5248OOI WEI TSANG
30
Access Time
T = Nmax(ts + tr + Btt)
Depends on:block size Bdisk characteristic ts,tr,tt
server design Nd
workload characteristic Nc, N(i,j)_
![Page 31: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/31.jpg)
NUS.SOC.CS5248OOI WEI TSANG
31
Model Verification
ServiceTime
Block Size
![Page 32: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/32.jpg)
NUS.SOC.CS5248OOI WEI TSANG
32
Imbalance and Overhead I = 1 – Tavg/Tmax
O = 1 – NmaxBTt/Tmax
![Page 33: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/33.jpg)
NUS.SOC.CS5248OOI WEI TSANG
33
Effects of Block Size
Block Size
Imbalance
Overhead
~optimum
![Page 34: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/34.jpg)
NUS.SOC.CS5248OOI WEI TSANG
34
Effects of Clients
Block Size
Imbalance
Overhead
Nc incr, I decr, opt B incr
![Page 35: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/35.jpg)
NUS.SOC.CS5248OOI WEI TSANG
35
Effects of Clients
BlockSize
Nc
![Page 36: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/36.jpg)
NUS.SOC.CS5248OOI WEI TSANG
36
Effects of Num of Disks
Nd incr, N(i,j) decr, I incr, opt B decrBlockSize
Nd
![Page 37: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/37.jpg)
NUS.SOC.CS5248OOI WEI TSANG
37
Choosing Block Size
Given Nc, assume rest is fixed find B that minimize T
if T < duration of a round incr Nc and try again
![Page 38: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/38.jpg)
NUS.SOC.CS5248OOI WEI TSANG
38
There is more..
redundant arrayfinding optimum degree of
striping
![Page 39: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/39.jpg)
39
Cello: Disk Scheduler
![Page 40: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/40.jpg)
NUS.SOC.CS5248OOI WEI TSANG
40
Problem
How to co-exist with other applications?
![Page 41: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/41.jpg)
NUS.SOC.CS5248OOI WEI TSANG
41
Classes of ApplicationsReal-time
hard/softperiodic/aperiodic
Best-effortinteractivetroughput-intensive
![Page 42: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/42.jpg)
NUS.SOC.CS5248OOI WEI TSANG
42
Attempt 1
Priority-based schedulerAlways schedule real-time tasks
ahead of best-effort tasks
![Page 43: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/43.jpg)
NUS.SOC.CS5248OOI WEI TSANG
43
Attempt 2
Partition time slots into real-time and best-effort
![Page 44: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/44.jpg)
NUS.SOC.CS5248OOI WEI TSANG
44
Attempt 3
Assign weight to application class, based on their priority
Service based on weight
![Page 45: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/45.jpg)
NUS.SOC.CS5248OOI WEI TSANG
45
Attempt 3: Cello
Two-level scheduling
![Page 46: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/46.jpg)
NUS.SOC.CS5248OOI WEI TSANG
46
Class Independent Scheduler
![Page 47: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/47.jpg)
NUS.SOC.CS5248OOI WEI TSANG
47
Allocating Disk BandwidthAllocate in proportion to timeAllocate in proportion to bytes
![Page 48: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/48.jpg)
NUS.SOC.CS5248OOI WEI TSANG
48
Notations
P : Interval of a roundI : Total idle time so far in this
roundUi : Time allocated so far to class
iwi : Weight for class iW : Total weight for all class
![Page 49: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/49.jpg)
NUS.SOC.CS5248OOI WEI TSANG
49
Proportionate Time-Allocation
Ui < P-I
![Page 50: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/50.jpg)
NUS.SOC.CS5248OOI WEI TSANG
50
Proportional Time Allocation
Ui < (P-I)wi/W
![Page 51: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/51.jpg)
NUS.SOC.CS5248OOI WEI TSANG
51
Example
Weights 1:1:2P = 100I = 30
10 20 16 14
![Page 52: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/52.jpg)
NUS.SOC.CS5248OOI WEI TSANG
52
Pick a class i which is underutilized, i.e.
Ui < (P-I)wi/W
![Page 53: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/53.jpg)
NUS.SOC.CS5248OOI WEI TSANG
53
r
next
prev
Class specific schedule will return a request to be inserted,plus to location in the scheduled queue for insertion.(r, prev, next)
![Page 54: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/54.jpg)
NUS.SOC.CS5248OOI WEI TSANG
54
Need to Make Sure..
Does not exceed share for class i
Ui + T < (P-I)wi/W
service time for new request
![Page 55: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/55.jpg)
NUS.SOC.CS5248OOI WEI TSANG
55
Need to Make Sure..
Total service time does not exceed available time
Ui + T + T’next – Tnext < P-I
![Page 56: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/56.jpg)
NUS.SOC.CS5248OOI WEI TSANG
56
Update and Repeat
next
prev
![Page 57: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/57.jpg)
NUS.SOC.CS5248OOI WEI TSANG
57
What if..
a class have no pending request?put in no_work group
a class violates one of the constraints?put in too_much_work group
![Page 58: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/58.jpg)
NUS.SOC.CS5248OOI WEI TSANG
58
Improving Utilization
When disk is idle, distribute service time among classes in too_much_work group
Pick a request from this class and put into scheduled queue
![Page 59: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/59.jpg)
NUS.SOC.CS5248OOI WEI TSANG
59
Free Time Utilization
![Page 60: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/60.jpg)
NUS.SOC.CS5248OOI WEI TSANG
60
Class Specific SchedulerHow to pick (r, prev, next)?
![Page 61: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/61.jpg)
NUS.SOC.CS5248OOI WEI TSANG
61
Slack Time
li = Latest time you must begin servicing request i
ei = Earliest time you may begin servicing request i
di = Deadline for request isi = Slack time for i = li - ei
![Page 62: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/62.jpg)
NUS.SOC.CS5248OOI WEI TSANG
62
Slack Time
li = min(di, li+1) – Ti
ei = ei-1 + Ti-1
Special case when request is at the beginning of queue or end of queue.
![Page 63: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/63.jpg)
NUS.SOC.CS5248OOI WEI TSANG
63
Slack Time
i
i
![Page 64: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/64.jpg)
NUS.SOC.CS5248OOI WEI TSANG
64
Class 1: Interactive Best Effort
next
prev
pick r at front of pending queue, insert into earliest positionin scheduled queue without violating deadlines of others.
![Page 65: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/65.jpg)
NUS.SOC.CS5248OOI WEI TSANG
65
Class 2: High Throughput Best Effort
next
prev
pick r at front of pending queue, insert into tail of scheduled queue. Order consecutive best effort request by SCAN order.
![Page 66: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/66.jpg)
NUS.SOC.CS5248OOI WEI TSANG
66
Class 3: Real Time Application
next
prev
pick r in EDF order. Insert into scheduled queue such that it will meet its deadline and will not violate deadlines of others
![Page 67: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/67.jpg)
NUS.SOC.CS5248OOI WEI TSANG
67
Evaluationsresponsetimeof besteffortapp
# video
SCAN
Cello
![Page 68: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/68.jpg)
NUS.SOC.CS5248OOI WEI TSANG
68
Evaluations%bandwidth
time
50%
No video
text app
video app
![Page 69: Multimedia Storage Issues](https://reader035.vdocuments.site/reader035/viewer/2022062410/5681600c550346895dcf0ba2/html5/thumbnails/69.jpg)
NUS.SOC.CS5248OOI WEI TSANG
69
Cello Summary
OS disk schedulerSupport multiple classesProtect classes from each otherWork conservingUse two-level scheduling