borje justrell director ict department national archives, sweden 2005
DESCRIPTION
The Digital Black Hole Reflections on Costs for Long Term Storage of Digital Information. Based on an Article by Jonas Palm, Head of Department of Preservation. Borje Justrell Director ICT department National Archives, Sweden 2005. “I’ve invented a machine which can answer all questions!”. - PowerPoint PPT PresentationTRANSCRIPT
The Digital Black Hole
Reflections on Costs for Long Term Storage of Digital Information. Based on an
Article by Jonas Palm, Head of Department of Preservation
The Digital Black Hole
Reflections on Costs for Long Term Storage of Digital Information. Based on an
Article by Jonas Palm, Head of Department of Preservation
Borje JustrellDirector
ICT departmentNational Archives, Sweden
2005
Borje JustrellDirector
ICT departmentNational Archives, Sweden
2005
“I’ve invented a machine which can answer all questions!”
…it knows all the answers but I can’t come up with the questions….
The digital black hole
$ €SEK
DKK
$
GBP
YEN SEK€
AUD
BTN
…the digital information may be lost for ever. It will be as secretly kept as anything captured by the
Black Holes in space.
Initiatives - lots of energy
The Life Cycle of Digitization
projects
Life on the Web
PlanningFunding
Project up and running
The Good Ideas
Termination
Survival of the fittest
“Which
project…?”
Digitization project costs…
1. The transfer to a digital format
2. Preserving the digital information
and keeping it accessible
Scanned material
Born digital
Analog audio and video magnetic tape
Should be stored digitally
Must be digitized to survive
Stored analog on film for future use
Left in the digital domain until it “dies”
Stored digitally for future use
Some reflections…
Storage demands for different kinds of
information; for comparison - approx 1 minute of
informationFile size of approx 1 minute digitally stored information
0
10
20
30
40
50
60
1
Format
Mb
Text A4
Image RGB 8bit 300dpi A4
Audio (CD quality) 1min, 16Bit 44100sampl freq
Video+sound (DVDmovie) 1 min
Harvard University Library, 2004
Comparison of costs for storing the same information on different media
including the original on paper
National Archives, Sweden 2005
Comparison of costs for storing the same information on different media
including the original on paper
Costs for storing the same information on different media
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
1Medieformer
COM 35 mm film
331 pages/ 165,5 sheets) inarchival box
693 Mb digital image files (1A4 = 600 dpi b/w (1-bit))
1,35 Gb digital image files(1 A4 = 300 dpi greyscale(8-bit))
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
COM 332 pages 1-bit 600 dpi 8-bit 300 dpi
RA
Harvard
Comparison of costs for storing the same information on different media
including the original on paper
National Archives and Harvard University Library
USD
Decrease in storage costs for 1 Mb
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
År
€
Decrease in price for computer processing same kind of information
0.0
500.0
1000.0
1500.0
2000.0
2500.0
År
€
Expected decrease in price of HSM-system
050000
100000150000200000250000300000350000400000
År
€
Storage Costs in HSM systemall costs in SEK
Storage Costs in HSM systemall costs in SEK
Storage costs in HSM system 1st year 2nd year 3rd year 4th year 5th year 5 years Cost/year 1 HSM storage system price 2003 + 3% interest/ year 948189 923791 899392 874994 850595 4496961 Price 4066430: - spread over 5 years; 813286: -/year
Storage medium 40 Tb/år 179300 112959 71164 44833 28244 436500 Staff 60% , operation, à 400000 :- incl all costs 240000 252000 264000 276600 289800 1322400 Staff 40% , input of data, à 400000 :- incl all costs 160000 168000 176000 184400 193200 881600
Premises 100m2 à 1260:- 126000 129150 132379 135688 139080 662297
Service/s upport 227000 289000 289000 289000 289000 1383000 Yearly Storage Cost (equipment, operation, service) 1541189 1593941 1584771 1576282 1568475 7864658 Yearly storage cost (equipment, operation, service) Gb/ år 38.53 19.92 13.21 9.85 7.84 89.35 Yearly Input Cost (staff, storage medium) 339300 280959 247164 229233 221444 1318100 Yearly input cost (staff, storage medium) Gb/ år 8.48 7.02 6.18 5.73 5.54 32.95
Tota l cost Gb/ år 47.01 26.95 19.39 15.58 13.38 122.31
To al cost 1 GB for 5 years 122.31
Costs for equipment and storage medium
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
1000000
1styear
2ndyear
3rdyear
4thyear
5thyear
SEK
Equipment
Storage medium 40Tb/år
Storage costs per Gb
0
5
10
15
20
25
30
35
40
45
50
1 2 3 4 5
Year
SEK
Yearly storage cost(equipment,operation, service)Gb/år
Yearly input cost(staff, storagemedium) Gb/år
Total cost Gb/år
Operating Costs
0
100000
200000
300000
400000
500000
600000
1 2 3 4 5
Year
SEK
Staff à 400000:-incl all costs
Premises 100m2 à1260:-
Service/support
Distribution of costs for storage in HSM
system at the National Archives, Sweden
Distribution of costs for storage in HSM
system at the National Archives, Sweden
…but the scanning…?What does that cost?…but the scanning…?What does that cost?
Distribution of scanning costs
Costs based on production costs at the scanning facility MKC, National Archives,
Sweden
Activities involved in feed scanning 1-bit 600 dpi A4 page
4%
3%
17%
33%12%
2%
14%
15%Registration
Archiving
Preparation
Scanning
Image control
Corrections
Extras
Administration
Distribution of scanning costs
Costs based on production costs at the scanning facility MKC, National Archives,
Sweden
Activities involved in scanning 8-bit 297 dpi greyscale of A2 page
2%
2%
2%
65%
4%
1%
5%
18%
RegistrationArchivingPreperationScanningImage controlCorrectionsExtrasAdmnistration
Total cost ≈€ 90,000,000
Digitizing costs for A/V as suggested in Swedish report covering 4,5 million hours on tape
30% audio tapes = 1,350,000 h = 810Tb in CD-quality
70% videotapes = 3,150,000 h = 7,245Tb in DVD quality
“Due to condition and technical circumstances transfer should be made
within the next ten years”
Total of 8,055Tb if digitized to CD- and DVD format
Facts from report published by The Ministry of Culture, Sweden, 2004, Bevara ljud och rörlig bild (SOU 2004:53)
(Preserving sounds and images)
Digitizing costs for A/V (1 year)
Digitizing A/V material 805,000Gb/year
€ 9,000,000
€ 11.18/Gb
Facts from report published by The Ministry of Culture, Sweden, 2004, Bevara ljud och rörlig bild (SOU 2004:53)
(Preserving sounds and images)
Activities involved in digitizing A/V objects
2%3%10%
75%
1%
2%
5%
2%
Registration
Archiving
Preparation
Transfer to digital
Control
Corrections
Extras
Administration
Scanning in Gb in Swedish scanning projects
Amount of Gb in Swedish digitization projects including A/V
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
Gb
A4 1-bit images
A2 8-bit greyscale images
A/V
Scanning costs costs per Gb
0
5
10
15
20
25
30
35
€
prod cost € per Gb 1-bit
prod cost € per Gb 8-bit
prod cost € per Gb A/V
Scanning costs compared
Swedish project will allow lesser quality digitization
e.g.
One person should be able run up to 12 taperecorders
Transfer from analog to digital at double speed
Facts from report published by The Ministry of Culture, Sweden, 2004, Bevara ljud och rörlig bild (SOU 2004:53)
(Preserving sounds and images)
Huge investments in digitizing must be met with adequate arrangements for preserving the work done
$
GBP
YEN SEK€
AUD
BTN
The Future
2D bar code2D bar code
May store any kind of digital information on film - text, image, audio, video
2D bar code2D bar code
1 m of 16mm film can today store 30Mb
The Rosetta stone
Greek text
Demotic text (Egyptian text)
Hieroglyphs
With 2D bar code solution each film will begin with plain readable text on what the film contains and what is needed to access its information content.
Conclusions…
…depends on your agenda...