preservation-worthy digital video: hope you brought your checkbook
DESCRIPTION
Preservation-Worthy Digital Video: Hope You Brought Your Checkbook. Jerome McDonough New York University November 8, 2014. Digital Video Basics. A video signal consists of luminance and chrominance information Luminance – brightness, varying from white to black (abbreviated as Y) - PowerPoint PPT PresentationTRANSCRIPT
Preservation-Worthy Digital Video: Hope You Brought Your
Checkbook
Jerome McDonoughNew York University
April 20, 2023
Digital Video Basics
A video signal consists of luminance and chrominance informationLuminance – brightness, varying from white to black (abbreviated as Y)Chrominance – color (hue & saturation), conveyed as a pair of color difference signals: R-Y (hue & saturation for red, without luminance) B-Y (hue & saturation for blue, without
luminance)
Digital Video Basics
Where’s the green?Spectral response of the human eye peaks in the green frequencies. The perceived brightness of an item can be constructed using weighted values for its red, green and blue components: Y = 0.299R + 0.587G +0.114B, or
G Y R B 0 3 0 110 5 9. .
.
Digital Video Basics
4:2:2 sampling 4:2:0 sampling
4:1:1 sampling
4:2:2 – High End DV (DigitalBetacam, DVCPro50)
4:2:0 – MPEG 1 & 24:1:1 – DV and DVCAM
Digital Video Basics
Why not 4:4:4 sampling? 720 x 486 resolution = 349,920 pixels per frame 349,920 pixels x 10 bits/sample x 3 samples/pixel =
10,497,600 bits per frame 10,497,600 bits/frame X 29.97 frames/second =
314,613,072 bits per second 314,613,072 bps x 3600 seconds = ~141.58 GB/hour For 1920x1080 HDTV, more like 840 GB/hour
4:2:2 sampling drops that rate by a third with almost no perceptible difference in quality. 4:2:0 and 4:1:1 drop it in half.
Digital Video Basics
MPEG 2 Compression Further subsampling
down sample to 8 bits/sample down sample to 4:2:0 sampling regime
Discrete Cosine Transformation + Requantizing of coefficients from DCT
Variable Length Encoding & Run Length Encoding Interframe compression (motion compensation)
all of which can take a 209 mbps video rate (for 4:2:2 video) and reduce it to around 8 mbps with no apparent visual loss.
Digital Video Basics
Raw digital video is extremely storage and bandwidth intensive.As a result, almost all digital video processing systems employ a mix of lossless and lossy compression mechanisms.
Preservation-Worthy Digital Video
Desired characteristics for digital video we feel we can preserve include: Content can be migrated to new formats
and new media without introducing artifacts Stored in non-proprietary, standard format
which is openly documented Easy to produce derivatives for end-user
distribution Minimize costs of production, distribution &
migration
Sampling, Migration & Artifacts
As in still image digitization, employing lossy compression can lead to artifacting when you migrate.Unlike still images, lossy compression is assumed in almost all video processing technology today.
One River Media Codec Test Image
Images courtesy of Marco Solorio, One River Media
Black Magic 8-Bit 4:2:2 Uncompressed Codec
Images courtesy of Marco Solorio, One River Media
Black Magic 8-Bit: 10th Generation
Images courtesy of Marco Solorio, One River Media
One River Codec Test
Images courtesy of Marco Solorio, One River Media
Digital Voodoo 10-bit Codec
Images courtesy of Marco Solorio, One River Media
Digital Voodoo 10-bit: 10th Gen.
Images courtesy of Marco Solorio, One River Media
One River Codec Test
Images courtesy of Marco Solorio, One River Media
Apple 4:4:4 “None”: 10th Gen.
Images courtesy of Marco Solorio, One River Media
Sampling, Migration & Artifacts
4:4:4 sampling fulfills the digital promise of perfect copies across generations, butMost video equipment doesn’t actually support it. Most high-end video editing packages on computers do and will store 4:4:4 to disk.Lesson: if you want to store 4:4:4 uncompressed video, prepare to buy a lot of disk (or HSM), and abandon videotape.
Storing 4:4:4 Uncompressed Video
QuickTime – Proprietary, but publicly documented and does the task, and software support is availableMJPEG 2000 – Open Standard (ISO/IEC 15444), supports 4:4:4 uncompressed. Software support iffy, but growing.Material Exchange Format – Open Standard, but software support is weak, and some vendor issues
Storing 4:4:4 Uncompressed Video
Videotape is a non-starter. D1 tape decks for uncompressed video cost $200K, and use 4:2:2.Disk vs. HSM Tape Cost vs. Speed Opportunities to detect bit rot, ability to
migrate, time to produce derivatives
Waiting on grid storage….
Mind Games I: Conversion
NYU has approximately 30,000 hours of moving image material, undigitized, in its special collections. Let’s digitize 1/10 of that.9 Digitization/Editing workstations: $690,0009 conversion staff full time for 1 year: $350,000 425 TB of Disk Storage @ $10k/TB = $4,250,000Grand Total: $5,290,000FYI, according to ARL, that’s nearly half our entire 2002 materials budget
Mind Games II: On-going costs
Assume migration every 10 years. Assume time to migrate = 2x time of source material. 6,000 hours x staff salary = ~$120K / 10 = $12k/year
Assume new derivatives every 5 years, and time to migrate = 2x time of source material 12,000 hours x staff salary = ~$240k / 10 = $24k/year
3% disk loss/year x 425 TB = 12.75 TB replaced/year. Assuming disk prices are halved every two years, for next 10 years we’d have $38,750 total replacement costs, or $3,875/year.Grand Total: ~$40K/year maintenance costs
Mind Games III
On-going maintenance costs for 3,000 hours of video on disk aren’t particularly bad.Initial conversion costs, however, are nightmarish.If you don’t spend the money, however, your digital video is unlikely to prove any more preservation-worthy than analog.
NYU Costs: Capture Hardware
Complete system cost: ~$125,000.00
NYU Costs: Conversion Personnel
Currently conversion takes approximately 8 hours for every hour of tape.Minimum personnel cost of ~$150/hour (staff time + benefits) for conversionHope to lower conversion time with practice (and better equipment), but at best, probably around $100/hour of tape.
NYU Costs: Repository Hardware
Sun Enterprise 15K w/L700 Tape Backup: ~$400K/yearSun T3 Disk Arrays (10 TB): ~$100K ($10K/TB)
NYU Costs: Repository Personnel
Fractional part of NYU ITS Unix SysAdmin, Network Support Specialist, Tape Backup support, equivalent to about 1 FTE~$75K/year
NYU Costs: Summary
About $475K/year to keep our server alive and happy; $10K to add another terabyteAbout $125K to add a new video capture/editing workstationAbout $100-150 per hour of video to pay staff capture/conversion costs
The Good vs. the Perfect
Good: 1 hour of video dubbed to Digital Betacam (w/duplicate master) and converted to DVD & MPEG4 streaming derivatives $150 staff time + $70 (2 digibeta tapes) + $3 (DVD-
R blank w/case) + $24.75 (disk for MPEG4) = $247.75
Perfect: 1 hour of video converted to 4:4:4 uncompressed on disk (w/replicated backup) w/DVD & MPEG4 streaming derivatives $150 staff time + $2,860 (disk for master and
backup) + $3 (DVD-R) + $24.75 (disk for MPEG4) = $3,037.75
Affordable Perfection
Disk prices from 1992 to 2000 fell at about 45% per year.If that holds, by 2010, the 1 TB which costs us $10,000 today will cost $276.Storing 143 GB (one hour of 4:4:4 uncompressed video w/audio) will cost $39.50 ($4.50 more than a 60 minute Digital Betacam tape today).