high dynamic range in vr · oled based on electrophosphorescence large viewing angle the power...
TRANSCRIPT
![Page 1: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/1.jpg)
Rafał MantiukDept. of Computer Science and Technology, University of Cambridge
High dynamic range
in VR
![Page 2: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/2.jpg)
These slides are a part of the tutorial
Cutting-edge VR/AR Display Technologies (Gaze-, Accommodation-,
Motion-aware and HDR-enabled)
Presented at IEEE VR in Reutlingen on the 18th of March 2018
The latest version of the slides and the slides for the remaining
part of the tutorial can be found at:
https://vrdisplays.github.io/ieeevr2018/
Material is copyright © Rafał Mantiuk, 2018, except where
otherwise noted.
Rafał Mantiuk, Univ. of Cambridge3
![Page 3: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/3.jpg)
HDR & VR ?
Do we have HDR VR headsets?
OLED contrast 1,000,000:1
Rafał Mantiuk, Univ. of Cambridge5
http://www.oculusvr.com/
![Page 4: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/4.jpg)
ToC & Benefits
HDR in a nutshell
Display technologies in VR
Perception & image quality
Rafał Mantiuk, Univ. of Cambridge6
![Page 5: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/5.jpg)
Slide 7
Dynamic range
max L
min L
(for SNR>3)
Luminance
Rafał Mantiuk, Univ. of Cambridge
![Page 6: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/6.jpg)
Dynamic range (contrast)
As ratio:
Usually written as C:1, for example 1000:1.
As “orders of magnitude”
or log10 units:
As stops:
C =Lmax
Lmin
C10 = log10
Lmax
Lmin
C2 = log2
Lmax
Lmin
One stop is doubling
of halving the amount of light
8 Rafał Mantiuk, Univ. of Cambridge
![Page 7: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/7.jpg)
High dynamic range (HDR)
Luminance [cd/m2]
10-6 10-4 10-2 100 102 104 106 108 Dynamic
Range
1000:1
1500:1
30:1
9 Rafał Mantiuk, Univ. of Cambridge
![Page 8: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/8.jpg)
Visible colour gamut
The eye can perceive more colours
and brightness levels than
a display can produce
a JPEG file can store
The premise of HDR:
Visual perception and not the
technology should define accuracy
and the range of colours
The current standards not fully
follow to this principle
10 Rafał Mantiuk, Univ. of Cambridge
![Page 9: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/9.jpg)
Luminous efficiency function
(weighting)
Light spectrum (radiance)
Luminance
Luminance – how bright the surface will appear
regardless of its colour. Units: cd/m2
Luminance
11
𝐿𝑉 = න
350
700
𝑘𝐿 𝜆 𝑉 𝜆 𝑑𝜆 𝑘 =1
683.002
Rafał Mantiuk, Univ. of Cambridge
![Page 10: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/10.jpg)
Luminance and Luma
Luminance
Photometric quantity
defined by the spectral
luminous efficiency function
L ≈ 0.2126 R + 0.7152 G +
0.0722 B
Units: cd/m2
Luma
Gray-scale value computed
from LDR (gamma
corrected) image
Y = 0.2126 R’ + 0.7152 G’+ 0.0722 B’
R’ – prime denotes gamma
correction
Unitless
R ' = R1/g
12 Rafał Mantiuk, Univ. of Cambridge
![Page 11: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/11.jpg)
Linear vs. gamma-corrected values
Rafał Mantiuk, Univ. of Cambridge13
![Page 12: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/12.jpg)
Sensitivity to luminance
Weber-law – the just-noticeable difference
is proportional to the magnitude of a
stimulus
The smallest
detectable
luminance
difference
Background
(adapting)
luminance
Constant
L
ΔLTypical stimuli:
Ernst Heinrich Weber[From wikipedia]
14 Rafał Mantiuk, Univ. of Cambridge
![Page 13: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/13.jpg)
Consequence of the Weber-law
Smallest detectable difference in luminance
Adding or subtracting luminance will have different visual
impact depending on the background luminance
Unlike LDR luma values, luminance values are not
perceptually uniform!
L ΔL
100 cd/m2 1 cd/m2
1 cd/m2 0.01 cd/m2
15
For k=1%
Rafał Mantiuk, Univ. of Cambridge
![Page 14: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/14.jpg)
How to make luminance (more)
perceptually uniform?
Using “Fechnerian” integration
luminance - L
response
-R
1
ΔL
dR
dl(L) =
1
DL(L)Derivative of
responseDetection
threshold
R(L) 1
L(l)dl
0
L
Luminance
transducer:
16 Rafał Mantiuk, Univ. of Cambridge
![Page 15: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/15.jpg)
Assuming the Weber law
and given the luminance transducer
the response of the visual system to light is:
R(L) 1
L(l)dl
0
L
17 Rafał Mantiuk, Univ. of Cambridge
![Page 16: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/16.jpg)
Fechner law
Response of the visual system to luminance
is approximately logarithmic
The values of HDR pixel values are much
more intuitive when they are plotted /
considered / processed in the logarithmic
domain
Gustav Fechner[From Wikipedia]
R(L) aln(L)
18 Rafał Mantiuk, Univ. of Cambridge
![Page 17: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/17.jpg)
But…the Fechner law does not hold for
the full luminance range
Because the Weber law does not hold either
Threshold vs. intensity function:
L
ΔL
The Weber law
region
19 Rafał Mantiuk, Univ. of Cambridge
![Page 18: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/18.jpg)
Weber-law revisited
If we allow detection threshold to vary with luminance
according to the t.v.i. function:
we can get more accurate estimate of the “response”:
R(L) =1
tvi(l)dl
0
L
ò
L
ΔL tvi(L)
20 Rafał Mantiuk, Univ. of Cambridge
![Page 19: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/19.jpg)
Fechnerian integration and Stevens’ law
21
R(L) - function
derived from the
t.v.i. function
R(L) =1
tvi(l)dl
0
L
ò
Rafał Mantiuk, Univ. of Cambridge
![Page 20: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/20.jpg)
Tone-mapping problem
luminance range [cd/m2]
conventional display
simultaneouslyhuman vision
adapted
Tone mapping
22 Rafał Mantiuk, Univ. of Cambridge
![Page 21: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/21.jpg)
Why do we need tone mapping?
To reduce excessive dynamic range
To customize the look (colour grading)
To simulate human vision
for example night vision
To adapt displayed images to a display and viewing
conditions
To make rendered images look more realistic
Different tone mapping techniques achieve different goals
Rafał Mantiuk, Univ. of Cambridge23
![Page 22: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/22.jpg)
Tone-mapping in rendering
Any physically-based
rendering requires tone-
mapping
“HDR rendering” in games is
pseudo-physically-based
rendering
Goal: to simulate a camera or
the eye
Greatly enhances realism
24
LDR illumination
No tone-mapping
HDR illumination
Tone-mapping
Half-Life 2: Lost coast
Rendering
engine
Simulate
Linear RGB
Map
Rafał Mantiuk, Univ. of Cambridge
![Page 23: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/23.jpg)
Tone-curve
Image histogram
Best tone-
mapping is the
one which does
not do anything,
i.e. slope of the
tone-mapping
curves is equal
to 1.
25 Rafał Mantiuk, Univ. of Cambridge
![Page 24: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/24.jpg)
Tone-curve
But in practice
contrast (slope)
must be limited
due to display
limitations.
26 Rafał Mantiuk, Univ. of Cambridge
![Page 25: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/25.jpg)
Tone-curve
Global tone-
mapping is a
compromise
between clipping
and contrast
compression.
27 Rafał Mantiuk, Univ. of Cambridge
![Page 26: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/26.jpg)
Sigmoidal tone-curves
Very common in
digital cameras
Mimic the response
of analog film
Analog film has been
engineered over many
years to produce
good tone-reproduction
Fast to compute
Rafał Mantiuk, Univ. of Cambridge28
![Page 27: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/27.jpg)
Tone-curve as an optimization problemGoal: Minimize the visual difference between
the input and displayed images
29 Rafał Mantiuk, Univ. of Cambridge
![Page 28: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/28.jpg)
Illumination &
reflectance separation
Input
Illumination
Reflectance
30
Y = I ×RImage
Illumination Reflectance
![Page 29: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/29.jpg)
Illumination and reflectance
Reflectance Illumination
White ≈ 90%
Black ≈ 3%
Dynamic range < 100:1
Reflectance critical for
object & shape detection
Sun ≈ 109 cd/m2
Lowest perceivable
luminance ≈ 10-6 cd/m2
Dynamic range 10,000:1 or
more
Visual system partially
discounts illumination
31 Rafał Mantiuk, Univ. of Cambridge
![Page 30: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/30.jpg)
Reflectance & Illumination TMO
Hypothesis: Distortions in reflectance are more apparent
than the distortions in illumination
Tone mapping could preserve reflectance but compress
illumination
for example:
Tone-mapped image
Reflectance
Illumination
Tone-mapping
white
c
whited LLIRL )/(
32 Rafał Mantiuk, Univ. of Cambridge
𝐿𝑑 = 𝑅 ∙ 𝑇(𝐼)
![Page 31: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/31.jpg)
How to separate the two?
(Incoming) illumination – slowly changing
except very abrupt transitions on shadow boundaries
Reflectance – low contrast and high frequency variations
33 Rafał Mantiuk, Univ. of Cambridge
![Page 32: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/32.jpg)
Bilateral filter
Better preserves sharp edges
Still some blurring on the
edges
Reflectance is not perfectly
separated from illumination
near edges
Tone mapping result
[Durand & Dorsey, SIGGRAPH 2002]34 Rafał Mantiuk, Univ. of Cambridge
![Page 33: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/33.jpg)
Glare
“Alan Wake” © Remedy Entertainment
35 Rafał Mantiuk, Univ. of Cambridge
![Page 34: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/34.jpg)
Glare Illusion
36
PaintingPhotography
Computer Graphics
HDR rendering in gamesRafał Mantiuk, Univ. of Cambridge
![Page 35: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/35.jpg)
Scattering of the light in the eye
From: Sekuler, R., and Blake, R. Perception, second ed. McGraw- Hill, New York, 1990
37 Rafał Mantiuk, Univ. of Cambridge
![Page 36: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/36.jpg)
Ciliary corona and lenticular halo
*
=
=+ From: Spencer, G. et al.
1995. Proc. of
SIGGRAPH. (1995)
38 Rafał Mantiuk, Univ. of Cambridge
![Page 37: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/37.jpg)
Examples of simulated glare
[From Ritschel et al, Eurographics 2009]
39 Rafał Mantiuk, Univ. of Cambridge
![Page 38: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/38.jpg)
Glare (or bloom) in games
Convolution with large, non-separable filters is too slow
The effect is approximated by a combination of Gaussian
filters
Each filter with different “sigma”
The effect is meant to look good, not be be accurate
model of light scattering
Some games simulate
camera rather than the eye
40 Rafał Mantiuk, Univ. of Cambridge
![Page 39: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/39.jpg)
41 Rafał Mantiuk, Univ. of Cambridge
![Page 40: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/40.jpg)
42 Rafał Mantiuk, Univ. of Cambridge
Simulation of night vision [Wanat 2014]
![Page 41: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/41.jpg)
Age-adaptive night vision
43 Rafał Mantiuk, Univ. of Cambridge
![Page 42: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/42.jpg)
ToC & Benefits
HDR in a nutshell
Display technologies in VR
Perception & image quality
Rafał Mantiuk, Univ. of Cambridge44
![Page 43: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/43.jpg)
VR display technologies
TFT-LCD AMOLED
Contrast: <3000:1
Transmissive
Complex temporal
response
Arbitrary bright
Constant power at
constant backlight
Contrast: >10,000:1
Emmisive
Rapid response
Brightness affects longevity
Power varies with image
content
Rafał Mantiuk, Univ. of Cambridge45
TN, STN, MVA,
PVA, IPS
![Page 44: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/44.jpg)
LCD
color may change with the viewing angle
contrast up to 3000:1
higher resolution results in smaller fill-factor
color LCD transmits only up to 8% (more often close to
3-5%) light when set to full white
TN LCD
Rafał Mantiuk, Univ. of Cambridge46
![Page 45: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/45.jpg)
LCD temporal response
Experiment on an IPS LCD screen
We rapidly switched between two
intensity levels at 120Hz
Measured luminance integrated
over 1s
The top plot shows the difference
between expected (𝐼𝑡−1+𝐼𝑡
2) and
measured luminance
The bottom plot: intensity
measurement for the full
brightness and half-brightness
display settings
Rafał Mantiuk, Univ. of Cambridge47
![Page 46: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/46.jpg)
OLED
based on
electrophosphorescence
large viewing angle
the power consumption
varies with the brightness of
the image
fast (< 1 microsec)
arbitrary sizes
life-span is a concern
more difficult to produce
Rafał Mantiuk, Univ. of Cambridge48
![Page 47: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/47.jpg)
Low persistence displays
Most VR displays flash an
image for a fraction of
frame duration
This reduces hold-type
blur
And also reduces the
perceived lag of the
rendering
Rafał Mantiuk, Univ. of Cambridge49
HT
C V
ive
Ma
te 9
Pro
+ D
ayD
ream
![Page 48: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/48.jpg)
Lens in VR displays
Aberrations when viewing off-center
Chromatic aberration
Loss of resolution
Difficult to eliminate if the exact eye
position is unknown
Glare
Scattering of the light in the lens
From Fresnel fringes
Reduces dynamic range
Rafał Mantiuk, Univ. of Cambridge50
Exa
mp
les fro
m: h
ttp://d
oc-o
k.o
rg/?
p=
14
14
![Page 49: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/49.jpg)
High resolution
Colour Image
High Dynamic
Range Display
HDR Display
• Modulated LED array
• Conventional LCD
• Image compensationLow resolution
LED Arrayx =
Rafał Mantiuk, Univ. of Cambridge51
![Page 50: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/50.jpg)
HDR display
Desired
image
LCD imageDLP image
DLP blur
(PSF)Subject to:
52
![Page 51: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/51.jpg)
Resolution
Relevant units: pixels per visual degree [ppd]
Nyquist frequency in cycles per degree = ½ of ppd
PC & mobile resolution
1981: 12” 320x200 monitor @50cm: 10.9 ppd
1990: 12” 1024x768 monitor @50cm: 37 ppd
2011: 3.5” 960x640 iPhone @30cm: 68 ppd
2016: 31” 4K monitor @50cm: 50 ppd
2018: 6” phone @30cm: 117 ppd
VR resolution
2016 HTC Vive: 10 ppd
2018 HTC Vive Pro: 13 ppd
Rafał Mantiuk, Univ. of Cambridge53
![Page 52: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/52.jpg)
ToC & Benefits
HDR in a nutshell
Display technologies in VR
Perception & image quality
Rafał Mantiuk, Univ. of Cambridge54
![Page 53: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/53.jpg)
Contrast Sensitivity Function
Detection of barely
noticeable contrast on a
uniform background
Varies with luminance
CSF is NOT MTF of
visual system
Contrast constancy
There is little variation in
magnitude of perceived
contrast above the
detection threshold
Rafał Mantiuk, Univ. of Cambridge55
Ge
org
eson
an
d S
ulli
va
n.
19
75
. J. P
hsysio
iPhone 4
Retina display
HTC Vive
![Page 54: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/54.jpg)
Retinal velocity
Sensitivity drops rapidly once
images start to move
The eye tracks moving objects
Smooth Pursuit Eye Motion
(SPEM)
Stabilizes images on the retina
But tracking is not perfect
Loss of sensitivity mostly caused
by imperfect SPEM
SPEM worse at high velocities
Motion sharpenning
Relatively small effect
Rafał Mantiuk, Univ. of Cambridge56
Spatio-velocity contrast sensitivity
Kelly’s model [1979]
![Page 55: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/55.jpg)
Hold-on blur
The eye smoothly follows a moving object
But the image on the display is “frozen” for 1/60th of a
second
Rafał Mantiuk, Univ. of Cambridge57
Real-w
orld
Perf
ect
motion
Physical image + eye motion + temporal integration
![Page 56: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/56.jpg)
Hold-on blur
The eye smoothly follows a moving object
But the image on the display is “frozen” for 1/60th of a
second
Rafał Mantiuk, Univ. of Cambridge58
60 H
z d
ispla
y
Physical image + eye motion + temporal integration
![Page 57: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/57.jpg)
Hold-on blur
The eye smoothly follows a moving object
But the image on the display is “frozen” for 1/60th of a
second
Rafał Mantiuk, Univ. of Cambridge59
Bla
ck f
ram
e insert
ion
Physical image + eye motion + temporal integration
![Page 58: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/58.jpg)
Flicker
Critical Flicker Frequency
Strongly depends on
luminance – big issue for
HDR VR headsets
Increases with eccentricity
and stimulus size
It is possible to detect
flicker even at 2kHz
For saccadic eye motion
Rafał Mantiuk, Univ. of Cambridge60
[Hartmann et al. 1979]
![Page 59: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/59.jpg)
Simulation sickness
Conflict between vestibular
and visual systems
When camera motion
inconsistent with head motion
Frame of reference (e.g.
cockpit) helps
Worse with larger FOV
Worse with high luminance
and flicker
Rafał Mantiuk, Univ. of Cambridge61
![Page 60: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/60.jpg)
Summary
HDR for VR is a great idea because
It gives more realistic experience
Better quality with the same number of pixels
Additional depth cues
HDR for VR is bad idea because
Increased flicker visibility
Increased simulation sickness
Lens glare will reduce effective dynamic range
In both cases
Tone-mapping will become an important part of VR rendering
Rafał Mantiuk, Univ. of Cambridge62
![Page 61: High dynamic range in VR · OLED based on electrophosphorescence large viewing angle the power consumption varies with the brightness of the image fast (< 1 microsec) arbitrary sizes](https://reader033.vdocuments.site/reader033/viewer/2022041918/5e6ac6cd38be2541e13772cc/html5/thumbnails/61.jpg)
References Concise overview of high dynamic range imaging
Mantiuk, R. K., Myszkowski, K., & Seidel, H. (2015). High Dynamic Range Imaging. In Wiley Encyclopedia of
Electrical and Electronics Engineering (pp. 1–42). Hoboken, NJ, USA: John Wiley & Sons, Inc.
https://doi.org/10.1002/047134608X.W8265
Downloadable PDF: http://www.cl.cam.ac.uk/~rkm38/pdfs/mantiuk15hdri.pdf
Comprehensive book on display technologies
Hainich, R. R., & Bimber, O. (2011). Displays: Fundamentals and Applications. CRC Press.
https://goo.gl/RLe8nA
Book on HDR Imaging
Reinhard, E., Heidrich, W., Debevec, P., Pattanaik, S., Ward, G., & Myszkowski, K. (2010). High Dynamic
Range Imaging: Acquisition, Display, and Image-Based Lighting (2nd editio). Morgan Kaufmann.
Lecture on tone-mapping (Advanced Graphics, Univ. of Cambridge)
http://www.cl.cam.ac.uk/teaching/1718/AdvGraph/06_HDR_and_tone_mapping.pdf
http://www.cl.cam.ac.uk/teaching/1718/AdvGraph/materials.html
Simulation of night vision
Wanat, R., & Mantiuk, R. K. (2014). Simulating and compensating changes in appearance between day and
night vision. ACM Transactions on Graphics (Proc. of SIGGRAPH), 33(4), 147.
https://doi.org/10.1145/2601097.2601150
Rafał Mantiuk, Univ. of Cambridge63