signal processing challenges for digital holographic video ......illustration of the...

Vrije Universiteit Brussel

Signal processing challenges for digital holographic video display systemsBlinder, David; Ahar, Ayyoub; Bettens, Stijn; Birnbaum, Tobias; Symeonidou, Athanasia;Ottevaere, Prof. Dr. Ir. Heidi; Schretter, Colas; Schelkens, PeterPublished in:Signal Processing: Image Communication

DOI:10.1016/j.image.2018.09.014

Publication date:2019

License:CC BY

Document Version:Final published version

Link to publication

Citation for published version (APA):Blinder, D., Ahar, A., Bettens, S., Birnbaum, T., Symeonidou, A., Ottevaere, P. D. I. H., ... Schelkens, P. (2019).Signal processing challenges for digital holographic video display systems. Signal Processing: ImageCommunication, 70, 114-130. https://doi.org/10.1016/j.image.2018.09.014

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portalTake down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.

Download date: 10. Sep. 2021

https://doi.org/10.1016/j.image.2018.09.014

https://researchportal.vub.be/en/publications/66950f3c-943d-4f23-957b-24a7fde4cb7a


Signal Processing: Image Communication 70 (2019) 114–130

Contents lists available at ScienceDirect

Signal Processing: Image Communication

journal homepage: www.elsevier.com/locate/image

Signal processing challenges for digital holographic video display systemsDavid Blinder a,b,∗, Ayyoub Ahar a,b, Stijn Bettens a,b, Tobias Birnbaum a,b,Athanasia Symeonidou a,b, Heidi Ottevaere c, Colas Schretter a,b, Peter Schelkens a,b

a Dept. of Electronics and Informatics (ETRO), Vrije Universiteit Brussel (VUB), Pleinlaan 2, B-1050 Brussels, Belgiumb imec, Kapeldreef 75, B-3001 Leuven, Belgiumc Dept. of Applied Physics and Photonics, Brussels Photonics Team (B-PHOT),Vrije Universiteit Brussel (VUB), Pleinlaan 2, B-1050 Brussels, Belgium

A R T I C L E I N F O

MSC:00-0199-00

Keywords:Digital holographyComputer generated holographyHolographic visualizationHolographic imagingData compressionQuality assessment

A B S T R A C T

Holography is considered to be the ultimate display technology since it can account for all human visual cues suchas stereopsis and eye focusing. Aside from hardware constraints for building holographic displays, there are stillmany research challenges regarding holographic signal processing that need to be tackled. In this overview, wedelineate the steps needed to realize an end-to-end chain from digital content acquisition to display, involving theefficient generation, representation, coding and quality assessment of digital holograms. We discuss the currentstate-of-the-art and what hurdles remain to be taken to pave the way towards realistic visualization of dynamicholographic content.

1. Introduction

Holography is a technique that can record and reconstruct thefull wavefield of light. Its invention in 1948 is attributed to DenisGabor, who initially developed the technique for improving the qualityof electron microscopes [1]. At the time, no adequate coherent lightsource was available, forcing him to arrange everything along oneaxis, known today as in-line holography. Because image quality waspoor, holography remained obscure. With the invention of the laser andLeith and Upatnieks’ off-axis holography [2] in the 1960s, high-qualityholograms became possible.

However, holography was still purely analogue, since everythinghad to be recorded on photographic film. This did change in the late1980s: with the emergence of Spatial Light Modulator (SLM) technolo-gies, Charged-Coupled Device (CCD) image sensors and increasinglypowerful computers, it became finally possible to digitally capture,process and display holograms. Digital holography is used today forvarious purposes, such as microscopy [3], interferometry [4], surfacemeasurements [5,6], storage [7] and three-dimensional (3D) displaysystems [8]. In this paper, we particularly focus on the realization ofthe latter.

3D displays enable depth perception, thereby showing a scene in its3 dimensions. Contrary to conventional two-dimensional (2D) displaysor stereoscopic displays, which are only providing a monoscopic ora stereoscopic view, 3D displays provide different visual informationdepending on the viewer’s eye position and gaze. This technology is

∗ Corresponding author at: Dept. of Electronics and Informatics (ETRO), Vrije Universiteit Brussel (VUB), Pleinlaan 2, B-1050 Brussels, Belgium.E-mail address: [email protected] (D. Blinder).

an essential component in many visualization applications in entertain-ment, medicine and industry.

Current solutions for 3D displays like those based on optical lightfields [9,10] can only provide a subset of the required visual cuesdue to inherent limitations, such as a limited amount of discreteviews. By contrast, digital holography [11] is considered to be theultimate display technology since it can account for all visual cues,including stereopsis, occlusion, non-Lambertian shading and continuousparallax. Representing the full light wavefront avoids eye vergence–accommodation conflicts [12].

However, current holographic displays do not yet offer the resolu-tions and angular Field-of-View (FoV) required for acceptable visualquality — but this may change soon. Although significant technologicalhurdles still need to be overcome, steady developments in photonics,microelectronics and computer engineering have led to the prospectto realize full parallax digital holography displays with acceptablerendering quality [13,14]. The extreme required image resolutions,paired with the video frame rates needed for dynamic video holographycould lead up to terabytes-per-second data rates in its plain form [15].

Much of prior research efforts on holographic display systems fo-cused on the hardware challenges, namely designing and developingthe relevant optical and electronic technologies. Examples include high-bandwidth Spatial Light Modulators (SLM) and specialized optics [16–18]. Thorough overviews of the latest developments of 3D displays canbe found in recent papers [8,14,19–21].

https://doi.org/10.1016/j.image.2018.09.014Received 14 May 2018; Received in revised form 24 September 2018; Accepted 27 September 2018Available online 4 October 20180923-5965/© 2018 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).


http://www.elsevier.com/locate/image

http://www.elsevier.com/locate/image

http://crossmark.crossref.org/dialog/?doi=10.1016/j.image.2018.09.014&domain=pdf

mailto:[email protected]


http://creativecommons.org/licenses/by/4.0/

D. Blinder et al. Signal Processing: Image Communication 70 (2019) 114–130

Fig. 1. Pipeline summarizing the challenges in signal processing that should be solved to enable high-quality holographic display systems.

An aspect which has received much less attention by comparisonto hardware designs is how the potentially huge data volumes shouldbe handled for efficient generation, storage and transmission. Thesignal statistics of a hologram differ considerably from regular naturalimage and video. Hence, conventional representation and encodingalgorithms, such as the standard JPEG and MPEG families of codecs,are suboptimal. When generating synthetic holographic content by de-ploying Computer Generated Holography (CGH) techniques, algorithmsare required that differ considerably from those needed for classicalsynthetic image rendering that are based on ray-space physics and aparticle model for light transport. Indeed, holography adheres to a wave-based transport model of coherent light instead.

This overview paper focuses on the signal processing challengesneeded for the realization of holographic video systems from sourcesignal generation to display. We identify challenges in hardware andsoftware and collect a set of important research targets to addressthem. The steps that need to be undertaken are illustrated in Fig. 1;a global comprehensive approach is required for designing a consistentholographic video display system.

The body of this document is organized as follows:

• Digital holographic display systems are currently very premature andheterogeneous in design, so there exists no established standardthat specifies how to supply holographic data to the display.Hence, it is important to have a signal processing framework thatis sufficiently generic as to allow for compatibility across displaysystems (see Section 2).

• Recording digital holograms at high resolutions is difficult. Not onlydo these acquisitions need specialized optical setups, but theyrequire expertise to build and operate. Furthermore, the size ofobjects that can be recorded is limited (see Section 3).

• Computer-generated holography (CGH) is much more calculation-intensive than classic image rendering: every point in the scenecan potentially affect every hologram pixel. This many-to-manyrelationship compounded with the large hologram resolutions istoo costly to compute by brute-force (see Section 4).

• Novel transforms and coding technologies are needed for digitalholograms. Conventional transforms and representations performsuboptimally because they do not match the statistical propertiesof holographic signals. Standard codecs such as JPEG and MPEGare solely designed for natural photography and video but operatepoorly on holographic content; novel transforms and motionestimation & compensation techniques need to be integrated (seeSection 5).

• Modelling perceptual visual quality for holographic content requiresaccurate models for the subjective quality assessment. Thesemetrics will be needed to steer the other components of the holo-graphic pipeline (CGH, coder/decoders, displays) as to optimizethe global quality of experience (see Section 6).

It is important to understand that all challenges outlined aboveare further complicated by the large computational load and mem-ory/signalling bandwidth associated with the realization offull-parallax, wide viewing angle dynamic digital holographic displaysystems [22]. Before digging deeper into the physics of holography(Section 3), we first address the high-level properties of holographicdisplays to better position the challenges that are faced by the modulesthat are processing the holographic signal prior to display.

2. Holographic displays

At present, most commercially available 3D displays are stereo-scopic: two different 2D images are fed to the viewer, one per eye (seeFig. 2(a)). This is either achieved with auto-stereoscopic displays or withspecialized eye wear, e.g. with passive glasses (anaglyph, polarized),active glasses (shutters), or Head-Mounted Displays (HMD). However,the 3D effect is still limited, because several cues of the human visualsystem are not accounted for. Such displays lead to eye strain andnausea after prolonged wear. A mismatch between the vergence andaccommodation of the eyes during viewing leads to the ‘‘vergence–accommodation conflict’’, which results in discomfort, see Fig. 3.

The important cues needed for 3D perception [19,23] can be dividedinto two categories: monocular and binocular. Monocular cues are:

• Accommodation — controlled by muscle’s strain for adjusting theeye’s focal length;

• Motion parallax — the observed relative velocity of objects withrespect to the eye’s position;

• Occlusion — indicate the relative distance of objects to the viewerdue to the observed partial overlap;

• Shading — providing visual information on the shape and 3Dposition of objects.

Binocular cues are:

• Vergence — the angular difference between the gaze direction ofthe eyes as a function of viewing depth;

• Stereopsis — emerging from correspondences between the pro-jected image on the retina of both eyes.

115


Fig. 2. Different 3D display types: (a) stereoscopic display, (b) multi-view or light-fielddisplay, (c) holographic display.

Fig. 3. Illustration of the vergence–accommodation conflict. In (a), e.g. in holographicdisplays, the accommodation and vergence always match up. In (b), e.g. in stereoscopicdisplays, accommodation is constrained at a fixed focal distance yielding a mismatch.

Many types of 3D display technologies exist that attempt to betteraddress the important cues with varying degrees of efficacy. 3D displayscome in many forms, such as volumetric displays [24] and multi-focaldisplays [25,26]. One of the most popular are multi-view (or light-field) displays. They generalize the stereoscopy approach by increasingthe amount of 2D views from two to many more (Fig. 2(b)). Thesedisplays [27,28] discretize the light field into many views emitted inmultiple directions.

Because of the current physical limitations in terms of screen reso-lution, and consequently on how many views can be displayed at once,this will degrade the viewing experience. The main noticeable deficitsare the step-wise changes in the motion parallax, and accommodation–vergence mismatches. Furthermore, typically only horizontal parallax issupported.

Multi-view displays come in many forms, including 360◦ table-tops [28] and HMDs [29–31]. It has to be noted though that in principle,future light field displays which exhibit sufficiently high spatial andangular resolutions enable the super multi-view principle [32,33]: thishappens when the angular sampling pitch is small enough so thatmultiple rays emanating from the same screen point enter the viewer’spupil, hence alleviating the above deficits. Nonetheless, the ray bundlesize dictates a trade-off between the maximal spatial and angularresolution, which is not present in holography [34].

However, holographic displays have the promise to account for all vi-sual cues. In principle they can display virtual objects indistinguishablyfrom real ones. But, realizing high-quality holographic display systemscomes with many software and hardware challenges.

Fig. 4. Illustration relating the dimensions, pixel pitch, viewing angle and resolution ofholographic displays. This diagram assumes a screen ratio of 16:9 and a wavelength of𝜆 = 460 nm (blue light). The graphs indicate how the parameters are related for displaydiagonals ranging from 1 to 40 inch. The green regions approximately indicate the desiredproperties of holographic displays with screen sizes for HMD, mobile phones, tablets andtable top.

2.1. Types of holographic displays

We will focus on dynamic digital pixel-based holographic displays,also known as ‘‘electro-holographic displays’’. Contrary to e.g. printedor analogue film holograms, they can be electronically updated at videorates. Further guidance on holographic displays and alternatives canbe found in the introductory paper from Dodgson [35], the recentsummary [34], or the elaborate review [20].

One of the first digital holographic displays was developed byMIT Media Lab [36]. It utilizes bulk acousto-optical modulators tocreate successive horizontal-parallax-only line holograms for bandwidthreduction. From then on, electro-holographic displays have come inmany shapes and sizes, varying widely in resolution, FoV and imagesize. Today’s SLM market is dominated Liquid-Crystal On Chip (LCOS)and Micro-ElectroMechanical Systems (MEMS) elements [17].

Although a true holographic display should modulate both theamplitude and phase of a wavefront, full complex modulation in asingle device is still hard to achieve, but some examples exist [37]. MostSLMs can either modulate amplitude or phase [17]. The latter is oftenpreferred, since phase-only SLMs have the advantage of high diffractionefficiency, which theoretically can reach 100%. The phase componentof a hologram is also generally more important for image quality thanthe amplitude.

Current commercially available SLMs typically have resolutions ofup to 1920 × 1080 (Full HD) or 3840 × 2160 (4K UHD), which do notyet reach the resolutions needed for large, full-parallax realistic displays:depending on the screen dimensions and viewing angle requirements,the needed resolution can surpass 1012 pixels/frame. Moreover, pixelpitches of commercial solutions are currently limited to approx. 4 μm,cf. Fig. 4; the pixel pitch being the distance between the centres of twoadjacent pixels. More details are given in Section 2.3. Hence, engineersneed to be resourceful with the available bandwidth. This has led toseveral types of holographic displays, which we can categorized in threegroups (see Figs. 5 and 6). The three following sections describe each ofthem.

116


Fig. 5. Different types of holographic displays: (a) head-mounted holographic display,(b) external single-user holographic desktop display (with gaze tracking), (c) multi-userholographic display.

Fig. 6. Classification of various holographic displays based on their pixel count and screensize.Source: Bilkent [38], ETRI [39], Microsoft [34], NICT [40], NVIDIA [41], Seereal [42],UTokyo [43], WUT [44].

2.1.1. Head-mounted displays (HMD)HMDs have applications in personalized virtual and augmented

reality. Holographic head-up displays have already been proposed earlyon for aircrafts [45], cars [46] or to aid medical diagnostics andsurgery [47]. More recent usage scenarios [48] involve human–machineinterfaces for visual content creation, computer games, CAD/CAE de-sign, training simulators, education and 3D-TV. Virtual environmentscan be rendered with high fidelity, and virtual objects can be seam-lessly integrated with the real environment, providing an immersiveexperience. Usually these display types use two monoscopic holographicdisplays [34,41,49,50]. Because only a small subset of the informationneeds to be transported per eye to the user’s eyebox, the requiredbandwidth is drastically reduced. Head-mounted holographic displayswill thus most likely be the first kind of holographic displays that willbe available to consumers in the near future.

2.1.2. Single-user desktop displays (SUDD)These displays are similar to the previously mentioned class. These

devices are either too small in display size or field-of-view to be effec-tively used simultaneously by multiple users [38,44,51]. Some displayseven steer the content solely to the eyes of a viewer by tracking his/her

Fig. 7. Diagram of the valid viewing zone of a holographic display by illuminating theSLM with either (a) planar wave or (b) spherical wave illumination. The opening angle ofthe blue cones is given by the grating equation 𝜃𝑚𝑎𝑥, and their intersection forms the validviewing zone. Note that for the same viewing distance, a larger 𝜃𝑚𝑎𝑥 is needed for planarwave illumination (and hence a smaller pixel pitch 𝑝).

gaze in real-time [42,52]. Due to screen size limitations and/or trackingarea those displays are meant to be operated by a single user with arather limited motion range, e.g. when seated or standing in a specificarea. The range of applications extends most usages of head-mounteddisplays by scenarios where the display does not have to be wearable,so larger optical setups (and bandwidth) can be allowed for; examplesinclude high-end 3D-TV or simulations needing advanced calculation-intensive CGH algorithms making the HMDs portability requirement ahindrance. This class is projected to mature shortly after head-mounteddisplays are commercially available, because many of the challengesoverlap.

2.1.3. Multi-user displays (MUD)These displays can present full-parallax holograms at extreme res-

olutions to multiple viewers at once. They are highly useful whenmultiple users need to simultaneously observe and interact with 3Dmodels, especially over long periods of time, enabling applications indesign, manufacturing, medicine, sports, etc. [53]. Currently horizontal-parallax-only tabletop displays such as [43,54,55] can give a firstimpression of this ultimate class of holographic displays. However,present implementations are still limited in resolution and can oftenonly display pre-rendered holograms of small objects, a few centimetresacross. They are mostly based on many multiplexed SLMs [40,56,57],or ultra-fast SLMs and rotating optical elements [39].

2.2. Limits of holographic displays

Although steady progress is made in SLM technology, satisfactoryholographic displays are still not realizable. The large amount of infor-mation contained in a hologram brings current hardware to its limits:Giga/Tera-pixel resolutions, pixel sizes approaching the wavelength oflight (i.e. < 1 μm), high interconnection bandwidths and sufficientswitching speeds in the case of time multiplexed methods (requiringup to 10 kHz refresh rates) are not yet supported.

The relationship between the pixel pitch 𝑝 and the supported angularFoV is given by the grating equation: for every spatial frequency 𝜈, theassociated diffraction angle 𝜃 is given by:

𝜆𝜈 = sin(𝜃) (1)

117


Fig. 8. Diffraction of an incoming planar wave going through slits, resulting in point-spread functions. Regions where the wavefield 𝑈 has phase 𝜑 = 2𝑛𝜋, 𝑛 ∈ Z are drawnwith red lines. (a) single point-spread function. (b) Young’s double slit experiment, withalternating constructive and destructive interference.

The maximal spatial frequency determined by the hologram’s pixel pitch𝑝 will thus correspond to a maximum diffraction angle 𝜆

2𝑝 = sin(𝜃𝑚𝑎𝑥).In this context, we also define the space-bandwidth product (SBP),

which actually defines in Wigner domain the product of the spatial andspectral footprint of the holographic signal [58,59]:

𝑆𝐵𝑃 =𝑊𝑥 ×𝑊𝑦

𝑝𝑥 × 𝑝𝑦(2)

where 𝑊𝑥 and 𝑊𝑦 are representing respectively the spatial width andheight of the hologram and 𝑝𝑥 and 𝑝𝑦, respectively the pixel height andwidth, i.e. pixel pitch in both dimensions. A large spatial size (e.g. a 44-inch display) and a large pixel pitch of e.g. 127 μm basically results in a2D 4K colour television screen, while a small pixel pitch of e.g. 230 nmwould produce a 10 Terapixel holographic display with a maximumangular FoV of 180◦ (see Fig. 4).

The maximum diffraction angle bound will impose limits on the validviewing zone geometry. This zone is delineated by all positions whichare reachable for rays emitted from every hologram point: namely theintersection of all cones placed at every hologram point with an angle𝜃𝑚𝑎𝑥. Their orientations (and by consequence, the resulting shape ofthe valid viewing zone) will depend on the shape of the illuminatingreference wave. This is shown in Fig. 7 for two typical examples: planarwave and spherical wave illumination. The former will have an infinitelyextended viewing zone, but at a larger minimal viewing distance. Thelatter allows for a wider viewing angle closer to the holographic screen,but generally limits the valid viewing zone to a small, fixed region. Thatis why spherical wave illumination is better suited for HMDs, whileplanar illumination might be preferred for full-parallax holographicdisplays.

2.3. Challenges

The following challenges in the design of holographic display impactsignal processing requirements:

Ultra-high resolution. A very small pixel pitch is required for the displayof macroscopic objects at a reasonable level of detail. Sub-wavelengthpixel sizes enable the display of higher frequencies, i.e. finer resolvedfringe patterns, which allow for large viewing angles. At the same timesmaller pixel sizes further intensify the resolution problem by increasingthe number of pixels required for a hologram of the same physical size(see Fig. 4). This search for high-resolution and wide-angle hologramsresults in large space-bandwidth products.

Light modulation efficiency. Digital SLMs have a limited number ofdegrees of freedom per pixel, expressed by the number of bits usedper pixel to specify amplitude or phase, i.e. bit-depth, and to whatextent phase and amplitude can be modulated, if at all. Furthermore, the

Fig. 9. Simplified diagram of a holographic on-axis recording setup in transmission mode.A laser beam is expanded as to fully illuminate the sample, then split into a reference beamand an object beam. Both beams are recombined after the object beam passes through theobject and form an interference pattern on the image sensor.

diffraction efficiency of a device will affect the display quality. Otherissues pop up too when the pixel density increases, such as cross-talkand noise. There is some trade-off between modulation efficiency andpixel count and density. All those aspects will need to be taken intoconsideration when optimizing these variables for designing the bestpossible display. Accurate simulations of these variables could help tomodel their impact on visual quality. Moreover, this may require theapplication and design of (new) algorithms to post-process the hologramas to obtain better visual quality, such as dithering, or the Gerchberg–Saxton algorithm [60] to generate phase-only holograms.

Multi-colour display. The laws of diffraction are wavelength-dependentand colours are not interchangeable. Blue light has the shortest wave-length, meaning it will have a lower diffraction angle for the samemaximal SLM spatial frequency than the other colours, which can lead toproblems at large viewing angles. Another consequence is that higherwavelengths can carry more information per unit area. An additionalproblem is that lasers have (by design) a very narrow bandwidth in theelectromagnetic spectrum. This can lead to perceptual colour distortionand to highly visible chromatic aberrations, because the individualcolour components will be much better distinguishable than in theincoherent white light case.

Speckle-noise reduction. Highly coherent light sources give rise tospeckle noise, which can be mitigated though by various approaches.Partially coherent light sources can reduce speckle, but will cause someblur and some loss of depth perception due to their reduced coherencelength. More information on the relation between the coherence andimage quality can be found in [61]. Other solutions involve diffusers,temporal modulation of the laser beams, frequency shifting over time,superposition of multiple reconstructions, and quickly repositioning thelaser beam.

Embedded signal processing. Due to vast number of pixels that needsto be processed, it is to be expected that internal signalling channelswith intrinsic bandwidth constraints will be under high stress. Hence,processing – e.g. decoding and rendering – will need to be parallelizedand hierarchically distributed such that data can be locally processed(cf. JPEG Pleno requirements [62]), as it is also the case for high-endlight field displays [33].

3. Digital hologram recording

The holograms to be rendered on the holographic display can beobtained either by recording the hologram on an optical capturingsetup, either by numerically calculating the hologram using computer-generated holography algorithms. In this section, we will focus on thefirst approach, Section 4 addresses the latter. To understand the require-ments and challenges, we first summarize the underlying fundamentalphysics. Thereafter, the acquisition of high-resolution holograms andthe connected inverse reconstructions methods are discussed.

118


Fig. 10. These diagrams illustrate the solution spaces, per pixel, that are consistentwith the measurements. (a) For a known value of 𝑅 and a measured intensity 𝐼 , thevalid solutions for 𝑂 lie on a circle centred at 𝑅 with radius

√

𝐼 . (b) When taking twomeasurements, there still is some ambiguity (two valid solutions), so (c) at least threedifferent 𝑅𝑖 are needed to find a unique solution for 𝑂.

3.1. Physics of holography

Holography relies on the wave model of light governed by the laws ofdiffraction. This representation is dual to the ray (or particle) model oflight. The wave nature of light is captured by Young’s famous double-slitexperiment, see Fig. 8. Although electromagnetic waves are generallydescribed by Maxwell’s equations, in a sufficiently large, homogeneous,dielectric, isotropic medium, the equations can be simplified to a scalarmodel.

Holography is thus typically described by scalar diffraction theory:the wavefield of a monochromatic coherent light source is modelled bya scalar field of complex-valued amplitudes, describing any componentof the electromagnetic field for every point in space. The magnitudedenotes the amplitude, and the angle denotes the (relative) phase delayw.r.t. to some reference wavefield. This contrasts with conventionalimaging techniques, which only measure intensity (squared amplitude),but lack phase data to encode depth.

Because holograms can capture all visible features of light waves,they can provide all needed 3D cues. An illuminated object in space willinduce interference patterns that can be captured by the hologram toencode the light field emanating from the object.

Mathematically, diffraction can be expressed as a (infinite) collectionof point emissions. A single point spread function is given by:

𝑂(𝐩) = 𝑈 (𝐱) 𝑒𝑖𝑘‖𝐩−𝐱‖

‖𝐩 − 𝐱‖(3)

evaluating the scalar field 𝑈 at a point 𝐩 in the wavefield 𝑂 on thehologram plane, for a spherical wave centred at a luminous point 𝐱; 𝜆is the wavelength of the light, and 𝑘 = 2𝜋

𝜆 is the wave number. For anarbitrary surface, we can integrate over all points using the Huygens–Fresnel principle [59]:

𝑂(𝐩) = 1𝑖𝜆 ∬𝑆

𝑈 (𝐱) 𝑒𝑖𝑘‖𝐩−𝐱‖

‖𝐩 − 𝐱‖2𝐧 ⋅ (𝐩 − 𝐱)𝑑𝐱 (4)

where 𝐱 ∈ 𝑆 are points on a surface 𝑆 over which is integrated and 𝐧 isthe surface normal on 𝑆 in 𝐱.

Since no physical detector can measure the phase directly, as phasecarries no energy, this information needs to be captured indirectly bymeans of interference. A typical holographic recording setup (Fig. 9)consists of a laser, whose beam is expanded as to be able to fullyilluminate the object to be recorded. Next, the beam is split into a knownreference beam 𝑅 and the sought object beam 𝑂, which will illuminatethe object and encode its information. These beams are superimposedand we measure the intensity of the two interfering beams on thedetector. The measured 2D intensity fringe pattern 𝐼 is given as:

𝐼 = |𝑅 + 𝑂|

2 = |𝑅|2 + |𝑂|

2 + 𝑅𝑂∗ + 𝑅∗𝑂 (5)

where ∗ denotes the complex conjugate. The last term 𝑅∗𝑂 containsthe sought 𝑂. Since 𝑅 is known, we can in principle retrieve the

Fig. 11. (a) The intensity of an off-axis hologram of a speckle phantom, (b) its Fourierrepresentation with annotated terms and (c) reconstructed hologram amplitude.Source: [63].

complex-valued 𝑂. The interferogram further consists of several otherundesired terms. For example, since the sought 𝑂 is complex-valued,the squared magnitude term |𝑂|

2 will not contain any of the importantphase information.

Because of the multiple valid solutions for 𝑂 (see Fig. 10), additionalsteps need to be taken to unambiguously retrieve 𝑂. The types of setupscan be classified into two categories: single- and multiple-exposuremethods. One of the most popular single-exposure techniques is called‘‘off-axis holography" [2] (Fig. 11). By using a tilted plane wave for 𝑅,the terms in Eq. (5) will get separated in the frequency domain; thedesired term can the subsequently be extracted using a bandpass filter.The disadvantage of this technique is that the bandwidth of 𝑂 will beonly a fraction of the camera bandwidth, because most of it will bediscarded by the filter.

Multi-exposure methods on the other hand, consist of taking severalmeasurements in succession. The best known example is phase-shiftingdigital holography [64], where generally at least 3 exposures are neededto unambiguously retrieve the complex amplitudes (cf. Fig. 10(c)). Moreexposures can be taken to improve noise robustness. The main draw-back is that such methods cannot easily capture dynamic phenomena,reducing their application domain. Combinations such as parallel phase-shifting holography [64] exist as well.

3.2. Acquiring high-resolution holograms

One of the current bottlenecks for acquiring high-quality hologramsis the limited image sensor resolutions: typical digital sensors consistof multiple megapixels, which is several orders of magnitude below theresolutions needed for large, high angular FoV displays.

A straightforward optical solution is to use multiplexing: by using anarray of sensors and/or moving the camera between successive acquisi-tions, a large synthetic aperture can be created by stitching numericallymany sub-holograms together [65,66]. This has applications beyonddisplay, such as in holographic tomography and microscopy [67,68].However, the setup will become very complex and acquisitions time-consuming, thereby precluding the capture of dynamic scenes. Otheroptical solutions involve modifications to the capturing system as toextract more information from the recorded intensity pattern, such asutilizing a coded aperture [69,70].

On the other hand, by accounting for the statistical features ofholographic signals, it is possible to enhance the resolution of acquiredholograms without modifying the recording setup. This can be done withinverse methods, whose goal is to extract a maximum of informationfrom relatively small sets of intensity measurements, thereby improvingthe resolution without increasing acquisition setup complexity. Thenumber of intensity measurements is generally considered insufficientaccording to the Shannon–Nyquist sampling theorem, resulting in manyholographic signals that can explain the measurements.

In this context, three questions arise. First, we have to identify whatconstitutes a good set of measurements. Secondly, we have to select oneof the holographic signals that best explains the intensity measurements(e.g. the maximum likelihood solution). To make a substantiated choice,prior information about the holographic signal may be used by inverse-problem reconstruction approaches .

119


Fig. 12. Illustration of the virtual setup geometry for CGH with planar illumination. Sceneobjects should be positioned inside the aliasing-free zone. Objects outside of this zone willemit rays with incident angles on the hologram surpassing 𝜃𝑚𝑎𝑥, causing aliasing.

3.3. Inverse reconstruction methods

One example of a class of inverse methods is compressed sensing(CS), which is a generic theory in signal processing that deals withsparse signal recovery from an underdetermined set of linear measure-ments [71]. Free space diffraction provides a natural signal mixingoperator, which makes CS an excellent fit for holography.

CS has been successfully applied for holographic tomography [72],3D scene segmentation [73] and measuring 3D refractive index dis-tributions of cells. In recent work [74,75], the groundwork was laidon how to determine optimal measurement conditions for holographicrecording setups and what CDF wavelet bases are best suited for imagereconstruction in compressed sensing.

In the case of off-axis geometries, a separation is created in theFourier domain. Direct image reconstruction approaches will manipu-late the frequency domain and apply a spatial filter for recovering onlythe complex amplitude information of the object beam, discarding thecomplex conjugate twin image as well as the zero-order illumination.These frequency-domain manipulations techniques are straightforwardif the object wavefield is bandlimited [76] and may yield exact recon-struction. However, in practice, the sample contains fine structures anddetails, independent of the lateral resolution of the detector. Recently,regularized inverse reconstruction approaches have been developedfor holographic signals and impressive reconstruction quality may beobtained [77–79], at the price of an increase in computation.

3.4. Challenges

Optically recording digital holograms still comes with several chal-lenges:

Resolution limitations. The maximum obtainable resolution is limited bythe issued detector. It can be extended by deploying inverse methods ifa good signal model can be defined, exploiting its sparsity properties.However, results will be still limited by the signal statistics.

Physical limitations. Aside from resolution these setups have physicallimitations on the types of objects and scene size that can be captured.Tomographic solutions can be utilized for static scenes, but thesetechniques are hardly applicable to dynamic scenes. Hence, recordingholograms requires specialized equipment and optically captured holo-grams for holographic display might only be suitable for very particularuse cases, e.g. (bio)medical imaging and non-destructive testing.

Optical distortions. Recorded holograms are typically contaminatedwith speckle noise and by optical aberrations. Hence, additional de-noising and aberration compensation techniques need to be deployed.

Fortunately, there is another way of acquiring holographic data,namely by numerically synthesizing holograms from a 3D scene modelusing computer-generated holography. Its benefits and challenges willbe discussed in the next section.

4. Computer-generated holography (CGH)

Computer generated holography (CGH) is an alternative to opticalhologram acquisition. It has the important advantage that the objectinformation can be obtained by means of conventional (multi-)camerasetups, point cloud data, or even computer graphics. One of the mainchallenges of CGH is to generate realistic holograms within acceptablecomputation times. This can be inferred from the Huygens–Fresnel prin-ciple in Eq. (4): computing the hologram using brute-force is inefficient,since the integral has to be evaluated over all scene points, for everypixel.

4.1. Types of CGH methods

The computational load of the many-to-many mapping of the diffrac-tion model is compounded with another difficulty: realistic hologramsneed very high resolutions. This fact can be derived from Eq. (1):suppose we have a green laser (𝜆 = 532 nm), and a desired viewingangle of 𝜃𝑚𝑎𝑥 = 40◦. This amounts to a pixel pitch 𝑝 of at least 0.414 μm.For a display of 10×10 cm, we then need a massive ultra-high definitiondisplay of 58 Gigapixels.

Aside from the valid viewing zone, this also imposes restrictions onthe scene geometry and the CGH calculations: incident light at too largeangles will otherwise result in aliasing, see Fig. 12. Furthermore, Eq. (4)does only account for free space propagation, but not for effects such asocclusion, shadows, refraction, etc. That is why over the years, manydifferent techniques were developed to accelerate and/or improve therealism of CGH. They can be classified as follows.

4.1.1. Point cloud methodsPoint clouds (Fig. 13(a)) may represent objects as a collection of

discrete luminous points [80–84]. This will discretize Eq. (4) into asummation over all points. These methods can be augmented usinglook-up tables with precomputed point-spread functions, or even surfaceelements with a precomputed light distribution. Occlusion can becumbersome to handle, but this can be achieved e.g. by attributing asmall volume to each point blocking incident light.

4.1.2. Polygon methodsSurface elements such as triangles (Fig. 13(b)) leverage the fact

that diffraction between planes can be efficiently computed using aconvolution [85–87], e.g. the Angular Spectrum Method (ASM). Thesemethods are particularly efficient when the amount of hologram pixelsgreatly exceeds the polygon count. Generally, the polygons will have anassociated complex-valued wavefield, where the amplitude will encodethe texture and the phase will encode the light emission distribution(e.g. diffuse/specular). Occlusion is more straightforward, but can becomputationally demanding.

4.1.3. RGB+Depth methodsWith a depth map (Fig. 13(c)), we can encode holograms with

pairs of colour images and per-pixel depth information [88]. Theyare computationally efficient and can be of high quality, but theyare restricted in the supported viewing angles. For example, objectscannot be placed behind others with a depth map. These methods areespecially suitable when the viewer should gaze at the display froma fixed position. This shortcoming may be addressed though by usingmore depth maps for several viewpoints.

120


Fig. 13. Comparison of various types of CGH algorithms. The object (blue) emits wavefronts (red) that propagate to the virtual hologram (black) at the bottom. There is a dependencebetween scene representations and their associated numerical light propagation methods.

4.1.4. Ray-based methodsRay-space representations (Fig. 13(d)) will approximate the holo-

gram by a discretized light field, which is converted into a holo-gram by assigning an incident-angle-dependent impulse response toevery ray [89–94]. One notable subset are the phase-added stereogrammethods: the hologram is subdivided into blocks, each individuallyrepresented in its FFT domain (similar to the blockwise DCT). Everyfrequency component within a single block will approximately corre-spond to a ray of light. That way, a set of rays can be sampled for everypoint/coefficient, followed by a block-wise inverse Fourier transform.This mutual conversion between the hologram and discrete light fieldrepresentation is also known as ‘‘ray-wavefront conversion’’ [95]. Theadvantage is that those methods are comparatively fast and can useexisting ray-tracing software, discretizing the light field will reduce thequality of the generated hologram, degrading some of the 3D cues suchas continuous parallax.

4.2. Sparse acceleration techniques

CGH algorithms can be accelerated further using sparsifying trans-forms. Sparsity is a well-known concept in signal processing literature.It refers to the property that signals drawn from a particular source canbe well-approximated by a few coefficients in a certain, sparse basis.This property is very useful for many applications, such as compression,denoising, and compressed sensing. Particularly, the notion of sparsity ishighly useful for CGH. The following subsections describe three commonaccelerations strategies: (1) using an intermediate wavefront recording(WRP) plane, (2) using a stack of WRPs and (3) computing PSFs (PointSpread Functions) in the STFT domain.

4.2.1. Single wavefront recording plane accelerationAlthough the wavefield in the hologram plane is very dense, it can

become highly sparse when expressed in the right transform domain.This can be leveraged for computational efficiency, since then only asmall subset of the coefficients needs to be updated. The most wellknown example is called Wavefront Recording Planes (WRPs) [82]. Itutilizes the property that points close to the hologram plane have asmall spatial footprint. By placing a WRP close to the object in 3Dspace, only a small subset of the WRP pixels needs to be updated(Fig. 14), resulting in high sparsity. After all PSF additions, the WRP canbe efficiently propagated to the hologram plane using e.g. the angularspectrum method.

4.2.2. Multiple wavefront recording planes accelerationThe WRP methods can be extended further by using multiple parallel

equidistant WRPs, so as to optimize calculation times. An efficient CGHmethod which exploits multiple WRPs has been proposed in [83]. Asshown in Fig. 15 the object is segmented in WRP zones and the pointsare allocated to their closest WRP. The WRPs are computed sequentially

Fig. 14. Diagram illustrating how distance to a plane will affect the point spread functionsupport size. The maximum diffraction angle 𝜃𝑚𝑎𝑥 will determine the opening angle of thecone of influence, 2 ⋅ 𝜃𝑚𝑎𝑥, shown on the left. The classic WRP method will reduce thesupport of point spread function w.r.t. the hologram plane, but placing the WRP in themiddle of the point cloud is even more sparse.

starting from the one furthest to the hologram plane. Furthermore, thismethod is combined with an occlusion processing technique locally ap-plied per point. The wavefield is progressively processed from the backto the front WRP and finally propagated to the hologram plane [96,97].

Another important acceleration technique relies on look-up tables(LUT). That way, scene elements can be precomputed and stored in anytransform domain. For example, PSFs quantized at discrete levels aroundthe WRP can be reused for all the WRP zones. But LUTs are not limitedto storing PSFs. This approach has been extended for diffuse surfacesby modulating the PSFs with a random phase factor [98], resulting intoa simulated diffuse scattering of the light. More recently, an extensionwhich utilizes the Phong illumination model and shadow mapping hasbeen proposed [97]. Here, each point contributes to the WRP accordingto the properties of its associated surface, enabling the generation ofhigh quality colour holograms, as shown in Fig. 16.

4.2.3. Wavelet and STFT-based accelerationsA more recent development is to compute PSFs in non-spatial

transform domains. PSFs are sparse in the wavelet domain (WASABImethod) [100], and even sparser in the Short-Time Fourier Transform

121


Fig. 15. Illustration of the multiple WRP CGH method showing the smaller supportrequired per point, compared to the single WRP methods in Fig. 14. Additionally, thevariation of the support of the PSF per depth level of the LUT is shown, which is determinedby the distance to the WRP and the maximum diffraction angle.

Fig. 16. View reconstructions extracted from a 65,536-by-65,536 pixel hologram gen-erated with the multiple WRP method. Reconstructions (a) and (b) are two views takenwith a small aperture (6.5 mm – 8𝑘 × 8𝑘 pixels), the two reconstructions (c) and (d) arerefocused respectively at the front and back using a larger aperture (13.1 mm – 16𝑘× 16𝑘pixels). The Biplane model is courtesy of ScanLAB Projects [99].

(STFT) domain [101]. High realism can be achieved using less than 2%of the highest-magnitude coefficients, with reported 30-fold speedups.Actually, the (phase-added) stereogram method can be viewed as aparticular case of sparse STFT-based CGH. Note that the aforementioned

classes are not mutually exclusive, and many existing methods arehybrids [95,102].

4.3. Challenges

CGH methods are a compromise between calculation time, realismand supported scene parameters (e.g. dimensions, viewing angles, occlu-sion, lighting, transparency). Optimizing CGH methods both for offline(video) rendering and for real-time display are still very active domainsof research. The main challenges to overcome are:

Synthesizing large CGH. The CGH generation must be effective com-putationally, especially for full-parallax holographic displays. Gener-ating holograms in real-time at video rates requires specialized soft-ware/hardware implementations. Solutions involve adapting the algo-rithms for multi-threading [103], implementations on Graphics Process-ing Units (GPUs), [84,91], high-performance computer clusters [104] orFPGA systems [105].

Local CGH updates. Local memory and caching methods are indispens-able regarding computational complexity and principles of efficienttransforms and sparse representations may be used. This challengeoverlaps with the transform design problem for compression(cf. Section5).

Realistic scene rendering. The aim is to obtain photo-realistic CGH.Although steady progress has been made, the realism is still limitedcompared to its state-of-the-art counterpart in computer graphics. Aim-ing to go beyond Phong shading would require adequate models forshadows, transparency and refraction, complex lighting, volumetriclight scattering, and many others.

5. Transforms and coding

High-resolution holograms need large volumes of data to be repre-sented, especially for dynamic holography: e.g. the considered 10×10 cmdisplay in Section 4 needs 58 Gigapixels. Supposing we have a framerate of 30 fps and a phase-only SLM with 1 byte per pixel bit-depth,the required bandwidth would amount to 1.75 TB/s. Because of thesedata rates, dynamic hologram transmission will be unfeasible withcurrently available (or near-future) hardware unless we have adequatecompression techniques. Efficiently coding holograms will thus be ofutmost importance for viewing dynamic content.

Conventional image and video codecs are unfortunately suboptimalfor holograms, notably for macroscopic objects; this is because the char-acteristics of holographic signals differ substantially from natural imagecontent. This becomes immediately clear when one tries to decompose ahologram with wavelets: in Fig. 17, the decomposition is not sparse. Thisis why novel codec architectures need to be developed to adequatelyrepresent holographic data. One of the core codec components that needto be modified is the chosen transform. Furthermore, the relationshipbetween the objective hologram distortion and the perceived subjectivedistortion is still poorly understood.

5.1. Coding of static holograms

A number of different coding solutions for static holograms havebeen proposed over the years, and up to recently most of them concernedrelatively rudimentary solutions [22]. We can categorize these codecsin four classes, coding: (1) the input content for CGH, (2) the complexamplitude data in the hologram plane, (3) the backpropagated hologramin the object plane and (4) an intermediate representation.

122


Fig. 17. Example of a hologram transformed with a dyadic wavelet transform. Thetransform coefficients are clearly not sparse, which explains why conventional imagecodecs will not perform well on holographic data. Hologram: courtesy of b-com [106].

5.1.1. Coding of CGH inputThese methods encode the 3D scene representations (e.g.

RGB+Depth, meshes, point clouds, light fields) to circumvent theproblem of efficiently compressing the holographic signals. Thereafter,CGH will generate the output hologram. For example; the RGB+Depthmethod can be used with compressed textures and depth maps togenerate holograms on-the-fly [107]. The advantage is that the 3Dcontent itself can be efficiently compressed with existing solutions,resulting in small file sizes. The main disadvantage is that the hologramgeneration is shifted to the display side: CGH has a high computationalload, making real-time holography only possible for simple scenes.Many types of holograms (especially optically acquired ones) cannot beencoded with these methods. This solution will thus be impractical forbroadcast scenarios. Moreover, the encoded 3D content will still containredundancies, since only a part of the scene’s information will reach thehologram due to e.g. occlusion.

5.1.2. Coding in hologram planeThese methods on the other extreme treat the raw holograms as

a special type of image. These are generally existing image or video

codecs, which are modified and/or extended to better match thestatistics of holographic signals as to improve their rate–distortionbehaviour w.r.t. the default unmodified codec. They mostly involvetransforms that are better tailored to handle high frequencies andstrong directionality, caused by the interference fringes. For staticholograms, various transform extensions were proposed: directional-adaptive wavelets and arbitrary packet decompositions [108], vectorquantization lifting schemes [109], wavelet–bandelets [110], the waveatom transform [111], and mode dependent directional transform-basedHEVC [112,113]. Another family concerns Gabor wavelets, which ob-tain an optimal compromise between spatial and angular resolution per-mitted by the Heisenberg principle [114]. It was shown by Viswanathanet al. [115] that this transform can be used for adaptive view-dependentcoding. El Rhammad et al. specified a matching pursuit approach foran overcomplete Gabor wavelet dictionary [116] and further expandedtheir support for efficient view-point dependent decoding in support ofholographic HMDs [117].

Many holographic displays are driven by phase-only SLMs. For thesedisplays, one can discard the amplitude since we need to code only theargument of the complex-valued wavefield, also known as the ‘‘wrappedphase’’, typically represented by values between 0 and 2𝜋. Because thewrapped phase behaves non-linearly, it should not be processed withconventional linear transforms such as the DCT or wavelets, since theywill incorrectly treat phase wraps as edges and wrongly model phasedistance. For example, for some very small 𝜖 > 0, the values 0 and (2𝜋−𝜖)will be treated as highly different intensity values, even though they arevery close in phase distance.

One possible solution is to use phase unwrapping [118] to linearizethe signal, but this is an ill-posed problem, computationally intensiveand can drastically increase the dynamic range of the signal. Othersolutions are the use of nonlinear phase-only transforms, such as modulowavelets [119,120]. The drawback is that lossy coding is more tricky;care has to be taken when quantizing the signal because of the trans-forms’ non-linearity.

Methods coding the hologram plane data can in principle representany content and are fast, since they leverage existing coding architec-tures, and do not involve any generation of holographic data. Their maindrawback is that it becomes much harder to efficiently compress data,

Fig. 18. Amplitudes of a reconstructed dice hologram using various compression methods at 0.25 bpp. The upper row of subfigures is focused at the back plane, the lower row is focusedat the front plane. Hologram: courtesy of b-com [106].Source: [101].

123


primarily because the transforms lack awareness of the 3D scene con-tent. This problem becomes even trickier for holographic video: unlikefor regular video content, local (e.g. block-based) motion estimation andcompensation techniques are inadequate rendering them ineffective,because small motion in the 3D space will influence all hologram pixels.Another important issue is the rate–distortion behaviour: as the contentis modelled indirectly, the relationship between objective and perceived(subjective) distortions is not straightforward.

5.1.3. Coding in object planeThis class of compression methods utilizes a single backpropagation,

typically numerical Fresnel diffraction or ASM, which is efficientlycomputable with 1 or 2 FFTs. When the hologram has a large depth-of-focus (small aperture, large pixel pitch) or a scene which is (nearly)flat, then a backpropagation to the right depth will effectively refocusthe entire scene. That way, the refocused hologram will resemble animage, making it subsequently better compressible by conventionalcodecs [121]. This parallels the use of the WRP in CGH, which utilizesits sparsity in the object plane for simplifying computations.

A notable example of this approach are the Fresnelets [122]: they area closed-form mathematical expression of Fresnel diffraction combinedwith B-spline wavelets, with demonstrated use for compression [123].Object-plane coding was recently investigated by combining it withmodern codecs as well [113,124]. However, an important limitation ofusing a global Fresnel (or ASM) transform is that it is ill-suited for deepscenes: multiple objects or single extended-depth objects cannot all becompletely brought in focus simultaneously.

5.1.4. Coding of intermediate representationsCoding in the object plane works well for flat scenes like for example

microscopic data; however, when encoding deep scenes, it is impossibleto identify one single object plane that is suitable for globally encodingthe data. Choosing a particular object plane results in large parts of thescene being out of focus. Consequently, this gives rise to poor encodingperformance due to a weak match with natural image statistics –significant fringe patterns are still present – and to loss of the depthinformation. This is illustrated for the Fresnel backpropagation case inFig. 18(b) where numerical reconstructions of the decoded holograms,with either focus on back or front planes of the scene, are poor.For comparison purpose, encoding in the hologram plane with plainJPEG 2000 is shown as well.

To discuss the design of an intermediate representation that allowsfor the preservation of depth information, we need to introduce thetime–frequency representation – a.k.a. space–frequency representation,the Wigner representation or phase space – in which the properties ofholographic signals can elegantly be represented visually and math-ematically. It is a key tool to build a deeper understanding of thecharacteristics of interferometric holographic signals. For simplicity, werestrict in Fig. 19 the graphical portrayal to 1D signals. Similarly to amusic score, time (or space) and frequency are jointly represented onthe horizontal (𝑥) and vertical (𝜔) axes, respectively.

Because frequencies correspond to diffraction angles given byEq. (1), points in the time–frequency domain will correspond to in-dividual rays. Note that we cannot have perfect localization in bothspace and frequency simultaneously due to the Heisenberg uncertaintyprinciple, so in practice we cannot extract individual rays from a wave-based representation such as holography.

A mathematically elegant solution to address the intermediate rep-resentation problem is to use Linear canonical transforms (LCTs). Theyform a class of unitary transforms, i.e., linear operations on the time–frequency domain. LCTs can be used to model Fresnel free-space back-propagation as a simple global shear in the time–frequency domain.Unfortunately, LCTs cannot adequately represent deep scenes, similarlyto the backpropagation case.

To that end, Blinder et al. [101] derived the reversibility conditionson the scene depth map needed for having valid nonlinear canonical

Fig. 19. Representations of a holographic signal propagating at various depths, withtheir associated time–frequency diagram. The energy in the time–frequency domain isdelineated by a blue polygon, whose shape will progressively shear as the signal ispropagating forward. The red and green light rays will correspond to points in the time–frequency domain, as shown on the diagram.

Fig. 20. Diagram of how non-planar diffraction warps the time–frequency domain. Theleft column shows the depth (map) of a surface. The right column shows the correspondingtime–frequency warping. The top row shows a nonlinear canonical transform using apiece-wise linear approximation of the surface. The bottom row shows the closest LCTby taking the average constant depth, which is insufficient for efficient coding.Source: [101].

transforms. Furthermore, they proposed an efficiently computable sub-set using a piecewise linear approximation of the depth profile of thescene surface (Fig. 20). This was achieved by segmenting the hologramand applying a series of (complementary) unitary transforms based onthe depth profile of the object or scene and as such seamlessly tile thetime–frequency domain.

This will result in an intermediate representation where all segmentsare brought into focus and hence a classic image or video codingstrategy can be deployed. These transforms do not exhibit redundancy,but can be more computationally costly and have some restrictionson the depth maps it can represent. The proposed transform showsmarkedly improved reconstructions and demonstrates the importance ofmodelling non-linearities for successfully capturing depth informationfrom scenes (see Fig. 18(c)).

5.2. Coding of dynamic holograms

Conventional video codecs will attempt to compensate for motionby using localized prediction schemes (e.g. block matching algorithms).

124


This is ineffective for holography, because even small motions will causepervasive changes to the signal because of the nature of diffraction,i.e. for a larger angular FoV, one can expect that every pixel containsinformation about every point in the scene, and hence also informationabout all motion present in the scene. Consequently, motion estimationwill fail. This fact was e.g. noticeable in the work of Xing et al. [125],where holographic videos were compressed with a standard HEVCcodec. Inter-frame decorrelation brought little benefit for compression.

Hence, due to the problematic nature of motion estimation, it makesmore sense to inherit motion information from the source data thatwas used to computer-generate the holograms. The motion vectors canthen subsequently be used to steer the motion compensation process.For translational and rotational motion parallel to the hologram planethis can easily be facilitated, but out-of-plane rotations/translationsneed operations in the frequency domain. Note that since full-parallaxholographic data contains information for different viewing angles,inherently information is present to support compensation for rotationsaround the horizontal and vertical axes defining the hologram plane.Translational motion on the axis perpendicular to the hologram planeresults in shearing of the spectrum in the time–frequency domain (cf.Fig. 19).

A solution recently proposed by Blinder et al. [126] is to generalizeLCTs using affine symplectic time–frequency transforms, enabling tomodel all small rigid-body motions (6 DOF) in 3D space as particularunitary transforms. That way, the transform can be used for motioncompensation (Fig. 21). This was successfully utilized for holographicvideo coding, resulting in substantial gains surpassing 5dB over standardHEVC codecs using a simple global motion compensation scheme [126].Even higher gains are expected when using more advanced motion com-pensation architectures. Furthermore, the same scheme can compensatefor the motion of users when viewing dynamic holographic content withHMDs.

5.3. Challenges

There is as of yet no generic transform or representation that isapplicable to holograms with arbitrary scene content. Ultimately, thegoal is to build a codec tuned for static and dynamic digital holograms,efficient both in terms of compression performance as well as inencoding/decoding time. To conclude, we list some of the importantchallenges to be tackled concerning hologram coding:

Efficient representations. Coding strategies based on intermediate rep-resentations are a promising approach, particularly those based onthe recently proposed nonlinear canonical transforms. However, moreresearch is necessary to lay sound mathematical foundations as well astime–frequency domain segmentation approaches. In addition, furtherinvestigation of sparse signal representations is a necessity, and researchis welcomed on intra-frame prediction techniques as deployed in manyimage and video coders but then particularly tuned to holographicsignals.

Motion compensation and estimation. Accounting for scene-space motionis needed because every 3D scene point can potentially affect everyhologram pixel and classical local motion models on the hologram (suchas block-based motion compensation) will be ineffective. Novel algo-rithms will be needed to compensate for arbitrary motion of multipleindependently moving objects. Moreover, optically acquired hologramspresent a challenge because the ground-truth motion is unknown, andwill thus have to be estimated. Automated general motion estimation indigital holography for complex scenes has not been solved yet either.

Coding complex values. Holograms are complex-valued and new ap-proaches are needed for efficiently coding wrapped phase data forphase-only holograms, or for the phase components of complex-valuedsignals. An additional difficulty will be achieving lossy coding, becauseof the inherent non-linearity of the phase information.

Fig. 21. Motion compensation for dynamic holographic video using affine time–frequency transforms. Both object and viewer motion can be accurately compensated,resulting in sparse difference frames.

Speckle reduction. Speckles arises on diffuse surface elements and theyare difficult to capture due to the random phase distribution at pointof emission. It is hard to separate the speckle noise realization from theunderlying signal, which will not only impact compression performancebut quality assessment as well. For example, diffuse surfaces have arandom phase delay distribution which will significantly increase theentropy of the hologram, but this cannot be eliminated as it wouldremove the diffuseness, even though the precise random instance isunimportant for visualization. Potential solutions involve encodingthe phase probability distribution instead, from which many randominstances can be generated, thereby reducing entropy considerably.

Scalable coding. View and position dependent coding will be importantto support holographic video streams across displays with differingscreen sizes and angular resolutions. This is especially relevant forSUDDs and HMDs, where only a fraction of the holographic signalwill reach the viewer’s pupil. This implies that any encoding schemesatisfying these requirements will require sufficient locality both in timeas well as in frequency.

Computational complexity. Reducing computation through adapted ap-proximations is of course a generic requirement returning for everycomponent of the holographic processing pipeline. Parallelization ofmethods and distributed processing are additional requirements.

6. Quality assessment

Assessing the impact of distortions introduced in the holographic sig-nal processing pipeline (Fig. 1) on the visual quality of the reconstructedhologram is not as straightforward as for regular images. In the lattercase, distortions introduced in the spatial or frequency domain can oftenbe easily linked to reconstruction artefacts. As previously discussed,for holographic content, this connection is much more obfuscated.Information related to one particular point source in space is spreadthroughout the hologram, particularly for large angular FoVs.

Hence, when it comes to quality prediction, digital holography yethas an extra level of complexity compared to regular imaging modalities.When the wavefield gets distorted by some form of image processingor compression, one is not primarily interested in changes of theholographic fringe pattern, but rather in how the decoded objects willappear after reconstruction (cf. Section 6.4). Furthermore, the visualquality depends on the chosen perspective as well: e.g. the highestfrequency components contain information for the largest viewingangles; quantizing these components more coarsely will result in aninhomogeneous distortion of viewing angles, and potentially a reductionof the angular FoV.

Unfortunately, so far only initial attempts have been made to developvisual quality metrics for holographic data and only few efforts have

125


Fig. 22. Numerical reconstructions of holograms from various databases: (a) b-comhologram repository [106], (b) Interfere-III database [127], (c) EmergImg-HoloGrail v2database [128].

been undertaken to design subjective quality assessment procedures.Both are also requiring suitable holographic test data. Let us take a closerlook at these three aspects.

6.1. Holographic test data

Over the past years, several research groups have setup open publicdatabases containing various holograms [106,127,128], see Fig. 22.Their goal is to provide a representative data set of high-resolutionholograms, both optically recorded and computer-generated, containingboth static and dynamic content. At the time of writing, the databasesare still modest in size and content: each one contains maximum adozen holograms, consists almost entirely of static holograms, and oftenencode simple scenes with humble resolutions ranging up to 8K and onlysporadically up to 16K or larger. An important goal is to expand thesedatabases by adding more holograms with higher resolutions and moreheterogeneous and dynamic content. Currently, the JPEG Pleno stan-dardization initiative [129], initiated by the JPEG committee, attemptsto setup a database ‘‘JPEG PlenoDB’’ [99], collecting publicly availabledata sets and to define guidelines for selection of suitable test content.

6.2. Objective visual quality measures

Objective visual quality measures are typically classified into twocategories: (1) signal fidelity measures that have a clear physical meaning,but which are typically poor in terms of predicting the quality perceivedby the human visual system and (2) perceptual visual quality measures,typically modelled and trained via subjective quality experiments [130].

In digital holography, popular signal fidelity measures are Euclideandistance-based error measures, in particular Mean Square Error (MSE)and Peak Signal-to-Noise Ratio (PSNR). These are used predominantlyfor both error analysis and as a measure for reconstruction quality.However, MSE technically intends only to measure the global energyvariation. Other well-known regular imagery measures like StructuralSimilarity Index Measure (SSIM) have been used separately on theCartesian components of the complex wavefield [109,131]. Even tougha polar representation would be mathematically equivalent, directlyapplying fidelity measures on those channels were found to be lessaccurate [113] (e.g. due to the non-linear response of wrapped phaseto distortions).

This behaviour is already known for regular image processing [132–135] and persists for holograms, where the phase encodes amongothers the depth information. The Cartesian representation is muchless sensitive to compression artefacts as it avoids non-linearities bysplitting holographic information into real and imaginary parts [113].In addition, the quality can be measured in the object plane, hologramplane and after numerical reconstruction. The first two approaches havethe advantage that they measure the global quality of the hologram withrespect to the reference hologram in terms of PSNR or SSIM. The lattercan be deployed if the numerical reconstruction quality for a particularviewing angle and focus plane needs to be examined [131].

Recently in [136], a new versatile similarity measure (VSM) wasintroduced, aiming to address the mentioned shortcomings for complex

valued error measurement. Another novel quality metric, the SparsenessSignificance Ranking Measure (SSRM) [137], utilizing sparse coding anda ranking system based on the coefficient magnitudes, was shown to bemore effective than the PSNR or SSIM on holographic data as well [138].

To the knowledge of the authors, no perceptual visual qualitymeasures have been proposed yet for digital holography. To constructthem, subjective test procedures are a necessity.

6.3. Subjective test procedures

A standardized testing methodology to measure the perceptual visualquality – i.e. the visual quality as perceived by the Human Visual System(HVS) – for holography is still an open problem. The challenge is two-fold: (1) the heterogeneity of currently existing holographic displaysystems, differing in many aspects, such as light source properties,type of SLMs, resolution, pixel pitch, viewing zone geometry, opticalaberrations; and (2) the high complexity of 3D input data, with contentobservable from many angles and at different focal depths. We considertwo categories: direct assessment, where users directly view and rateholograms on an holographic display, and indirect assessment whichrelies on a substitute display (e.g. a light field display) showing viewsextracted from the hologram to be analysed.

For direct assessment to be reproducible, one would have to establishwhat the most relevant parameters are. On top of the existing require-ments for quality assessment on regular screens, aspects to considerare: (1) minimal screen capabilities (eyebox dimensions, viewing angle,bit-depth) and (2) viewer positioning (should the viewer move freely,go to specific positions or stay still?). However, given that holographicdisplays are evolving rapidly, care should be taken that the requirementsare sufficiently general to remain relevant in the years to come.

It is also possible to reliably display high-quality synthetic hologramsup to even hundreds of Gigapixels by printing them in e.g. glassplates [139]. Although this provides an experience which is very closeto what a future high-end holographic display could show, it becomesimpractical and expensive to print a large quantity of instances to com-pare e.g. various codecs, distortions, CGH algorithms etc. Furthermore,this medium cannot be used for displaying dynamic content.

Alternatively, these obstacles can be partially overcome using in-direct assessment techniques, by utilizing conventional 2D displaysor even light-field displays to render various reconstructions of thehologram. Nonetheless, this will come at the cost of ignoring severalvisual cues, especially the ones pertaining to depth perception andparallax. Initial work on this problem was done in [136,140], where alarge 4K 2D screen was used to rate the holograms of the public data setfound in [127]. In this test, the goal was to rate only the overall qualityof central view reconstructions for the holograms. Such a setup allowsfor adapting to the currently available standard testing methodologieswith minimal modifications.

For more advanced and thorough quality assessments, one may takeinspiration from quality assessment procedures used for light fields: mul-tiple views and focal depths can be shown in succession in a video formatto simulate scene perception from many different viewpoints [141]. Theadvantage of this approach is that the testing conditions for subjectivequality measurements have been meticulously standardized for classical2D displays [142–144]. Another similar approach would be to allowusers to actively explore hologram content. This can be achieved on ascreen as well, or could be experienced more naturally with HMDs.

In recent efforts [98], a high-end light field display was used torender a set of colour CGHs. The display provided the possibility ofrendering a wide field of view but only with horizontal parallax. Sucha display setup can be utilized for subjective tests, where the observerswill be able to freely move within the FoV of the screen, observe thereconstructed hologram objects, and score the perceptual visual qualityfrom different points of view.

126


6.4. Challenges

Consequently, quality assessment of holographic data still facesmany challenges to be resolved:

Public holographic databases. Extended databases should be created byadding more holograms with higher resolutions and more heteroge-neous and dynamic content. These holograms should depict a goodutilization of the time–frequency space. If large swathes of the time–frequency domain are zero, this means that the hologram capacity isunderused. However, solely looking at the spatial or Fourier representa-tion can be misleading, since both can have a dense signal yet havea very sparse time–frequency representation. This will happen whene.g. an object surface is insufficiently diffuse. Secondly, objects shouldbe placed close enough to the hologram plane. This is needed to enableproper evaluation of depth cues such as accommodation and parallax.Finally, the bounds of the aliasing-free zone should be respected (cf. Fig. 7).The hologram parameters dictate the allowed scene geometries [145].Care should be taken to correctly place objects in the scene, otherwisethe useable viewing zone will shrink and unwanted artifacts will appear.For example, objects too close to the hologram plane will vanish at steepviewing angles, having the effect as if the viewer is looking at the scenethrough a keyhole.

These databases can subsequently be deployed to develop qualitymetrics that are targeted at assessing the perceptual visual quality. Thesemetrics face the following challenges.

Visualization artefacts. Visual distortions are entirely different fromnatural imagery: e.g. because of the non-local nature of the hologram’sinformation content, the effect of local point defects on the hologramwill be less visible than for natural imagery and have a more distributedimpact on the reconstructed hologram [15]. This requires an extra effortto study the effect of different distortion types affecting the wavefieldon the reconstructed hologram. Examples include: errors induced bylossy codecs, display limitations (bit-depth, phase/amplitude only) andoptical aberrations.

Speckle noise perception. Speckles arise naturally in holography and willaffect the assessment of any reconstructed views, whether opticallyrecorded or synthetic, by reducing the perceived reconstruction quality.Quality metrics should either account for or ignore this noise dependingon the use case. As explained in the previous section, two generatedholograms can possess highly different signals because of the differentrandom speckle instances for diffuseness, yet be visually nearly identicalafter reconstruction, making hologram comparison difficult.

View position and angle dependencies. Although traditionally a singlequality score is expected to represent the overall visual quality, thismay not suffice for 3D content. For wide FoV holograms of deep scenes,a human subject may experience a different perceptual visual qualitydepending on the viewing position, angle and focal depth. For example,standard JPEG compression with perceptual quantization will favourlow frequencies, thereby better preserving the centre view than largerviewing angles [123]. Visual quality measures are not only expected toprovide overall quality prediction per hologram, but also perspective-dependent predictions as well. As far as concerning depth resolution,compression technologies might affect the depth of field as well, seeFig. 18.

Subjective quality assessment. To model and train perceptual qualitymetrics, extensive subjective testing is required to facilitate their design.New procedures on holographic display should be developed thataccount for aspects such as minimal screen capabilities and viewingposition. An additional challenge would be to design these assessmentprocedures such that they are sufficiently flexible to support successivegenerations of holographic displays.

Alternative displays. Adapting subjective quality assessment procedureswill be required as long as sufficiently mature holographic displaysare lacking. These methodologies are based on displaying numericallyreconstructed holograms on nowadays available displays ranging fromclassical 2D displays to light field displays, the latter being capa-ble rendering large, wide angle, numerical hologram reconstructions.Modelling the relationship between quality scores collected via such‘‘pseudo-holographic’’ setups with quality scores obtained on earlyholographic displays will be a necessity to validate predictions fromthese experiments.

7. Conclusions

Realizing digital holographic display systems still faces many chal-lenges on the signal processing side. In this paper, we provided anoverview on the state-of-the-art in holographic displays, computer-generated holography, efficient transforms and coding solutions andquality assessment for digital holograms. Because of its multidisciplinarynature, several challenges remain to be addressed:

Holographic displays do not yet reach the requisite visual quality, screensizes and viewing angles needed for truly high quality experiences.Various attributes of holographic displays will need to be leveragedto get the best signal given the technological constraints. Examplesare light modulation efficiency, cross-talk, bit-depth, amplitude/phasemodulation, colour display accuracy, speckle noise and embedded signalprocessing within the display.

Recording digital holograms at high resolutions especially for largescenes remains cumbersome. Hence, for holographic display, computer-generated holography solutions are preferred. CGH algorithms have toovercome many hurdles for reaching an acceptable computationalcost. Numerical evaluations of diffraction is highly computationallyexpensive for modelling realistic-looking content, all while needingresolutions orders of magnitudes larger than current 2D displays. Goodcandidates are sparse CGH methods, which exploit spatial, temporal andfrequency redundancies in holographic signals without compromisingquality.

Transforms and codecs require new efficient representations, whichremains an open problem for holographic signals with large space-bandwidth products. Optimized transforms and prediction schemes willhave to strike a balance between sparsity, computational efficiency,3D scene-awareness and scalability requirements. Because holographicsignals are highly non-stationary, we expect that the best transforms willmodel scene content in phase space, i.e. the time–frequency domain. Notonly will this track the behaviour of diffraction more closely, but thiswill also allow for selective ROI decoding of the time–frequency spaceso as to only process the displayable subset of the holographic scene orto extract specific views.

Quality assessment is important for designing holographic systems. Ade-quate quality measures will help to steer the joint design of holographiccodecs, CGH algorithms and display systems. The first steps to achievethis goal will be (1) to design large, varied public data sets containingholograms of highly realistic objects that can account for all humanvisual cues; (2) to design initial quality assessment procedures fornumerically reconstructed holograms on regular 2D screens or lightfield displays at various viewing angles and focal depths; (3) to eval-uate the relevant hardware parameters affecting quality perception inholographic display systems aiding the establishment of standardizedtesting procedures.

In this context, it is encouraging that also standardization bodies arealready addressing some challenges. Particularly, the JPEG committeeis taking the lead by developing the JPEG Pleno standard that will facili-tate the capturing, representation and exchange of not solely light field,but also point cloud and holographic imaging modalities [129,146,147].The standard aims at (1) defining tools for improved compression while

127


providing advanced functionalities at system level, (2) supporting dataand metadata manipulation, editing, random access and interaction,protection of privacy and ownership rights as well as other securitymechanisms and providing methodologies for quality assessment.

In summary, we can state that holographic display systems aresteadily coming within reach [8,14,19–21]. However, to enable this typeof visualization, significant efforts are still required, not solely from theoptics, photonics and nano-electronics communities, but also from thesignal processing community. Moreover, all developed algorithms willhave to run at an acceptable computational cost, which in the end mightbe one of the highest hurdles to take. Nonetheless, while accounting forthese many challenges, we can still expect on a relatively short-termholographic head-mounted displays to hit the market, competing withlight field inspired HMD solutions for augmented reality applications.However, high-resolution multi-user displays with large angular FoVsupport, will at least still take a decade before commercial solutionswill become available.

Acknowledgements

The authors would like to thank b-com, ScanLAB Projects and NicolasPavillon for providing the holograms and 3D models used in this paper.

Funding

The research leading to these results has received funding fromthe European Research Council under the European Union’s SeventhFramework Programme (FP7/2007–2013)/ERC Grant Agreement Nr.617779 (INTERFERE), and from the Research Foundation – Flanders(FWO), Belgium project Nr. G024715N.

References

[1] D. Gabor, A new microscopic principle, Nature 161 (4098) (1948) 777–778,http://dx.doi.org/10.1038/161777a0.

[2] E.N. Leith, J. Upatnieks, Reconstructed wavefronts and communication theory, J.Opt. Soc. Amer. 52 (10) (1962) 1123–1128, http://dx.doi.org/10.1364/JOSA.52.001123.

[3] E. Cuche, P. Marquet, C. Depeursinge, Simultaneous amplitude-contrast andquantitative phase-contrast microscopy by numerical reconstruction of Fresneloff-axis holograms, Appl. Opt. 38 (34) (1999) 6994–7001, http://dx.doi.org/10.1364/AO.38.006994.

[4] F. Charrière, J. Kühn, T. Colomb, F. Montfort, E. Cuche, Y. Emery, K. Weible, P.Marquet, C. Depeursinge, Characterization of microlenses by digital holographicmicroscopy, Appl. Opt. 45 (5) (2006) 829–835, http://dx.doi.org/10.1364/AO.45.000829.

[5] E. Cuche, F. Bevilacqua, C. Depeursinge, Digital holography for quantitativephase-contrast imaging, Opt. Lett. 24 (5) (1999) 291–293.

[6] I. Yamaguchi, J.-i. Kato, S. Ohta, Surface shape measurement by phase-shiftingdigital holography, Opt. Rev. 8 (2) (2001) 85–89.

[7] H.J. Coufal, D. Psaltis, G.T. Sincerbox, et al., Holographic Data Storage, Vol. 8,Springer, 2000.

[8] F. Yaraş, H. Kang, L. Onural, State of the art in holographic displays: A survey, J.Display Technol. 6 (10) (2010) 443–454, http://dx.doi.org/10.1109/JDT.2010.2045734.

[9] M. Levoy, P. Hanrahan, Light field rendering, in: Proceedings of the 23rd AnnualConference on Computer Graphics and Interactive Techniques, SIGGRAPH ’96,ACM, New York, NY, USA, 1996, pp. 31–42, http://dx.doi.org/10.1145/237170.237199.

[10] M. Yamaguchi, Light-field and holographic three-dimensional displays (Invited),J. Opt. Soc. Amer A 33 (12) (2016) 2348–2364, http://dx.doi.org/10.1364/JOSAA.33.002348.

[11] Stephan Reichelt, Holographic 3-d displays - electro-holography within the graspof commercialization, in: R. Haussler, N. Leister (Eds.), Advances in Lasersand Electro Optics, IntechOpen, Rijeka, 2010, http://dx.doi.org/10.5772/8650,(Chapter 29).

[12] D.M. Hoffman, A.R. Girshick, K. Akeley, M.S. Banks, Vergenceaccommodationconflicts hinder visual performance and cause visual fatigue, J. Vis. 8 (3) (2008)33–33.

[13] J. Son, W. Son, S. Kim, K. Lee, B. Javidi, Three-dimensional imaging for creatingreal-world-like environments, Proc. IEEE 101 (1) (2013) 190–205, http://dx.doi.org/10.1109/JPROC.2011.2178052.

[14] M. Lucente, The first 20 years of holographic videoand the next 20, 2011.

[15] C. Slinger, C. Cameron, M. Stanley, Computer-generated holography as a genericdisplay technology, Computer 38 (8) (2005) 46–53.

[16] G. Lazarev, A. Hermerschmidt, S. Krger, S. Osten, LCOS spatial light modulators:trends and applications, Opt. Imaging Metrol.: Adv. Technol. (2012) 1–29.

[17] T. Haist, W. Osten, Holography using pixelated spatial light modulators - part 1:theory and basic considerations, J. Micro/Nanolithogr. MEMS MOEMS 14 (2015)041310, http://dx.doi.org/10.1117/1.JMM.14.4.041310.

[18] T. Haist, W. Osten, Holography using pixelated spatial light modulators - Part2: applications, J. Micro/Nanolithogr. MEMS MOEMS 14 (4) (2015) 041311,http://dx.doi.org/10.1117/1.JMM.14.4.041311.

[19] J. Geng, Three-dimensional display technologies, Adv. Opt. Photon. 5 (4) (2013)456–535, http://dx.doi.org/10.1364/AOP.5.000456.

[20] J.Y. Son, H. Lee, B.R. Lee, K.H. Lee, Holographic and light-field imaging as future3-d displays, Proc. IEEE 105 (5) (2017) 789–804, http://dx.doi.org/10.1109/JPROC.2017.2666538.

[21] D. Smalley, T.-C. Poon, H. Gao, J. Kvavle, K. Qaderi, Volumetric displays: Turning3-D inside-out, Opt. Photonics News 29 (6) (2018) 26–33, http://dx.doi.org/10.1364/OPN.29.6.000026.

[22] F. Dufaux, Y. Xing, B. Pesquet-Popescu, P. Schelkens, Compression of digitalholographic data: an overview, Proc. SPIE 9599 (2015) http://dx.doi.org/10.1117/12.2190997.

[23] E. Goldstein, Sensation and Perception, Cengage Learning, 2013.[24] B.G. Blundell, 3D Displays and Spatial Interaction: From Perception to Technology

Volume I Exploring the Science, Art, Evolution and Use of 3D Technologies,Walker & Wood Limited, New Zealand, 2011.

[25] K. Akeley, S.J. Watt, A.R. Girshick, M.S. Banks, A stereo display prototype withmultiple focal distances, ACM Trans. Graph. 23 (3) (2004) 804–813, http://dx.doi.org/10.1145/1015706.1015804.

[26] N. Padmanaban, R. Konrad, T. Stramer, E.A. Cooper, G. Wetzstein, Optimizingvirtual reality for all users through gaze-contingent and adaptive focus displays,Proc. Natl. Acad. Sci. 114 (9) (2017) 2183–2188, http://dx.doi.org/10.1073/pnas.1617251114.

[27] T. Balogh, P.T. Kovács, Z. Dobrányi, A. Barsi, Z. Megyesi, Z. Gaál, G. Balogh, Theholovizio system-new opportunity offered by 3d displays, in: Proceedings of theTMCE, 2008.

[28] S. Yoshida, fvision: 360-degree viewable glasses-free tabletop 3d display com-posed of conical screen and modular projector arrays, Opt. Express 24 (12) (2016)13194–13203, http://dx.doi.org/10.1364/OE.24.013194.

[29] T.H. Yoshihiro Kajiki, Hiroshi Yoshikawa, Hologramlike video images by 45-view stereoscopic display, Proc. SPIE 3012 (1997) http://dx.doi.org/10.1117/12.274452.

[30] Y. Takaki, K. Tanaka, J. Nakamura, Super multi-view display with a lowerresolution flat-panel display, Opt. Express 19 (5) (2011) 4129–4139, http://dx.doi.org/10.1364/OE.19.004129.

[31] H. Mizushina, J. Nakamura, Y. Takaki, H. Ando, Super multi-view 3d displaysreduce conflict between accommodative and vergence responses, J. Soc. Inf. Disp.24 (12) (2016) 747–756, http://dx.doi.org/10.1002/jsid.520.

[32] Y. Takaki, High-density directional display for generating natural three-dimensional images, Proc. IEEE 94 (3) (2006) 654–663, http://dx.doi.org/10.1109/JPROC.2006.870684.

[33] Z. Alpaslan, H. El-Ghoroury, P. Schelkens, Information processing challenges offull parallax light field displays, in: SPIE Photonics Europe, Optics, Photonicsand Digital Technologies for Multimedia Applications, Vol. 10679, SPIE, 2018,http://dx.doi.org/10.1117/12.2316659.

[34] A. Maimone, A. Georgiou, J.S. Kollin, Holographic near-eye displays for virtualand augmented reality, ACM Trans. Graph. 36 (4) (2017) 85:1–85:16, http://dx.doi.org/10.1145/3072959.3073624.

[35] N.A. Dodgson, Autostereo displays: 3D without glasses, in: Electronic InformationDisplays, 1997.

[36] J.S. Kollin, S.A. Benton, M.L. Jepsen, Real-time display of 3-d computed hologramsby scanning the image of an acousto-optic modulator, Proc. SPIE 1136 (1989)http://dx.doi.org/10.1117/12.961683.

[37] S. Reichelt, R. Häussler, G. Fütterer, N. Leister, H. Kato, N. Usukura, Y. Kanbayashi,Full-range, complex spatial light modulator for real-time holography, Opt. Lett. 37(11) (2012) 1955–1957, http://dx.doi.org/10.1364/OL.37.001955.

[38] F. Yaraş, H. Kang, L. Onural, Circular holographic video display system, Opt.Express 19 (10) (2011) 9147–9156, http://dx.doi.org/10.1364/OE.19.009147.

[39] Y. Lim, K. Hong, H. Kim, H.-E. Kim, E.-Y. Chang, S. Lee, T. Kim, J. Nam, H.-G.Choo, J. Kim, J. Hahn, 360-degree tabletop electronic holographic display, Opt.Express 24 (22) (2016) 24999–25009, http://dx.doi.org/10.1364/OE.24.024999.

[40] H. Sasaki, K. Yamamoto, K. Wakunami, Y. Ichihashi, R. Oi, T. Senoh, Large sizethree-dimensional video by electronic holography using multiple spatial lightmodulators, Sci. Rep. 4 (2014) 6177, http://dx.doi.org/10.1038/srep06177.

[41] L. Shi, F.-C. Huang, W. Lopes, W. Matusik, D. Luebke, Near-eye light fieldholographic rendering with spherical waves for wide field of view interactive3d computer graphics, ACM Trans. Graph. 36 (6) (2017) 236:1–236:17, http://dx.doi.org/10.1145/3130800.3130832.

[42] R. Häussler, Y. Gritsai, E. Zschau, R. Missbach, H. Sahm, M. Stock, H. Stolle, Largereal-time holographic 3D displays: enabling components and results, Appl. Opt.56 (13) (2017) F45–F52, http://dx.doi.org/10.1364/AO.56.000F45.

128

http://dx.doi.org/10.1038/161777a0

http://dx.doi.org/10.1364/JOSA.52.001123



http://dx.doi.org/10.1364/AO.38.006994

http://dx.doi.org/10.1364/AO.38.006994

http://dx.doi.org/10.1364/AO.38.006994

http://dx.doi.org/10.1364/AO.45.000829

http://dx.doi.org/10.1364/AO.45.000829

http://dx.doi.org/10.1364/AO.45.000829

http://refhub.elsevier.com/S0923-5965(18)30485-5/sb5









http://dx.doi.org/10.1109/JDT.2010.2045734



http://dx.doi.org/10.1145/237170.237199

http://dx.doi.org/10.1145/237170.237199

http://dx.doi.org/10.1145/237170.237199

http://dx.doi.org/10.1364/JOSAA.33.002348



http://dx.doi.org/10.5772/8650






http://dx.doi.org/10.1109/JPROC.2011.2178052









http://dx.doi.org/10.1117/1.JMM.14.4.041310

http://dx.doi.org/10.1117/1.JMM.14.4.041311

http://dx.doi.org/10.1364/AOP.5.000456




http://dx.doi.org/10.1364/OPN.29.6.000026



http://dx.doi.org/10.1117/12.2190997

http://dx.doi.org/10.1117/12.2190997

http://dx.doi.org/10.1117/12.2190997







http://dx.doi.org/10.1145/1015706.1015804

http://dx.doi.org/10.1145/1015706.1015804

http://dx.doi.org/10.1145/1015706.1015804

http://dx.doi.org/10.1073/pnas.1617251114








http://dx.doi.org/10.1364/OE.24.013194

http://dx.doi.org/10.1117/12.274452

http://dx.doi.org/10.1117/12.274452

http://dx.doi.org/10.1117/12.274452

http://dx.doi.org/10.1364/OE.19.004129

http://dx.doi.org/10.1364/OE.19.004129

http://dx.doi.org/10.1364/OE.19.004129

http://dx.doi.org/10.1002/jsid.520




http://dx.doi.org/10.1117/12.2316659

http://dx.doi.org/10.1145/3072959.3073624

http://dx.doi.org/10.1145/3072959.3073624

http://dx.doi.org/10.1145/3072959.3073624




http://dx.doi.org/10.1117/12.961683

http://dx.doi.org/10.1364/OL.37.001955

http://dx.doi.org/10.1364/OE.19.009147

http://dx.doi.org/10.1364/OE.24.024999

http://dx.doi.org/10.1038/srep06177

http://dx.doi.org/10.1145/3130800.3130832

http://dx.doi.org/10.1145/3130800.3130832

http://dx.doi.org/10.1145/3130800.3130832

http://dx.doi.org/10.1364/AO.56.000F45


[43] T. Inoue, Y. Takaki, Table screen 360-degree holographic display using circularviewing-zone scanning, Opt. Express 23 (5) (2015) 6533–6542, http://dx.doi.org/10.1364/OE.23.006533.

[44] T. Kozacki, G. Finke, P. Garbat, W. Zaperty, M. Kujawińska, Wide angle holo-graphic display system with spatiotemporal multiplexing, Opt. Express 20 (25)(2012) 27473–27481, http://dx.doi.org/10.1364/OE.20.027473.

[45] D. Close, M. Ernstoff, W. Hoffman, E. Opittek, R. Winner, Holographic lens andliquid crystal image source for head-up display, US Patent 3, 915, 548, 1975.

[46] G.E. Moss, Holographic display panel for a vehicle windshield, US Patent 4, 790,613 A, 1987.

[47] W.J.A. Charles, I. Dumoulin, Robert D. Darrow, Display system for enhancingvisualization of body structures during medical procedures, US Patent 5, 526, 812A, 1995.

[48] H. Peng, D. Cheng, J. Han, C. Xu, W. Song, L. Ha, J. Yang, Q. Hu, Y. Wang,Design and fabrication of a holographic head-up display with asymmetric fieldof view, Appl. Opt. 53 (29) (2014) H177–H185, http://dx.doi.org/10.1364/AO.53.00H177.

[49] H.-J. Yeom, H.-J. Kim, S.-B. Kim, H. Zhang, B. Li, Y.-M. Ji, S.-H. Kim, J.-H. Park,3D holographic head mounted display using holographic optical elements withastigmatism aberration compensation, Opt. Express 23 (25) (2015) 32025–32034,http://dx.doi.org/10.1364/OE.23.032025.

[50] G. Li, D. Lee, Y. Jeong, J. Cho, B. Lee, Holographic display for see-throughaugmented reality using mirror-lens holographic optical element, Opt. Lett. 41(11) (2016) 2486–2489, http://dx.doi.org/10.1364/OL.41.002486.

[51] J. Khan, C. Can, A. Greenaway, I. Underwood, A real-space interactive holographicdisplay based on a large-aperture HOE, Proc. SPIE 8644 (2013) http://dx.doi.org/10.1117/12.2021633.

[52] R. Hussler, S. Reichelt, N. Leister, E. Zschau, R. Missbach, A. Schwerdtner, Largereal-time holographic displays: from prototypes to a consumer product, Proc. SPIE7237 (2009) http://dx.doi.org/10.1117/12.805873.

[53] M.L. Huebschman, B. Munjuluri, H.R. Garner, Dynamic holographic 3-d imageprojection, Opt. Express 11 (5) (2003) 437–445, http://dx.doi.org/10.1364/OE.11.000437.

[54] X. Xia, X. Liu, H. Li, Z. Zheng, H. Wang, Y. Peng, W. Shen, A 360-degree floating3d display based on light field regeneration, Opt. Express 21 (9) (2013) 11237–11247, http://dx.doi.org/10.1364/OE.21.011237.

[55] J. Kim, Y. Lim, K. Hong, E.-Y. Chang, H. gon Choo, 360-degree tabletop colorholographic display, in: Digital Holography and Three-Dimensional Imaging,Optical Society of America, 2017, p. W3A.1, http://dx.doi.org/10.1364/DH.2017.W3A.1.

[56] M. Oikawa, T. Shimobaba, T. Yoda, H. Nakayama, A. Shiraki, N. Masuda, T. Ito,Time-division color electroholography using one-chip rgb led and synchronizingcontroller, Opt. Express 19 (13) (2011) 12008–12013, http://dx.doi.org/10.1364/OE.19.012008.

[57] W. Zaperty, T. Kozacki, M. Kujawińska, Multi-SLM color holographic 3D displaybased on RGB spatial filter, J. Display Technol. 12 (12) (2016) 1724–1731.

[58] A.W. Lohmann, R.G. Dorsch, D. Mendlovic, Z. Zalevsky, C. Ferreira, Space-bandwidth product of optical signals and systems, J. Opt. Soc. Amer A 13 (3)(1996) 470–473, http://dx.doi.org/10.1364/JOSAA.13.000470.

[59] J.W. Goodman, Introduction to Fourier Optics, Roberts & Company, 2004.[60] R.W. Gerchberg, W.O. Saxton, A practical algorithm for the determination of the

phase from image and diffraction plane pictures, Optik 35 (1972) 237–246.[61] Y. Deng, D. Chu, Coherence properties of different light sources and their effect

on the image sharpness and speckle of holographic displays, Sci. Rep. 7 (1) (2017)5893.

[62] WG1, JPEG Pleno Scope, Use Cases and Requirements Ver.1.7, P. Schelkens (Ed.),Doc. N74020, 74th JPEG Meeting, Geneva, Switzerland, 2017.

[63] N. Pavillon, C. Arfire, I. Bergoënd, C. Depeursinge, Iterative method for zero-ordersuppression in off-axis digital holography, Opt. Express 18 (15) (2010) 15318–15331, http://dx.doi.org/10.1364/OE.18.015318.

[64] I. Yamaguchi, T. Zhang, Phase-shifting digital holography, Opt. Lett. 22 (16)(1997) 1268–1270, http://dx.doi.org/10.1364/OL.22.001268.

[65] V. Mico, Z. Zalevsky, P. García-Martínez, J. García, Synthetic aperture superres-olution with multiple off-axis holograms, J. Opt. Soc. Amer A 23 (12) (2006)3162–3170, http://dx.doi.org/10.1364/JOSAA.23.003162.

[66] A. Pelagotti, M. Paturzo, M. Locatelli, A. Geltrude, R. Meucci, A. Finizio, P.Ferraro, An automatic method for assembling a large synthetic aperture digitalhologram, Opt. Express 20 (5) (2012) 4830–4839, http://dx.doi.org/10.1364/OE.20.004830.

[67] A.E.T. James R. Fienup, Gigapixel synthetic-aperture digital holography, Proc.SPIE 8122 (2011) http://dx.doi.org/10.1117/12.894903.

[68] M. Kujawinska, Spatio-temporal multiplexing in digital holographic microscopyand holographic displays, in: Digital Holography and Three-Dimensional Imaging,Optical Society of America, 2017, p. W1A.1, http://dx.doi.org/10.1364/DH.2017.W1A.1.

[69] M. Paturzo, F. Merola, S. Grilli, S.D. Nicola, A. Finizio, P. Ferraro, Super-resolutionin digital holography by a two-dimensional dynamic phase grating, Opt. Express16 (21) (2008) 17107–17118, http://dx.doi.org/10.1364/OE.16.017107.

[70] R. Horisaki, Y. Ogura, M. Aino, J. Tanida, Single-shot phase imaging with a codedaperture, Opt. Lett. 39 (22) (2014) 6466–6469, http://dx.doi.org/10.1364/OL.39.006466.

[71] D. Donoho, Compressed sensing, IEEE Trans. Inform. Theory 52 (4) (2006) 1289–1306, http://dx.doi.org/10.1109/TIT.2006.871582.

[72] L. Williams, G. Nehmetallah, P.P. Banerjee, Digital tomographic compressive holo-graphic reconstruction of three-dimensional objects in transmissive and reflectivegeometries, Appl. Opt. 52 (8) (2013) 1702–1710, http://dx.doi.org/10.1364/AO.52.001702.

[73] D.J. Brady, K. Choi, D.L. Marks, R. Horisaki, S. Lim, Compressive holography, Opt.Express 17 (15) (2009) 13040–13049, http://dx.doi.org/10.1364/OE.17.013040.

[74] S. Bettens, C. Schretter, N. Deligiannis, P. Schelkens, Bounds and conditionsfor compressive digital holography using wavelet sparsifying bases, IEEE Trans.Comput. Imaging 3 (4) (2017) 592–604, http://dx.doi.org/10.1109/TCI.2017.2761742.

[75] S. Bettens, H. Yan, D. Blinder, H. Ottevaere, C. Schretter, P. Schelkens, Studies onthe sparsifying operator in compressive digital holography, Opt. Express 25 (16)(2017) 18656–18676, http://dx.doi.org/10.1364/OE.25.018656.

[76] N. Pavillon, C.S. Seelamantula, J. Kühn, M. Unser, C. Depeursinge, Suppressionof the zero-order term in off-axis digital holography through nonlinear filter-ing, Appl. Opt. 48 (34) (2009) H186–H195, http://dx.doi.org/10.1364/AO.48.00H186.

[77] J.A. Fessler, S. Sotthivirat, Simplified digital holographic reconstruction usingstatistical methods, in: 2004 International Conference on Image Processing(ICIP’04), Vol. 4, IEEE, 2004, pp. 2435–2438, http://dx.doi.org/10.1109/ICIP.2004.1421593.

[78] A. Bourquard, N. Pavillon, E. Bostan, C. Depeursinge, M. Unser, A practicalinverse-problem approach to digital holographic reconstruction, Opt. Express 21(3) (2013) 3417–3433, http://dx.doi.org/10.1364/OE.21.003417.

[79] C. Schretter, D. Blinder, S. Bettens, H. Ottevaere, P. Schelkens, Regularized non-convex image reconstruction in digital holographic microscopy, Opt. Express 25(14) (2017) 16491–16508, http://dx.doi.org/10.1364/OE.25.016491.

[80] M.E. Lucente, Interactive computation of holograms using a look-up table, J.Electron. Imaging 2 (1) (1993) 28–34, http://dx.doi.org/10.1117/12.133376.

[81] S.-C. Kim, E.-S. Kim, Effective generation of digital holograms of three-dimensional objects using a novel look-up table method, Appl. Opt. 47 (19) (2008)D55–D62, http://dx.doi.org/10.1364/AO.47.000D55.

[82] T. Shimobaba, N. Masuda, T. Ito, Simple and fast calculation algorithm forcomputer-generated hologram with wavefront recording plane, Opt. Lett. 34 (20)(2009) 3133–3135, http://dx.doi.org/10.1364/OL.34.003133.

[83] A. Symeonidou, D. Blinder, A. Munteanu, P. Schelkens, Computer-generated holo-grams by multiple wavefront recording plane method with occlusion culling, Opt.Express 23 (17) (2015) 22149–22161, http://dx.doi.org/10.1364/oe.23.022149.

[84] D. Blinder, P. Schelkens, Accelerated computer generated holography using sparsebases in the STFT domain, Opt. Express 26 (2) (2018) 1461–1473, http://dx.doi.org/10.1364/OE.26.001461.

[85] K. Matsushima, S. Nakahara, Extremely high-definition full-parallax computer-generated hologram created by the polygon-based method, Appl. Opt. 48 (34)(2009) H54–H63, http://dx.doi.org/10.1364/AO.48.000H54.

[86] K. Matsushima, M. Nakamura, S. Nakahara, Silhouette method for hidden surfaceremoval in computer holography and its acceleration using the switch-back tech-nique, Opt. Express 22 (20) (2014) 24450–24465, http://dx.doi.org/10.1364/OE.22.024450.

[87] H. Nishi, K. Matsushima, Rendering of specular curved objects in polygon-basedcomputer holography, Appl. Opt. 56 (13) (2017) F37–F44, http://dx.doi.org/10.1364/AO.56.000F37.

[88] N. Okada, T. Shimobaba, Y. Ichihashi, R. Oi, K. Yamamoto, M. Oikawa, T. Kakue,N. Masuda, T. Ito, Band-limited double-step fresnel diffraction and its applicationto computer-generated holograms, Opt. Express 21 (7) (2013) 9192–9197, http://dx.doi.org/10.1364/OE.21.009192.

[89] T. Yatagai, Stereoscopic approach to 3-d display using computer-generated holo-grams, Appl. Opt. 15 (11) (1976) 2722–2729, http://dx.doi.org/10.1364/AO.15.002722.

[90] M. Yamaguchi, H. Hoshino, T. Honda, N. Ohyama, Phase-added stereogram:calculation of hologram using computer graphics technique, Proc. SPIE 1914(1993) 25–31, http://dx.doi.org/10.1117/12.155027.

[91] R.H.-Y. Chen, T.D. Wilkinson, Computer generated hologram with geometricocclusion using gpu-accelerated depth buffer rasterization for three-dimensionaldisplay, Appl. Opt. 48 (21) (2009) 4246–4255, http://dx.doi.org/10.1364/AO.48.004246.

[92] H. Zhang, Y. Zhao, L. Cao, G. Jin, Fully computed holographic stereogram basedalgorithm for computer-generated holograms with accurate depth cues, Opt.Express 23 (4) (2015) 3901–3913, http://dx.doi.org/10.1364/OE.23.003901.

[93] L. Shi, F.-C. Huang, W. Lopes, W. Matusik, D. Luebke, Near-eye light fieldholographic rendering with spherical waves for wide field of view interactive3d computer graphics, ACM Trans. Graph. 36 (6) (2017) 236:1–236:17, http://dx.doi.org/10.1145/3130800.3130832.

[94] S. Igarashi, T. Nakamura, K. Matsushima, M. Yamaguchi, Efficient tiled calculationof over-10-gigapixel holograms using ray-wavefront conversion, Opt. Express 26(8) (2018) 10773–10786, http://dx.doi.org/10.1364/OE.26.010773.

[95] K. Wakunami, H. Yamashita, M. Yamaguchi, Occlusion culling for computergenerated hologram based on ray-wavefront conversion, Opt. Express 21 (19)(2013) 21811–21822, http://dx.doi.org/10.1364/OE.21.021811.

129

http://dx.doi.org/10.1364/OE.23.006533

http://dx.doi.org/10.1364/OE.23.006533

http://dx.doi.org/10.1364/OE.23.006533

http://dx.doi.org/10.1364/OE.20.027473

http://dx.doi.org/10.1364/AO.53.00H177



http://dx.doi.org/10.1364/OE.23.032025

http://dx.doi.org/10.1364/OL.41.002486

http://dx.doi.org/10.1117/12.2021633

http://dx.doi.org/10.1117/12.2021633

http://dx.doi.org/10.1117/12.2021633

http://dx.doi.org/10.1117/12.805873

http://dx.doi.org/10.1364/OE.11.000437

http://dx.doi.org/10.1364/OE.11.000437

http://dx.doi.org/10.1364/OE.11.000437

http://dx.doi.org/10.1364/OE.21.011237

http://dx.doi.org/10.1364/DH.2017.W3A.1



http://dx.doi.org/10.1364/OE.19.012008

http://dx.doi.org/10.1364/OE.19.012008

http://dx.doi.org/10.1364/OE.19.012008














http://dx.doi.org/10.1364/OE.18.015318

http://dx.doi.org/10.1364/OL.22.001268


http://dx.doi.org/10.1364/OE.20.004830

http://dx.doi.org/10.1364/OE.20.004830

http://dx.doi.org/10.1364/OE.20.004830

http://dx.doi.org/10.1117/12.894903




http://dx.doi.org/10.1364/OE.16.017107

http://dx.doi.org/10.1364/OL.39.006466

http://dx.doi.org/10.1364/OL.39.006466

http://dx.doi.org/10.1364/OL.39.006466

http://dx.doi.org/10.1109/TIT.2006.871582

http://dx.doi.org/10.1364/AO.52.001702

http://dx.doi.org/10.1364/AO.52.001702

http://dx.doi.org/10.1364/AO.52.001702

http://dx.doi.org/10.1364/OE.17.013040

http://dx.doi.org/10.1109/TCI.2017.2761742



http://dx.doi.org/10.1364/OE.25.018656




http://dx.doi.org/10.1109/ICIP.2004.1421593



http://dx.doi.org/10.1364/OE.21.003417

http://dx.doi.org/10.1364/OE.25.016491

http://dx.doi.org/10.1117/12.133376

http://dx.doi.org/10.1364/AO.47.000D55

http://dx.doi.org/10.1364/OL.34.003133

http://dx.doi.org/10.1364/oe.23.022149

http://dx.doi.org/10.1364/OE.26.001461

http://dx.doi.org/10.1364/OE.26.001461

http://dx.doi.org/10.1364/OE.26.001461


http://dx.doi.org/10.1364/OE.22.024450

http://dx.doi.org/10.1364/OE.22.024450

http://dx.doi.org/10.1364/OE.22.024450




http://dx.doi.org/10.1364/OE.21.009192

http://dx.doi.org/10.1364/OE.21.009192

http://dx.doi.org/10.1364/OE.21.009192

http://dx.doi.org/10.1364/AO.15.002722

http://dx.doi.org/10.1364/AO.15.002722

http://dx.doi.org/10.1364/AO.15.002722

http://dx.doi.org/10.1117/12.155027

http://dx.doi.org/10.1364/AO.48.004246

http://dx.doi.org/10.1364/AO.48.004246

http://dx.doi.org/10.1364/AO.48.004246

http://dx.doi.org/10.1364/OE.23.003901

http://dx.doi.org/10.1145/3130800.3130832

http://dx.doi.org/10.1145/3130800.3130832

http://dx.doi.org/10.1145/3130800.3130832

http://dx.doi.org/10.1364/OE.26.010773

http://dx.doi.org/10.1364/OE.21.021811


[96] A. Symeonidou, D. Blinder, B. Ceulemans, A. Munteanu, P. Schelkens, Three-dimensional rendering of computer-generated holograms acquired from point-clouds on light field displays, Proc. SPIE 9971 (2016) http://dx.doi.org/10.1117/12.2237581.

[97] A. Symeonidou, D. Blinder, P. Schelkens, Colour computer-generated holographyfor point clouds utilizing the Phong illumination model, Opt. Express 26 (8) (2018)10282–10298, http://dx.doi.org/10.1364/OE.26.010282.

[98] A. Symeonidou, D. Blinder, A. Ahar, C. Schretter, A. Munteanu, P. Schelkens,Speckle noise reduction for computer generated holograms of objects with diffusesurfaces, Proc. SPIE 9896 (2016) http://dx.doi.org/10.1117/12.2225201.

[99] ISO/IEC JTC1/SC29/WG1 (JPEG) Website – JPEG Pleno Database, https://jpeg.org/jpegpleno/plenodb.html (Accessed: 10.08.18).

[100] T. Shimobaba, T. Ito, Fast generation of computer-generated holograms usingwavelet shrinkage, Opt. Express 25 (1) (2017) 77–87, http://dx.doi.org/10.1364/OE.25.000077.

[101] D. Blinder, C. Schretter, H. Ottevaere, A. Munteanu, P. Schelkens, Unitarytransforms using time-frequency warping for digital holograms of deep scenes,IEEE Trans. Comput. Imaging 4 (2) (2018) 206–218, http://dx.doi.org/10.1109/TCI.2018.2813167.

[102] A. Gilles, P. Gioia, R. Cozot, L. Morin, Computer generated hologram fromMultiview-plus-Depth data considering specular reflections, in: 2016 IEEE In-ternational Conf. Multimedia Expo Workshops (ICMEW), 2016, pp. 1–6, http://dx.doi.org/10.1109/ICMEW.2016.7574699.

[103] T. Shimobaba, H. Nakayama, N. Masuda, T. Ito, Rapid calculation algorithmof Fresnel computer-generated-hologram using look-up table and wavefront-recording plane methods for three-dimensional display, Opt. Express 18 (19)(2010) 19504–19509, http://dx.doi.org/10.1364/OE.18.019504.

[104] N. Takada, T. Shimobaba, H. Nakayama, A. Shiraki, N. Okada, M. Oikawa, N.Masuda, T. Ito, Fast high-resolution computer-generated hologram computationusing multiple graphics processing unit cluster system, Appl. Opt. 51 (30) (2012)7303–7307, http://dx.doi.org/10.1364/AO.51.007303.

[105] T. Sugie, T. Akamatsu, T. Nishitsuji, R. Hirayama, N. Masuda, H. Nakayama, Y.Ichihashi, A. Shiraki, M. Oikawa, N. Takada, et al., High-performance parallelcomputing for next-generation holographic imaging, Nat. Electron. 1 (4) (2018)254.

[106] b-com hologram repository, URL https://hologram-repository.labs.b-com.com/#/hologram-repository.

[107] T. Senoh, K. Wakunami, Y. Ichihashi, H. Sasaki, R. Oi, K. Yamamoto, Multiviewimage and depth map coding for holographic tv system, Opt. Eng. 53 (11) (2014)112302, http://dx.doi.org/10.1117/1.OE.53.11.112302.

[108] D. Blinder, T. Bruylants, H. Ottevaere, A. Munteanu, P. Schelkens, JPEG 2000-based compression of fringe patterns for digital holographic microscopy, Opt. Eng.53 (12) (2014) 123102, http://dx.doi.org/10.1117/1.OE.53.12.123102.

[109] Y. Xing, M. Kaaniche, B. Pesquet-Popescu, F. Dufaux, Adaptive nonseparablevector lifting scheme for digital holographic data compression, Appl. Opt. 54 (1)(2015) A98–A109, http://dx.doi.org/10.1364/AO.54.000A98.

[110] L.T. Bang, Z. Ali, P.D. Quang, J.-H. Park, N. Kim, Compression of digital hologramfor three-dimensional object using Wavelet-Bandelets transform, Opt. Express 19(9) (2011) 8019–8031, http://dx.doi.org/10.1364/OE.19.008019.

[111] T. Birnbaum, D. Blinder, C. Schretter, P. Schelkens, Compressing macroscopicnear-field digital holograms with wave atoms, in: Imaging and Applied Optics2018, Optical Society of America, 2018, p. DW2F.5, http://dx.doi.org/10.1364/DH.2018.DW2F.5.

[112] J.P. Peixeiro, C. Brites, J. Ascenso, F. Pereira, Holographic data coding: Bench-marking and extending hevc with adapted transforms, IEEE Trans. Multimed. 20(2) (2018) 282–297, http://dx.doi.org/10.1109/TMM.2017.2742701.

[113] M.V. Bernardo, P. Fernandes, A. Arrifano, M. Antonini, E. Fonseca, P.T. Fiadeiro,A.M. Pinheiro, M. Pereira, Holographic representation: Hologram plane vs. objectplane, Signal Process., Image Commun. 68 (2018) 193–206, http://dx.doi.org/10.1016/j.image.2018.08.006.

[114] L.M. Kartik Viswanathan, Patrick Gioia, Wavelet compression of digital holo-grams: Towards a view-dependent framework, Proc. SPIE 8856 (2013) http://dx.doi.org/10.1117/12.2027199.

[115] K. Viswanathan, P. Gioia, L. Morin, A framework for view-dependent hologramrepresentation and adaptive reconstruction, in: 2015 IEEE International Confer-ence on Image Processing (ICIP), 2015, pp. 3334–3338, http://dx.doi.org/10.1109/ICIP.2015.7351421.

[116] A. El Rhammad, P. Gioia, A. Gilles, A. Cagnazzo, B. Pesquet-Popescu, Color digitalhologram compression based on matching pursuit, Appl. Opt. 57 (17) (2018)4930–4942, http://dx.doi.org/10.1364/AO.57.004930.

[117] Anas El Rhammad, Patrick Gioia, Antonin Gilles, Marco Cagnazzo, BatricePesquet-Popescu, View-Dependent Compression of Digital Hologram Based onMatching Pursuit, Vol. 10679, SPIE, 2018, p. 106790L, http://dx.doi.org/10.1117/12.2315233.

[118] D.C. Ghiglia, M.D. Pritt, Two-Dimensional Phase Unwrapping: Theory, Algo-rithms, and Software, Vol. 4, Wiley New York, 1998.

[119] D. Blinder, Efficient Representation, Generation and Compression of DigitalHolograms (Ph.D. thesis), Vrije Universiteit Brussel, 2018.

[120] D. Blinder, H. Ottevaere, A. Munteanu, P. Schelkens, Efficient multiscale phaseunwrapping methodology with modulo wavelet transform, Opt. Express 24 (20)(2016) 23094–23108, http://dx.doi.org/10.1364/OE.24.023094.

[121] E. Darakis, J.J. Soraghan, Reconstruction domain compression of phase-shiftingdigital holograms, Appl. Opt. 46 (3) (2007) 351–356, http://dx.doi.org/10.1364/AO.46.000351.

[122] M. Liebling, T. Blu, M. Unser, Fresnelets: New multiresolution wavelet bases fordigital holography, Trans. Image Proc. 12 (1) (2003) 29–43, http://dx.doi.org/10.1109/TIP.2002.806243.

[123] E. Darakis, T.J. Naughton, J.J. Soraghan, Compression defects in different recon-structions from phase-shifting digital holographic data, Appl. Opt. 46 (21) (2007)4579–4586, http://dx.doi.org/10.1364/AO.46.004579.

[124] M.P. Marco, V. Bernardo, Antnio M.G. Pinheiro, Benchmarking coding standardsfor digital holography represented on the object plane, Proc. SPIE 10679 (2018)http://dx.doi.org/10.1117/12.2315361.

[125] Y. Xing, B. Pesquet-Popescu, F. Dufaux, Compression of computer generatedphase-shifting hologram sequence using avc and hevc, Proc. SPIE 8856 (2013)http://dx.doi.org/10.1117/12.2027148.

[126] D. Blinder, C. Schretter, P. Schelkens, Global motion compensation for com-pressing holographic videos, Opt. Express 26 (20) (2018) 25524–25533, http://dx.doi.org/10.1364/OE.26.025524.

[127] D. Blinder, A. Ahar, A. Symeonidou, Y. Xing, T. Bruylants, C. Schretter, B.Pesquet-Popescu, F. Dufaux, A. Munteanu, P. Schelkens, Open access databasefor experimental validations of holographic compression engines, in: 2015 Sev-enth International Workshop on Quality of Multimedia Experience (QoMEX),2015, pp. 1–6, http://dx.doi.org/10.1109/QoMEX.2015.7148145, URL http://erc-interfere.eu/downloads.html.

[128] EmergImg-holograil holographic database. URL http://emergimg.di.ubi.pt/downloads.html.

[129] T. Ebrahimi, S. Foessel, F. Pereira, P. Schelkens, Jpeg pleno: Toward an efficientrepresentation of visual reality, IEEE MultiMedia 23 (4) (2016) 14–20, http://dx.doi.org/10.1109/MMUL.2016.64.

[130] W. Lin, C.-C. Jay Kuo, Perceptual visual quality metrics: A survey, J. Vis. Commun.Image Represent. 22 (4) (2011) 297–312, http://dx.doi.org/10.1016/j.jvcir.2011.01.005.

[131] A. Ahar, D. Blinder, T. Bruylants, C. Schretter, A. Munteanu, P. Schelkens, Sub-jective quality assessment of numerically reconstructed compressed holograms,in: SPIE Optical Engineering+ Applications, International Society for Optics andPhotonics, 2015, 95990K–95990K.

[132] A.V. Oppenheim, J.S. Lim, The importance of phase in signals, Proc. IEEE 69 (5)(1981) 529–541.

[133] W.H. Hsiao, R.P. Millane, Effects of fourier-plane amplitude and phase errors onimage reconstruction. i. small amplitude errors, J. Opt. Soc. Amer A 24 (10) (2007)3180–3188.

[134] X.S. Ni, X. Huo, Statistical interpretation of the importance of phase informationin signal and image reconstruction, Statist. Probab. Lett. 77 (4) (2007) 447–454.

[135] M. Narwaria, W. Lin, I.V. McLoughlin, S. Emmanuel, L.T. Chia, Fourier transform-based scalable image quality measure, IEEE Trans. Image Process. 21 (8) (2012)3364–3377.

[136] A. Ahar, T. Birnbaum, C. Jaeh, P. Schelkens, A new similarity measure for complexvalued data, in: Digital Holography and Three-Dimensional Imaging, OpticalSociety of America, 2017, p. Tu1A.6, http://dx.doi.org/10.1364/DH.2017.Tu1A.6.

[137] A. Ahar, A. Barri, P. Schelkens, From sparse coding significance to perceptualquality: A new approach for image quality assessment, IEEE Trans. Image Process.27 (2) (2018) 879–893, http://dx.doi.org/10.1109/TIP.2017.2771412.

[138] A. Ahar, T. Birnbaum, D. Blinder, A. Symeonidou, P. Schelkens, Performanceevaluation of sparseness significance ranking measure (ssrm) on holographiccontent, in: Imaging and Applied Optics 2018, Optical Society of America, 2018,p. JTu4A.10, http://dx.doi.org/10.1364/3D.2018.JTu4A.10.

[139] T.M. Lehtimäki, M. Niemelä, R. Näsänen, R.G. Reilly, T.J. Naughton, Usingtraditional glass plate holograms to study visual perception of future digitalholographic displays, in: Imaging and Applied Optics 2016, Optical Society ofAmerica, 2016, p. JW4A.20, http://dx.doi.org/10.1364/3D.2016.JW4A.20.

[140] A. Ahar, D. Blinder, T. Bruylants, C. Schretter, A. Munteanu, P. Schelkens, Sub-jective quality assessment of numerically reconstructed compressed holograms,Proc. SPIE 9599 (2015) http://dx.doi.org/10.1117/12.2189887.

[141] T.E. Irene Viola, A new approach to subjectively assess quality of plenopticcontent, Proc. SPIE 9971 (2016) http://dx.doi.org/10.1117/12.2240279.

[142] Rec. ITU-R P.910 Subjective video quality assessment methods for multimediaapplications, International Telecommunication Union, 2008.

[143] Rec. ITU-R P.913 Methods for the subjective assessment of video quality, audioquality and audiovisual quality of internet video and distribution quality televisionin any environment, International Telecommunication Union, 2016.

[144] Rec. ITU-R BT.500-13 Methodology for the subjective assessment of the qualityof television pictures, International Telecommunication Union, 2012.

[145] G. Finke, M. Kujawińska, T. Kozacki, Visual perception in multi SLM holographicdisplays, Appl. Opt. 54 (12) (2015) 3560–3568, http://dx.doi.org/10.1364/AO.54.003560.

[146] P. Schelkens, Z.Y. Alpaslan, T. Ebrahimi, K.-J. Oh, F.M.B. Pereira, A.M.G. Pinheiro,I. Tabus, Z. Chen, JPEG Pleno: a standard framework for representing andsignalling plenoptic modalities, in: Proc. of SPIE, Optics and Photonics 2018,Applications of Digital Image Processing XLI, Vol. 10752.

[147] ISO/IEC JTC1/SC29/WG1 (JPEG) Website, https://www.jpeg.org.

130

http://dx.doi.org/10.1117/12.2237581

http://dx.doi.org/10.1117/12.2237581

http://dx.doi.org/10.1117/12.2237581

http://dx.doi.org/10.1364/OE.26.010282

http://dx.doi.org/10.1117/12.2225201

https://jpeg.org/jpegpleno/plenodb.html



http://dx.doi.org/10.1364/OE.25.000077

http://dx.doi.org/10.1364/OE.25.000077

http://dx.doi.org/10.1364/OE.25.000077




http://dx.doi.org/10.1109/ICMEW.2016.7574699



http://dx.doi.org/10.1364/OE.18.019504

http://dx.doi.org/10.1364/AO.51.007303








https://hologram-repository.labs.b-com.com/#/hologram-repository



http://dx.doi.org/10.1117/1.OE.53.11.112302

http://dx.doi.org/10.1117/1.OE.53.12.123102

http://dx.doi.org/10.1364/AO.54.000A98

http://dx.doi.org/10.1364/OE.19.008019

http://dx.doi.org/10.1364/DH.2018.DW2F.5



http://dx.doi.org/10.1109/TMM.2017.2742701

http://dx.doi.org/10.1016/j.image.2018.08.006



http://dx.doi.org/10.1117/12.2027199

http://dx.doi.org/10.1117/12.2027199

http://dx.doi.org/10.1117/12.2027199




http://dx.doi.org/10.1364/AO.57.004930

http://dx.doi.org/10.1117/12.2315233

http://dx.doi.org/10.1117/12.2315233

http://dx.doi.org/10.1117/12.2315233







http://dx.doi.org/10.1364/OE.24.023094

http://dx.doi.org/10.1364/AO.46.000351

http://dx.doi.org/10.1364/AO.46.000351

http://dx.doi.org/10.1364/AO.46.000351

http://dx.doi.org/10.1109/TIP.2002.806243



http://dx.doi.org/10.1364/AO.46.004579

http://dx.doi.org/10.1117/12.2315361

http://dx.doi.org/10.1117/12.2027148

http://dx.doi.org/10.1364/OE.26.025524

http://dx.doi.org/10.1364/OE.26.025524

http://dx.doi.org/10.1364/OE.26.025524

http://dx.doi.org/10.1109/QoMEX.2015.7148145

http://erc-interfere.eu/downloads.html



http://emergimg.di.ubi.pt/downloads.html



http://dx.doi.org/10.1109/MMUL.2016.64



http://dx.doi.org/10.1016/j.jvcir.2011.01.005


























http://dx.doi.org/10.1364/DH.2017.Tu1A.6




http://dx.doi.org/10.1364/3D.2018.JTu4A.10

http://dx.doi.org/10.1364/3D.2016.JW4A.20

http://dx.doi.org/10.1117/12.2189887

http://dx.doi.org/10.1117/12.2240279

http://dx.doi.org/10.1364/AO.54.003560

http://dx.doi.org/10.1364/AO.54.003560

http://dx.doi.org/10.1364/AO.54.003560

https://www.jpeg.org

signal processing challenges for digital holographic video ......illustration of the...

Documents