a multi-modal stereo microscope based on a spatial light...

11
A multi-modal stereo microscope based on a spatial light modulator M. P. Lee, 1, G. M. Gibson, 1 R. Bowman, 2 S. Bernet, 3 M. Ritsch-Marte, 3 D. B. Phillips 4 and M. J. Padgett 1 1 School of Physics and Astronomy, SUPA, University of Glasgow, G12 8QQ, UK 2 Queens’ College, University of Cambridge, CB3 9LE, UK 3 Division for Biomedical Physics, Innsbruck Medical University, A-6020 Innsbrusck, Austria 4 H. H. Wills Physics Laboratory, University of Bristol BS8 1TL, UK [email protected] http://www.gla.ac.uk/schools/physics/research/groups/optics Abstract: Spatial Light Modulators (SLMs) can emulate the classic mi- croscopy techniques, including differential interference (DIC) contrast and (spiral) phase contrast. Their programmability entails the benefit of flexibil- ity or the option to multiplex images, for single-shot quantitative imaging or for simultaneous multi-plane imaging (depth-of-field multiplexing). We report the development of a microscope sharing many of the previously demonstrated capabilities, within a holographic implementation of a stereo microscope. Furthermore, we use the SLM to combine stereo microscopy with a refocusing filter and with a darkfield filter. The instrument is built around a custom inverted microscope and equipped with an SLM which gives various imaging modes laterally displaced on the same camera chip. In addition, there is a wide angle camera for visualisation of a larger region of the sample. © 2013 Optical Society of America OCIS codes: (110.0180) Microscopy; (110.6880) Three dimensional image acquisition; (230.3720) Liquid-crystal devices. References and links 1. D. B. Murphy, Fundamentals of Light Microscopy and Electronic Imaging, 1st ed. (Wiley-Blackwell, 2001). 2. C. Maurer, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “What spatial light modulators can do for optical mi- croscopy," Laser Photon. Rev. 5(1), 81–101 (2011). 3. T. J. McIntyre, C. Maurer, S. Bernet, and M. Ritsch-Marte, “Differential interference contrast imaging using a spatial light modulator," Opt. Lett. 34(19), 2988–2900 (2009). 4. C. Maurer, S. Khan, S. Fassl, S. Bernet, and M. Ritsch-Marte, “Depth of field multiplexing in microscopy," Opt. Express 18(3), 3023–3034 (2010). 5. S. Fürhapter, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Spiral phase contrast imaging in microscopy," Opt. Express 13(3), 689–694 (2005). 6. J. A. Davis, D. E. McNamara, D. M. Cottrell, and J. Campos, “Image processing with the radial Hilbert transform: theory and experiments," Opt. Lett. 25(2), 99–101 (2000). 7. C.-S. Guo, Y.-J. Han, J.-B. Xu, and J. Ding, “Radial Hilbert transform with Laguerre-Gaussian spatial filters," Opt. lett. 31(10), 1394–1396 (2006). 8. G. Situ, G. Pedrini, and W. Osten, “Spiral phase filtering and orientation-selective edge detection/enhancement," J. Opt. Soc. Am. A. 26(8), 1788–1797 (2009). 9. R. Bowman, D. Preece, G. Gibson, and M. Padgett, “Stereoscopic particle tracking for 3D touch, vision and closed-loop control in optical tweezers," J. Opt. 13, 044003 (2011). #189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013 (C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16541

Upload: others

Post on 11-Jul-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

A multi-modal stereo microscope basedon a spatial light modulator

M. P. Lee,1,∗

G. M. Gibson,1 R. Bowman,2 S. Bernet,3

M. Ritsch-Marte,3 D. B. Phillips4 and M. J. Padgett1

1School of Physics and Astronomy, SUPA, University of Glasgow, G12 8QQ, UK2Queens’ College, University of Cambridge, CB3 9LE, UK

3Division for Biomedical Physics, Innsbruck Medical University, A-6020 Innsbrusck, Austria4H. H. Wills Physics Laboratory, University of Bristol BS8 1TL, UK

[email protected]

http://www.gla.ac.uk/schools/physics/research/groups/optics

Abstract: Spatial Light Modulators (SLMs) can emulate the classic mi-croscopy techniques, including differential interference (DIC) contrast and(spiral) phase contrast. Their programmability entails the benefit of flexibil-ity or the option to multiplex images, for single-shot quantitative imagingor for simultaneous multi-plane imaging (depth-of-field multiplexing). Wereport the development of a microscope sharing many of the previouslydemonstrated capabilities, within a holographic implementation of a stereomicroscope. Furthermore, we use the SLM to combine stereo microscopywith a refocusing filter and with a darkfield filter. The instrument is builtaround a custom inverted microscope and equipped with an SLM whichgives various imaging modes laterally displaced on the same camera chip.In addition, there is a wide angle camera for visualisation of a larger regionof the sample.

© 2013 Optical Society of America

OCIS codes: (110.0180) Microscopy; (110.6880) Three dimensional image acquisition;(230.3720) Liquid-crystal devices.

References and links1. D. B. Murphy, Fundamentals of Light Microscopy and Electronic Imaging, 1st ed. (Wiley-Blackwell, 2001).2. C. Maurer, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “What spatial light modulators can do for optical mi-

croscopy," Laser Photon. Rev. 5(1), 81–101 (2011).3. T. J. McIntyre, C. Maurer, S. Bernet, and M. Ritsch-Marte, “Differential interference contrast imaging using a

spatial light modulator," Opt. Lett. 34(19), 2988–2900 (2009).4. C. Maurer, S. Khan, S. Fassl, S. Bernet, and M. Ritsch-Marte, “Depth of field multiplexing in microscopy," Opt.

Express 18(3), 3023–3034 (2010).5. S. Fürhapter, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Spiral phase contrast imaging in microscopy," Opt.

Express 13(3), 689–694 (2005).6. J. A. Davis, D. E. McNamara, D. M. Cottrell, and J. Campos, “Image processing with the radial Hilbert transform:

theory and experiments," Opt. Lett. 25(2), 99–101 (2000).7. C.-S. Guo, Y.-J. Han, J.-B. Xu, and J. Ding, “Radial Hilbert transform with Laguerre-Gaussian spatial filters,"

Opt. lett. 31(10), 1394–1396 (2006).8. G. Situ, G. Pedrini, and W. Osten, “Spiral phase filtering and orientation-selective edge detection/enhancement,"

J. Opt. Soc. Am. A. 26(8), 1788–1797 (2009).9. R. Bowman, D. Preece, G. Gibson, and M. Padgett, “Stereoscopic particle tracking for 3D touch, vision and

closed-loop control in optical tweezers," J. Opt. 13, 044003 (2011).

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16541

Page 2: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

10. J. S. Dam, I. R. Perch-Nielsen, D. Palima, and J. Glückstad, “Three-Dimensional optical multi-beam microma-nipulation," Opt. Express 16(10), 7244–7250 (2008)

11. M. Hasler, T. Haist, and W. Osten, “Stereo vision in spatial-light-modulator–based microscopy," Opt. Lett.37(12), 2238–2240 (2012).

12. G. Grover, K. DeLuca, S. Quirin, J. DeLuca, and R. Piestun, “Super-resolution photon-efficient imaging by nano-metric double-helix point spread function localization of emitters (SPINDLE)," Opt. Express 20(24), 26,681–26,695 (2012).

13. Z. Wang, L. Millet, M. Mir, H. Ding, S. Unarunotai, J. Rogers, M. U. Gillette, and G. Popescu, “Spatial lightinterference microscopy (SLIM)," Opt. Express 19(2), 1016–1026 (2011).

14. B. Bhaduri, D. Wickland, R. Wang, V. Chan, R. Bashir, and G. Popescu, “Cardiomyocyte imaging using real-timespatial light interference microscopy (SLIM)," PloS one 8(2), e56930 (2013).

15. P. M. Blanchard and A. H. Greenaway, “Simultaneous multiplane imaging with a distorted diffraction grating,"Appl. Opt. 38(32), 6692–6699 (1999).

16. H. Sierra, J.-Y. Zheng, B. Rabin, and N. N. Boustany, “Measurement of object structure from size-encodedimages generated by optically-implemented Gabor filters," Opt. Express 20(27), 28,698–28,706 (2012).

17. A. Greengard, Y. Y. Schechner, and R. Piestun, “Depth from diffracted rotation," Opt. lett. 31(2), 181–183 (2006).18. M. D. Lew, S. F. Lee, M. Badieirostami, and W. Moerner, “Corkscrew point spread function for far-field three-

dimensional nanoscale localization of pointlike objects," Opt. lett. 36(2), 202–204 (2011).19. L. Camacho, V. Micó, Z. Zalevsky, and J. Garcí, “Quantitative phase microscopy using defocusing by means of

a spatial light modulator," Opt. Express 18(7), 6755–6766 (2010).20. G. M. Gibson, R. W. Bowman, A. Linnenberger, M. Dienerowitz, D. B. Phillips, D. M. Carberry, M. J. Miles,

and M. J. Padgett, “A compact holographic optical tweezers instrument," Rev. Sci. Instrum. 83, 113107 (2012).21. D. G. Grier, “A revolution in optical manipulation," Nature 424(6950), 810–816 (2003).22. M. P. Lee and M. J. Padgett, “Optical tweezers: a light touch," J. Microsc. 248(3), 219–222 (2012).23. E. R. Dowski, W. T. Cathey, “Extended depth of field through wave-front coding," Appl. Opt. 34(11), 1859–1866

(1995).24. T.-C. Poon and M. Motamedi, “Optical/digital incoherent image processing for extended depth of field," Appl.

Opt. 26(21), 4612–4615 (1987).25. G. A. Siviloglou, J. Broky, A. Dogariu, and D. N. Christodoulides, “Observation of accelerating airy beams,"

Phys. Rev. Lett. 99, 213,901 (2007).26. G. Sinclair, P. Jordan, J. Leach, M. Padgett, and J. Cooper, “Defining the trapping limits of holographical optical

tweezers," J. Mod. Opt. 51(3), 409–414 (2004).27. J. Leach, K. Wulff, G. Sinclair, P. Jordan, J. Courtial, L. Thomson, G. Gibson, K. Karunwi, J. Cooper, Z. J.

Laczik, and M. Padgett, “Interactive approach to optical tweezers control," Appl. Opt. 45(5), 897–903 (2006).28. F. Zernike, “How I discovered phase contrast," Science 121(3141), 345–349 (1955).29. C. Maurer, A. Jesacher, S. Bernet, and M. Ritsch-Marte, “Phase contrast microscopy with full numerical aperture

illumination," Opt. Express 16(24), 19,821–19,829 (2008).30. R. Bowman, V. D’Ambrosio, E. Rubino, O. Jedrkiewicz, P. Di Trapani, and M. Padgett, “Optimisation of a low

cost SLM for diffraction efficiency and ghost order suppression," Eur. Phys. J. Special Topics 199, 149–158(2011).

31. K. D. Wulff, D. G. Cole, R. L. Clark, R. Di Leonardo, J. Leach, J. Cooper, G. Gibson, and M. J. Padgett,“Aberration correction in holographic optical tweezers," Opt. Express 14(9), 4170–4175 (2006).

32. R. W. Bowman, A. J. Wright, and M. J. Padgett, “An SLM-based Shack–Hartmann wavefront sensor for aberra-tion correction in optical tweezers," J. Opt. 12(12), 124,004 (2010).

33. D. Phillips, S. Simpson, J. Grieve, R. Bowman, G. Gibson, M. Padgett, J. Rarity, S. Hanna, M. Miles, andD. Carberry, “Force sensing with a shaped dielectric micro-tool," Europhys. Lett. 99(5), 58,004 (2012).

1. Introduction

Optical microscopes are indispensable tools in a vast range of laboratories [1]. Spatial LightModulators (SLMs) offer remarkable control over a given light field. In optical microscopy,this control has been used to create forms of microscopy more typically implemented withdifferent physical apertures and filters in the optical path. SLMs positioned in the Fourier planebring a flexibility to optical microscopy which would be difficult to realise with traditionaltechniques [2–8]. In parallel with these developments, we reported a new approach to stereomicroscopy [9]. Normally, a stereo microscope comprises of two objective lenses and a singlelight source. In our previous approach we used a single objective lens and two light sourcescombined with a bi-prism in the Fourier plane. This has also been achieved with two differentcolours of illumination [10]. More recently, the bi-prism was replaced by an SLM, albeit with a

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16542

Page 3: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

Cam

era

PolarisingBeamsplitter

Tube Lensf =120mm

x40, 1.35NAOlympus Objective

Z-Axis

Z-Axis

BandpassFilter

SampleSlide

Light Guides

f = 150mm

f =200mm

Camera

f = 50mm f = 50mm

SLM

Iris

SLM

Camera

Camera

TubeLens

Objective

Fig. 1. The configuration of our SLM microscope. Shown at 45◦ is the z-axis with the in-verted microscope and illumination. The polarising beamsplitter is used so that the verticalpolarisation of the light goes through the intermediate image iris and on to the SLM, wherethe Fourier filter is displayed. There is then a 10nm bandpass filter so that dispersion iskept to a minimum in the sample images recorded by the camera. The horizontally po-larised light forms the full image of the sample on the wide angle camera. Inset shows a 3Ddrawing of the system at a scale of approximately 10 : 1.

conventional light source [11]. In this work, we use two light sources and a SLM acting as a bi-prism to create a stereo microscope that is also capable of darkfield and defocus in conjunctionwith stereo microscopy. This new technique improves 3D visualisation, extends the depth offield and could allow stereo microscopy to be used for more weakly scattering objects. Ourset up is compatible for using other filters, such as spiral phase contrast [5], double-helix pointspread functions [12], Spatial Light Interference Microscopy (SLIM) [13, 14], depth of fieldmultiplexing [4,15] and Gabor filters [16] amongst others [17–19]. Not only can different filtersbe applied without changing any hardware, but several filters can be applied at the same timeto give multiple modalities simultaneously, offering a unique advantage over any traditionaloptical element.

2. Design

The inverted microscope design, Figs. 1 and 2, is similar to that of our recent work developinga compact optical tweezer system [20]. Focussing is achieved by moving the objective lens ona motorised stage (LS-50-M, ASI) which is mounted vertically within the aluminium frame ofthe microscope. Similarly, translation of the microscope sample is achieved using a motorisedX-Y stage (MS-2000, ASI). Both these stages are controlled with the ASI MS-2000-WK Multi-

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16543

Page 4: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

Fig. 2. Photograph of the system. Visible is the optical fibre illumination, motorised stage,SLM and CMOS cameras. The footprint of the system is 45cm × 30cm, and the height isapproximately 35cm.

Axis Stage Controller. High power red LEDs (LUXEON Rebel) coupled through the condenserusing a short length of acrylic fibre (core diameter = 1.5mm) provide the illumination. A crit-ical illumination condenser provides a uniform illumination of the sample. The output face ofthe fibre is imaged on to the sample, with a variable aperture to give brightness and spatialcoherence control. An alternative illumination consists of three acrylic fibres positioned a fewmillimetres from the sample: one on axis, and two at 30◦ for stereoscopic operation [9]. For thisillumination, some control of the Numerical Aperture (NA) is given by raising and lowering thefibres, however careful consideration of scattered light from outside the field of view is neededat low NA. Switching between the on axis and stereo illumination is achieved through USBcontrol.

A polarising beamsplitter (PBS) splits the light between two arms, one containing a camerafor wide field viewing of the sample and the other arm containing the SLM (Boulder Nonlin-ear Systems, X-Y Series (512× 512 pixels)). The reflected light from the PBS is the correctpolarisation for the SLM. In order to avoid problems with dispersion when illuminating theSLM using a LED, a bandpass interference filter (636nm, 10nm) is placed in the optical pathafter the SLM. The light diffracted from the SLM is imaged on to a second CMOS camera(Dalsa Genie, HM1020) which has a large sensor area, capable of imaging all diffracted im-ages simultaneously. The overall optical configuration is similar to that of holographic opticaltweezers [21,22], where the diffraction from the SLM creates multiple, laterally displaced trap-ping beams. Here, the SLM now produces displaced images and, just as the trapping beams canbe of different beam types, the images can correspond to different Fourier filter functions.

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16544

Page 5: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]̂_̀abcdefghijklmnopqrstuvwxyz{|}~

Fig. 3. Illustrative intensity distributions of a 5 μm silica particle in water. The particle isstuck to the coverslip and is moved through the focus of the microscope. A series of imageswere recorded at 0.5 μm spacing in the axial direction, then four intensity isosurfaces wereinterpolated from the data (shown in shades of red). In the x− y plane of each plot is anillustrative phase hologram used to Fourier filter the image. A darkfield filter has been usedin (a), an annulus filter has been used in (b) and a cubic filter has been used in (c). Theannular filter extends the depth of field of the imaging system. The cubic filter also extendsthe depth of field but in addition introduces a curved intensity profile. All these point spreadfunctions were introduced by the SLM.

3. SLM Fourier filters

Fourier filtering is a common technique applied images to enhance desired information of asample object [23]. The role of the SLM in our system is to generate different imaging modali-ties by modulating both the phase and amplitude of the light in the Fourier plane. For example,Fig. 3 shows the image intensity as a function of defocus whilst displaying a darkfield filter, anannular filter or a cubic filter on the SLM. The annular filter is useful as it gives an extendeddepth of field [24], while the cubic filter produces an Airy point spread function [25]. For eachof the following patterns, we use a grating to diffract an image far enough from the zero order(undiffracted) image so that it does not overlap with it. For this to be possible, we first needto partially close the intermediate image iris so that the cone angle of each diffracted image isless than the diffraction angle given by the SLM. As with holographic optical tweezers, wherethe effective pixel size of the SLM sets an upper limit to the field of view over which traps canbe positioned [26], the SLM resolution sets the maximum field of view of the images not tooverlap in the camera plane.

As previously reported [27], the ideal kinoform describing the pattern to be displayed on theSLM for a particular imaging mode, φm, is given by,

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16545

Page 6: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]̂_̀abcdefghijklmnopqrstuvwxyz{|}~

Combine

Displayed on SLM

Camera image

a)

b)

c)

d)

e)

Fig. 4. Shown here are results from combining the Fourier filters as described in the textwhile imaging three 5 μm silica spheres suspended in optical adhesive. (a) - (c) Shows indi-vidual phase patterns for three imaging modes, namely, double-helix point spread function,defocus and darkfield, respectively. Aberration correction has been incorporated with thesefilters and each also includes a grating to diffract the image. The greyscale correspondsto 0− 2π phase changes. (d) The SLM display, showing the combined filter as describedby Equation 2. (e) The resulting image detected by the camera. We extract the subregionsof the image indicated, each 220× 220 pixels. The subregions correspond to each filter;top left: double helix, bottom left: defocus, right: darkfield, centre: undiffracted zero orderimage.

φm = arg[ei(αu+βv+φFilter)], (1)

where u and v are, respectively, the vertical and horizontal distances from the centre of the SLM,α and β are constants which define the angle of diffraction and φFilter is the filter function.There are a wide range of different filter types ranging from simple implementations of defocus[4] and darkfield to more elaborate phase contrast techniques [13, 28, 29] and vortex phasemasks [5].

One advantage of the SLM implementation is that it is possible to diffract several imagemodalities at different angles by combining their phase patterns. Several images can be pro-duced simultaneously by the complex addition of individual kinoforms, the argument of whichgives the combined kinoform and the simultaneous images,

φcombined = arg

[M

∑m=1

Rmeiφm

], (2)

where m is the imaging mode produced by the SLM, M is the number of diffracted images,and R2

m is the relative intensity of the image. The potential interference between overlappingimages is minimised by suppression of ghost diffraction orders by directing unwanted lightinto the zero order [30]. This multi-modal approach is made possible by the availability oflarge sensor arrays capable of working over a large dynamic range inherently arising fromthe different filter types. The design of the SLM filter to produce these images is exactly theapproach used in holographic optical tweezers to produce multiple trapping beams, where each

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16546

Page 7: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]̂_̀abcdefghijklmnopqrstuvwxyz{|}~

10μm

a)

b) c)

d)e)

f)

Fig. 5. (a) A darkfield image of a dry sample of 200nm diameter gold particles stuck to acoverslip. (b) and (c) Images of a two photon polymerised star shape in water. Image (b) hashad no aberration correction, while (c) has had aberration correction. (d) is a cross sectionthroughout the point spread function of the microscope, obtained by imaging a 200nmgold particle in darkfield. Aberration correction is seen to improve contrast by 30%. Theinsets in (d) show contrast enhanced lateral point spread functions where (e) is aberrationcorrected and (f) uncorrected. A horizontal line indicates the cross sections in (d).

beam type can be different. Figure 4 shows a set of multi modal images.A possible drawback of a microscope incorporating an SLM in the optical path is that the

SLM itself introduces aberrations as it is not optically flat. However, this can be corrected forwith a corresponding phase pattern displayed on the SLM. Not only can we correct for aberra-tions of the SLM itself, but also other aberrations in the optical train or even within a sample.We do this by manually tuning coefficients of Zerinke polynomials whilst imaging a knownobject, typically a 2 μm silica bead. With adjusting the focus of the microscope and iteratingthe coefficients, we were able to dramatically improve the image and remove the worst of thesystem’s aberration [31]. Figure 5 gives details of the aberration correction. Whilst successfulin correcting the system aberrations, imaging an unknown object through a dynamic aberratinglayer would require some kind of reference, such as a focused laser spot in the object plane.The system could then be used as a SLM-based Shack-Hartman wavefront sensor [32].

3.1. Holographic stereo microscopy

As discussed, Bowman et al. [9] created a novel microscope, optimised for 3D visualisationand particle tracking. The approach uses a single microscope objective lens and a custom illu-mination consisting of two optical fibres at an angle around 30◦ to the optical axis, yielding twoviews of the sample. This method enables the three dimensional tracking of particles withoutthe need for complex beam steering optics or the requirement for pre-recorded template images.It is also easily scaled to track multiple particles simultaneously, and has been employed to trackthe motion of non-spherical particles in a novel form of scanning probe microscopy [33].

The two images are from different angles of illumination, and, just as parallax vision givesus depth perception, depth information can be obtained from the images. A particle moving inz direction would move from left to right in one image and from right to left in the other. It issimple then to attain the z displacement, δ z, as,

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16547

Page 8: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]̂_̀abcdefghijklmnopqrstuvwxyz{|}~

a)

b)

2μm

c)

d)

5μm

e) f)g)

Fig. 6. 3D position coordinates can be obtained using the software’s particle tracking ca-pabilities. (a) An image of the SLM display where the greyscale represents 0− 2π phasechange. Two apertures can be seen corresponding to the two views of the sample, each witha different grating so that the images from these angles don’t overlap in the image plane. (b)and (c) The different images extracted from an image recorded on the CMOS camera. Theimages are of the same object, 2 μm polystyrene spheres undergoing Brownian motion inwater. (d) Stereo visualisation of particle positions. The images (b) and (c) are colour codedand overlapped. With 3D stereo glasses, the user can visualise the scene with depth percep-tion. Images (e) and (f) are of the same cheek epithelial cell nucleus as viewed with thestereo microscope. Viewing the sample from two directions affords additional geometricinformation about the cell structure.

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16548

Page 9: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

δ z =δx

2tanθ, (3)

where δx is the difference in the x position of the particle in the two images and θ is the angleof the optical fibres to the vertical axis (i.e. the fibres are separated by 2θ ).

Illuminating at two angles corresponds to two positions in the Fourier plane and aroundthese positions is the scattered light from the particles in the sample. For each of these lightfields, a wedge prism deflects the light so that they do not overlap in the image plane. Also,a twin circular aperture needs to be placed just in front of the prisms in order to restrict highangled scattering from one view contributing to the image for the other view. In [11], Hasleret al. an SLM was used to create stereo images with just a single, on axis, illumination. Asdiscussed by the authors, this approach leads to an inversion of the image for some samplesowing to the division of the scattered field in the Fourier plane. By using two light sources weeffectively have two separate light fields which, in the Fourier plane, only overlap at high spatialfrequencies. By blocking these areas with the twin circular aperture we can obtain imagessuitable for particle tracking and free from inversion, albeit with a high frequency cutoff.

The twin aperture needs to be designed for a particular magnification or illumination angle,θ , as these set the position of the light fields in the Fourier plane. With the use of an SLM here,we can make the stereo microscope more flexible, see Fig. 6. With no mechanical change to thesystem described in section 2, the SLM can be used in place of the twin apertures and prisms.The size and position of the apertures on the SLM are now reconfigurable, meaning we canswitch between microscope objectives and have corresponding SLM phase patterns ready tocompensate for the new position and size of light fields for each view. We can also experimentwith different angles of illumination and adjust the SLM display accordingly, which may bemore suited to particular applications. The approach works well for particle tracking spheres,but it’s also possible to image extended objects, Figs. 6(e) and 6(f). Viewing from two differentdirections can give additional information of the shape of the structure, however, the largeseparation angle between the two views means 3D visualisation of such objects is difficult (ifthe images were to be colour coded and overlapped).

4. Combination of holographic stereo microscopy with holographic lenses

As a demonstration of the flexibility of the system, we combine our holographic stereo mi-croscopy with holographic lenses and darkfield imaging. We image a 5 μm silica bead withstereo illumination and an objective 100×, NA = 1.3 (Zeiss), Figs. 7(a) and 7(b). With stereomicroscopy, particle tracking accuracy and precision is limited by the defocusing of the trackedobject: the more the particle moves away from the focal plane, the less reliably can we mea-sure its position. Adjusting the focus of the objective lens to keep the particle in focus duringa measurement is not always desirable. Adjusting the focus holographically has the advantageof having no mechanical movements, as well as maintaining the parallax displacement if sep-arate lenses are aligned to the axis of each view. This means we can focus the image withoutcausing a lateral shift, i.e. the measured x position in each view is unchanged. The range of thistechnique is now limited by the field of view, rather than the depth of focus of the microscope.In Fig. 8, we show the improvement to the accuracy of the z position measurement as a func-tion of distance from the focal plane. Without the lens correction, the tracking error increasesquickly outside of the depth of focus of the microscope [9]. However, with the application ofholographic lens correction, the error falls to a mean of 5.5nm across a 50 μm range.

We note that this stereo visualisation approach has parallels with human vision, where eacheye has its own lens. This addition of a lens could be of benefit to 3D tracking with a closed loopadjustment of the focus to optimise the trackability of a particle without changing its position

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16549

Page 10: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]̂_̀abcdefghijklmnopqrstuvwxyz{|}~

Fig. 7. Recorded images of a 5 μm silica bead fixed in position on a microscope slide. Weilluminated the bead with stereo illumination, so that the bead translates as we adjust thefocus of the microscope objective over a 35 μm range. We diffract the left and right views todifferent positions on the camera then extract, colour and overlap the images. (a) The SLMdisplay along with stereo images as the bead goes through the focus of the microscope. (b)the SLM display used to refocus the bead as we change the focus of the objective lens. Therefocusing does not change the position of the bead. A different holographic lens is neededfor each focal position, shown is an example of the lens used at the 10 μm position. (c)Darkfield stereo images. (d) Darkfield stereo with focus correction.

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16550

Page 11: A multi-modal stereo microscope based on a spatial light ...bnonlinear.com/pub/microscopy/Multi_modal_stereo... · A multi-modal stereo microscope based on a spatial light modulator

!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]̂_̀abcdefghijklmnopqrstuvwxyz{|}~

Fig. 8. Measurement of the accuracy of z position measurement as a function of distancefrom the focal plane. The error is calculated from measuring the standard deviation ofa series of position measurements of a 5 μm silica particle stuck on the coverslip. Theimprovement is due to the increased contrast brought about by keeping the particle in focus.

in the image.Again, the flexibility of the SLM implementation means that we can introduce a darkfield

filter to the stereo imaging with no additional system complexity, Figs. 7(c) and 7(d). A useof the dark-field stereo could be improving the contrast for the 3D trajectory tracking of indi-vidual water-borne single-celled organisms. In principle, all the benefits of using an SLM inthe Fourier plane can be transferred to the stereo microscope, however care may be neededregarding the coherence of the illumination.

5. Conclusions

We have developed a multi-modal microscope capable of stereo imaging, aberration correc-tion and a range of other imaging modalities. The microscope uses a spatial light modulatorto Fourier filter the images and generate different imaging modalities laterally displaced onthe same camera. Depending on the application, existing filters can be optimised or new filterscreated and combined with stereo imaging. Novel Fourier filters can be developed with soft-ware and tried out without additional hardware components. As an example, we demonstratedholographic stereo microscopy, and used the flexibility of the SLM to improve the range of thetechnique with the addition of a holographic lens, and holographic stereo darkfield microscopy,which could allow stereo microscopy to be used with weakly scattering objects.

Acknowledgments

MJP acknowledges support from EPSRC, Boulder Nonlinear Systems, and the Royal Society.

#189935 - $15.00 USD Received 6 May 2013; revised 11 Jun 2013; accepted 14 Jun 2013; published 2 Jul 2013(C) 2013 OSA 15 July 2013 | Vol. 21, No. 14 | DOI:10.1364/OE.21.016541 | OPTICS EXPRESS 16551