Shadow Removal from Image of Stained Glass Windows

Shanmugalingam Suganthan,1 Lindsay MacDonald2

1 Smart Sensors Ltd, University of Bath Innovation Centre, Bath, UK

2 London College of Communication, University of Arts London, London, UK

Received 2 July 2008; revised 15 April 2010

ABSTRACT: Shadows may be formed on stained glass windows by

structural bars supporting the leaded panels, or by external protectivewire grilles, or by masonry, such as mullions or buttresses, or external

objects, such as trees. The eye tends to ‘‘discount’’ such shadow for-

mations when viewing the actual windows even though in the photo-graphic images they are very clearly visible. This article introduces a

method to remove shadow effects on stained-glass windows; the

observed image, as captured by the camera, may be modeled math-

ematically as a combination of a ‘‘true stained glass image’’ and a‘‘grille/bar image.’’ A mixture model is derived, based on a theoretical

model of image formation, leading to a conjectured relationship

between ‘‘shadow’’ pixels and the neighboring ‘‘nonshadow’’ pixels.

The resulting mixture model assumes a multiplicative relationship. Ifthis mixture can be separated into its original components, then it

should be possible to remove the unwanted shadow component from

the captured image to produce the desired image of the stained glasswithout the shadows. The digital modeling techniques enable the

shadows to be characterized and removed with a reasonable degree

of success. VVC 2010 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 20,

223–236, 2010; View this article online at wileyonlinelibrary.com. DOI 10.1002/

ima.20241

Key words: stained glass windows; shadow removal; mixturemodel; image segmentation

I. INTRODUCTION

The term ‘‘stained glass’’ refers either to the material of colored

glass or to the art and craft of working with it. Throughout its 1000

years history, the term stained glass was applied almost exclusively

to the windows of churches, cathedrals, and other significant build-

ings. Stained glass has been an important medium for the illustra-

tion of European culture and history and is still used today in archi-

tecture and the decorate arts. The properties of transparency, malle-

ability, and hardness make it perfect to be painted, thus creating a

window with wide gamut of brightly colored artwork seen against a

constantly changing daylight backdrop. These colorful painted win-

dows were used as principal design features especially in church

architecture. Stained glass windows were originally designed to fit

into gaps within the structural masonry of churches.

Stained glass windows are continued to be built today. The ear-

liest know stained glass windows date from the 7th century

(Osborne, 1997) and while the craft had its hcyday in the 14th cen-

tury (Armitage, 1959; Osborne, 1997). Although glassmaking tech-

nology has changed a great deal since the middle ages, the proce-

dures for designing windows have changed. Then, as now, the full

outline of the design, or cartoon, for a stained glass window is

drawn before construction (Brown, 1998). In the past, these car-

toons were sketched directly onto tabletops and are now drawn on

article. Within the cartoon, the designer is able to indicate not only

the principal outlines of the work but also the shape and color of the

individual pieces of glass to be used, along with the position of the

lead strips (called calmes from the Latin calmus (Sloan, 1993)) that

hold the work together. In time, a unique range of colors could be

achieved using enamels of differing pigments, which allow rich

details to be painted using clear glass window possesses a distinc-

tive style partly because of the unique color ranges produced

through the interaction of color enamels, glass, and light (see Fig.

1). The imposition of calmes further separates the appearance of

stained glass from other mediums. In computer graphics literatures

(Mould, 2003; Brooks, 2006; Seo et al., 2007) addressed transform-

ing an arbitrary image into a stained-glass version of that image;

the key issues in designing a simulation of the stained glass are the

title boundaries and tile colors.

Stained glass has been photographed over the past 150 years

using a range of techniques (Findlater and MacDonald, 2001a).

Stained glass panels can be photographed either under controlled

lighting conditions in a studio, for example when they have been

removed for restoration, or in their normal architectural setting in a

building. In a studio, the illumination can be selected and con-

trolled, although with limited space the window may have to be

photographed as separated panels or tracery segments. The alterna-

tive of photography in situ usually enables each stained glass win-

dow to be photographed entirely in a single image.

The physical properties of glass may influence the imaging pro-

cess. These could include any color change over a period of time,

corrosion, erosion of paint, breakages, fragility, and durability. Par-

adoxically, whilst glass can be very fragile and may break or shatter

very easily, unbroken glass can remain durable for centuries. Rele-

vant physical properties of stained glass include opacity, transpar-

ency, color appearance, size and thickness, chemical composition,Correspondence to: Shanmugalingam Suganthan; e-mail: s.suganthan@ieee.org

' 2010 Wiley Periodicals, Inc.

refractive index, age, and method of production. In addition, surface

properties include hardness, flatness, gloss, and susceptibility to

corrosion. There are various ways in which glass can be colored

using additives and heating methods as well as direct surface paint-

ing. The physical color achieved depends on the properties and

thickness of the paints and glass used (Weyl, 1976). The general

appearance of stained glass depends on both interior and exterior

illumination, including variable daylight conditions; the properties

of the glass itself and the effects of any obstructions to the light. All

these properties affect image capture through both photographic and

digital means, including the following (Findlater and MacDonald,

2001b): the glass absorbs varying amounts of light, affecting its

tone and color; the properties of backlight illumination affect the

appearance of the glass; the transparent medium allows some of the

background to be seen; Shadows may be cast onto the surface from

exterior objects, such as wire grilles, buttresses, trees, and so forth;

Ambient interior lighting can render surface detail of the stained

glass, such as the paint layer and the surrounding lead work.

Heritage conservators are interested in using digital images in

the analysis of stained glass windows. Image of stained glass are

significantly different from those of most other subjects because the

colors of stained glass windows are indeed dominantly caused by

light transmission, however, there are also relatively small amount

of reflection because of ambient light or inter reflection in the

observer’s place, which is usually an indoor room. In many cases

the background is visible through the glass, typically trees, foliage,

sky, or other buildings. Because glass is translucent, image of

stained glass windows taken with external illumination often con-

tain shadows, moreover, cast by structures, such as support bars

called ‘‘ferramenta’’ and protective wire grilles (Fig. 1). The eye

tends to ‘‘discount’’ such shadow formations when viewing the

actual windows, but in photographic images they are clearly visible.

The physical structures producing the shadows are often irremov-

able, because they are difficult to access or constitute structural ele-

ments of the window. It is thus necessary to provide a suitable set of

image processing tools to remove shadows from the digital images.

Shadow removal is a critical problem in image processing. In

some literatures (Prati et al., 2003; Salvador et al., 2004; Levine

and Bhattacharyya, 2005; Finlayson et al., 2006; Leone et al.,

2006), the problem of shadow detection and removal in both still

images and video sequences has been intensely investigated in the

last few years within different application contexts. A shadow is

cast in a scene when an object lies in the path of the direct illumina-

tion source. If a scene is illuminated by two or more sources, then

the shadow and nonshadow regions of an object may differ not only

in terms of their relative brightness but also in terms of their relative

color.

In this article is organized as follows. In Section II, image seg-

mentation of stained glass described. The multiplicative mixture

model for shadows is described in Section III. The bra shadow re-

moval and grill shadow removal discussed in Section IV and V,

respectively. Finally, Section VI we draw the conclusions.

II. IMAGE SEGMENTATION OF STAINED GLASS

Image segmentation provides useful information for discriminating

regions of glass and metallic lead (calmes) within the images. Once

this information is available, the development of image processing

algorithms for images of stained glass windows may be achieved

under less stringent constraints. The information is also useful to

assist conservators in the analysis and restoration of stained glass

panels, by allowing them to visualize the effects of repainting and

reconstruction before physically undertaking the work.

Digital images of stained glass panels were investigated to deter-

mine the morphological calmes structure, with a view to segmenta-

tion of the panels into their constituent pieces of glass. This process

is complicated by dark painted lines and textured areas on the glass.

In a purely transmissive image these may easily be confused with

the contours of the lead calmes, which are opaque, and therefore,

appear black. In practice the metallic lead also reflects some of the

ambient light, making it off-black (dark grey, brown, or blue) in the

image. The calmes structure is characterized by different tiles of

colored glass, with rather arbitrary shapes but limited number of

transmissive body colors, and the linear patterns of the opaque

calmes. The paintwork overlaid on the glass has a higher level of

detail and minimal color variation (typically dark grey or brown).

This gives a set of specific problems in terms of pattern recognition

and image processing.

The investigation described in this section tackled the task using

well-known image processing techniques. Section IIA describes the

threshold technique. Template matching and the Gabor filter are

described in Section IIB and IIC, respectively.

A. Thresholding. To segment the structure formed by the calmes

in transmissive light, several image processing strategies suggest

themselves. The colored glass tiles clearly define bounded clusters

of relatively high saturation and luminosity, whereas the calmes

present a very low level of luminosity, and a bluish, desaturated tint

given by the typically weak ambient light reflected from the surface

of the lead. Therefore, thresholding techniques (Sahoo et al., 1988)

in some suitable color space may prove sufficient to segment the

calmes from the glass background.

This approach suffers from major limitations. It is overly sim-

plistic to treat the glass tiles as areas of uniform color characteris-

tics, when in fact corrosion, dirt. and the deposition of opaque ele-

ments weakens the transmitted light and spreads the distribution of

the pixel lightness values to over lap those of the calmes. On the

other hand, scattering phenomena and diffuse illumination could

cause the tint of the calmes to drift far from the expected value,

especially at a boundary with a bright region. Paintwork represents

the other major limitation of this approach, as the values in the

color space belonging to the painted area might not be separable

from those of the calmes.

Figure 1. Shadows of horizontal support bars in a stained glass

window. The window is in the chancel of the Church of St. Mary,

Studley Royal, Yorkshire, UK. [Color figure can be viewed in the onlineissue, which is available at wileyonlinelibrary.com.]

224 Vol. 20, 223–236 (2010)

B. Template Matching. One of the main characteristics of the

calmes is that they have a nearly constant width. Therefore, one

should explore ways of exploiting this information, which is not

generally characteristic of the background areas that overlap the

calmes in the color space. A very simple method of segmenting

structures of give width is represented by Template Matching tech-

niques (Jain et al., 1995), by defining a binary template mask repre-

senting the average calmes width in a properly sized window and

successively convolving the image with this mask. It is necessary to

define as many template masks as the number of orientations one

wishes to detect. Figure 2 shows eight template masks, representing

eight possible calmes orientations.

Template matching is simple technique with an intuitive rela-

tionship with the goal of the segmentation. From a more analytical

point of view, however, template matching may not offer enough

flexibility. A more principled approach is constituted by use of the

Gabor filter (Gabor, 1946).

C. Gabor Filter. The Gabor filter has received considerable atten-

tion because the characteristics of certain cells in the visual cortex

of some mammals can be approximated by these filters. Gabor fil-

ters have been widely applied in texture analysis (Dunn et al., 1994;

Weldon et al., 1996) to extract spatial information from the image.

Of relevance for stained glass window segmentation is the applica-

tion of the Gabor filter to fingerprint ridge structures (Hong et al.,

1996) and to craquelure analysis of paintings on canvas (Abas and

Martinez, 2002). Unlike the template matching masks illustrated

above, the Gabor filter can be tuned in both space and frequency.

The Gabor filter may be expressed by:

hðx; y; f ; uÞ ¼ exp � 1

2

x02

d2xþ y02

d2y

!" #cosð2pfx0Þ ð1Þ

where:

x0 ¼ x sinðuÞ þ y cosðuÞy0 ¼ x cosðuÞ � y sinðuÞ

So the Gabor filter kernel is a sine wave of frequency f, rotatedby an angle y and enveloped by an elliptical Gaussian function with

width parameters dx and dy. Figure 3 shows the Gabor filter kernel.

The corresponding frequency spectrum (in the first quadrant of the

Fourier domain) is given by:

h fx; fy; f ; u� � ¼ d2xd

2yexp 2

d2x fx � fxcos uð Þ½ �þd2y fy � fysin uð Þ� �� �� �

ð2Þ

Therefore, a Gabor filter can be considered to be band-limited in

both space and frequency. The spatial bandwidth and the frequency

bandwidth are expressed, respectively, by dx, dy and dy21, dy

21.

Eight values were used for y, corresponding to the orientations

y 5 08, 22.58, 458, 67.58, 908, 112.58, 1358, and 157.58. The eight

results were combined by taking the maximum output value for

each pixel, and finding the optimal threshold strategy to binarize

the image. Figure 4 shows two of the eight Gabor Filter outputs,

after thresholding, demonstrating the orientation and width selectiv-

ity of the Gabor filter. Figure 5 shows the results obtained using the

threshold technique, template matching and the Gabor filter

method. Satisfactory results are obtained using Gabor filters

analysis.

One of the problems to tackle when designing a Gabor filter is

the determination of the parameters and the filter aperture—a prob-

lem that in principle is nonlinear. Once the parameters are deter-

mined, a filter bank may be created with as many Gabor filters as

the number of orientations one wishes to detect. It has been sug-

gested by Xiong and Shafer (1994) that the parameters dx, dy shouldbe approximately equal to the inverse of f to include enough cycles

of the sine wave in the kernel. The filter frequency in general should

be fixed to half the inverse of the average calmes width (Xiong and

Shafer, 1994). This information can be provided by the user, but an

automatic detection of this parameter would be preferable.

III. MULTIPLICATIVE MIXTURE MODEL FOR SHADOWS

The shadows problem may be modeled in terms of mixture of

images. The stained glass window image, the shadows of the pro-

tective grille and the shadows of the ferramenta (bar) are assumed

to be ‘‘mixed’’ together to form the observed image, plus a superim-

posed ‘‘noise’’ component. The goal is to separate this mixture

into its original components. The general problem of image separa-

tion can be stated as follows: given a set of m unknown images S1,

Figure 2. Template masks (a) y 5 08, (b) y 5 22.58, (c) y 5 458, (d) y 5 67.58, (e) y 5 908, (f) y 5 112.58, (g) y 5 1358, (h) y 5 157.58; Templatemasks of size 503 50 pixels with feature width5 20 pixels.

Figure 3. Gabor filter kernel with parameter f 5 20, y 5 08, dx 5 dy5 7 and window size 50 3 50. [Color figure can be viewed in the

online issue, which is available at wileyonlinelibrary.com.]

Vol. 20, 223–236 (2010) 225

S2, . . . Sm and a set of n observed images that are formed as a

composition f of the sources, find the inverse function f21 that

produces an estimate of the m sources from the n observed images

O1, O2, . . . On.

The observed image may be modeled as a combination of a

‘‘true stained glass image’’ and a ‘‘grille/bra image.’’ The goal is to

separate this mixture into its original components. Therefore, the

nature of this combination must be investigated to produce a valid

model.

We consider the image formation model. The RGB sensor

responses of the digital camera can be represented as (MacDonald

and Ji, 2002):

R ¼XNn¼1

EðknÞSðknÞrðknÞ

G ¼XNn¼1

EðknÞSðknÞgðknÞ

B ¼XNn¼1

EðknÞSðknÞbðknÞ

ð3Þ

Where: E(kn) is the power of illumination at wavelength kn; S(kn)is the object’s transmittance wavelength kn; r(kn), g(kn), b(kn) arethe sensitivities of the three camera channels; k spans the whole

Figure 4. Thresholded Gabor filter output (a) y5 908 (b) u 5 458.

Figure 5. (a) Original image; (b) Image segmented by threshold techniques; (c) Image segmented by template matching; (d) Image segmented

by combined Gabor filter

226 Vol. 20, 223–236 (2010)

visible spectrum, typically from 380 to 760 nm; N is the sampling

ratio chosen to represent the continuous function of wavelength

with sufficient accuracy.

A shadow is produced when a surface element (grille or bar)

acts as a blocker of some of the light emitted by another surface ele-

ment (light source). The blocker stops some of the radiant energy

along its path, so that the energy is then absorbed or redirected.

Because there is usually a gap between the grille or bar and the win-

dow, the shadowed region of the glass still receives partial illumina-

tion from the nonpoint source of the sun’s disc at the edges (penum-

bra) and from the diffused skylight overall. This suggests that the

response of the camera in the shadowed region should be related to

the response in the neighboring unshadowed region. As the effect

of the blocker is to reduce the amount of illumination E(kn) uni-formly over kn (at least to a first approximation), it is clear from Eq.

(3) that there should be a multiplicative relationship between the

pixel value PS corresponding to a grille line and its closest nonshad-

owed pixel value PNS:

PS ¼ að�ÞPNS að�Þ 2 ½0; 1� ð4Þ

Note that in Eq. (4). a is assumed to depend on a not-yet-speci-

fied set of parameters. This is because the reduction of illumination

may depend on factors such as the distance between the blocker and

the glass, on the thickness of the blocker and on the kind of glass.

For example, an opalescent glass will tend to scatter light, therefore

yielding values of a closer to unity. On the other hand, a very trans-

parent glass will produce a more distinct and possibly darker

shadow, therefore yielding values of a closer to zero. The observed

image Io(x,y) may then be modeled as:

Ioðx; yÞ ¼ aðx; yÞ � Iðx; yÞ ð5Þ

where: x,y are the image coordinates in pixels; Io is the observed

stained glass image; I is the true stained glass image; a is the bar/

grille shadow image; a(x,y) [ [0,1] if x, y lies on a bar/grille line,

else a(x,y)5 1.

IV. BAR SHADOW REMOVAL

Because glass is translucent, image of stained glass windows taken

with external illumination very often contain shadows cast by struc-

tural support bars called ferramenta (Fig. 1). A thick horizontal

structural bra, which runs across the entire window, creates a strong

horizontal shadow. The eye tends to ‘‘discount’’ such shadow for-

mations when viewing the actual windows, but in photographic

images they are very clearly visible. In this section, we presented an

algorithm to remove the bra shadows. Section IV.A describes a

physical model of shadow formation. Removing the bar shadows

removal described in Section IV.B.

A. Physical Model for Bar Shadows. A physically based

model can be viewed as the simulation of the propagation of light

in an environment. It should include sources that emit light energy

into the environment; materials that scatter, reflect, refract, and

absorb light; and sensors that record the quantity of light at different

positions and angles. A prerequisite for the development of a physi-

cally based shadow computation is knowledge about the distribu-

tion of light in a scene from each of the illuminating sources. Figure

6 illustrates the concept of an illuminating hemisphere (Rogers,

1989).

The illuminating hemisphere is a notational convenience for

describing the illumination events above or below a surface. These

events, such as light sources or other reflecting surfaces, are pro-

jected onto the hemisphere. Typically a hemisphere of radius 1 is

used for computational convenience. A solid angle defines the

amount of the illuminating hemisphere that is covered by the pro-

jection of the illumination event onto the illuminating hemisphere.

A differential solid angle is defined as a projected differential sur-

face element on the hemisphere (projected from the illuminating

event) divided by the square of the radius of the hemisphere. The

solid angle of an illumination event is determined by integrating the

differential solid angle over the bounds of the projection of the

event.

Physical models can explain the formation of ‘‘simple’’ shad-

ows. The most powerful tool for analyzing this problem is to think

about what a source looks like from the surface. This technique ena-

bles us to give a qualitative description of ‘‘brightness.’’ An ideal-

ized form is the Wall-Bar model, where a bar of infinite length (bar)

casts a shadow onto an infinite surface (wall) illuminated by a per-

fectly diffused light source (overcast sky) with the geometry shown

in Figure 7. We wish to know the brightness distribution at the base

of an infinitely long bar and infinitely high wall.

The solid angle of the bar is determined by projecting it onto

illuminating hemisphere above the surface and integrating the area

of the projection. Figure 8 sketches the appearance of the infinitely

long bar form at two points A and B on the infinitely wall. The bar

projected onto the illuminating hemisphere looks like the segment

of an orange, converging to a polar point as the bar recedes to infin-

ity. All points on a given horizontal line on the wall see the same

input hemisphere, and so must have the same ‘‘brightness’’ but

points along a vertical line see different amounts of the input hemi-

sphere (Fig. 9).

Figure 6. The illuminating hemisphere. [Color figure can be viewed

in the online issue, which is available at wileyonlinelibrary.com.]

Figure 7. Brightness of the shadow cast by an infinitely long bar

onto an infinitely long wall, under a uniformly overcast sky.

Vol. 20, 223–236 (2010) 227

The angle b defines the projected outline of the bar and can be

expressed as a function of the vertical position y on the wall as:

b ¼ arctanad

a2 þ y2 � d2

4

!ð6Þ

where a is the distance of the centre of the bar from the wall and dis the bar diameter. Integrating over the bounds of the projected

area of the bar, the solid angle x is given by:

x ¼ 2

Zp2

0

Zb0

sin udud/ ¼ 2b ð7Þ

The brightness B is defined as:

B ¼ 2p� -2p

¼ 2p� 2b

2p¼ p� b

pð8Þ

Based on this technique, the typical shadow profile is calculated

as shown in Figure 10, plotting brightness as a function of vertical

position on the wall, with bar distance as a parameter. Figure 11

shows the actual profile measured from a series of test images cap-

tured under controlled illumination, in which only the distance of

the bar was changed and all other parameter were kept constant.

The resulting shadow profile is a bell-shaped curve. To deter-

mine the exact shape of the curve, it is necessary to know the dis-

tance of the bar from the wall and the diameter of the bar. This sim-

ple model provides a useful first approximation, but for a more so-

phisticated model of transmissive media one should consider other

effects, such as multiple reflections within the glass, the homogene-

ity of the glass itself, and scattering of light by impurities. Other

extensions could include the restriction of the illuminating hemi-

sphere to the upper half (sky), and the dual outdoor illumination of

direct sunlight (yellowish), and reflected skylight (bluish).

B. Removing the Bar Shadows. An idealized test image was

captured by using a high-resolution digital camera, the Rollei

60008i with a Jenoptik eyelike MF digital back (MacDonald et al.,

2002), to photograph a Victorian (c.1860) stained glass panel and a

metal bar placed on a large light table. A detail from the RGB test

image was cropped to 600 3 600 pixels in size, and then trans-

formed into Hue-Saturation-Value (HSV) components, using the

standard computer graphics model (Foley et al., 1990). Figure 12

shows the resulting image components. It is clear that the shadow

Figure 8. Projection of the bar onto the illuminating hemisphere.

Figure 9. Projected outline of the bar.

Figure 10. Physical model predictions: family of intensity curveswith the bar distance as a parameter.

Figure 11. Measured shadow profile on a sample window undercontrolled illumination: family of curves with the distance as a

parameter.

228 Vol. 20, 223–236 (2010)

mainly affects the V channel rather than H and S channels, as

shown by the profile of pixel values Figure 13.

In the case of the shadow produced by a horizontal bar across a

stained glass window, if the position and diameter of the bar are

roughly known then the physical model can be used to predict the

shadow profile. Stained glass windows are commonly photographed

under diffuse lighting rather than direct light. Here, we assume that

the stained glass window panel transmittance process reduces the

intensity of illumination by the same amount over all parts of the

glass, facilitating computation of the intensity of illumination arriv-

ing on the exterior surface of the window. The wall-bar model is

well suited to this problem. Recall from Eq. (5) that the conjectured

mixture model for the observed window image is the true window

image multiplied by the profile of the shadow produced by the bar.

Using the wall bar model, the a value may be estimated then the in-

tensity I recovered.The algorithm was applied to three test images, taken from dif-

ferent parts of the same stained glass test panel, captured under con-

trolled laboratory illumination (using a flash light with soft box and

diffuser as a backlight source). Figure 14 shows the results com-

pared to the original image. Figure 15 shows details of four images

taken under exterior daylight illumination in the chancel of the

Church of St.Mary, Studley Royal, Yorkshire.

It is evident that the features painted on the glass are success-

fully retained, whereas the shadow has been reduced. This is true

for both laboratory images (Fig. 14) and real world images (Fig.

15). This provides some evidence that the multiplicative mixture

model is valid, at least as an initial step, but is over-simplistic. At

the centre of the bar the model gives encouraging results. The

shadow is mostly removed, and the features of the glass hidden by

the shadow are fully restored. The multiplication of the model has

the effect of amplifying the (unwanted) noise in the shadow areas

Figure 12. Test image: (a) RGB-image; (b) H-component; (c) S-component; (d) V-component. [Color figure can be viewed in the

online issue, which is available at wileyonlinelibrary.com.]

Figure 13. H, S and V profiles of pixels down a vertical line of the

test image. The shadow is evident only in the V component.

Figure 14. Results of bar shadow removal algorithm (right) com-

pared to the original images (left) captured under controlled lab illumi-nation (using soft box). [Color figure can be viewed in the online issue,

which is available at wileyonlinelibrary.com.]

Vol. 20, 223–236 (2010) 229

of the image as well as the (wanted) texture of the glass and paint

work.

Near the periphery of the shadow, however, obvious artifacts

have been introduced into the image, in the form of light or dark ha-

lation. This shows that the modeled profile does not fit so well with

actual profile, because the gradient of the shadow predicted by the

model is not exactly the same as in the real-world image.

Figure 16 shows the actual shadow profile and model fitting. It

is clear the model does not fit in the tail part well with actual

shadow edge profile because it never considers the glass transpar-

ency and internal scattering. If a realistic model were developed,

this problem might be overcome. A comprehensive method for

removing the shadow perfectly would need more real world infor-

mation, such as bra position, calibre of the bar, distance between

the bra and glass, properties of the glass, window size, position, and

so forth. We will investigate whether the model parameters can be

fitted to the actual shadow edge profile observed in the image, or

whether a more sophisticated blending model might be used (Giani

and MacDonald, 2003).

V. GRILLE SHADOW REMOVAL

Protective window grilles are typically constructed from a regular

mesh of small diameter iron or copper wire fixed to a frame 10–15

cm away from the glass. The grille casts a soft shadow because the

scattering of light produces a shadow that is not completely black,

but in fact inherits some chromaticity from the color of the glass

onto which the shadow is cast. In other words, the shadow line in

the image is ‘‘dark greenish’’ where the interposed glass is green;

‘‘dark bluish’’ where the interposed glass is blue and so forth.

Although exceptions may be found, this can be considered as a gen-

eral behavior in typical illumination conditions. Another character-

istic of this kind of shadow is derived directly from the typical peri-

odic structure of a grille. The pattern of horizontal and vertical lines

repeats across the window, although its tint changes across the dif-

ferent glass tiles. Figure 17 (left) shows the test image, in which

focus the attention on a horizontal line of 1 pixel width. The lumi-

nance profile for pixels on this horizontal line is show in Figure 17

(right) and the corresponding R, G, B channel profile are shown in

Figure 18. The grille shadow profile is approximately V-Shaped,

and its periodicity is clearly evident.

The multiplicative mixture model described in Section III, the

conjectured mixture model is Ioij 5 aij � Iij, where i, j are the image

coordinates in pixels, Io is the observed stained glass image, I is the

true stained glass image, a is the grille image with aij [ [0,1] if i, j

is a grille line, one elsewhere. Therefore the task is to recover I.

The proposed algorithm can be divided in two steps.

The first step produces a binary map (grille map) where a null

value denotes the presence of a grille line and a unitary value

denotes the absence of a grille line (clear glass). This is a two-class

classification problem representing the object of interest (grille) and

the remaining part of the image. Based on this grille map an esti-

mate of aij may be produced.

The proposed algorithm exploits the conjecture that the value of a

pixel in correspondence of a grille line is a fraction of the value of the

neighboring nonshadowed pixels. Two convolution windows are

implemented in correspondence with the vertical and horizontal grille

lines. The vertical convolution window is the average of column j(say Sij) over the averages of the neighboring columns (say NSij). The

quantities are defined on a 53 5 window as shown in Figure 19:

SðVÞij ¼ 1

5

Xiþ2

k¼i�2

fkj ð9Þ

NSðVÞij ¼ 1

20

Xiþ2

k¼i�2

Xj�1

l¼j�2

fkl þXjþ2

l¼jþ1

fkl

" #ð10Þ

where f is the input image.

Similarly, we define the horizontal quantities:

SðHÞij ¼ 1

5

Xjþ2

k¼j�2

fik ð11Þ

NSðHÞij ¼ 1

20

Xjþ2

k¼j�2

Xi�1

l¼i�2

flk þXiþ2

l¼iþ1

flk

" #ð12Þ

Figure 15. Results of bar shadow removal algorithm (right) com-

pared to the original images (left) captured under daylight illumination.

[Color figure can be viewed in the online issue, which is available at

wileyonlinelibrary.com.]

230 Vol. 20, 223–236 (2010)

Figure 17. Test image and horizontal sampling line (left)Luminance profile of pixels across the horizontal line (right). [Color figure can be viewed

in the online issue, which is available at wileyonlinelibrary.com.]

Figure 16. The blue line represents the luminance profile of a vertical line in original image. The green line represents the estimated a value,

using the wall-bar model. The red line represents the luminance profile of a vertical line in the resulting image. [Color figure can be viewed in theonline issue, which is available at wileyonlinelibrary.com.]

Vol. 20, 223–236 (2010) 231

where Sij(H) is the average value of the five neighboring pixels on

row i, and NSij(H) is the average of a rectangular region surrounding

the pixel (i,j) (i.e., the average of the neighboring rows).

The following ratios are then defined, based on Eq. (9–12):

Vertical Luminance Ratio:

VIRij ¼SðVÞij

NSðVÞij

ð13Þ

Horizontal Luminance Ratio:

HIRij ¼SðHÞij

NSðHÞij

ð14Þ

First, consider a test image as shown in Figure 20. The caliber of

the grill line is 3 pixels. The vertical convolution window is applied

to the test image and VIRij is estimated. One can see that the

shadow profile has been successfully detected, but the edges of the

shadow profile are affected. Given that aij [ [0,1] in the multiplica-

tive mixture model [Eq. (5)], the VIRij value can be clipped outside

this range. One can see that the profile of a look similar to the true

image luminance profile. The window size (w 3 w) is function of

grill calibre (say c). Empirically the relationship w 5 2c 1 1 is

chosen.

Note from Figure 18 that there is in principle a different aij for

each color channel (named as aijR, aij

G aijB). Figure 21 shows flow-

chart of the algorithm, where HIRijV and VIRij

V are convolution win-

dows applied on the value V-channel (HSV color space). The grille

map (Mij(H) is a horizontal grille map; Mij

(V) is a vertical grille map.)

is computed by checking if HIRijV < 1 or VIRij

V < 1 as shown in

Figure 21. HIRijR, HIRij

G, and VIRijB are estimates based on the hori-

zontal convolution window applied to the R, G, B channels. Simi-

larly VIRijR, VIRij

G, and VIRijB are estimates based on the vertical

convolution window applied to the R, G, B channels.

The algorithm was applied to the test image and the resulting

‘‘grille maps’’ for the vertical and horizontal orientations are shown

in Figure 22. In this test image the grille caliber is 2–3 pixels (c 53). The size of the convolution windows was therefore 7 3 7 pixels

(w 5 2c 1 1). One can see that the grille has been successfully

detected, but unfortunately this approach fails because there are

other features in the image, which have similar characteristics to

the shadow, namely the lead calmes and painted details. Figure 23

shows that the algorithm not only succeeds in substantially reducing

the grille shadow but also removes some detail, for example, in the

centaur’s beard. To solve this problem it is necessary to discrimi-

nate the window features (to be retained in the image) from the

grille shadows (to be removed).

Extensions to the basic algorithm enable its performance to be

improved by using knowledge about the structure of stained glass:

A. Exclusion of Calmes.. It is clear from Figure 17 that the

grille and calmes profiles have a similar ‘‘V’’ shape. In purely trans-

missive image taken in direct sunlight the grille shadows may easily

be confused with the contours of the lead calmes, which are opaque,

Figure 18. The pixel values across a horizontal image line (R, G, Bchannels).

Figure 19. Vertical (left) and horizontal (right) convolution windows.

232 Vol. 20, 223–236 (2010)

and therefore, appear black (although in practice the metallic lead

also reflects some of the ambient light, making it off-black in the

image). In diffused daylight the grille shadows are softer and

lighter, whereas the calmes have a lower luminance level, close to

black.

In Section II, we present the image segmentation of stained

glass, several approaches have applied to a digital image of stained

glass window in order to segment the image to match the window’s

physical structure of separate pieces of glass joined by strips of lead

(calmes), satisfactory results obtained when using a tuned set of

Gabor filter. Using the image segmentation results to correct the

grille map, then the a value may be estimated.

B. Exclusion of Painted Features. The painted features cover

a wide range of spatial frequencies, whereas the typical grille struc-

ture is periodic. The pattern of horizontal and vertical lines is

repeated across the window. So for a rectilinear wire grille, the

grille map should have only horizontal and vertical lines.

Several line detection techniques can be found in the literature,

based for instance on template matching (Jain et al., 1995) or on the

Hough transform (Nixon and Aguado, 2002). The template match-

ing techniques enable detection of the lines to correct the grille map

then estimate the a value.

C. Reduction of Artifacts. Remaining shadow artifacts can be

further reduced in two ways: first to increase the convolution win-

dow size; second to use multiple iterations. In the current imple-

mentation, the user sets the number of iterations, although it would

b easy to implementation, the user sets the number of iterations,

although it would be easy to implement other termination criteria.

The resulting algorithm (see Fig. 24 gives a flowchart) is fast

because it usually converges in three to five iterations and each iter-

ation calculates the a value on a grille region only (the grille map is

known already) and not on the entire image. The results of succes-

sive iterations are shown in Figure 25.

VI. CONCLUSIONS

Shadows are cast onto stained glass windows by close external

structures, such as wire grille and support bars. We have show that

it is feasible to reduce the visible effects of these shadows in digital

images of stained glass windows, by a series of image processing

algorithms. First the shadowed image is modeled as a product of

Figure 20. Test image (left); Profile (right) and a value based on 7 3 7 window.

Figure 21. Details of grille shadow removal algorithm.

Vol. 20, 223–236 (2010) 233

the luminance of the ‘‘true’’ unshadowed window image and an

achromatic shadow image. The broad bar shadows can be approxi-

mated by a physically based shadow formation model, the parame-

ters of which can be tuned to the actual edge profile of the shadow

found in the image. The results show that the bar shadows can be

very effectively compensated in their central regions, but that to

obtain optimum performance at the edges it is necessary to tune the

shadow formation model to match the intensity gradient observed

in the image.

Shadows cast by wire grilles have different characteristics,

namely that they are narrow, lighter and periodic, and must there-

fore be dealt with in a different way from the bars. The shadows

were detected by computation of horizontal and vertical grille

shadow map by a one-dimensional convolution process. An algo-

rithm was developed to generate a composite grille shadow map

then apply the multiplicative mixture model to remove the shadows

Figure 22. Detected vertical grille map (left), and horizontal grille map (right).

Figure 23. Comparison between the original image (left) and the

results of the algorithms (right). [Color figure can be viewed in the

online issue, which is available at wileyonlinelibrary.com.] Figure 24. Details of grille shadow removal algorithm.

234 Vol. 20, 223–236 (2010)

from the image. Three refinements were subsequently made to the

algorithm to improve its performance.

In general, the more information there is available about a given

image processing problem, the easier it is to solve by modeling the

characteristics of the wanted and unwanted image features. Re-

moval of shadows is no exception, but getting the necessary infor-

mation is not always easy. If the bar and grille structures were per-

fectly regular geometrically and the windows were always illumi-

nated by a perfectly diffused uniform backlight, the problem would

be easy. In practice, real stained glass windows are nonplanar and

the bars and grilles are distorted, corroded, and dirty. Illumination

varies with weather conditions and time of day, and is affected by

reflections and occlusions from nearby masonry, vegetation, trees,

and buildings. Ultimately it is the user who must decide how the

image of the windows are to be reproduced, and therefore, how

(un)acceptable the visibility of shadows will be. We have shown in

this study that it is feasible to reduce or remove shadows from

images of stained glass by various techniques.

Figure 25. Results of applying the Shadow -removal algorithm with multiple iterations. [Color figure can be viewed in the online issue, which is

available at wileyonlinelibrary.com.]

Vol. 20, 223–236 (2010) 235

There are further challenges for image processing in the investi-

gation of stained glass. Some engaging research questions are as

follows: Can background be effectively removed from image of

stained glass? Glass is translucent and the image captured is a com-

bination of the glass itself and the scene behind the window. In typi-

cal church environments, the latter may include sky, foliage, other

building etc. This article addressed the multiplicative mixture

model; could be extended to the general case of an arbitrary back-

ground, but it is not easy to obtain an image of the ‘‘unwanted’’

background from the camera view point.

REFERENCES

F.S. Abas and K. Martinez, Craquelure analysis for content based retrieval,

Proceedings of IEEE 14th International Conference on Digital Signal Proc-

essing, 2002, pp 111–114.

E.L. Armitage, Stained glass: History, Technology and Practice, Charles T.

Branford Company, Newton, 1959.

S. Brooks, Image-based stained glass, IEEE Trans visual Comput Graph

12 (2006), 1547–1558.

G. Brown, Stained Glass Windows, Celtic Cross Press, 1998.

D. Dunn, W.E. Higgins, and J. Wakelely, Texture segmentation using 2-D gabor

elementary function, IEEE Trans Pattern Anal Mach Intell 16 (1994), 130–149.

K. Findlater and L.W. MacDonald, Imaging Properties of Stained Glass,

Report 1, AHRB Project Direct Digital Image Capture of Stained Glass,

University of Derby, Derby, UK, January 2001a.

K. Findlater and L.W. MacDonald, Photographic Practices, Report 2, AHRB

Project Direct Digital Image Capture of Stained Glass, University of Derby,

Derby, UK, March 2001b.

G.D. Finlayson, S.D. Hordley, C. Lu, and M.S. Drew, On the removal of shad-

ows from images, IEEE Trans Pattern Anal Mach Intell 28 (2006), 59–68.

J.D. Foley, A. Van Dam, S.K. Feiner, and J.F. Hughes, Computer Graphics—

Principles and Practice, 2nd Ed, Addison-Wesley, 1990, pp. 590–592.

D. Gabor, Theory of Communication, J IEE 93 (1946), 429–457.

A. Giani and L.W. MacDonald, Blending Images of Stained Glass Windows,

Proc. IS&T/SID 11th Color Imaging Conf., Scottsdale, 2003.

L. Hong, A.K. Jain, S. Pankanti, and R. Bolle, Fingerprint enhancement,

Proceedings of 1st IEEE WACV, Sarasota FL, 1996, pp . 202–207.

R. Jain, R. Kasturi, and B.G. Schunck, Machine Vision, McGraw-Hill Inter-

national, McGraw-Hill, New York, 1995.

A. Leone, C. Distante, and F. Buccolieri, A shadow elimination approach in

video-surveillance context, Pattern Recogn Lett 27 (2006), 345–355.

M.D. Levine and J. Bhattacharyya, Removing Shadows, Pattern Recogn

Lett 26 (2005), 251–265.

L.W. MacDonald and W. Ji, Color Characterization of a High-Resolution

Digital Camera, The First IS&T European Conference on Color in Graphics,

Imag Vis (CGIV), Poitiers, France, April 2–April 5, 2002.

L.W. MacDonald, W. Ji, S. Bouzit, and K. Findlater, Characterization of a

Digital Camera for Heritage Photography, Proceeding of the International

Congress of Imaging Science (ICIS’02), Tokyo, May 2002, pp. 345–346.

D. Mould, A Stained Glass Image Filter, Proceedings of the 14th Euro-

graphics workshop on Rendering, Leuven, Belgium, June 25–27, 2003; pp

20–27.

M. Nixon and A. Aguado, In Feature Extraction and Image Processing, Aca-

demic Press, 2nd edition. Newnse, 2002.

J. Osborne, Stained Glass in England, Alan Sutton Publishing, Phoenix Mill,

1997.

A. Prati, I. Mikic, M.M. Trivedi, and R. Cucchiara, Detecting moving shad-

ows algorithms and evaluation, IEEE Trans Pattern Anal Mach Intell

25 (2003), 918–923.

D.F. Rogers, Illumination and Color in Computer Generated Imagery,

Springer-Verlag, 1989.

P.K. Sahoo, S. Soltani, and A.K.C. Wong, A survey of thresholding techni-

ques, Comput Vis Graph Image Process 41 (1988), 233–260.

E. Salvador, A. Cavallaro, and T. Ebrahimi, Cast shadow segmentation using

invariant color features, Comput Vis Image Understand 95 (2004), 238–259.

S. Seo, H. Lee, H. Nah, and K. Yoon, Stained Glass Rendering with Smooth

Tile Boundary, Proceedings of the 7th International Conference on Compu-

tational Science (ICCS 2007), LNCS, Beijing, China, 2007, Vol. 4488, pp.

162–165.

J.L. Sloan, Conservation of Stained Glass in America, Wilmington: Art in

Architecture Press, 1993.

T.P. Weldon, W.E. Higgins, and D.F. Dunn, Efficient gabor filter design for

texture segmentation, Pattern Recogn 29 (1996), 2005–2015.

W.A. Weyl, Colored Glasses, Society of Glass Technology, Sheffield, 1976.

Y. Xiong and S.A. Shafer, Variable Window Gabor Filters and Their use in

Focus and Correspondence, Technical Report CMU-RI-TR-94–06, The

Robotic Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania,

March 1994.

236 Vol. 20, 223–236 (2010)