a survey on algorithms of shadow removal in vehicle detection · a survey on algorithms of shadow...

A Survey on Algorithms of Shadow Removal in

Vehicle Detection Padmavathi.S1, Abylash Kumar.KT2

1,2Department of Computer Science, Amrita School of Engineering, Coimbatore.

Abstract

A shadow appears on an area when the light from a source

cannot reach the area due to obstruction by an object. Shadow

detection and removal in real time images is always a challenging

task. In contrast with the rapidly expanding and continuous

interests on this area, Shadow detection is critical for robust and

reliable vision-based systems for traffic flow analysis. In our case

Vehicle Detection, Shadow Removal is used to prevent moving

shadows being misclassified as moving objects (or parts of them),

thus avoiding the merging of two or more objects into one. The

environment considered is Local Highway with multiple lanes

and a fixed camera. This paper is aimed at giving comprehensive

algorithms to detect and remove vehicle shadows in Real-time

Videos.

Terms used – Shadow Detection, cast shadow, Shadow

Detection, Shadow Removal

I. INTRODUCTION

Many Computer applications involving video segments

require detecting and tracking moving objects. For Vehicle

detection and speed estimation, Vehicles are to be detected for

which various techniques are explained in the paper [19]. We

identify shadow removal as a critical step for improving object

detection and tracking. Shadows can be classified into self-

shadows and cast shadows. Neither motion segmentation nor

change detection methods can distinguish between moving

objects and moving shadows. Our job is to detect the cast

shadow and not the self-shadow because removing the later

will result in parts of the object removed. A frame may contain

a vehicle with or without shadow or multiple vehicles

connected by shadows. The key success for the shadow

removal is achieving the robustness and efficiency of the

images or sequences that are been used for progress. Proper

segmentation is required for the correct isolation of objects and

shadows in the foreground regions. The previous works on

shadow detection are reviewed in [1]. Many of these methods

involve using background subtraction technique. Results vary

depending on the algorithms used. Tracking of dynamic objects

can be taken by image frames extraction from the video

sequences and shadow detection can work at pixel level. Fact is

that illumination changes based on Sunlight and Street lights at

night. First step computed for shadow detection is the

difference between the current frame and a reference image.

Second, the assumption is used to distinguish between shadow

and object in changed image regions. Third, shadows smooth

the background they cover. Many of the works published

locally exploit pixel appearance change due to cast shadows. A

possible approach is to compute the ratio between the

luminance of pixels after and before shadow appears. E.

Salvador et al proposed an approach to detect and classify

shadows for still images [2]. They exploit invariant color

features to classify cast and self- shadows.

A. Prati et al [3] conducted a survey on detecting moving

shadows; algorithms dealing with shadows are classified in two

divisions as Deterministic approach and Statistical approach.

The first classification considers whether the decision process

introduces and exploits uncertainty. Deterministic approaches

use an on/off decision process, whereas statistical approaches

use probabilistic functions. As the parameter selection is a

crucial problem for statistical methods, the authors further

divided statistical methods into parametric and non-parametric

methods. For deterministic approaches, algorithms are

classified by whether or not the decision can be supported by

model-based knowledge. The authors reviewed four

representative methods for the categories of his taxonomy and

argued that Deterministic Model-based methods [4] rely so

much on models of the scene that they inevitably become too

complex and time-consuming. The survey of A. Prati et al

mainly focuses on the moving shadow detection, typically they

concentrate the attention on umbra, considering the penumbra

as a particular case of umbra. It is because the distance between

the objects and the background is negligible compared to the

distance of illumination sources to the objects in a highway

scene and most or all of the shadows are umbra or strong

shadow.

T. Horprasert et al’s method [5] is a statistical

nonparametric approach and the authors denote it with symbol

SNP. This approach uses color information and a trained

classify to distinguish between object and shadows. I. Mikic et

al [6] proposed a statistical parametric approach (SP) and

utilized both spatial and local features, which improved the

detection performance by imposing spatial constraints. R.

Cucchiara et al’s method (DNM1) [7] and J. Stauder et al’s

work (DNM2) [8] were representatives of deterministic non-

model based method. DNM1 is based on an assumption that

shadows in image do not change the hue of surfaces. The

reason why the author reviewed DNM2 is that it is the only

work that handles the penumbra regions in image.

In section II deals with the previous works on shadow

detection algorithms. By section III a survey on effective

algorithms used in shadow detection and concludes with

section IV with the conclusion of the Paper along with the

References...

Abylash Kumar KT et al, Int.J.Computer Technology & Applications,Vol 5 (2),518-521

IJCTA | March-April 2014 Available [email protected]

518

ISSN:2229-6093

II. SHADOW DETECTION

Most of the shadow removal algorithms use the background

subtraction and temporal differences in case of using stationary

cameras. Yang et al. [9] described the ratio of intensities

between a shaded pixel and its neighboring shaded pixel in the

current image, and this intensity ratio was found to be close to

that in the background image. They also made use of the slight

change of intensities on the normalized R and G channels

between the current and background image. Hsieh et al. [10]

used the histograms of vehicles and the calculated center of

lane to detect the lane markings, and also developed a

horizontal and vertical line-based method to remove shadows

by characteristics of those lane markings. This method might

become ineffective in case of no lane markings. Song et al.

[11] applied Gaussian model to representing the constant RGB-

color ratios, and determined whether a moving pixel belonged

to the shadow or foreground-object by setting ±1.5 standard

deviation as a threshold. Mohammed et al. [12] proposed their

method by using division image analysis and projection

histogram analysis. Image division operation was performed on

the current and reference frames to highlight the homogeneous

property of shadows. They afterwards eliminated the left pixels

on the boundaries of shadows by using both column- and row-

projection histogram analyses. Geometric model attempted to

remove shadowed regions or the shadowing effect by

observing the geometric information of objects.

III. EFFECTIVE ALGORITHMS

An algorithm is said to be effective if it uses least resources

such as Processing time, power consumption with better

results.

A. Shadow removal from Single image:

1) Vague Shadow estimation

Vague shadow refers to the shadows or shadings in images

which have no clear boundaries and gradually changed

intensity. In real scenes, self-shadows and shadings usually

have smooth changes and can be classified as vague shadow.

However, even in the cast shadow regions, we can separate the

gradually changed components. For instance, the penumbra

regions of the cast shadows are more likely to be vague shadow

components. Unlike the Land’s work on illumination removal

[13], our approach works in the gradient space rather than

intensity space. It is mainly because that it is easier to combine

the vague and hard shadow removal in the gradient domain.

Therefore, we first transfer the raw logarithmic image into

gradient domain:

( ) || ( )||)) Eq. (1)

Where (x,y) represents the logarithmic image, k denotes

the color channel: k = 1,2,3 and is the gradient

response. Since vague shadows are expected to be spatially

smooth, we clip out the high derivative peaks corresponding to

the object boundaries and obtain the vague shadow mask. To

determine the clipping threshold, the following criteria is

applied:

T= * * ( )++ Eq.(2)

Where is a predetermined value and T represents a

threshold. After deriving threshold T, we could do the

following clipping operation to obtain a binary vague shadow

mask for further process:

( ) { ( ( ))

Eq.(3)

The VS(x,y) is the binary mask. It could either be applied

directly to the derivatives to remove the vague shadows in

image or be combined with the hard shadow mask to perform

the full shadow removal task.

B. SAKBOT and ATON:

a) SAKBOT

SAKBOT (Statistical And Knowledge-Based Object Tracker)

is a deterministic non-model based approach and it aims to be

a general purpose object detector and tracker. It performs

object detection by means of Background Suppression.

Sakbot first converts pixel information from the RGB color space to the HSV color space. HSV color space corresponds closely to the human perception of color [14] And it has revealed more accuracy to distinguish shadows. Then, it tries to estimate how the occlusion due to shadow changes the value of H, S and V. The rationale is that cast shadow's occlusion darkens the background pixel and saturate its color. The first assumption is well evident. The second assumption helps distinguish the object points from the shadow points. Another interesting point is that shadows often lower the saturation of the points. Sakbot exploits these assumptions to define the shadow point mask S as follows,

{

( )

( )

Eq. (4)

b) ATON

The moving shadow detection algorithm developed

for the ATON project is described in detail in [15]. It is based

on the detection of umbra of moving cast shadow in outdoor

traffic video scene. It deals with non-modeled environments

and it takes into account most of the assumptions described in

Section II. Its key novelty is the use of three sources of

information to help in detecting shadows and objects:

Local – based on appearance of the individual

pixel.

Spatial – information based on the assumption that

objects and shadows are compact regions in the

scene.

Temporal – information that states that object and

shadow position can be predicted from previous

frames.

The local information is exploited assuming that if , - is the value of the pixel not shadowed, a shadow



519

ISSN:2229-6093

changes the color appearance by means of an approximated linear transformation , where

( ) ( ) is a diagonal

matrix obtained by experimental evaluation. The

corresponding values under shadows are

and

, with i R,G,B. A pixel is classified maximizing

the probability of the class membership.

(

. ⁄ / ( )

∑ . ⁄ / ( ) Eq.(5)

ATON improves the detection performance using spatial

constraints. Experimental results are shown in Table 1

TABLE 1

FN FP SHADOW

ACCURAC

Y

OVERALL

ACCURAC

Y

SAKBOT

W/O

COLOUR

150

8

319

0

60.59

91.65

SAKBOT

WITH

COLOUR

180

6

177

8

68.56 93.10

ATON W/O

POST

PROCESSIN

G

252

9

152

7

57.37 92.09

ATON WITH

POST

PROCESSIN

G

265

4

115

9

60.67 92.65

C. Analysis of Results

From Table 1 I is clear that SAKBOT and ATON have

almost same accuracy. If SAKBOT is applied for colour video

then the discussion favors it.The overall accuracy is more than

90%. In conclusion there is a trade-off between Effectiveness

and accuracy of the system and this must be taken into account

during parameter selection.

D. Illumination Assessment

The objective of Illumination assessment is to decide

whether there are shadows present in an image. Wixon et al

[16][17] have proposed approaches for evaluating the

distribution of illumination over an Image. Here two values,

Brightness and energy values, are calculated for each image

pixel. Based on these the distribution of illumination over the

images are asserted. The brightness value of a pixel is defined

as the average of intensity values of neighbors of the pixel. The

energy value is defined as the average of intensity differences

between image pixels around the pixel. Rather than apply to

the entire input image, try to apply illumination assessment to

a selected foreground figure. Two criteria are employed for the

selection: size and value, where

Eq.(6)

Here and represent the number of bright and Dark

Pixels. If a foreground pixel has its R,G,B values all smaller

than those of the corresponding background pixel, the

foreground pixel is regarded as 'dark' pixel; otherwise regarded

as a 'bright' pixel. Energy value is calculated by the formula

∑

, j {b, d} Eq.(7)

E. Finlayson et. Al[18]

Two independent steps are involved in the shadow

removal method of Finlayson et al. [18]. The first step consists

in performing operations to get an illumination invariant

image, which is a gray scale representation proportional to the

color reflectance of the surfaces. To work properly, this step

requires some color calibration. The second step consists in

finding shadow edges and removing them from the color

image. Shadow edges are defined as edges that are part of the

intensity image, but not of the illumination invariant image.

With these shadow edges, it is possible to stamp these edges

off from the gradient of each color channel, and to recover a

shadow less color image by solving a linear system

approximating the Poisson’s equation to integrate the now

shadow less image from the modified gradient information.

∫ ( ) ( ) ( ) where k=R,G,B Eq.(8)

and E( ) is the spectral power distribution of the

illumination, S( ) is the surface reflectance distribution, and

( ) is the sensor sensitivity distribution.The Shadow edges

algorithm is as ( ) { ( )

( ) Eq.(9)

IV. CONCLUSION

This paper presented a critical review on the algorithms and

techniques used in detecting shadows in dynamic videos and

methods to remove them. Even if shadow removal is one of the

steps in achieving vehicle tracking; we now understand the

importance of Shadow removal in Vehicle Tracking and Speed

Estimation. Even though the topic is bit complex in nature

many great minds have dedicated their work in this field. If the

shadows are removed with right algorithms then the accuracy

of Vehicle tracking will be increased dramatically.



520

ISSN:2229-6093

REFERENCES

[1] Huerta, I.; Holte, M.; Moeslund, T.; Gonzàlez, J.; "Detection and removal of chromatic moving shadows in surveillance scenarios," Computer Vision, 2009 IEEE 12th International Conference on, vol., no., pp.1499-1506, Sept. 29 2009-Oct. 2 2009..

[2] E. Salvador, P. Green, and T. Ebrahimi. Shadow identification

and classification using invariant color models. In Proceedings

of ICASSP 01, volume 3, pages 1545--1548. IEEE, 2001.

[3] A. Prati, I. Mikic, M. Trivedi, and R. Cucchiara, Detecting

moving shadows: Algorithms and evaluation, IEEE Transactions

on Pattern Analysis and Machine Intelligence, 25, pp. 918--923,

2003..

[4] D. Koller, K. Danilidis, H.-H. Nagel, Model-based object

tracking in monocular image sequences of road traffic scenes,

Int. J. Comput. Vis. 10 (3) (1993) 257–281.

[5] T. Horprasert, D. Harwood, and L.S. Davis, ―A Statistical

Approach for Real-Time Robust Background Subtraction and

Shadow Detection,‖ Proc. IEEE Int’l Conf. Computer Vision

’99 FRAME-RATE Workshop, 1999.

[6] I. Mikic, P. Cosman, G. Kogut, and M.M. Trivedi, ―Moving

Shadow and Object Detection in Traffic Scenes,‖ Proc. Int’l

Conf. Pattern Recognition, vol. 1, pp. 321-324, Sept. 2000.

[7] R. Cucchiara, C. Grana, G. Neri, M. Piccardi, and A. Prati, ―The

Sakbot System for Moving Object Detection and Tracking,‖

Video-Based Surveillance Systems—Computer Vision and

Distributed Processing, pp. 145-157, 2001.

[8] J. Stauder, R. Mech, and J. Ostermann, ―Detection of Moving

Cast Shadows for Object Segmentation,‖ IEEE Trans.

Multimedia, vol. 1, no. 1, pp. 65-76, Mar. 1999.

[9] Yang, M.-T.; Lo, K.-H.; Chiang, C.-C.; Tai, W.-K., "Moving

cast shadow detection by exploiting multiple cues," Image

Processing, IET , vol.2, no.2, pp.95-104, April 2008.

[10] Kai-Tai S.; Jen-Chao T., "Image-Based Traffic Monitoring With

Shadow Suppression," Proceedings of the IEEE , vol.95, no.2,

pp.413-426, Feb. 2007.

[11] Jun-Wei H.; Shih-Hao Y.; Yung-Sheng C.; Wen-Fong H.,

"Automatic traffic surveillance system for vehicle tracking and

classification," IEEE Transactions on Intelligent Transportation

Systems, vol.7, no.2, pp. 175-187, June 2006.

[12] Mohammed Ibrahim M; Anupama R., ―Scene Adaptive Shadow

Detection Algorithm‖, Proceedings Of World Academy Of

Science, Engineering and Technology, vol. 2, pp. 1307-6884,

Jan. 2005.

[13] E.H. Land, ―The Retinex Theory of Color Vision‖, Scientific

American, vol. 237, pp. 108-128, 1977.

[14] N. Herodotou, K.N. Plataniotis, and A.N. Venetsanopoulos, \A

color segmentation scheme for object-based video coding," in

Proceedings of the IEEE Symposium on Advances in Digital

Filtering and Signal Processing, 1998, pp. 25{29.

[15] I. Mikic, P. Cosman, G. Kogut, and M.M. Trivedi, \Moving

shadow and object detection in tra scenes," in Proceedings of

Int'l Conference on Pattern Recognition, Sept. 2000.

[16] L. Wixson, "Illumination Assessment for Vision-Based Traffic

Monitoring", proc. of 13th Int'l Conf. on Pattern Recognition,

Vol.3, pp. 56-62, 1996.

[17] L. Wixson, K. Hanna and D. Mishra, "Improved Illumination

Assessment for Vision-Based Traffic Monitoring", proc. of

IEEE Workshop on Visual Surveillance, pp. 34-41, 1998

[18] Finlayson, G. D., S. D. Hordley, M. S. Drew. Removing

Shadows from Images. - In: Proceedings of 7th European

Conference on Computer Vision - Part IV, ECCV’02, London,

UK, Springer-Verlag, 2002, 823-836

[19] Z. Sun, G.Bebis, and R.Miller, ―On-road vehicle detection: A

review,‖ IEEE Transactions Pattern Analysis and Machine

Intelligence, vol. 28, no. 5, pp. 694 –711, May 2006.



521

ISSN:2229-6093

a survey on algorithms of shadow removal in vehicle detection · a survey on algorithms of shadow...

Documents