COLORIMAGE

PROCESSINGMethods and Applications

Edited by

Rastislav LukacKonstantinos N. Plataniotis

To be published by CRC Press.

TBA: The proper content for this page will be added by CRC Press.

This book contains information obtained from authentic and highly regarded sources. Reprinted material

is quoted with permission, and sources are indicated. A wide variety of references are listed. Reasonable

efforts have been made to publish reliable data and information, but the author and the publisher cannot

assume responsibility for the validity of all materials or for the consequences of their use.

Neither this book nor any part may be reproduced or transmitted in any form or by any means, electronic or

mechanical, including photocopying, microfilming, and recording, or by any information storage or retrieval

system, without prior permission in writing from the publisher.

All rights reserved. Authorization to photocopy items for internal or personal use, or the personal or internal

use of specic clients, may be granted by ...

Color television! Bah, I won’t believe it until I see it in black

and white.

—Samuel Goldwyn

Movie producer

To my dear parents whose constant love and support have made

my achievements possible.

R. Lukac

To the loving memory of my father.

K. N. Plataniotis

Preface

Over the last two decades, we have witnessed an explosive growth in both the diversity of techniques and the

range of applications of image processing. However, the area of color image processing is still sporadically

covered, despite having become commonplace with consumers choosing the convenience of color imaging

over traditional gray-scale imaging. With advances in imaging sensors, digital TV, image databases, video

and multimedia systems, and with the proliferation of color printers, color image displays, DVD devices,

and especially digital cameras and image-enabled consumer electronics, color image processing appears to

become the main focus of the image processing research community. Processing color images or, more

generally, processing multichannel images, such as satellite images, color filter array images, microarray

images, and color video sequences, is a nontrivial extension of the classical gray-scale processing. Indeed,

the vectorial nature of multichannel images suggests a different approach — that of vector algebra and

vector fields — should be utilized in approaching this research problem. Recently, there have been many

color image processing and analysis solutions, and many interesting results have been reported concerning

filtering, enhancement, restoration, edge detection, analysis, compression, preservation, manipulation and

evaluation of color images. The surge of emerging applications, such as single-sensor imaging, color-based

multimedia, digital rights management, art and biomedical applications indicate that the demand for color

imaging solutions will grow considerably in the next decade.

The purpose of this book is to fill the existing literature gap and comprehensively cover the system,

processing and application aspects of digital color imaging. Due to the rapid developments in specialized

areas of color image processing, this book has the form of a contributed volume where well-known experts

address specific research and application problems. It presents the state-of-the-art as well as the most recent

trends in color image processing and applications. It serves the needs of different readers at different levels.

It can be used as textbook in support of a graduate course in image processing or as stand-alone reference

for graduate students, researchers and practitioners. For example, the researcher can use it as an up-to-date

reference since it offers a broad survey of the relevant literature. Finally, practicing engineers may find it

useful in the design and the implementation of various image and video processing tasks.

This book details recent advances in digital color imaging and multichannel image processing methods

i

ii Color Image Processing: Methods and Applications

and explores emerging color image, video, multimedia and biomedical processing applications. The first

few chapters focus on color fundamentals, targeting three critical areas: color management, gamut mapping

and color constancy. The remaining chapters explore color image processing approaches across a broad

spectrum of emerging applications ranging from vector processing of color images, segmentation, resizing

and compression, halftoning, secure imaging, feature detection and extraction, image retrieval, semantic

processing, face detection, eye tracking, biomedical retina image analysis, real-time processing, digital

camera image processing, spectral imaging, enhancement for plasma display panels, virtual restoration of

artworks, image colorization, super-resolution image reconstruction, video coding, video shot segmentation

and surveillance.

Chapters 1 to 3 discuss the concepts and technology which are essential to ensure constant color appear-

ance in different devices and/or media. This part of the book covers issues related to color management,

color gamut mapping and color constancy. Given the fact that each digital imaging device exhibits unique

characteristics, its calibration and characterization using a color management system are of paramount im-

portance to obtain predictable and accurate results when transferring the color data from one device to

another. Similarly, each media has its own achievable color gamut. This suggests that some colors can often

not be reproduced to precisely match the original, thus requiring gamut mapping solutions to overcome the

problem. Since the color recorded by the eye or a camera is a function of the reflectances in the scene and

the prevailing illumination, color constancy algorithms are used to remove color bias due to illumination

and restore the true color information of the surfaces.

Chapters 4 through 7 are intended to cover the basics and overview recent advances in traditional color

image processing tasks such as filtering, segmentation, resizing and halftoning. Due to the presence of noise

in many image processing systems, noise filtering or estimation of the original image information from noisy

data is often used in order to improve perceptual quality of an image. Since edges convey essential infor-

mation about a visual scene, edge detection allows imaging systems to better mimic the human perception

of the environment. Modern color image filtering solutions which rely on the trichromatic theory of color

are suitable for both above tasks. Image segmentation refers to partitioning the image into different regions

that are homogeneous with respect to some image features. It is a complex process involving components

relative to the analysis of color, shape, motion, and texture of objects in the visual data. Image segmenta-

tion is usually the first task in the lengthy process of deriving meaningful understanding of the visual input.

Image resizing is often needed for the display, storage, and transmission of images. Resizing operations are

usually performed in the spatial domain. However, as most images are stored in compressed formats, it is

more attractive to perform resizing in a transform domain, such as the discrete cosine transform domain used

in most compression engines. In this way, the computational overhead associated with the decompression

and compression operations on the compressed stream can be considerably reduced. Digital halftoning is

Preface iii

the method of reducing the number of gray-levels or colors in a digital image while maintaining the visual

illusion that the image still has a continuous-tone representation. Halftoning is needed to render a color

image on devices which cannot support many levels or colors, e.g., digital printers and low-cost displays.

To improve a halftone image’s natural appearance, color halftoning relies heavily on the properties of the

human visual system.

Chapter 8 introduces secure color imaging using secret sharing concepts. Essential encryption of private

images, such as scanned documents and personal digital photographs, and their distribution in multimedia

networks and mobile public networks, can be ensured by employing secret sharing based image encryption

technologies. The images, originally available in a binary or halftone format, can be directly decrypted

by the human visual system at the expense of the reduced visual quality. Using the symmetry between

encryption and decryption functions, secure imaging solutions can be used to restore both binarized and

continuous-tone secret color images in their original quality.

Chapters 9 to 11 address important issues in the areas of object recognition, image matching, indexing

and retrieval. Many of the above tasks rely on the use of discriminatory and robust color feature detec-

tion to improve color saliency and determine structural elements such as shadows, highlights and object

edges/corners. Extracted features can help to group the image into distinctive parts to associate them with

individual chromatic attributes and mutual spatial relationships. The utilization of both color and spatial in-

formation in image retrieval ensures effective access to archives and repositories of digital images. Semantic

processing of color images can potentially increase the usability and applicability of color image databases

and repositories. Application areas such as surveillance and authentication, content filtering, transcoding,

and human and computer interaction can benefit directly from improvements of tools and methodologies in

color image analysis.

Chapters 12 to 14 cover face and eye-related color image processing. Color cues have been proven

to be extremely useful in facial image analysis. However, the problem with color cue is its sensitivity to

illumination variations which can significantly reduce the performance of face detection and recognition

algorithms. Thus, understanding the effect of illumination and quantifying its influence on facial image

analysis tools has become an emerging area of research. As the pupil and the sclera are different in color

from each other and from the surrounding skin, color can be seen a useful cue also in eye detection and

tracking. Robust eye trackers usually utilize the information from both visible and invisible color spectra

and are used in various human computer interaction applications such as fatigue and drowsiness detection

and eye typing. Apart from biometrics and tracking applications, color image processing can be helpful in

biomedical applications such as automated identification of diabetic retinal exudates. Diagnostic analysis

of retinal photographs by an automated computerized system can detect disease in its early stage and reduce

the cost of examination by an ophthalmologist.

iv Color Image Processing: Methods and Applications

Chapters 15 through 18 address the important issue of color image acquisition, real-time processing

and displaying. Real-time imaging systems comprise a special class of systems which underpin important

application domains including industrial, medical, and national defense. The understanding of the hardware

support is often fundamental to the analysis of real-time performance of a color imaging system. How-

ever, software, programming language and implementation issues are also essential elements of a real-time

imaging system as algorithms must be implemented in some programming language and hardware devices

interface with the rest of the system using software components. The typical example of the real-time color

imaging system is a digital camera. In the most popular camera configuration, the true color visual scene is

captured using a color filter array-based single image sensor and the acquired data must be pre-processed,

processed and post-processed to produce the captured color image in its desired quality and resolution. Thus,

single-sensor camera image processing typically involves real-time interpolation solutions to complete de-

mosaicking, enhancement and zooming tasks. Real-time performance is also of paramount importance in

spectral imaging for various industrial, agricultural and environmental applications. Extending three color

components up to hundreds or more spectral channels in different spectral bands requires dedicated sensors

in particular spectral ranges and specialized image processing solutions to enhance and display the spectral

image data. Most display technologies have to efficiently render the image data in the highest visual qual-

ity. For instance, plasma display panels use image enhancement to faithfully reproduce dark areas, reduce

dynamic false contours and ensure color fidelity.

Chapters 19 to 21 deal with other applications of color image enhancement. Recent advances in elec-

tronic imaging have allowed for virtual restoration of artworks using digital image processing and restora-

tion techniques. The usefulness of this particular kind of restoration consists of the possibility to use it as a

guide to the actual restoration of the artwork or to produce a digitally restored version of the artwork, as it

was originally. Image and video colorization adds the desired color to a monochrome image or movie in a

fully automated manner or based on a few scribbles supplied by the user. By transferring the geometry of the

given luminance image to the three dimensional space of color data, the color is inpainted, constrained both

by the monochrome image geometry and the provided color samples. Apart from the above applications,

super-resolution color image reconstruction aims to reduce the cost of optical devices and overcome the res-

olution limitations of image sensors by producing a high-resolution image from a sequence of low-resolution

images. Since each video frame or color channel may bring unique information to the reconstruction pro-

cess, the use of multiple low-resolution frames or channels provides the opportunity to generate the desired

output in higher quality.

Finally, Chapters 22 through 24 discuss various issues in color video processing. Coding of image se-

quences is essential in providing bandwidth efficiency without sacrificing video quality. Reducing the bit

rate needed for the representation of a video sequence enables the transmission of the stream over a com-

Preface v

munication channel or its storage in an optical medium. To obtain the desired coding performance, efficient

video coding algorithms usually rely on motion estimation and geometrical models of the object in the visual

scene. Since the temporal nature of video is responsible for its semantic richness, temporal video segmen-

tation using shot boundary detection algorithms is often a necessary first step in many video processing

tasks. The process segments the video into a sequence of scenes, which are subsequently segmented into a

sequence of shots. Each shot can be represented by a key-frame. Indexing the above units allows for effi-

cient video browsing and retrieval. Apart from traditional video and multimedia applications, processing of

color image sequences constitutes the basis for the development of automatic video systems for surveillance

applications. For instance, the use of color information assists operators to classify and understand complex

scenes, detect changes and objects on the scene, focus attention on objects of interest and track objects of

interest.

The bibliographic links included in the various chapters of the book provide a good basis for further

exploration of the topics covered in this edited volume. The volume includes numerous examples and

illustrations of color image processing results, as well as tables summarizing the results of quantitative

analysis studies. Complementary material including full-color electronic versions of results reported in this

volume are available online at the

http://www.dsp.utoronto.ca/ColorImaging

We would like to thank the contributors for their effort, valuable time and motivation to enhance the

profession by providing material for a fairly wide audience while still offering their individual research

insights and opinions. We are very grateful for their enthusiastic support, timely response, and willingness

to incorporate suggestions from us, from other contributing authors, and from a number of colleagues in the

field who served as reviewers. Particular thanks are due to the reviewers whose which helped to improve

the quality of contributions. Finally, a word of appreciation for CRC Press for giving us the opportunity to

edit a book on color image processing. In particular, we would like to thank Dr. Phillip A. Laplante for his

encouragement, Nora Konopka for initiating this project, Helena Redshaw for her support and assistance at

all times.

Rastislav Lukac and Konstantinos N. Plataniotis

University of Toronto, Toronto, ON, Canada

lukacr@dsp.utoronto.ca, kostas@dsp.utoronto.ca

About the Editors

Rastislav Lukac (http://www.dsp.utoronto.ca/∼lukacr) re-ceived the M.S. (Ing.) and Ph.D. degrees in Telecommuni-

cations from the Technical University of Kosice, Slovak Re-

public in 1998 and 2001, respectively. From February 2001

to August 2002 he was an Assistant Professor with the De-

partment of Electronics and Multimedia Communications at

the Technical University of Kosice. During August 2002

to July 2003 he was a Researcher with the Slovak Image

Processing Center in Dobsina, Slovak Republic. From Jan-

uary 2003 to March 2003 he was a Postdoctoral Fellow with

the Artificial Intelligence and Information Analysis Labo-

ratory, Aristotle University of Thessaloniki, Greece. Since

May 2003 he has been a Post-doctoral Fellow with the Ed-

ward S. Rogers Sr. Department of Electrical and Computer Engineering, University of Toronto in Toronto,

Canada. He is a contributor to four books and he has published over 200 papers in the areas of digital camera

image processing, color image and video processing, multimedia security, and microarray image processing.

Dr. Lukac is a Member of the IEEE, EURASIP, and IEEE Circuits and Systems, IEEE Consumer Elec-

tronics, and IEEE Signal Processing societies. He is a Guest Co-Editor of the Real-Time Imaging, Special

Issue on Multi-Dimensional Image Processing, and of the Computer Vision and Image Understanding, Spe-

cial Issue on Color Image Processing for Computer Vision and Image Understanding. He is an Associate

Editor for the Journal of Real-Time Image Processing. He serves as a Technical Reviewer for various sci-

entific journals and he participates as a Member of numerous international conference committees. In 2003,

he was the recipient of the NATO/NSERC Science Award.

vi

About the Editors vii

Konstantinos N. Plataniotis (http://www.dsp.utoronto.ca/∼kostas)received the B. Engineering degree in Computer Engineer-

ing from the Department of Computer Engineering and In-

formatics, University of Patras, Patras, Greece in 1988 and

the M.S and Ph.D degrees in Electrical Engineering from the

Florida Institute of Technology (Florida Tech), Melbourne,

Florida in 1992 and 1994 respectively. From August 1997 to

June 1999 he was an Assistant Professor with the School of

Computer Science at Ryerson University. He is currently an

Associate Professor at the Edward S. Rogers Sr. Department

of Electrical & Computer Engineering where he researches

and teaches image processing, adaptive systems, and multi-

media signal processing. He co-authored, with A.N. Venet-

sanopoulos, a book on ”Color Image Processing & Applications”, Springer Verlag, May 2000, ISBN 3-540-

66953-1, he is a contributor to seven books, and he has published more than 300 papers in refereed journals

and conference proceedings in the areas of multimedia signal processing, image processing, adaptive sys-

tems, communications systems and stochastic estimation.

Dr. Plataniotis is a Senior Member of IEEE, an Associate Editor for the IEEE Transactions on Neural

Networks, a past member of the IEEE Technical Committee on Neural Networks for Signal Processing. He

was the Technical Co-Chair of the Canadian Conference on Electrical and Computer Engineering (CCECE)

2001, and CCECE 2004. He is the Technical Program Chair of the 2006 IEEE International Conference

in Multimedia and Expo (ICME 2006), the Vice-Chair for 2006 IEEE Intelligent Transportation Systems

Conference (ITSC 2006), and the Image Processing Area Editor for the IEEE Signal Processing Society

e-letter. He is the 2005 IEEE Canada “Outstanding Engineering Educator” Award recipient and the co-

recipient of the 2006 IEEE Transactions on Neural Networks Outstanding Paper Award.

Contributors

SAVVAS ARGYROPOULOS

Aristotle University of Thessaloniki

Thessaloniki, Greece

savvas@iti.gr

YANNIS AVRITHIS

National Technical University of Athens

Zografou, Greece

iavr@image.ece.ntua.gr

STEFANO BERRETTI

Università degli Studi di Firenze

Firenze, Italy

berretti@dsi.unifi.it

NIKOLAOS V. BOULGOURIS

King’s College London

London, United Kingdom

nikolaos.boulgouris@kcl.ac.uk

VITO CAPPELLINI

University of Florence

Firenze, Italy

cappellini@lci.det.unifi.it

MATTHIAS F. CARLSOHN

Engineering and Consultancy Dr. Carlsohn

Bremen, Germany

Matthias.Carlsohn@t-online.de

ZUZANA CERNEKOVA

University of Thessaloniki

Thessaloniki, Greece

zuzana@zeus.csd.auth.gr

COSTAS COTSACES

University of Thessaloniki

Thessaloniki, Greece

cotsaces@zeus.csd.auth.gr

NIRANJAN DAMERA-VENKATA

Hewlett-Packard Labs

Palo Alto, CA, USA

damera@exch.hpl.hp.com

STAMATIA DASIOPOULOU

Aristotle University of Thessaloniki

Thessaloniki, Greece

viii

Contributors ix

dasiop@iti.gr

ALESSIA DE ROSA

University of Florence

Firenze, Italy

derosa@lci.det.unifi.it

ALBERTO DEL BIMBO

Università degli Studi di Firenze

Firenze, Italy

delbimbo@dsi.unifi.it

BRIAN L. EVANS

The University of Texas

Austin, Texas, USA

bevans@ece.utexas.edu

GRAHAM D. FINLAYSON

University of East Anglia

Norwich, United Kingdom

graham@cmp.uea.ac.uk

THEO GEVERS

University of Amsterdam

Amsterdam, The Netherlands

gevers@science.uva.nl

ABDENOUR HADID

University of Oulu

Oulu, Finland

hadid@ee.oulu.fi

FRED A. HAMPRECHT

University of Heidelberg

Heidelberg, Germany

fred.hamprecht@iwr.uni-heidelberg.de

HU HE

State University of New York at Buffalo

Buffalo, NY, USA

huhe@eng.buffalo.edu

MICHAEL KELM

University of Heidelberg

Heidelberg, Germany

michael.kelm@iwr.uni-heidelberg.de

ANDREAS KERCEK

Carinthian Tech Research AG

Villach/St. Magdalen, Austria

andreas.kercek@ctr.at

CHOON-WOO KIM

Inha University

Incheon, Korea

cwkim@inha.ac.kr

YU-HOON KIM

Inha University

Incheon, Korea

g2041410@inhaian.net

YIANNIS KOMPATSIARIS

Informatics and Telematics Institute

x Color Image Processing: Methods and Applications

Thessaloniki, Greece

ikom@iti.gr

HIROAKI KOTERA

Chiba University

Chiba, Japan

kotera@faculty.chiba-u.jp

LISIMACHOS P. KONDI

State University of New York at Buffalo

Buffalo, NY, USA

lkondi@eng.buffalo.edu

PHILLIP A. LAPLANTE

Penn State University

Malvern, PA

plaplante@psu.edu

RAIMUND LEITNER

Carinthian Tech Research AG

Villach/St. Magdalen, Austria

raimund.leitner@ctr.at

RASTISLAV LUKAC

University of Toronto

Toronto, ON, Canada

lukacr@dsp.utoronto.ca

BIRGITTA MARTINKAUPPI

University of Joensuu

Joensuu, Finland

jbm@cs.joensuu.fi

BJOERN H. MENZE

University of Heidelberg

Heidelberg, Germany

bjoern.menze@iwr.uni-heidelberg.de

SANJIT K. MITRA

University of California

Santa Barbara, CA, USA

mitra@ece.ucsb.edu

VISHAL MONGA

Xerox Innovation Group

El Segundo, CA, USA

vishal.monga@xeroxlabs.com

JAYANTA MUKHERJEE

Indian Institute of Technology

Kharagpur, India

jay@cse.iitkgp.ernet.in

NIKOS NIKOLAIDIS

University of Thessaloniki

Thessaloniki, Greece

nikolaid@zeus.csd.auth.gr

ALIREZA OSAREH

Chamran University of Ahvaz

Ahvaz, Iran

A Osareh@cua.ac.ir

HENRYK PALUS

Contributors xi

Silesian University of Technology

Gliwice, Poland

hpalus@polsl.pl

MATTI PIETIKÄINEN

University of Oulu

Oulu, Finland

mkp@ee.oulu.fi

IOANNIS PITAS

University of Thessaloniki

Thessaloniki, Greece

pitas@zeus.csd.auth.gr

ALESSANDRO PIVA

University of Florence

Firenze, Italy

piva@lci.det.unifi.it

STEFANO PIVA

University of Genoa

Genoa, Italy

piva@dibe.unige.it

KONSTANTINOS N. PLATANIOTIS

University of Toronto

Toronto, ON, Canada

kostas@dsp.utoronto.ca

GERRIT POLDER

Wageningen University

Wageningen, The Netherland

Gerrit.Polder@wur.nl

CARLO S. REGAZZONI

University of Genoa

Genoa, Italy

carlo@dibe.unige.it

RYOICHI SAITO

Chiba University

Chiba, Japan

saito@office.chiba-u.jp

GUILLERMO SAPIRO

University of Minnesota

Minneapolis, MN, USA

guille@umn.edu

HWA-SEOK SEONG

Samsung Electronics Co.

Gyeonggi-Do, Korea

hs78.seong@samsung.com

ABHAY SHARMA

Ryerson University

Toronto, ON, Canada

sharma@ryerson.ca

BOGDAN SMOLKA

Silesian University of Technology

Gliwice, Poland

smolka@ieee.org

xii Color Image Processing: Methods and Applications

MARCELLA SPIRITO

University of Genoa

Genoa, Italy

marcella@ginevra.dibe.unige.it

EVAGGELOS SPYROU

National Technical University of Athens

Zografou, Greece

espyrou@image.ece.ntua.gr

HARRO STOKMAN

University of Amsterdam

Amsterdam, The Netherlands

stokman@science.uva.nl

MICHAEL G. STRINTZIS

Informatics and Telematics Institute

Thessaloniki, Greece

strintzi@eng.auth.gr

NIKOLAOS THOMOS

Informatics and Telematics Institute

Thessaloniki, Greece

nthomos@iti.gr

ANASTASIOS N. VENETSANOPOULOS

University of Toronto

Toronto, ON, Canada

anv@dsp.toronto.edu

PAMELA VERCELLONE-SMITH

Penn State University

Malvern, PA, USA

pav115@gv.psu.edu

JOOST VAN DE WEIJER

INRIA

Grenoble, France

Joost.van-de-weijer@inrialpes.fr

DAN WITZNER HANSEN

IT University of Copenhagen

Copenhagen, Denmark

witzner@itu.dk

LIRON YATZIV

Siemens Corporate Research

Princeton, NJ, USA

liron.yatziv@siemens.com

Contents

Preface i

About the Editors vi

Contributors viii

Contents xiii

1 ICC Color Management: Architecture and Implementation 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 The Need for Color Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Closed-Loop Color Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.2.2 Open-Loop Color Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 CIE Color Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3.1 CIE Color Matching Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.3.2 CIE XYZ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3.3 CIE x,y Chromaticity Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3.4 CIE LAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.4 ICC Specification and Profile Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

1.4.1 Profile Header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.4.2 Profile Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.4.3 Scanner Profile Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.4.4 Monitor Profile Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.4.5 Printer Profile Tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.5 Device Calibration and Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.5.1 Scanner Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.5.2 Monitor Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

xiii

xiv Color Image Processing: Methods and Applications

1.5.3 Printer Characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2 Versatile Gamut Mapping Method Based on Image-to-Device 34

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2.2 Gamut Boundary Descriptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.2.1 Description of Image Gamut Shell . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.2.2 Compact GBD by Compression of r-Image . . . . . . . . . . . . . . . . . . . . . . 37

2.2.3 Quantization Error in r-Image by Segmentation . . . . . . . . . . . . . . . . . . . . 39

2.2.4 Image Gamut Reconstruction from Reduced DCT and SVD Parameters . . . . . . . 40

2.2.5 SVD Parameters for Reconstruction . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.2.6 r-Image for GBD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.3 Compression-Based GMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.3.1 Focal Point . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.3.2 Printer GBD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

2.3.3 Application to I-D GMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.3.4 Psychophysical Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.3.5 Experimental Test for Location of Focal Point . . . . . . . . . . . . . . . . . . . . . 44

2.4 Expansion-Based GMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.4.1 Gaussian Histogram Specification for Image . . . . . . . . . . . . . . . . . . . . . 47

2.4.2 Histogram Stretching for Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.5 Versatile GMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

2.5.1 Histogram Rescaling Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.5.2 Wide Color Gamut Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.5.3 Gamut Rescaling to Destination Device . . . . . . . . . . . . . . . . . . . . . . . . 51

2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3 3-, 2-, 1- and 6-D Color Constancy 65

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.2 3-Dimensional Color Constancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.3 2-Dimensional Chromaticity Constancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.4 1-Dimensional Scalar Constancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.5 6-D constancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Contents xv

3.5.1 Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.5.2 Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

3.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

4 Noise Reduction and Edge Detection in Color Images 88

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

4.2 Noise Reduction in Color Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

4.2.1 Vector Median Based Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

4.2.2 Fuzzy Adaptive Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

4.2.3 Switching Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

4.2.4 Application of Anisotropic Diffusion to Color Images . . . . . . . . . . . . . . . . 98

4.3 Edge Detection in Color Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.3.1 Vector Gradient Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.3.2 Vector Field Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

4.3.3 Vector Order-Statistics Edge Operators . . . . . . . . . . . . . . . . . . . . . . . . 104

4.3.4 Edge Detection Based on Hypercomplex Convolution . . . . . . . . . . . . . . . . 105

4.3.5 Evaluation of the Edge Detection Efficiency . . . . . . . . . . . . . . . . . . . . . . 106

4.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

5 Color Image Segmentation: Selected Techniques 121

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.2 Clustering in the Color Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

5.3 Region Growing for Color Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

5.3.1 Seeded Region Growing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

5.3.2 Unseeded Region Growing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

5.4 Postprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

5.5 Shadows and Highlights in the Image Segmentation Process . . . . . . . . . . . . . . . . . 131

5.6 Quantitative Evaluation of the Segmentation Results . . . . . . . . . . . . . . . . . . . . . . 132

5.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

xvi Color Image Processing: Methods and Applications

6 Resizing of Color Images in the Compressed Domain 155

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

6.2 Image Resizing Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

6.2.1 Using Linear, Distributive and Unitary Transform Properties . . . . . . . . . . . . . 157

6.2.2 Using Convolution-Multiplication Properties . . . . . . . . . . . . . . . . . . . . . 158

6.2.3 Using Sub-band DCT Approximation . . . . . . . . . . . . . . . . . . . . . . . . . 158

6.3 Image Halving and Image Doubling Algorithms Revisited . . . . . . . . . . . . . . . . . . 162

6.3.1 Image Halving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

6.3.2 Image Doubling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

6.4 Resizing with Arbitrary Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.4.1 Resizing with Integral Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

6.4.2 Computational Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

6.4.3 Resizing with Rational Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

6.5 Color Image Resizing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

6.6 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

Appendix: Mathematical Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

6.6.1 DCT: Definitions and Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

6.6.2 Down-sampling and Up-sampling Properties of the DCTs . . . . . . . . . . . . . . 173

6.6.3 Sub-band Relationship of the Type-II DCT . . . . . . . . . . . . . . . . . . . . . . 174

6.6.4 Recomposition and Decomposition of the DCT Blocks . . . . . . . . . . . . . . . . 175

6.6.5 Symmetric Convolution and Convolution-Multiplication Properties in DCT Domain 177

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178

7 Color Image Halftoning 185

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

7.2 Screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187

7.2.1 Classification of Screening Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 188

7.2.2 Heuristic Stochastic Screen Design . . . . . . . . . . . . . . . . . . . . . . . . . . 189

7.2.3 Halftone Statistics and Optimum AM-FM Screens . . . . . . . . . . . . . . . . . . 190

7.2.4 Optimum Donut Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191

7.3 Error Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

7.3.1 Gray-Scale Error Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

7.3.2 Color Error Diffusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

7.4 Iterative Approaches to Color Halftoning . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

Contents xvii

7.4.1 Color Direct Binary Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

7.4.2 Training Based Halftone Structures Via Iterative Methods . . . . . . . . . . . . . . 205

7.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

8 Secure Color Imaging 219

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

8.2 Visual Secret Sharing of Color Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

8.2.1 Visual Cryptography Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . 222

8.2.2 Color Visual Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223

8.3 Perfect Reconstruction-Based Image Secret Sharing . . . . . . . . . . . . . . . . . . . . . . 224

8.3.1 Color Image Secret Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225

8.3.2 Secret Sharing Solutions for Various Image Formats . . . . . . . . . . . . . . . . . 226

8.4 Cost-Effective Private-Key Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

Appendix: Basis Matrices of Some Popular Threshold Configurations . . . . . . . . . . . . . . . 230

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

9 Color Feature Detection 241

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

9.2 Color Invariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

9.2.1 Dichromatic Reflection Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

9.2.2 Color Invariants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

9.2.3 Color Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245

9.3 Combining Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

9.3.1 The Color Tensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

9.3.2 Color Tensor-Based Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

9.4 Color Feature Detection: Fusion of Color Derivatives . . . . . . . . . . . . . . . . . . . . . 251

9.4.1 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

9.4.2 Feature Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252

9.4.3 Corner Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

9.5 Color Feature Detection: Boosting Color Saliency . . . . . . . . . . . . . . . . . . . . . . . 254

9.6 Color Feature Detection: Classification of Color Structures . . . . . . . . . . . . . . . . . . 256

9.6.1 Combining Shape and Color . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

9.6.2 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

xviii Color Image Processing: Methods and Applications

9.6.3 Detection of Highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

9.6.4 Detection of Geometry/Shadow Edges . . . . . . . . . . . . . . . . . . . . . . . . . 258

9.6.5 Detection of Corners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258

9.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

10 Color Spatial Arrangement for Image Retrieval by Visual Similarity 267

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

10.1.1 Related Work on Modelling Techniques for Representing Spatial Relationships . . . 271

10.2 Modelling Spatial Arrangements of Color . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

10.2.1 Representing Spatial Relationships between Color Clusters . . . . . . . . . . . . . . 274

10.3 Efficient Computation of Weights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279

10.4 Graph Representation and Comparison of Spatial Arrangements . . . . . . . . . . . . . . . 283

10.5 A Retrieval System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285

10.5.1 Retrieval Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

10.6 User-Based Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286

10.6.1 A Benchmark Database of Basic Spatial Arrangements of Color . . . . . . . . . . . 287

10.6.2 Ground Truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

10.6.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

10.6.4 A Benchmark Database of Real Images . . . . . . . . . . . . . . . . . . . . . . . . 290

10.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292

11 Semantic Processing of Color Images 307

11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307

11.2 State of the Art . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

11.3 Knowledge-Assisted Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

11.4 Knowledge-Assisted Analysis using MPEG-7 and Semantic Web Technologies . . . . . . . 316

11.4.1 Overview of MPEG-7 Visual Descriptors . . . . . . . . . . . . . . . . . . . . . . . 316

11.4.2 Ontology Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320

11.4.3 Domain Ontologies Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

11.4.4 Semantic Multimedia Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

11.4.5 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

11.5 Conclusions and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 328

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329

Contents xix

12 Color Cue in Facial Image Analysis 338

12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338

12.2 Color Cue and Facial Image Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339

12.3 Color Appearance for Color Cameras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340

12.3.1 Color Image Formation and the Effect of Illumination . . . . . . . . . . . . . . . . 340

12.3.2 The Effect of White Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 341

12.4 Skin Color Modelling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344

12.4.1 Color Spaces for Skin . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344

12.4.2 Skin Color Model and Illumination . . . . . . . . . . . . . . . . . . . . . . . . . . 345

12.4.3 Mathematical Models for Skin Color . . . . . . . . . . . . . . . . . . . . . . . . . 346

12.5 Color Cue in Face Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348

12.5.1 Overview of Color-based Face Detection Methods . . . . . . . . . . . . . . . . . . 348

12.5.2 Case Study: Face Detection using Skin Locus and Refining Stages . . . . . . . . . . 350

12.6 Color Cue in Face Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353

12.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355

13 Using Colors for Eye Tracking 369

13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369

13.2 Using the IR Colors for Eye Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

13.3 Method Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 372

13.3.1 State Model and Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373

13.4 Observation Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 374

13.4.1 Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

13.4.2 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 375

13.4.3 Likelihood of the Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

13.4.4 Gray-Scale Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376

13.4.5 EM Contour Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379

13.4.6 Color Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 380

13.5 Tracking Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 381

13.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383

xx Color Image Processing: Methods and Applications

14 Automated Identification of Diabetic Retinal Exudates in Digital Color Images 390

14.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390

14.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 391

14.3 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392

14.4 Previous Works on Exudates Identification . . . . . . . . . . . . . . . . . . . . . . . . . . . 393

14.5 Data Collection and Pre-processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395

14.5.1 Retinal Color Normalization and Contrast Enhancement . . . . . . . . . . . . . . . 395

14.6 Region-level Exudate Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

14.6.1 Retinal Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

14.6.2 Color Space Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397

14.6.3 Retinal Image Coarse Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . 398

14.6.4 Fine Segmentation using Fuzzy C-Means Clustering . . . . . . . . . . . . . . . . . 400

14.6.5 Segmentation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401

14.6.6 Feature Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403

14.6.7 Region-Level Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404

14.7 Summary and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412

15 Real-time Color Imaging Systems 419

15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

15.1.1 Real-time Imaging Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 419

15.1.2 Previous Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 420

15.2 Hardware and Display Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421

15.2.1 Color Representation and Real-time Performance . . . . . . . . . . . . . . . . . . . 422

15.2.2 Buffering for Speed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422

15.3 Language Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423

15.3.1 Java . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

15.3.2 Color Image Processing in Java3D . . . . . . . . . . . . . . . . . . . . . . . . . . . 424

15.4 Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

15.4.1 Test Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427

15.4.2 Experiment 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427

15.4.3 Experiment 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 428

15.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 430

Contents xxi

16 Single-Sensor Camera Image Processing 434

16.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434

16.2 Digital Camera Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 436

16.2.1 Consumer-Grade Camera Hardware Architecture . . . . . . . . . . . . . . . . . . . 437

16.2.2 Color Filter Array (CFA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438

16.3 Camera Image Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439

16.3.1 CFA Data Imaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441

16.3.2 Structural Information-Based Image Processing . . . . . . . . . . . . . . . . . . . . 441

16.3.3 Spectral Information-Based Image Processing . . . . . . . . . . . . . . . . . . . . . 442

16.3.4 Generalized Camera Image Processing Solution . . . . . . . . . . . . . . . . . . . . 442

16.4 Edge-Sensing Mechanism (ESM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443

16.4.1 Aggregation Concept-Based ESM . . . . . . . . . . . . . . . . . . . . . . . . . . . 444

16.4.2 Neighborhood Expansion-Based ESM . . . . . . . . . . . . . . . . . . . . . . . . . 445

16.5 Spectral Model (SM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445

16.5.1 Modelling Assumption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446

16.5.2 Advanced Design and Performance Characteristics . . . . . . . . . . . . . . . . . . 447

16.6 Demosaicking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448

16.6.1 Bayer CFA-Based Demosaicking Procedure . . . . . . . . . . . . . . . . . . . . . . 448

16.6.2 Universal Demosaicking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449

16.7 Demosaicked Image Postprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 450

16.8 Camera Image Zooming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 451

16.8.1 Spatial Interpolation of Demosaicked Images . . . . . . . . . . . . . . . . . . . . . 452

16.8.2 Spatial Interpolation of CFA Images . . . . . . . . . . . . . . . . . . . . . . . . . . 453

16.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455

17 Spectral Imaging and Applications 470

17.1 Introduction into Spectral Imaging (SI) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 470

17.1.1 Spectral Imaging as a Generalization of Color Imaging . . . . . . . . . . . . . . . . 471

17.1.2 Analysis of Spectral Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476

17.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

17.2.1 Calibration of SI Equipment in Measuring of Biochemicals in Food . . . . . . . . . 478

17.2.2 SI Systems for Industrial Waste Sorting . . . . . . . . . . . . . . . . . . . . . . . . 482

17.2.3 Classification of Magnetic Resonance Spectroscopic Images . . . . . . . . . . . . . 489

xxii Color Image Processing: Methods and Applications

17.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 492

18 Image Enhancement for Plasma Display Panels 504

18.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 504

18.2 Pulse Number Modulation and Dynamic False Contour . . . . . . . . . . . . . . . . . . . . 505

18.3 Smooth Gray Level Reproduction in Dark Areas . . . . . . . . . . . . . . . . . . . . . . . . 507

18.3.1 Error Diffusion Based Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . 509

18.3.2 Dithering Based Technique . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 512

18.4 Color Reproduction on PDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514

18.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517

19 Image Processing for Artworks Virtual Restoration 530

19.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 530

19.2 Color Cleaning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532

19.2.1 Cleaning Based on Two Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . 533

19.2.2 Cleaning Based on One Image . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534

19.3 Color Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535

19.4 Cracks Removal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537

19.4.1 A Semi-Automatic Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 537

19.4.2 Automatic Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538

19.5 Lacuna Filling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541

19.5.1 A Method Based on Restoration Schools . . . . . . . . . . . . . . . . . . . . . . . 541

19.5.2 A Method Based on Texture Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . 542

19.6 Image Mosaicing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543

19.7 Image Registration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545

19.7.1 Cultural Heritage Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547

19.8 Edge Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 548

19.8.1 Cultural Heritage Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 549

19.9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552

20 Image and Video Colorization 559

20.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559

Contents xxiii

20.2 Fast Colorization Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 561

20.2.1 Colorization Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563

20.2.2 Recolorization and Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565

20.3 Inpainting the Colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 565

20.3.1 Inpainting Colors from Gradients and Boundary Conditions . . . . . . . . . . . . . 566

20.3.2 Comments on Different Variational Formulations . . . . . . . . . . . . . . . . . . . 568

20.4 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 568

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569

21 Super-Resolution Color Image Reconstruction 579

21.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 579

21.2 Previous Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 580

21.3 Generalized Acquisition Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584

21.4 Joint MAP Registration Algorithm with Gaussian-Markov Random Field as Image Prior . . 585

21.5 Regularized Cost Function in Multi-channel Form . . . . . . . . . . . . . . . . . . . . . . . 588

21.6 Estimation of the Regularization Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . 589

21.7 Extension to the Color Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 590

21.8 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591

21.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 594

22 Coding of 2D and 3D Color Image Sequences 602

22.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 602

22.2 Overview of Color Video Coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603

22.2.1 Color Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603

22.2.2 Quantization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605

22.2.3 Motion Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 605

22.2.4 Reconstruction Quality Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . 607

22.3 H.264/MPEG 4 Part 10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607

22.3.1 Video Coding Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607

22.4 Flexible 3D Motion Estimation for Multiview Image Sequence Coding . . . . . . . . . . . . 611

22.4.1 Rigid 3D Motion Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612

22.4.2 3D Motion Tracking Using Kalman Filtering . . . . . . . . . . . . . . . . . . . . . 614

22.4.3 Estimation and Tracking of Flexible Surface Deformation Using PCA . . . . . . . . 615

22.4.4 Estimation of Flexible Surface Deformation . . . . . . . . . . . . . . . . . . . . . . 616

xxiv Color Image Processing: Methods and Applications

22.4.5 Flexible 3D Motion Tracking Using Kalman Filtering . . . . . . . . . . . . . . . . 616

22.4.6 3D Flexible Motion Compensation . . . . . . . . . . . . . . . . . . . . . . . . . . . 617

22.4.7 Experimental Results for Real Multiview Images . . . . . . . . . . . . . . . . . . . 618

22.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620

23 Color-based Video Shot Boundary Detection 628

23.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 628

23.2 Typology of Shot Boundary Detection Algorithms . . . . . . . . . . . . . . . . . . . . . . . 631

23.2.1 Features Used for Shot Boundary detection . . . . . . . . . . . . . . . . . . . . . . 631

23.2.2 Feature Similarity Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 632

23.3 Survey of Shot Boundary Detection Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 633

23.4 TREC Shot Boundary Detection Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 637

23.4.1 Description of the Evaluation Procedure . . . . . . . . . . . . . . . . . . . . . . . . 637

23.4.2 Algorithms Participating in TRECVID . . . . . . . . . . . . . . . . . . . . . . . . 638

23.5 Performance Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639

23.6 Information Theory-based Shot Cut/Fade Detection . . . . . . . . . . . . . . . . . . . . . . 640

23.6.1 Background and Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641

23.6.2 Shot Detection using Entropy Measures . . . . . . . . . . . . . . . . . . . . . . . . 642

23.6.3 Experimental Results and Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 643

23.7 Shot Transition Detection using Singular Value Decomposition . . . . . . . . . . . . . . . . 645

23.8 Feature Level Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 646

23.9 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 648

24 The Use of Color Features in Automatic Video Surveillance Systems 656

24.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656

24.2 Automatic Vision-based Monitoring Systems . . . . . . . . . . . . . . . . . . . . . . . . . 658

24.2.1 Color AVS Systems Logical Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . 658

24.2.2 Color Features in AVS Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 660

24.3 Color-based Processing in Video Surveillance Applications . . . . . . . . . . . . . . . . . . 661

24.3.1 Low Level Algorithms: Filtering and Shadow Removal . . . . . . . . . . . . . . . . 661

24.3.2 Medium Level Algorithms: Object Tracking . . . . . . . . . . . . . . . . . . . . . 663

24.3.3 High Level Algorithms: Classification and Grouped People Splitting . . . . . . . . . 665

24.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 668

Contents xxv

Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669

Index 678