optimum color filters for ccd digital cameras

Optimum color filters for CCD digital cameras

Kai Engelhardt and Peter Seitz

A procedure for the definition of optimum spectral transmission curves for any solid-state (especiallysilicon-based CCD) color camera is presented. The design of the target curves is based on computersimulation of the camera system and on the use of test colors with known spectral reflectances. Colorerrors are measured in a uniform color space (CIELUV) and by application of the CommissionInternationale de l'Eclairage color difference formula. Dielectric filter stacks were designed by simulatedthermal annealing, and a stripe filter pattern was fabricated with transmission properties close to thespecifications. Optimization of the color transformation minimizes the residual average color error andan average color error of 1 just noticeable difference should be feasible. This means that colordifferences on a side-to-side comparison of original and reproduced color are practically imperceptible.In addition, electrical cross talk within the solid-state imager can be compensated by adapting the colormatrixing coefficients. The theoretical findings of this work were employed for the design andfabrication of a high-resolution digital CCD color camera with high calorimetric accuracy.

Key words: Colorimetry, dielectric filter, CCD, digital color camera.

1. IntroductionThe human visual system is a complex and largelystill ill-understood image data acquisition and process-ing system. It is a fortunate fact that the richhuman visual color perception is accurately describedby a simple trireceptor theory that involves the linearcombination of only three different photoreceptortypes with known spectral sensitivity. For this rea-son today's color measurement and color reproduc-tion technology has reached quite a high performance.Most modern techniques for acquisition and renderingof color imagery, however, still produce color picturesperceptibly different from the original scene. A ma-jor reason for this is the difficult selection andfabrication of transmission filter sets that are suit-able for incorporation into color television camerasand color scanners. The present work describes anattempt at designing and producing dielectric filtersets that are either optimum in an absolute, theoreti-cal sense, or close to this optimum and robust tofabrication tolerances at the same time.

What is normally done for consumer color videocameras and color scanners' is that, for a given sensorspectral response, the illumination's spectrum andthe chromaticities of the reproduction primaries [e.g.,

The authors are with the Paul Scherrer Institute Zurich,Badenerstrasse 569, CH-8048 Zurich, Switzerland.

Received 27 May 1992.0003-6935/93/163015-09$06.00/0.c 1993 Optical Society of America.

the phase alternation line (PAL) or national televisionsystems committee (NTSC) monitor's phosphors] the-oretically optimum filter triplet characteristics aredetermined, and-because of fabrication limitationsand cost-one tries to match them with dye mosaicfilters as well as possible. In broadcast cameras withtheir increased performance requirements three-chipsolutions are adopted by using prisms with standarddesign dielectric filters.

According to the trireceptor theory of human colorvision one needs only the signals from three differentreceptors with an all-positive response, the so-calledtristimulus or color-matching curves, to predict thecolor perception of the human visual system and toreproduce this color stimulus accurately by usingsuitable color primaries. Because of the linearity ofthe system, any linear combination of the color-matching curves is also sufficient, and it is alwayspossible to form linear combinations of these signals(so-called matrixing) that can be used directly for the(additive) reproduction of a color stimulus. In thisway perfect color reproduction is possible, in princi-ple, within the reproduction capabilities of the outputdevice. Since there are infinitely many possibilitiesof obtaining linear combinations of the color-match-ing curves with an all-positive transmission, thequestion to be answered in this work is which of theselinear combinations (for which ideal matrixing coeffi-cients can be determined for all perceptible colors)should be selected. Two selection criteria are inves-tigated in this work; first, the theoretical optimum

1 June 1993 / Vol. 32, No. 16 / APPLIED OPTICS 3015

with respect to minimum perceptible color noisecaused by the unavoidable photon noise, and second,small perceptible color noise (close to the theoreticaloptimum) but for which it is possible to fabricate gooddielectric transmission filters with their normal fabri-cation tolerances.

The presented theoretical studies were applied tothe development of the prototypes of an advanceddesktop publishing color camera system as part of theEuropean research initiative ESPRIT (European Stra-tegic Program for Information Technology) II project2103 MASCOT (Multienvironment Advanced Systemfor Colour Treatment). Dielectric filters were de-signed and fabricated according to target transmis-sion curves that were obtained from our simulations,taking into account the illumination's spectrum andthe sensor's spectral sensitivity. Residual colorimet-ric errors were minimized by matrixing, reducing theeffects of the nonideal filter fabrication and inaccura-cies in the measurement of the sensor's spectralsensitivity.

2. Baseline for Filter DesignThe goal of this work was to provide the theoreticalframework for the design of color filters for a high-resolution digital still camera with high calorimetricaccuracy. Such a camera would find many practicalapplications in desktop publishing, the graphic arts,the printing industry, and the medical area. Theseapplications determine the boundary conditions forwhich the performance of the camera should beoptimized.

Since color can be represented unambiguously by avector with only three components, three individualdetectors (pixel types), K, L, M, with suitably chosenspectral responses are sufficient to determine thelocus of each color within the space of perceivablecolors. If a camera should be capable of perfectcalorimetric reproduction of objects, three-color chan-nels must be established with spectral responsesequal to a set of linear combinations of the standardcolor-matching functions x, y, . Usually images ofhighly structured objects are acquired with the cam-era. Because the individual colored fields are viewedunder only a small angle, the 2 color-matchingfunctions3 [CIE (Commission Internationale del'Eclairage) 1931] apply.

The main fields of application for this color camerashare the availability of high levels of illuminancewith a suitable spectral distribution. Therefore, incontrast to consumer video cameras, the filter setshould be optimized for bright-light conditions toachieve minimum color errors from photon noise.

To achieve programmable, high spatial resolution,the camera incorporates a scanning mechanism withwhich the scene can be shifted laterally with respectto the 1k x 1k pixel CCD chip (Thomson THX 31156)employed. This is realized with an optical microscanmodule that forms part of the lens system, whichconsists of a parallel glass plate that can be tilted insmall steps around two orthogonal axes. In this

way, not only can the spatial resolution of the camerabe significantly enhanced but it also becomes possibleto expose a scene in three steps so that every detail ofthe scene can be seen through three different colorfilters, arranged in a certain pattern on top of thepixels. The arrangement of color filters chosen forthis purpose is simple vertical stripes, making cross-talk corrections easier and avoiding complex colordecoding algorithms and antialiasing measures suchas the ones required for the shift patterns."4 Thereason for selecting patterned color filters instead ofsequential exposure through uniform, large-area fil-ters was the desire to use the camera also for single-shot color exposures of moving objects, albeit atreduced spatial resolution. For a detailed descrip-tion of the complete camera system see Ref. 2.

3. Design of the Filter Transmission CurvesThe color filters were realized as multilayer dielectricfilters arranged as stripes on a glass substrate andglued onto the top of the CCD imager. The applica-tion of advanced filter design techniques by usingsimulated thermal annealing 5 6 allows almost anydesired filter transmission characteristic to beachieved. Only fabrication tolerances represent alimitation for the practicability of a certain filter set.

A. Simulation Model of the SystemThe camera was simulated by using the simplifiedmodel shown in Fig. 1. The object is illuminatedwith a known spectral power distribution S( X).Light reflected by the object is modified by thespectral reflectance p( X), and a certain amount passesthe lens and the infrared cutoff filter with spectraltransmittances Tiens( A) and TIR( A). The spectral trans-mittances and the spectral response sccD(X) of theCCD array are multiplied and normalized to obtainthe basic spectral response Sb/w(X) of the camerawithout color filters:

Sb/w(X) c Tlens(X) X TIR(X) X SCCD(X). (1)

Figure 2 displays these spectral distributions that arebasic to our filter design. At short wavelengths thespectral response Sb/w( A) is determined mainly by the

illumination S(A)

objectp(A)

/ /lens IR-cutoffnecn (A) rmR(A) .

COD(A

color filters

TK (A) r 0) TM 0)

_ - X R

..V- Y (G)

X (R.X)

matring

Fig. 1. Simulation model of the transmission chain of colorinformation.

3016 APPLIED OPTICS / Vol. 32, No. 16 / 1 June 1993

-

1.

0 . 8

IJ0)

Ea

PJ

0 . 6

. 4

0 . 2 StlfSCCD() X

4:0r 0 5:0:0 600 700wavelength A/nm

Fig. 2. Normalized basic spectral response Sb/w(X) of the cameracomposed of the spectral response sccD(X) of the CCD image sensor,and the spectral transmissions Tens(X) of the lens and TIR(X) of theinfrared cutoff filter.

spectral response of the CCD array and at longwavelengths by the infrared cutoff filter.

For each color channel K, L, M the spectral re-sponse sb/w(X) is further multiplied by the transmit-tances rK(X), rL( X), TM( X) of the color filters to yield thespectral responses SK(A), SL(X), SM(A). In theMASCOT digital CCD camera the video signal isdigitized pixel synchronously by a 10-bit analog-digital converter at a measured nonlinearity of 67dB.7 Color transformation and any subsequent sig-nal processing are performed in the digital domain.The effects of quantization on the colorimetric perfor-mance of the camera are discussed in Section 6.

B. Estimation of Colorimetric Performance and Matrixing

Given the transmission spectra TK( X), TL( X), TM( A) of aspecial filter triplet the calorimetric performance ofthe camera can be checked by computer simulation ifa set of test colors is provided with known spectralreflectances. Each test color will modify the spec-trum of illuminance according to its spectral reflec-tance p( X). For the spectrum of illuminance S( X) weused the spectrum of average daylight D65, a CIEstandard.3 In a first step the resulting spectrum ismultiplied by the corresponding values x( X), y( X), 2( X)of the standard color-matching functions and summedover wavelength X to obtain the (XR, YR, ZR) tristimu-lus values of the test color as a reference. Thesummation was carried out at intervals AX of 5 nmfrom 380- to 780-nm wavelength.

In a second step the values from the standardcolor-matching functions are substituted by valuesfrom the spectral responses of the camera sK( X), SL( X),SM( X) to obtain tristimulus values (K, L, M) measuredat the output of the CCD array. A linear transforma-tion matrix r is applied to color vector (K, L, M) toobtain tristimulus values (Xc, Yc, Zc) in the CIE 1931XYZ system at the output of the simulated camera.

If the responses of the color channels K, L, M showdeviations from a set of linear combinations of x, y, 2or if noise affects the tristimulus values (K, L, M),

then no linear transformation X can be found toproduce correct tristimulus values (Xc, Yc, Zc) for alltest colors simultanously.

To estimate the resulting color error for each testcolor we transformed the color vectors from bothsteps (nonlinearly) from the CIE 1931 XYZ space to auniform color space in which color differences areperceived approximately uniformly:

(XR, YR, ZR) '- (LR*, UR*, VR*),

(Xc, Ye, Zc) (Lc*, UC*, VC*).

In our simulation the CIE 1976 L*, u*, v* system3

(CIELUV) was employed but, in general, results thatwere obtained by using the CIE 1976 L*, a*, b*system3 (CIELAB) proved to be similar.

Next, the distance of both color vectors in theuniform color space was calculated by using the CIEcolor difference formula3 to obtain the colorimetricerror AE for the reproduction of each test color:

AE = [(LR* - Lc*)2 + (UR* - UC*) + (VR* VC-)]

(2)

If the spectral responses of the three-color channelsdeviate from any set of linear combinations of thestandard color-matching functions, optimization8 ofthe color transformation matrix will help to minimizethe systematic color error of the camera. For mini-mum perceivable color errors a nonlinear optimiza-tion in the uniform color space (CIELUV or CIELAB)must be carried out. The following procedure, whichis similar to that presented in Ref. 1, proved to beefficient: In a first step a color transformation ma-trix - is determined that minimizes the mean qua-dratic error of the reproduced test colors in XYZspace. In this linear optimization three systems oflinear equations have to be solved. In a second stepa nonlinear optimization based on the method ofsteepest descent is applied that uses a variable stepwidth to decrease the total number of steps. As aninitial transformation matrix for the nonlinear optimi-zation we use the matrix that resulted from linearoptimization. We obtain a matrix opt for the trans-formation of detector tristimulus vector (K, L, M) totristimulus vector (X, Y, Z) in the standard XYZspace that minimizes the perceivable color errors atthe output device. If, for example, colors should bedisplayed on a red-green-blue (RGB) monitor withPAL phosphor primaries, then the final transforma-tion matrix results from the matrix product RpAI21opt,where matrix 3TpAL performs the transformation tothe specific output device including an optional shiftin reference white.

The mean color error that results for a special set offilters strongly depends on the set of test colors that isused to optimize the camera's performance. Our setof test colors consists of approximately 300 carefullyselected colors at various relative luminous reflec-tances with most of them highly saturated. The lociin the CIE chromaticity diagram of the various test


colors are shown in Fig. 3. The set includes the CIEstandard test colors (indicated by * in Fig. 3) that arerequired to calculate the CIE general color renderingindex3 R0, optimal colors3 (indicated by + in Fig. 3) ofdifferent dominant wavelength, and luminous reflec-tance as well as a subset of 200 samples of test colors(indicated by A, 0, V in Fig. 3) from the commerciallyavailable COLORCURVE9 ATLAS. Optimal colorswere included in the simulation because these colorsrepresent the theoretic limit of object colors underthe given spectrum of illumination and, for mediumand low relative luminous reflectances, are localizedfar outside the color triangle defined by the chromatic-ity coordinates of the three primaries of the monitor.Such colors cannot be reproduced correctly on amonitor, but the camera can be used to measure theircolor coordinates. Colors from the Colorcurve sys-tem are especially suited for the design of color filtersby computer simulation because spectra of reflec-tance are provided for each color sample and, afterrealization, actual performance can be investigatedby using the real color samples. The spectra ofreflectance of the Colorcurve colors are provided from400 to 700 nm in steps of 20 nm accompanied bytristimulus values in the CIE 1964 XYZ system of the100 standard observer under illuminant D65. Oursimulation works with spectral data from 380 to 780nm in steps of 5 nm. Therefore, the original datamust be extrapolated and then interpolated. In Ref.10 various techniques for extrapolations of truncatedspectra are presented. In contrast to these auto-matic techniques that follow fixed rules, we prefer toadd reasonable looking values at both limiting wave-

y0 . 8 0

0 . 6 0

0 .4 0

0 .2 0

0.00

0 .0 0 0.20 0.40 0.60

Fig. 3. Loci of chromaticity in the CIE XYZ system of the set oftest colors used in simulation and optimization. The set consistsof the CIE standard test colors (), optimal colors (+) of differentluminous reflectances, and a subset of 200 samples of test colorsfrom the COLORCURVE ATLAS at different levels of luminousreflectance (A, 0, V').

lengths by hand for extrapolation and to interpolatethe spectra at 5-nm intervals by using a cubic splineinterpolation. Finally we checked if the manipu-lated spectra still represent the original colors thatexist as real samples and, if necessary, the procedurewas repeated by using improved values in the extrap-olation. For a check of the manipulated spectra, wecalculated their tristimulus values in the CIE 1964XYZ system under illuminant D65 and comparedthese results with the original data provided byColorcurve Systems.

The optimization of matrixing was carried out overthe subset of Colorcurve test colors only. All valuesAE of estimated average colorimetric performancethat are given in the following sections were obtainedafter nonlinear optimization of matrixing.

C. Filter Triplet for Optimum Noise PerformanceFor the target fields of application of the camera thefilter triplet should be optimized for high levels ofilluminance. Therefore one goal is to find threefilter transmission curves and the correspondingcolor transformation matrix that are matched to aspecific output device and show minimum color errorat the output that originates from temporal noise inthe photodetector. At high levels of illuminancePoisson-distributed photon noise predominates. Asoutlined before, the filter transmission curves mustbe chosen such that the basic spectral response Sb/w( X)of the camera is modified to yield spectral responsesof the color channels that are linear combinations ofthe color-matching functions.

Any linear combination of the color-matching func-tions can be addressed in the following way: Eachcolor channel is replaced by an imaginary primarystimulus that emits light of a spectral power distribu-tion that is equal to the spectral response. Thechromaticity coordinates (x, y, z) of the three imagi-nary primary stimuli must be selected to be linearlyindependent of each other. By solving a system oflinear equations the basic linear transformation 9can be derived from these chromaticity coordinates.The spectral response curves of the color channels areobtained if the inverse transformation matrix Z-' isapplied to the color-matching functions x- , .Nonnegative response curves that can be realized byusing a photodetector and filters are obtained auto-matically only if the triangle in the x-y plane formedby the chromaticity loci of the three imaginary pri-mary stimuli contains the spectrum locus completely.

The colorimetric quality of the individual filtertriplets that are determined by this procedure differsonly in the noise performance. To check noise perfor-mance in the simulation we disturbed the tristimulusvalues (K, L, M) of each test color by a set of 125disturbance vectors that approximate a Gaussianamplitude distribution in each color channel to simu-late the temporal noise from each detector element.The temporal noise consists mainly of photon noise,noise from a dark signal, and read noise that origi-nates from the on-chip output amplifier of the CCD


imager. The number nik, k = K, L, M of photogener-ated electrons of each pixel is proportional to thecomponent of the tristimulus value (Ki, Li, Mi) of testcolor i that corresponds to the specific color of thepixel; e.g., ni,K = ne X Ki.

The conversion factor n,- is given by the conven-tion that a tristimulus component of 100 is equivalentto a saturated detector element with a number ofphotoelectrons given by the full well capacity of theCCD imager. In addition the detector element isfilled with the number nd, of electrons that aregenerated by dark current that is sensitive to temper-ature and proportional to the sum of the integrationtime of the imager and the time required for read out.The temporal noise voltage of each pixel is propor-tional to the square root of the total number ofelectrons collected in the pixel. Because this noisevoltage is independent of the noise voltage that isgenerated by the output amplifier, these voltagesmust be added like vectors and the standard deviation(Ti,k of signal electrons from each pixel can be calcu-lated for each test color i; e.g., for color channel K

9iK = {[(niK + nd, )1/2]2 + n read 2}11/2 (3)

The number nread of noise electrons contributed bythe output amplifier is obtained if the noise voltage ofread noise is converted to electrons by using theconversion factor F of the output amplifier, whichgives the output voltage change per signal electronand which typically is of the order of 1 ,uV/e-.

The Gaussian amplitude distribution was approxi-mated by the five discrete amplitudes -1.4Uisk,-0. 5 3 9i,k, 0, +0.53riu, +1. 4 0iTk, which follow if thearea under the Gaussian function is divided into fiveregions of equal area and the center of gravity isdetermined for each of these regions. The combina-tion of these five discrete noise amplitudes for eachcomponent k gives a set of 125 different noise vectors,which backconverted to the KLM space, represent anisotropic disturbance of the tristimulus vector(Ki, Li, Mi) of test color i as shown schematically inFig. 4(a). To estimate the effect of temporal noise oncolor reproduction of the CCD still camera we musttransform the whole set of disturbed tristimulusvectors to the uniform color space for each test color.By using this transformation the isotropic distur-bance of the color vector in general is changed to ananisotropic disturbance that may look like the oneshown in Fig. 4(b). For each test color i the colorimet-ric effect of temporal noise can be described by adifference vector AEi in the uniform color space thatindicates the magnitude and direction of the per-ceived color shift by noise and that is calculated byaveraging over the color shift vectors AEij, j = 1 ...125:

1 125AE, =~ _5 , Ei3j

= 125 * A(j=)

AEi j =(ALi j*, Auij*, Avij*). (4a)

Fig. 4. (a) Illustration of an isotropic disturbance of the tristimu-lus vector (Ki, Li, Mi) in the detectors KLM space by photon noise,noise from a dark signal, and read noise. (b) Illustration of theanisotropic disturbance of the color vector resulting from thenonlinear transformation of the isotropic disturbances in KLMspace (a) to a uniform color space, for example, the CIELJV space.

From all the individual color shift vectors AEij wecan calculate the value of the average color error AEifrom temporal noise for test color i and finally thetotal color error AEnoie that is due to noise that isaveraged over the set of N test colors:

1 125AEi = -125 (5E,j Q5E,'j)1/2,

1 N

AEnoise = N AEi.i=1

(4b)

(4c)

This evaluation procedure is applied to many differ-ent filter sets and we find a filter triplet that, for thespecific CCD imager in our application, provides thbminimum color errors from temporal noise. InFig.5(a) the dotted curves show the target spectraltransmission characteristics of the filter triplet; thesolid curves show the transmission characteristics ofthe filter triplet that results from the design of thedielectric filter stacks by simulated thermal anneal-ing.5 ,6

Assuming an integration time of 250 ms, a readtime of 240 ms, and a maximum saturation of 80%full well, a total color error AEnoise of 1.1 jnd (CIELUV)for the subset of Colorcurve test colors results, whichis just above the human discrimination threshold forcolors and which would be difficult to detect. Ageneral color rendering index Ra of 95 is found if onlynoise is taken into account. The average color shift


(a) L

M

K

(b) L*

AE 1

1. 0

0 .8

0 .6

0 .4

0 . 2

0 .0

1. 0

0 . 8

0 .6

0 .4

0 . 2

0 .0

1. 0

0. 8

0. 6

0 .4

400 500 600 700

wavelength A/nm

400 500 600 700

wavelength A/nm

0 .2 7 I .

400 500 600 700

wavelength A/nm

Fig. 5. (a) Filter triplet for optimum noise performance: targetspectral transmission (dotted curve) and spectral transmissioncurves of the designed dielectric filter stacks (solid curve). (b)Optimum practical filter set: target spectral transmission (dottedcurve) and spectral transmission curves of the designed dielectricfilter stacks (solid curve). The additional global filter G (dashedcurve) is deposited onto a separate substrate and inserted into theimaging path to prefilter the light of all the color channels. (c)Optimum practical filter set: target spectral transmission (dottedcurve) and spectral transmission of the fabricated dielectric filtersK, L, M (solid curve) and the global filter G (dashed curve).

IAE, I was calculated to be extremely low and for nocolor i exceeds 0.02 jnd. Under the same conditionsthe signal-to-noise ratio was calculated for PAL RGBoutput signals and the spectrum of illuminant D65.

The signal to noise ratio was 43 dB for the redchannel, 50 dB for the green channel, and 45 dB forthe blue channel.

The transmission curves obtained from the dis-cussed filter design show only slight departures fromthe target transmission curves as can be seen in Fig.5(a). The application of the designed filter triplet'sspectra in the simulated camera gives a systematiccolor error AE of 0.8 jnd (CIELUV). The generalcolor rendering index Ra is calculated to be 97.

The design of the dielectric filter stacks shows thatthis filter triplet would be difficult to manufacturewith tight tolerances, especially if the filter stacks ofthe individual colors have to be deposited sequentiallyonto the same substrate to produce a patterned stripefilter. For each of the three filter stacks the designdemands 21 layers with total stack heights between1.44 and 1.77 Rxm. If fabrication tolerances aretaken into account this filter triplet surely is notoptimum.

D. Optimum Practical Filter Set: Global Filter Concept

In general, dielectric filters are easier to design andfabricate if the spectral region decreases in which thefilter must match a desired target transmission espe-cially at short wavelengths. Therefore we decided toadd a global filter to the triplet of color filters. Thisglobal filter covers all the detector elements and isdeposited on a separate substrate, which in our case isthe infrared cutoff filter. The global filter definesthe transmission edges of the whole filter set in theblue and in the red. Because of this additional filtertwo of the three color filters become much simplerand the sequential deposition of the patterned stripefilter can be performed with a higher precision.

Furthermore, we searched for a filter triplet with ablue filter M with a transmission of a few percent inthe long-wavelength spectral range because design ofthe dielectric filter stacks becomes less critical ifnearly zero transmission in this range is not required.The transmission characteristics of the filter set wefound are shown in Fig. 5(b); the dotted curvesdisplay the target transmissions, the solid curvesshow the transmission curves that result from thedesign of the filter stacks. The dashed curve illus-trates the spectral transmission of the designed glo-bal filter G that requires 12 dielectric layers with atotal thickness of 1.1 Am. The filter of the redchannel K is composed of 13 layers with a totalthickness of 0.74 pum. The filters of the green andblue channels L and M require 19 layers each with atotal thickness of 1.35 and 1.47 ,um, respectively.

If the target transmission characteristics are com-bined with the camera's basic spectral response Sb/w,the resulting spectral responses SK, SL, SM are nolonger a numerically exact linear combination of x, 5i,2 because the initial target filter transmission datawere smoothed by using cubic spline approximation.Target transmission data were obtained that showedno artificial sharp edges or spectral regions of con-stant transmission both of which would complicate


4)

;t!

a._

I)U

.:Z2

4)IU

Cu

8ct

EH

c_

the design of the dielectric stacks. Target and de-sign curves are in good agreement and from both setsof data a mean colorimetric error AE of 1.0 jnd(CIELUV) is calculated after nonlinear optimizationof matrixing. The general color rendering index Rais calculated to be 97.

Noise performance of this filter triplet is estimatedto be only slightly worse than that of the previoustheoretically ideal filter triplet. Under the sameconditions a total color error Anoise of 1.3 jnd(CIELUV) is calculated for the subset of Colorcurvetest colors. Color errors from noise only produce ageneral color rendering index Ra of 95. The averagecolor shift AEi is again lower than 0.02 jnd for everycolor i. The signal-to-noise ratio for PAL RGB out-put signals and D65 illuminant is calculated to be 41dB for the red channel, 49 dB for the green channel,and 45 dB for the blue channel.

4. Filter Fabrication and CharacterizationThe dielectric filters were fabricated by e-beam depo-sition of TiO2 and SiO2; see, for example, Ref. 11.First, the global filter was deposited onto the infraredcutoff filter. The dashed curve in Fig. 5(c) shows itsspectral transmission versus the dotted design charac-teristics. The color filters were deposited onto on athin glass substrate that was subsequently glueddirectly onto the CCD imager. For optimum colori-metric reproduction the green filter L was the mostcritical and was therefore fabricated as the first of thepatterned stripe filter to allow a maximum number oftrials. Figure 5(c) shows the spectral transmissioncharacteristics of the fabricated filters as solid curvesversus the design curves that are dotted. Design andrealization are in fairly good agreement.

5. Compensation of Color Cross TalkColor cross talk occurs from photons that enter thefilter of a specific color channel and the photogener-ated charge carriers travel inside the silicon of theCCD array such a distance that detection takes placein the pixel of a different color channel. The meandistance that a charge carrier will travel in silicon ishigher for longer wavelengths because the long-wavelength photons are absorbed deeper in the silicon.Therefore the amount of cross talk differs for photonsthat entered different color filters and color errorswill arise. The largest cross talk is expected fromchannel K; the lowest from channel M. Becausecolor cross talk is linearly dependent on intensity itcan be compensated almost completely by a lineartransformation that may be combined with the colortransformation matrix 7.

In all modes of the MASCOT camera each imagesampling area is digitized subsequently by a pixel ofeach of the three-color channels to obtain colorsamples K, L, and M of each sampling area. Sincecross talk influences each acquired image in itself, itshould be corrected for after each single microscan-ning step. In contrast, the color transformation willbe applied to tristimulus values (K, L, M) that come

from different microscanning steps. But if the ob-ject color in the image on the CCD array variessmoothly from pixel to pixel and the imaging condi-tions remain fixed, then, nevertheless, color cross talkcan be removed almost completely.

Because color cross talk is a linear effect it can bemeasured by the projection of a suitable spot of whitelight subsequently onto a K, an L, and an M pixel.The spot ideally should have the same size and shapeas the photosensitive part of a pixel. Since in ourcase the pixels were additionally masked with achrome mask that left only the center part of the pixelarea photosensitive (- 25% of the pixel area), it waseasy to achieve this in practice. With no cross talkpresent three tristimulus vectors (K, L, M) are ob-tained that, after normalization, have the values(1, 0, 0), (0, 1, 0), (0, 0, 1). From the three measuredtristimulus values (Kk', Lk', Mk') the coefficients Cik, i,k = K, L, M, of cross-talk matrix F can be calculated.Index k denotes the color of the pixel onto which thespot was projected and index i the color channel inwhich the signal was detected. For example, thefirst column of W follows from the measurement of apixel of color K:

CK = 1CLK CMK, CLK K + LK +M,

XK'CMK KF+=,M,(5)

KK + LK + MKI

Matrix W transforms the ideal tristimulus values(K, L, M) to real tristimulus values (K', L', M'), whichare disturbed by color cross talk:

K' 1- LK-CMK

L, - CLK

M CMK

CKL CKM

1-CKL-CML CLM

CML 1-Cal-CLM

X( . (6)

Because cross talk reduces contrast, all the coeffi-cients of matrix T are positive. Color cross talk canbe removed by the application of the inverse trans-form W - to the measured tristimulus vectors and thecolor transformation can be extended to includecross-talk compensation, RpALIptF-1 Thus colorcross talk will not degrade calorimetric performance,but, because of the increased coefficients in the colortransformation, a lower signal-to-noise ratio will re-sult.

6. Influence of QuantizationUp to now all the estimations of colorimetric perfor-mance have been done under the assumption thatmatrixing is performed with floating point precision.But in the MASCOT CCD digital camera the signalsare handled as integer values and quantization errorscan introduce additional color errors. Therefore,


different combinations of digital dynamic range in theMASCOT camera are simulated.

The video signal from the CCD imager is digitizedwith a quantization of n, bits to obtain digital tristim-ulus values (K', L', M'). Intermediate sums and out-put values of the color transformation are limited toan n2 bit and the final output values that appear atthe output lookup table are rounded to n3 bit. Table1 summarizes the results of the simulation for differ-ent digital dynamic ranges n, n2, n3. The meancolorimetric errors A hold for the transmissioncurves of Fig. 5(c) of the fabricated filters, illuminantD65, and the optimized transformation R'pALe2 opt forPAL RGB output data.

The camera was built with an input quantization n1of 10 bits, a quantization n2 of matrixing of 16 bits,and an output quantization n3 of 8 bits. As can beseen from Table 1 the limitation at the outputintroduces the main degradation in clorimetric per-formance, but the additional errors are so small thatthey can hardly be detected by human observers.

7. Experimental Results and DiscussionWe tried to determine the color cross-talk compensa-tion matrix 9-1 after the procedure detailed in Sec-tion 5 in two different ways. First we measured thecolor cross talk on a masked black-and-white (b/w)CCD imager by the projection of small spots of lightthat were suitably prefiltered to exhibit the spectrumof light that entered a K, L, or M filter on the colorimager. We obtained the following cross-talk matrix

[0.88 0.04 0.031Tbw= 10.06 0.92 0.03 I (7a)

[0.06 0.04 0.94]

This measurement was repeated with a white-lightspot on a color imager that, in addition, had the KLMcolored stripe filter on top and obtained the followingmatrix F'coIor:

[0.94 0.06 0.241W'color = 10.04 0.90 0.16 I (7b)

-0.02 0.04 0.60]

The two matrices differ substantially and, while

Table 1. Colorimetric Effect of Quantization In the MASCOT CCD DgitalCamera

AEni n2 n3 U~~~~~(nd) R

00 00 00 1.0 9712 16 10 1.0 9610 16 10 1.1 9612 16 8 1.2 9610 16 8 1.3 9610 14 8 1.3 968 16 8 2.0 92

matrix Wb/, looks quite reasonable, matrix Wcolor

shows a strong cross talk from the blue channel Minto the green and red channels, which is not under-stood. We decided to measure the tristimulus vec-tors (K', L', M') of a large number of Colorcurve colorsamples with known spectra of reflectance and undera known spectrum of illuminance and to determinethe cross-talk matrix by nonlinear optimization ofmatrixing. The color samples were measured under(45/0) illumination and viewing conditions at aviewing angle of 1.50, which corresponds to a field of

~3 300 detector pixels. We obtained the follow-ing cross-talk matrix W,'opt:[ 1.21 -0.01 0.141

Fapt = -0.09 0.91 0.30 L 0.08 0.03 0.431(7c)

Although F'opt differs largely from W'b/w and W'color, itconfirms the measurements we obtained from thecolor imager. In principle, the sums of the columnsof cross-talk matrix '9 must be equal to one, and eachcoefficient must be positive. The nonlinear optimiza-tion of matrixing did not notice these restrictions buttried to minimize the resultant colorimetric errors aswell as possible. In this way negative coefficientsappeared, and the sum of each column was differentfrom the valve one. Therefore, it was concludedthat available data on the CCD response SCCA(), asprovided by the CCD manufacturer at the beginningof this work, which was the only data we could notmeasure directly, were inaccurate, especially in theshort-wavelength region. This suspicion was con-firmed by a series of measurements of the relativesensitivity of the b/w imager at discrete wavelengthsby using narrow-bandpass interference filters.

As a consequence the target filter curves no longerprovide spectral responses of the color channels thatare linear combinations of the color-matching func-tions , 5i, ~. The mean clorimetric error of theMASCOT CCD digital camera prototype, which re-sulted after optimization of matrixing, was 2.65 jnd(CIELUV). This is slightly better than the perfor-mance of today's high-quality broadcast cameras.Visually the acquired images showed an exceptionallyhigh clorimetric fidelity when they were displayedon a high-quality color monitor as well as a gooduniformity over the whole field.

8. ConclusionA procedure for the development of color filters forprecise color reproduction has been presented, whichis based on clorimetric standards, computer simula-tion, and nonlinear optimization. Advanced designtechniques for the dielectric filters allowed the use offilter transmission curves that ideally would not showsystematic clorimetric errors and that were opti-mized to show no apparent color noise. The introduc-tion of an additional global filter helped to reduce therequirements for the individual color filters and thusfacilitated fabrication.


The transmission curves of the fabricated filterswere in close agreement with the target curves andwould cause a mean colorimetric error of only 1.3 jnd(CIELUV) if an optimized color transformation isapplied that minimizes the color errors from thesmall deviations of the transmission curves and ifquantization errors are taken into account.

The prototype camera showed a lower colorimetricperformance than that predicted by simulation.Investigations showed that this was due to the filterdesign being based on available but slightly inaccu-rate input data of the imager's spectral response.Based on subsequent experimental data, the colortransformation could be optimized to provide a meancalorimetric error of the actual camera system of

2.7 jnd (CIELUV).

We gratefully acknowledge the help of many peopleat the Paul Scherrer Institute Zurich who contrib-uted to the ESPRIT II MASCOT project and particu-larly to the present work. We express our apprecia-tion to K. Knop for many stimulating discussions andfor suggesting the global filter approach. We thankR. E. Kunz for providing the many computer designsthat were required to determine the optimum dielec-tric filters and H. Brunner for actually fabricatingthem. We also thank J. M. Raynor for designing andbuilding the hardware that was needed to take themeasurements for characterizing and optimizing theMASCOT camera. In addition, we thank M. T. Galefor his continuing support and management of theMASCOT project, as well as for critically reading thismanuscript. We also gratefully acknowledge thecross-talk measurements contributed by R. Zum-brunn and H. Gerber (Leica Heerbrugg). This workwas supported in part by the Swiss Commission forthe Advancement of Scientific Research, Project KWF-1804.2.

References1. S. Suzuki, T. Kusunoki, and M. Mori, "Color characteristic

design for color scanners," Appl. Opt. 29, 5187-5192 (1990).2. H. Brunner, K. Engelhardt, M. T. Gale, G. K. Lang, P. Metzler,

J. M. Raynor, P. Seitz, L. Brissot, G. Cilia, C. Gadda, and V.Leone, "High fidelity, programmable CCD colour camera fordesk-top publishing and the graphic arts," in Proceedings ofthe Annual ESPRIT Conference 1991 (Commission of theEuropean Communities, Luxembourg, 1991), pp. 439-454.

3. G. Wyszecki and W. S. Stiles, Color Science: Concepts andMethods, Quantitative Data and Formulae, 2nd ed. (Wiley,New York, 1982), Chap. 3.

4. K. Knop and R. Morf, "A new class of mosaic color encodingpatterns for single-chip cameras," IEEE Trans. ElectronDevices ED-32, 1390-1395 (1985).

5. R. Morf and R. E. Kunz, "Dielectric filter optimization bysimulated thermal annealing," in Surface Measurement andCharacterization, J. M. Bennett, ed., Proc. Soc. Photo-Opt.Instrum. Eng. 1019, 211-217 (1988).

6. R. Morf and R. E. Kunz, "Dielectric filter optimization bysimulated thermal annealing: a simulated zone-meltingapproach," in Optical Thin Films and Applications, R. Herr-mann, ed., Proc. Soc. Photo-Opt. Instrum. Eng. 1270, 11-17(1990).

7. J. M. Raynor and P. Seitz, "The technology and practicalproblems of pixel-synchronous CCD data acquisition for opti-cal metrology applications," in Close-Range PhotogrammetryMeets Machine Vision, E. Baltsavias and A. Gruen, eds., Proc.Soc. Photo-Opt. Instrum. Eng. 1395,96-103 (1990).

8. A. H. Jones, "Optimum color analysis characteristics andmatrices for color television cameras with three receptors," J.SMPTE 77,108-115 (1968).

9. Colorcurve MasterAtlas (Colorcurve Systems, Inc., Minneapo-lis, Minn., 1988).

10. D. L. MacAdam, Color Measurement, Vol. 27 of OpticalSciences (Springer-Verlag, Berlin, 1981), pp. 64-65.

11. M. T. Gale, H. W. Lehmann, H. Brunner, and K. Frick,"In-situ optical monitoring of thin films during evaporation,"in Surface Measurement and Characterization, J. M. Bennett,ed., Proc. Soc. Photo-Opt. Instrum. Eng. 1019, 90-95 (1988).


optimum color filters for ccd digital cameras

Documents