Evaluation of processes used in screen imperfection algorithms

Siavash A. Renani

Introduction

Screen compensation algorithm Divided in four parts

– Projector characterization– Camera characterization– Geometrical alignment– Screen compensation

“A Projection System with Radiometric compensation for Screen Imperfections”, Nayar et al.

“Making One Object Look Like Another: Controlling Appearance Using a Projector-Camera System”, Grossberg et al.

”Robust Content-Dependent Photometric Projector Compensation”, Ashdown et al.

Motivation

Screens increases the cost of projectors Screens takes up space Screens decreases projectors mobility

– And therefore decreases functionality.

Can alter color of objects (Virtual offices).

Index

Thesis– General– Goal

General model for characterization Projector Camera Geometrical alignment

Thesis-general

This thesis focus on the different steps of achieving screen independence.

Evaluated 2 projector characterization methods and established their parameters.

Evaluated 4 camera characterization methods and established their parameters.

Transformation of coordinates of the screen from the captured image to the original image.

Use of regression to compensate for the screens effect.

Thesis- general

Color I is projected

Camera captures projected colors.

Colors are again modified, this time by the camera

Colors are modified by the projector.

Colors are modified by the

screen

Thesis - general

Input and output devices are restricted by their sensors and/or ability to reproduce colors.

To be able to calculate how screens modify colors, we need to know how input and output devices modify them first.

Thesis-Goal

Evaluate characterization methods for camera

Evaluate characterization methods for projectors

Implement Geometrical alignment algorithm Investigate the effect of screen

compensation as the characterization error changes.

General model of characterization

RGB Linearization

Transformation to device-

independent values

Ex.Spline interpolation

B

G

R

AZYX

Projector –Resarch Questions

How many colors are needed for linearization using linear, spline and cubic interpolation?

How will PLCC compare against a characterization using regression?

How many colors in the training set is needed to for the color difference to be considered hardly visible, when regression is used?

Projector - Characterization methods

3 different interpolation techniques for linearization.

Piecewise Linear assuming constant chromaticity model (PLCC).

Regression

Projector-experiment

Gamut of the projector

150 Random colors33

colors pr

ramp

100 colors

for test-set

51 colors for the training-set

10 to 20colors

10 to 20colors

Color difference is calculated for different amount of colors used in linearization and as trainining-set.

PLCC do no require training-set.

Different interpolaiton techniques was used to linearize RGB.

Projector: conclusion

PLCC performed better than regression. With only 12 colors used in linearization acceptable result is achieved.– Possible threat: The assumptions of the PLCC model

is correct for the test-set but not for the whole gamut.

It is possible to achieve good result with regression using 12 or more colors for linearization and 12-18 colors in the training-set.

Camera Research questions

How many colors should be used for regression? What order of polynomial regression should we use? How will the use of only the cubic root function

before transformation to LAB perform? How will use of CIELAB compare to CIEXYZ? Will always the method that performs best in CIEXYZ

perform best also in CIELAB? How stabile are these methods?

Camera: characterization methods

Method name Method description

Method 1 Gamma method for linearization and regression into CIEXYZ space

Method 2 Polynomial fitting for linearization and regression into CIEXYZ space

Method 3 No linearization beyond a cubic root function and regression into CIELAB space

Method 4 Gamma method and a cubic root function for linearization and regression into CIELAB space

Method 5 Polynomial fitting and a cubic root function for linearization and regression not CIELAB space

dcxbxaxy 23

1

kIE

Camera: Experiment

Regression up to fourth order was used. Methods were tested 100 timer per training-

set. 180 random colors were measured 33 grey values were used for linearization.

Camera-Result

Size of regression Matrix

Method 1 Method 2 Method 3 Method 4 Method 5

3x3 10.35 7.77 19.66 9.03 7.80

3x5 8.11 7.18 16.29 8.21 6.18

3x10 6.20 3.97 6.58 3.51 3.75

3x20 4.52 2.24 2.82 1.79 2.53

3x35 3.20 1.40 1.34 1.10 1.37

Camere-conclusion

Number of colors used for regression was dependent on methods and order of regression.

Minimum order: Second order regression. Use of cubic root function proved to yield good results but was

very unstabile. CIELAB performed better than CIEXYZ and was more stabile. It’s not certain that method that perfoms well in CIEXYZ

performs as well in CIELAB. (Method 1 and 4 versus Method 2 and 5).

Stability was dependent on amount of colors in the training-set, order of regression and linearization method.

Geometrical alignment.

Geometrical alignment

The points are detected Each point are binary coded. Divided in blocks Regression for finding transformation matrix. Compensation:

– Divide image in blocks.– Multiply with the transformation matrix.

Dependent on size of the screen, the resolution of the camera and number of points and blocks.

Acknowledgement

I want to thank Mr. Hardeberg and HiG administration for giving me chance to visit Japan.

I want also to thank Tsukdada-san, Toda-san, Funyama-san, Inoue-san and rest of the NEC employees who have welcomed me warmly.

Resten av slides er bare i tilfelle jeg trenger dem.

Takk for hjelpen!

Projector:Mean Delta

Mean delta

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

10 12 14 16 18 20

Nr. of colors in the color channels

Mean

delt

a E

Spline

Linear

Cubic

Max delta

0

2

4

6

8

10

12

10 12 14 16 18 20

Nr. of colors in the color channels

Ma

x d

elt

a E

Spline

Linear

Cubic

Projector:Mean Delta

Projector: interpolation+regression

Comparison of mean Delta

0

0.5

1

1.5

2

2.5

10 12 14 16 18 20

Nr. Of Colors in the training-set

Me

an

De

lta

E

Spline

Linear

Cubic

Projector:Interpolation+regression

Comparison of Max Delta

0

2

4

6

8

10

12

14

10 12 14 16 18 20

Nr. Of Colors in the training-set

Max D

elt

a E

Spline

Linear

Cubic

Camera-standard deviance.