correcting projector distortions on planar screens via homography - newben-shahar/teaching... ·...
TRANSCRIPT
Correcting Projector Distortions on Planar Screens
via Homography
Method and Setup Proposal for Correcting Distorted Projected Images
on Planar Screens Under Difficult Conditions
Daniel Hirt
Introduction to Computational and Biological Vision Project
Computer-Science Department
Ben-Gurion University
2013
Abstract
In the following paper we will discuss and present an approach and implementation for an
automated setup for correcting a projection of an image from a projector device.
A single, user-friendly method will be established so that any user can operate such a setup
even at home, provided a projector device, from harsh (sometimes unimaginable) angles, and
get proper satisfactory results, i.e. a corrected image on the projection screen.
1. Introduction In our day and age we rely heavily on projector systems. Whether it is for educational (such
as presentation displaying), or even entertainment purposes (television, movies etc.), it is
clear that projector systems had become a popular, and sometimes essential, tool to fulfill our
needs in our daily lives. Moreover, costs of projectors had decreased dramatically over the
years, so that today an average person could afford one.
However, it seems that though costs for the projector unit alone had become affordable, in a
lot of cases the designated environment for it is far from being appropriate for its operation.
In a lot of cases the space in which the projector will be operating doesn‟t allow for an ideal
setup for proper viewing on the screen.
An ideal setup for the common commercial projector consists of an overhead mounting of the
projector, in a perpendicular angle to the projection screen. Any considerable deviations from
the recommended setup easily result in a distorted unattractive image displayed on the
projection screen.
The common projector device has built-in tweaking capabilities, allowing the user to correct
some distotion, but at a limited range.
Although an ideal recommended setup does have its virtues, aesthetically-wise, the main
claim is that what the common consumer wants is flexibility, especially when there is little
choice on where to place the projector device.
The following sections of this paper we will propose an approach and implementation for a
system capable of handling harsh scenarios for projector device placement with little user
intervention, avoiding preliminary requirements for such a solution from other methods.
2. Perspective Projection Layout
Consider the following layout depicting the course our original image goes through, starting
from our computer/projector to our flat projection screen.
Denote (x1, y1) be a point in our image on the computer, then (x2,y2) is the projection of that
point on the screen.
An ideal positioning of a projector i.e. perpendicular to the flat screen will result in an image
identical to the image on the computer in its proportions.
However, if the projector is located in a notable angle from the screen, the projection will
result in an uneven image on it.
Figure 1 - Projection of an Object on a Flat Screen from a Difficult Angle (not a real-life depiction)
(x1,y1)
(x2,y2)
Many common projectors offer correction for limited types of distortions, the most popular
being a “keystone-effect correction”.
Figure 2 - "Keystone effect" - when the projection axis is not perpendicular (below here) to the screen
In common projector devices such built-in correction mechanisms are not only limited, but
also require user intervention to manually determine the suitable amount of correction values.
Our approach will help both problems: provide corrections solutions for a wide range of
distortions and with little user-intervention.
However, there is another projection in our system – the projection to viewer‟s eye. We will
consider this projection as the projection from the screen to a camera device.
Eventually, we wind-up with a system as follows:
Figure 3 - an appropriate depiction of the common projector system
Projector device
Projector device
Viewers/camera
Each of the factors obey perspective geometry principles. Under regular representations of
the images we need to know information about each system. These parameters are both
intrinsic (focal length, principal point) and extrinsic (rotation, translation) parameters. With
that amount of unknowns we are bound to take another approach in order limit user-
intervention and preliminary actions.
3. Homography in a Nutshell
Recall that in our system we deal with planar projections – each of the images is projected on
a flat screen, thus z=0 for every point on the plane.
The camera model in homogenous form:
The properties of homogenous representation, where z=0 give us the following relationship:
Setting w=1, we end up with an 8-DOF equation system.
In order to solve an 8-DOF equation system of the algebraic distance ⃗ we require a
set of four points (x,y). A common method of solving this is by applying Least Squares on
the algebraic distance of the homogeneous system ⃗ , where ⃗ is a set of points.
In our approach we will apply the robust RANSAC algorithm to guarantee outlier exclusion,
while providing a sufficient amount of sample points (more than 10).
Every projection of the three in our setup can be represented by the above equation.
Moreover, the projection from a point in an image, which is itself a flat projection screen, to
another planar screen is reversible.
4. Approach and Method
In this section of the article we will draw the outlines of one-time procedure for a specific
placement of a projector. For any new placement of the projector, this procedure should be
reapplied. In any case, the procedure itself is easy and requires little as possible user
intervention. The implementation developed (in C++) provides a demonstration, but can be
easily improved with more user-friendly features to allow an easy user-experience.
This suggested method provides high flexibility even at an oriented projected device (though
up to a certain limit of rotation) without falling into ambiguities.
In the solution we require the following:
A Computer
A standard resolution webcam
A projector device
Some approaches suggested mild deviations in the projection axis from the projector to the
projection screen. Such approaches relied solely on the four corners of the projected
boundaries. If difficult angles such as some slight rotations were applied, it would be hard to
determine how the projector was rotated. Moreover, the mixture of an unknown webcam and
unknown projector will add impurities to the whole set of equations. Thus, using more than
four points in a sampling consensus to solve the equations seems logical, to get proper results
in difficult scenarios.
Note that in this method we rely that the viewer sits in an ideal position in front of the
projection screen. However, we do not limit the projection screen to be nothing more than a
flat surface (even a wall, which has no boundaries).
First, we project a specific pattern through the projector, to allow grabbing more than 4
correspondence points. Second, we compare an ideal projection of the same pattern the
viewer would‟ve expected to see if the projector was properly placed. Next, we calculate the
Homography matrix that transfer the points from the wall-projected image to the reference
(correct) image, denoted H, via a random sampling consensus (RANSAC). Finally, we apply
perspective warp with H on a chosen sample image. At this stage the image should be
properly warped to be displayed correctly under the projection conditions.
Figure 4 - initially we project the pattern
Pattern
Figure 5 - after applying the homography on the input pattern, the output is projected correctly
5. Implementation
The method suggested in this paper was implemented in a C++ program for demonstration
purposes and verifying results under tough conditions.
The code relies heavily on the OpenCV v2.4.31 open-source library in a linux operating
system. Source code 2for this implementation is provided.
The algorithm:
1. Verify camera device is connected, otherwise give error and exit
2. Provide the user a reference 6x8 chessboard pattern to project via the projector
3. In fixed frame intervals, look for chessboard pattern in the camera‟s view, and in each
time a pattern is found, update the display with numbering of all inner-corners of the
pattern. If numbering haven't been updated, the pattern was not yet found.
Also, wait for a key input „c‟ to capture a proper detected chessboard frame. If „c‟
was pressed, proceed to step 4.
Repeat step 3.
4. Create corresponding points of the expected 6x8 chessboard pattern3, denoted pts1, to
those detected at the end of step 3, denoted pts2. Expected corresponding pairs of
points (inner corners): 35.
5. Improve4 set of points pts1 by resizing with a factor denoted by the ratio d2/d1, where
di is the diagonal length between ptsi[0] to ptsi[11].
1 Currently, OpenCV 2.4.4 is the most up-to-date version
2 To build the program, consult the official OpenCV site: www.opencv.org
3 A high number of corners was used in order to establish more sampling at the corners of the projected screen,
where high distortions could appear in difficult positioning of the projector
Pattern
6. Calculate via RANSAC the Homography matrix from points set pts2 to pts1, denoted
H.
7. Apply (pre-) warp perspective: H∙sample_image, where “sample_image” is a sample
image provided by the user on execution of the program.
8. Display both original and warped images and prompt user to display (drag) it to the
projector screen.
9. End demonstration.
4 This step was added to maintain proportions of output image, as the captured pattern on the projection wall always
turns out smaller than the original pattern image, leading to an oversized result from the calculated homography.
6. Test Results
Provided are the test results produces by a setup of a low-cost projector device, a standard laptop,
and a standard 640x480 resolution webcam.
In each test we examine a different placement of the projector against the projection screen
(white wall in this case).
Pico Projector (low-cost) 1024x768
Laptop (Linux OS)
Standard 640x480 webcam
Original Image Fixed Image
7. Conclusions
Homography indeed offers a somewhat hassle-free approach to solve the adressed problem
with common projectors. We have managed to establish a process that approximates proper
corrections for a wide range of images and positions of the projector. However, as mentioned
there are some unwanted results, due to the fact that we can not model our system as a pure
homography system. Nevertheless, we can get closer to a universal solution if we adopt other
method than homography.
8. Future Development
Integrate the process to graphic display, so that the whole desktop is projected correctly,
and not only images.
Better patterns for better correspondence matching, to allow an almost full-rotation of the
projected plane. Right now we may get some ambiguities once we get close to 90 degrees
rotations.
Extend to non-planar screens - can‟t be solved with homography.
Handle cropping of corrected images: an optimal resized image in the distorted projection
screen.
9. References
9.1. Computer Vision, Algorithms and Application, Richard Szeliski
9.2. Smarter Presentations: Exploiting Homography in Camera-Projector Systems, Rahul S.,
Robert G. S. and Matthew D. M.
9.3. Introduction to Computer and Biological Vision Lecture Notes, Prof. Ohad Ben-Shahar,
Computer-Science Dept. ,Ben-Gurion University.