Journal of Geometry 0047-2468/83/020118-1351.50+0.20/0 Voi.21 (1983) �9 Birkh~user Verlag, Basel

AN ALTERNATE PROOF OF THE RECTILINEAR ART GALLERY THEOREM

Joseph O'Rourke

i. INTRODUCTION

In 1980, Kahn, Klawe, and Kleitman proved that Ln/~

guards are always sufficient and occasionally necessary to see

the entire interior of an n-wall room whose boundary is a simple

rectilinear polygon [KKK]. A rectilinear polygon is one whose

edges are either horizontal or vertical. A guard is a point g

that can "see" a point x iff the segment gx remains within

the interior or on the boundary of the polygon. This"rectilinear

art gallery theorem" is similar in spirit to Chv~tal's original

art gallery theorem, which solved the same problem for unre-

stricted simple polygons: in this case, Ln/~ guards are

necessary and sufficient [Chv] [Honsl].

Fisk offered a proof of Chv~tal's theorem that first

partitions the polygon into triangles, 3-colors the resulting

graph, and places guards at the vertices colored with the least

frequently used color [Fisk] [Hons2]. Kahn et al's proof is

similar: they first partition the rectilinear polygon into con-

vex quadrilaterals, 4-color the graph obtained by including the

quadrilateral's diagonals, and then place guards with the least

frequently used color. The proof that there always exists a con-

vex quadrilateralization is by no means simple, and involves a

"reducible configurations" argument.

Here an alternate proof is offered that does not detour

through a partition into convex quadrilaterals, but rather

Rourke 119

depends on a rectilinear partitioning. 1 The proof is somewhat

simpler, although it by no means has the elegance of Fisk's

proof.

2. ~IN INDUCTIVE ARGUMENT

The proof is phrased in terms of r , the number of

reflex vertices of the rectilinear polygon, rather than n , the

total number of vertices. A reflex vertex is one at which the

internal angle is >~ . This rephrasing is justified because

there is a fixed relationship between r and n :

LEMMA i. In a simple rectilinear polygon of n vertices, r of

which are reflex, n = 2r+4

Proof. Let c be the number of vertices at which the internal

angle is 7/2 ; clearly, n = c+r . Since the sum of the internal

angles of a simple polygon is (n-2) ~ , and since the angle at

each reflex verhex is 3~/2 ,

(n-2)z = c(~/2)+r(3~/2)

Solving for c and substituting into n=c+r yields n= 2r+4

Q.E.D.

Since Ln/4~ = L(2r+4)/4~ = Lr/2~+l , the Kahn, Klawe, Kleitman

theorem can be ~tated as follows:

THEOREM: Ir/21~l guards are necessary and sufficient to cover

the interior of a simple rectilinear polygon of r reflex

vertices.

The "comb" example of [KKK] establishes occasional necessity; an

alternate sufficiency proof follows.

Define a cut of a rectilinear polygon as an extension

of one of the two edges incident to a reflex vertex through the

interior of the polygon until it first encounters the boundary of

iThat a simpler proof might exist was suggested by Toussaint [Tous].

120 Rourke

the polygon (see Figure 1). A cut "resolves" its reflex in the

sense that the vertex is no longer reflex in either of the two

pieces of the partition determined by the cut. Clearly a cut

does not introduce any reflex vertices. The induction step of

the proof cuts the polygon in two, and applies the induction

hypothesis to each half. This will yield the formula of the

theorem if a cut can be found such that at least one of the halves

contains an odd number of reflex vertices. The only difficult

part of the proof is establishing that such an odd-cut always

exists. This sketch will now be formalized.

Proof of the Theorem

The theorem is clearly true for r ~ 1 : a single guard

suffices. So assume that r > 2 and that the theorem holds for

all r' < r Consider now two cases.

Case i: There exists a cut that resolves 2 reflex vertices.

This case occurs when two reflex vertices can "see" one

another along a vertical or horizontal line. Cut the polygon in

two along this line, and let L and R be the number of reflex

vertices in the two pieces produced. Since r =L+R+2 , the

formula to be proved is

I!/~+l = U~+R+2)/~+I

Applying the induction hypothesis to each half yields a coverage

of both polygons (and so the entire original polygon) with

LL/~+I+LR/~+I guards, which, by the above calculation, is

the formula to be established.

Case 2. No two reflex vertices can see one another along a

vertical or horizontal line.

Lemma 6 below will establish that in this case, there

exists an odd-cut: a cut such that one of the two pieces has an

odd number of reflex vertices. Let such a cut be chosen, and let

L and R be the number of reflex guards in the halves, with R

odd. As one reflex guard is resolved by the cut, r =L+R+I .

Rourke 121

L=5

n =22 r = 9

] R=3

Figure i. A recti- linear polygon with n=22 vertices, r=9 of which are reflex. The dashed line is a cut that partitions the polygon into two pieces that have L=5 and R=3 reflex ver- tices. Since the cut resolves one reflex vertex, r=L+R+I.

Figure 2. A polygon that does not admit an odd-cut, but that permits a single cut to re- solve two reflex vertices. In such cases, r=L+R+2.

1 L=2

i R=O

n=12

r = 4

J 1

? Y i n =14

r = 5

Figure 3. A polygon that has no horizontal odd-cut (H-odd-cut).

122 Rourke

The formula of the theorem can therefore be written as

~ / ~ + i : LCT ,+~+ I ) /~+ I = I_r ( R - l ) ) / 2 ] +2

>_ I ~ / ~ + L ( R - z ) / ~ + 2 .

Applying the induction hypothesis to each half yields coverage by

IL/21§ (since R is odd), which, by the above calcu-

lation, is < the formula to be proved.

Q.E.D.

3. EXISTENCE OF ODD-CUTS

The existence of odd-cuts will now be established.

First note that an odd-cut may not exist if reflex vertices can

see one another along horizontal or vertical lines (see Figure 2),

but that this falls under Case 1 of the proof above. Therefore

in this section it will be assumed that the vertices of the poly-

gon are in "general position" in the sense that no cut can

resolve two reflex vertices. Second note that the existence of

an odd-cut is trivial if r , the total number of reflex vertices,

is even; any cut partitions the reflex vertices according to

r = L+R+I , so one of L or R must be odd and the other even.

When r is odd, either L and R are both even or both odd;

the task is to show that a cut can be found such that they are

both odd. Finally note that a horizontal odd-cut does not always

exist: Figure 3 shows an example with r=5 . In this case, only

a vertical odd-cut exists. Thus cuts in both directions must be

considered; call a horizontal cut an H-cut and a vertical cut a

V-cut.

The proof depends on a particular rectilinear partition-

ing of the polygon, which will now be defined for H-cuts. Call

a reflex vertex H-isolated if the other endpoint of its incident

horizontal edge is not reflex, and otherwise call it a member of

an H-pair. Partition the polygon by forming an H-cut at each

reflex vertex that is a member of an H-pair (see Figure 4); the

Rourke 123

m

Figure 4. The H-graph associa- ted with a polygon records region adjacency in the parti- tion formed by cutting each member of an H-pair.

124 Rourke

only reflex vertices not resolved in this partitioning are 2

H-isolated vertices. It will be proved that either an H-odd cut

exists or there is precisely one H-isolated vertex. The proof

depends on a rather close analysis of the structure of this

partition, which will be explored in terms of its region adja-

cency graph, called its H-graph.

Each piece of the partition corresponds to a node of

this graph, and node A is connected by an ~rc directed to node

B iff (i) A and B are adjacent pieces, s~-parated by an H-cut, and

(2) the H-pair corresponding to the H-cut lies on the boundary of

the A piece. See ....... 4. The following lemma classifies the

nodes according to their incident arcs.

LEMMA 2. The H-graph corresponding to the above defined parti-

tion of a rectilinear polygon can have just four types of nodes:

name

leaf

branch

source

sink

total deqree incominq arcs outqoinq arcs

1 1 0

3 1 2

2 or 4 0 2 Or 4

2 2 0

Proof. The general position assumption prevents a single cut

linking two reflex vertices. Thus each region can have at most

2 H-pairs and therefore 4 H-cuts on its boundary. Thus the de-

gree of a node is < 4. The definition of "arc" implies that a

node cannot have just one outgoing arc. Thus a degree 1 node

must be a leaf. A degree 2 node can have 2 outgoing (source) or

2 incoming (sink) arcs; 1 outgoing arc is not possible. A degree

3 node must be a branch, again because 1 outgoing is not possible.

A degree 4 node must have 2 H-pairs on its boundary, which

implies that all 4 arcs are outgoing.

Q.E.D.

2This decomposition is a partition into pieces monotone with

respect to the y-axis [Sack].

Rourke 125

It wilil now be shown that the graph for a polygon that

does not admit an H-odd-cut must have a very specia ! structure.

LEMMA 3. If a polygon's H-graph contains a sink nodie, then it

admits an H-odd-cut.

Proof. Let S be the region corresponding to a si~k node, and

let C+ and C_ be the upper and lower H-cuts on ~he boundary

of S . Let S contain k H-isolated vertices, and let the

total number of reflex vertices in the portion of the original

polygon above C+ (not including the vertex forming C+) be u .

See Figure 5.

If u is odd, then C+ is an H-odd-cut. If u is

even, a cut at the highest H-isolated vertex in S(if k>0) or

C (if k=0) is an H-odd-cut.

I Q.E.D.

Thus, if a polygon does not admit an H-odd-cut, it

cannot have any sink nodes. This implies that such a graph has

just a single source node, as two source nodes can only interlink

via sinks. Thus the graph is a tree with a source ~oot node, and

otherwise binary directed towards the leaves.

LEMMA 4. If a polygon does not admit an H-odd-cut, then it has

exactly 1 H-isolated vertex located in its sole source region.

Proof. First it will beshown that all leaf and branch nodes

must be devoid Qf H-isolated vertices. The proof is by induction

on the number of arcs to the nearest leaf node.

Suppose some leaf L contains k>0 H-isolatedreflex

vertices. Let C be the H-cut corresponding to its single in-

coming arc (see Figure 6) ~. T~en if k is odd, C is an odd-cut,

and if k is even, then the H-isolated vertex in L closest to

C is an odd-cut. This establishes the basis of the induction.

Suppose now that all leaf and branch nodes with

frontier distance d'<d have no H-isolated vertices, and con-

sider a branch node B at distance d Let C be the H-cut

corresponding to its single incoming arc, and let k>0 be the

126 Rourke

l k=4 H-isoloted

Figure 5. A sink region S always permits an odd-cut.

Fi@ure 6. If a leaf region L has k>0 H-isolated vertices, then it admits an H-odd-cut.

J �9

\

k=4 H-isolated

Rourke 127

number of H-isolated vertices in B . If k is odd, then C

is an odd-cUt, since by the induction hypothesis none of the

descendants of B have any H-isolated vertices, and otherwise

the reflex vertices come in H-pairs. If k is even, then the

H-isolated vertex in B closest to C can form an odd-cut.

Finally it will be shown that the single source region

S must have exactly 1 H-isolated vertex to avoid an H-odd-cut.

Suppose S contains an even number k of H-isolated vertices,

and let C be a cut corresponding to one of S's outgoing arcs

(see Figure 7). Then C is an odd-cut, since it resolves 1

reflex vertex of an H-pair, and otherwise all other reflex ver-

tices are either in the even k H-isolated vertices or they

come in H-pairs. If k is odd and >l , then the second closest

one to C forms an odd-cut. Thus there must be exactly 1 H-

isolated vertex.

Q.E.D.

Clearly Lemmas 2-4 hold for V-cuts as well as H-cuts.

Thus, if a polygon does not admit an H-odd-cut nor a V-odd-cut,

then it must have a single H-isolated vertex h and a single V-

isolated vertex v , both located in source regions of the H-

and V-graphs respectively. That these conditions are impossible

to achieve is shown by the following lemma.

LEMMA 5. A polygon of r>3 reflex vertices cannot have exactly

1 H-isolated vertex in a region corresponding to a source node

of its H-graph, and exactly 1 V-isolated vertex in a region cor-

responding to a source node of its V-graph.

Proof. Let h and v be the H- and V-isolated vertices re-

spectively. All reflex vertices besides h and v (there is at

least one such since r~3) are members of both an H-pair and a V-

pair, else they would be isolated. This implies that they are

all adjacent, forming a contiguous chain of reflex vertices.

This chain cannot close upon itself without forming a hole, con-

tradicting the simplicity of the polygon. The polygon is there-

fore a spiral (having a single concave chain) whose endpoints

are h and v (see Figure 8). But then h is in a leaf region

128 Rourke

m

] C

k=3 H-isolated

Figure 7. A source region S admits an H-odd-cut if it contains k#l H-isolated vertices.

Figure 8. Reflex vertices that are members of both H- and V-pairs form a spiral chain whose endpoints are either H- or V-isolated.

u 7

O

J

Rourke 129

Of the H-graph an d v in a leaf region of the V-graph, contra-

dicting the requirement that they be located in source regions.

Q.E.D.

The existence of odd-cuts is now established.

LEMMA 6. A rectilinear polygon with an odd number r>3 of

reflex vertices, no two of which can see one another along a

vertical or horizontal line, admits an odd-cut.

4. DISCUSSION

The proof of Kahn et al leads to an O(n logn) algorithm

for finding convex quadrilateralizations [Sack], which in turn

yields an O(n log n) algorithm for locating the In/41 guards.

Similarly the proof presented in the previous section can be

used as the basis of a rather different, but equally fast, algo-

rithm for locating the Lr/~+l guards at reflex vertices [EOW].

Finally, the proof in this paper easily extends to

"multi-level" recflilinear polygons connected by ramps, as does

that in [KKK], bu~ it does not immediately extend to multiply-

connected rectilinear polygons (ones with rectilinear holes), a

problem left open in [KKK].

ACKNOWLEDGEMENTS

I thank Rao Kosaraju of Johns Hopkins University, and

Hossam EiGindy, Jorg Sack, and Godfried Toussaint of McGill

University, for clarifying discussions. The research was par-

tially supported by NSF Grant MSC-8117424.

REFERENCES

[Chv] V. Chv~tal, "A Combinatorial Theorem in Plane Geometry," Journal of Combinatorial Theory B, Vol. 18, 1975, pp.39-41.

[EOW] Edeisbrunner, E., O'Rourke, J., and Welzl, E., "Stationing Guards in Rectilinear Art Galleries," Computer, Vision, Graphics, and Image Processing, to appear, 1983.--

130 Rourke

[Fisk] S .... Fisk, '!A. Short Proof of Chv~tal's Watchman Theorem," Journal of Combinatorial Theory B, Vol. 24, 1978, p.374.

[Honsl] Honsberger, R., Mathematical Gems II, Mathematical Asso- ciation of America, 1976, pp.104-I10.

[Hons2] Honsberger, R., "Games, Graphs, and Galleries," in The Mathematical Gardner, ed. David A. Klarner, Prindle, Weber & Schmidt: Boston, 1981, pp,274-284.

[KKK] Kahn, J., Klawe, M., Kleitman, D., "Traditional Galleri@s Require Fewer Watchman," SIAM Journal on Algebraic and Discrete Methods, Vol. 4, N-o[ 2, June 1983, pp. 194-206.

[Sack] Sack, Jorg-R., "An O(nlogn) Algorithm for Decomposing Simple Rectilinear Polygons into Convex Quadrilaterals," Proceedings of the Twientieth Allerton Conference, Oct. 1982, pp. 64-74.

[Tous] Toussaint, G., personal communication, May 1982.

Department of Electrical Engineering and Computer Science The Johns Hopkins University Baltimore, MD 21218

(Eingegangen am10. Januar 1983)