The concept and automatic generation of theCurved Nonogram puzzle

by Tim de Jong

Master Thesis

ICA-4123689

Game and Media TechnologyUtrecht University

supervised byMarc van Kreveld

and Maarten Loffler

July 8, 2016

Abstract

In this paper we introduce a variation on the Nono-gram puzzle that we call the Curved Nonogram. Therules for colouring are similar, but instead of beingconstrained to a grid it is structured by arbitrarycurves that can take any shape. This allows for puz-zles that feel more unique and can have more beauti-ful images hidden in them.

Together with the introduction of this new puzzle,we define classes of solvability for it and provide mea-surements to judge its aesthetics. We also proposean algorithm that can generate these Curved Nono-grams, taking the desired solution image as input.We test this algorithm on 16 varied input images andfind that all of the outputs have a unique solution- an important requirement for puzzle-book puzzles,and at least 10 of them satisfy the aesthetic criteriasufficiently that they could be used in a puzzle book.

1 Introduction

1.1 Nonograms

Nonograms, also known as Paint-by-numbers puzzlesor Japanese puzzles, are a popular type of pencil-and-paper puzzle that can be found in puzzle booksaround the globe. The goal of the puzzle is to makea picture appear by colouring the correct cells in asquare grid, according to the given row- and columndescriptions. Figure 1 shows an example of a Nono-gram. The description of a row or column consistsof a sequence of numbers, which are the lengths ofthe sequences of coloured cells that appear in it. Forexample, the number 3 on the second row means thatthis row will have one uninterrupted sequence of threecoloured cells. The puzzler can use this informationto reason which cells should be coloured and whichshould be left empty.

A Nonogram starts as an empty square grid withdescriptions, and ends with an image revealed in thegrid. The discovery of the image acts a reward forsolving the puzzle.

1.2 Curved Nonograms

In this paper we introduce the Curved Nonogram,a new variant on the classic Nonogram puzzle. ACurved Nonogram consists of a set of arbitrary curvesenclosed by a border. An example is shown in Fig-ure 2. The only constraints we impose on the curvesare that (1) they lie completely within the border; (2)the endpoints of each curve lie on the border; and (3)

1 1 1

1 1 1

3

1

42 21 1

(a) Empty

1 1 1

1 1 1

3

1

42 21 1

(b) Solved

Figure 1: An example of a Nonogram. The puzzleruses the descriptions at each row and column to fillcells in the empty puzzle (a) in order to find the imageformed by the solved puzzle (b).

Page 1

that no more than two curves may intersect at thesame point.

Instead of row- and column descriptions, theCurved Nonogram has a description for both sidesof each curve. In the same way that the descrip-tions of a classic Nonogram give information aboutthe sequence of cells that form a row or column, thisdescription gives information about the sequence offaces that are incident to that side of the curve. Forexample, see the description ”1 1” in Figure 2. Fol-lowing this side of the curve from start to end, you willencounter one coloured face, one or more empty faces,and one coloured face again, in that order. Whenreading a description, one should start with the out-ermost number and end with the innermost number.For example, in Figure 2, the description on the rightside of the curve that starts at the bottom of thepuzzle should be read as ”2 1 1”.

The purpose of the three constraints given above isto make the Curved Nonogram clear and easy to graspfor puzzlers, especially if they already have experi-ence with classic Nonograms. Having a single bordermakes it clear which faces are part of the puzzle. Werequire the endpoints of the curves to lie on the borderto make it a clear start- and endpoint when tracingthe faces incident to the side of a curve. The rea-son for not allowing three or more curves to intersectin a point is also a practical one. At such a multi-intersection, some curves and faces are only incidentto each other in a single point. Whether this formof adjacency counts or not is not clear for the puz-zler, so it would have to be explicitly defined in therules of the puzzle book. Because we want CurvedNonograms to be understandable without requiringthe memorization of such specific rules, constraint (3)was added to prevent this situation from occurring.

While the rules of the Curved Nonogram are similarto those of the classic Nonogram, there are severalthings that distinguish the Curved Nonogram fromthe classic version.

First of all, because the classic Nonogram consistsof a square grid, the image formed by the solutionis necessarily very pixellated and abstract. A CurvedNonogram is not limited by such a grid, and the imagethat is formed by the solved puzzle can take nearlyany shape. This means that the image can more faith-fully represent the image that the puzzle designer hasenvisioned.

Second of all, the puzzler’s curiosity is triggeredas soon as they see the puzzle and try to visualisethe shape of the solution image hidden between the

1 0

1

14

112

1

12 1

1

(a) Empty

1 0

1

14

112

1

12 1

1

(b) Solved

Figure 2: An example of a Curved Nonogram. Thedescriptions are associated with one side of a curve,instead of a row or column. Again, the goal is tostart with the empty puzzle (a) and find the imageformed by the solution (b), which shows a house inperspective with smoke coming from the chimney.

Page 2

curves. This in contrast to a classical Nonogram,where the only moment of surprise comes at the end,when the puzzle has been solved and the image hasbecome clear.

Thirdly, Curved Nonograms offer new challenges topuzzlers, even if they are already familiar with classicNonograms. The one most obvious is that they willcarefully have to trace the path of each curve throughthe puzzle to see which faces are incident to it. Lessobvious but more impactful is the information thatcan be obtained when one side of a curve comes acrossthe same face more than once. Such an occurrencegives the puzzler a new type of clue that is not presentin classic Nonograms. While this aspect may be toosubtle or confusing for the beginning puzzler, it couldbe introduced over time in a puzzle book, with onlythe hardest puzzles really requiring this informationin order to progress. We choose to allow this situationin our rules, because we think that its occurrence isone of the factors that make Curved Nonograms standout from classic Nonograms - especially for the moreexperienced puzzler. Another interesting difference isthat a cell in a classic Nonogram is referred to in thedescriptions of exactly one row and one column. Aface in a Curved Nonogram, on the other hand, canbe referred to by many more descriptions, namely onefor each curve that bounds it.

1.3 Automated generation of CurvedNonograms

In addition to the concept of Curved Nonograms,we will also introduce an algorithm that generatesa Curved Nonogram from a given image in vectorgraphics format. This may prove useful for a puzzledesigner, who will only have to draw his envisionedsolution image and let the algorithm do the rest. Ourintent is also to study the output of this algorithm, inorder to get a better understanding of Curved Nono-grams. We would like to generate Curved Nonogramsthat satisfy the following conditions:

1. The generated puzzle should adhere to the threerules of Curved Nonograms described above

2. The generated puzzle should be visually unam-biguous

3. The generated puzzle should be aestheticallypleasing

4. The generated puzzle should have a unique solu-tion

5. The solution image of the generated puzzleshould look like the input image

6. The outlines of the solution image should be suf-ficiently obscured in the unsolved puzzle

The first condition is required to generate a validCurved Nonogram.

The second condition concerns potential sources ofvisual ambiguity such as small faces, intersections at asmall angle, and narrow parts of faces (i.e. where twocurves almost touch). The generated puzzle should besolvable by a human puzzler, which makes it impor-tant that the structure of the puzzle is clear to thenaked eye.

The third condition concerns the aesthetics asidefrom ambiguity, and is perhaps the hardest to definemathematically or be solved by a computer. We stillchoose to add it to the list, because this conditionwould certainly be a goal of a human puzzle designer;consequently, an algorithm that automates this partof the designer’s job should be judged by it. Examplesof this criterion include making fluid curves, and notmaking large empty faces.

The fourth condition requires the puzzle to havea unique solution. We will show later that not all(Curved) Nonograms have a solution, and that it isalso possible that a puzzle has multiple solutions,none of which is desirable.

The fifth condition requires the generated puzzle tohave a solution image that looks like the input image.The weight that is given to this condition determinesthe level of control that the user has over the output.

The sixth and last condition states that the out-lines of the solution image should be well hidden inthe unsolved puzzle. The risk that the puzzler canguess the image before they have solved the puzzleis inherent to Curved Nonograms. This risk shouldbe mitigated as much as possible in the generatedpuzzles.

The creation of an algorithm that satisfies all of theconditions is a difficult task that exceeds the scope ofthis paper. We choose to focus on the aesthetic side ofthe puzzle generation, because this side is the mostnew and unique aspect of Curved Nonograms. Wedevelop an algorithm that generates Curved Nono-grams that satisfy criteria (1), (2), (3) and (6). Wethen experiment with the evaluation function of thealgorithm to learn which terms give the best resultsaccording to these criteria. We also measure the per-centage of the generated puzzles that have a uniquesolution as per criterion (4), and judge how well thesolution images are obscured as per criterion (5).While we do not optimise for these last two criteria,the observations may prove useful to indicate if thesecriteria require more focus in possible future work.

Page 3

No solutionMultiple solutions

Unique solution

Simple

Figure 3: Relation between the four classes of solv-ability for (Curved) Nonograms

2 Related work

Nonogram solving Most of the literature onNonograms focuses on solving them as efficiently aspossible. Because the problem of solving a Nono-gram is NP-complete [1], there is no straightforwardsolution to this problem. Many different types of al-gorithm have been applied to solving them, based ontechniques from Discrete Tomography [2], chronolog-ical backtracking [3] and many others [4][5].

There are also many websites dedicated to solvingNonograms. Our algorithm uses an adapted versionof the well-documented classic Nonogram solver ofwebpbn.com [6] to determine the solvability of CurvedNonograms.

Classification of Nonograms There is also re-search on the classification of Nonograms [4][7]. Fromthis work we can learn that a Nonogram always fallsinto one of four classes of solvability : unsolvable, mul-tiple solutions, unique solution, and simple. Figure 3shows how these classes relate. A Nonogram is calledsimple if it can be solved by working on one row orcolumn at the time. Most Nonograms found in puz-zle books are simple, and this subset of Nonogramsis solvable in polynomial time. Because there are nobranching decisions while solving a simple Nonogram,they always have a unique solution. We will showthat these same classes apply to Curved Nonograms.Of course, the Curved Nonograms generated by ouralgorithm should ideally always have a unique solu-tion. While we do not implement this feature, we dodetermine the class of each generated puzzle in orderto analyse how the generated puzzles are distributedamong the classes.

Puzzle generation Perhaps due to the popularityof puzzles and puzzle games on mobile devices, a lotof recent research has been done on the automaticgeneration of puzzles. A Nonogram-related exampleis [7], which is about generating simple Nonogramsof a given difficulty - roughly defined as the numberof steps required to solve it. The author finds thatchanging the solution colour of a single face can havea large effect on the difficulty of the puzzle. OtherNonogram-generation related work explores differentquestions, such as how to generate them from digitalRGB images [8].

There are, of course, many studies of puzzle genera-tion for other puzzles than Nonograms. Examples arethe generation of Sudokus [9], Crossword puzzles [10],and Connect-The-Dots puzzles [11]. The Connect-The-Dots paper is especially relevant, because it alsodeals with ambiguities in a generated puzzle. How-ever, the specific ambiguities involved are quite dif-ferent from those in Curved Nonograms, because thepuzzle elements are dots instead of curves, faces andintersections.

Automated drawing aesthetics One of thethings that we are trying to accomplish in this paperis to generate clear and aesthetically pleasing CurvedNonograms. The aesthetics of automated drawingare much studied in the field of Graph drawing [12],where the input is a graph, which needs to be em-bedded in the plane. Many studies have been doneto find a set of aesthetic criteria for such an embed-ding [13][14][15]. Other papers offer algorithms thatcan be used to generate Graph drawings according tothese criteria. An example of an algorithm that usesforce-directed placement can be found in [13].

3 Definitions

We will start by giving our definition of the classicalNonogram in Section 3.1. This is followed by thedefinition of the Curved Nonogram in Section 3.2,which uses some of the same elements. Section 3.3then defines the classes of solvability for both typesof Nonogram. Finally, three measurements to judgethe aesthetic of a Curved Nonogram are introducedin Section 3.4.

3.1 Nonogram

A Nonogram consists of a grid of square cells, whichwe will also refer to as the faces, and a descriptionfor each row and column of the grid. See Figure 1a.

Page 4

A colouring C of the Nonogram is an assignmentof colours from the set Σ = {b, w} (black and white)to its faces.

Each row and column has a sequence of faces init. For brevity, we will refer to this as simply thesequence of a row or column. For a given colouringC, each of these sequences is associated with a colourstring over the alphabet Σ, which corresponds to thecolours assigned to the faces in the sequence underC. For a colour string s of a sequence with l faces:

s ∈ Σl. (1)

Note that a Nonogram with an n × m grid hasn+m sequences. The descriptions of the Nonogramare each associated with the sequence in the corre-sponding row or column. A description is a string ofpositive integers:

d = c1c2 . . . ck. (2)

A description may also be an empty string, which isdisplayed in the puzzle as a zero. For a given colour-ing, the colour string s of a sequence adheres to thedescription d if it satisfies the following regular ex-pression:

s ∈ w∗bc1w+bc2w+ . . . bckw∗. (3)

For readers unfamiliar with regular expressions: w∗

denotes a string of zero or more w’s; w+ denotes astring of one or more w’s; and bc denotes the stringconsisting of c times the b character. In other words,the pattern describes a sequence that contains k seg-ments of consecutive black coloured cells, of whichthe lengths and order are given by the integers in thedescription. These segments are separated by at leastone white cell, and the sequence may start and endwith a segment of white cells.

A colouring C is called a solution of the Nonogramif each of the colour strings adheres to the descrip-tion associated with its sequence. In Section 3.3, weshow that a Nonogram may have no solution, a singlesolution, or multiple solutions.

3.2 Curved Nonogram

A Curved Nonogram consists of a two-dimensional ar-rangement induced by curves, together with a collec-tion of descriptions. See Figure 2a for an example.

One curve is called the puzzle enclosure, which isa closed curve that encloses the other curves. Thesecond type of curve are the puzzle curves, definedas the collection P = {p1, p2, . . . , pn}. These curves

are associated with the descriptions, and their start-and endpoints lie on the puzzle enclosure. The thirdand final type of curve are the background curves,defined as B = {b1, b2, . . . , bm}. These curves arenot associated with any descriptions, and their onlyrestriction is that they are contained in the puzzleenclosure.

Like with a classical Nonogram, a colouring C of aCurved Nonogram is an assignment of colours fromthe set Σ = {b, w} to its faces, which are the faces ofthe arrangement.

The two sides of a puzzle curve are each incidentto a different sequence of faces. For this reason, weconceptually split each puzzle curve into two half-curves, each representing one side of the curve. Ahalf-curve is incident to one sequence of faces, whichwe call the sequence of the half-curve. Thus, a CurvedNonogram with n puzzle curves has 2n sequences.

The rest of the definition is analogous to that ofthe classic Nonogram. For a given colouring, the se-quences are associated with colour strings as definedin (1). Each sequence is associated with a descriptionas defined in (2). A sequence adheres to a descriptionif it matches the regular expression in (3). Finally, acolouring C of a Curved Nonogram is called a solutionif each of the colour strings adheres to the descriptionassociated with its sequence.

3.3 Solvability

The classification of classic Nonograms based on solv-ability was discussed in Section 2, including the dia-gram in Figure 3. We will give a more precise defini-tion of the solvability classes here, and extend them tothe concept of a Curved Nonogram. The definitionsof solvability will be given for both types of Nono-gram at once, using the word Nonogram to refer toboth.

If all colour strings of a colouring C adhere to theircorresponding descriptions, C is called a solution ofthe Nonogram. It is possible that there are multipledifferent colourings of the Nonogram for which this istrue. In that case, the Nonogram has multiple solu-tions.

It is also possible that no colouring C exists forwhich all colour strings adhere to their descriptions.In this case, the Nonogram has no solution.

Finally, if exactly one colouring C exists that is asolution, the Nonogram is said to have a unique so-lution. All Nonograms that appear in puzzle booksbelong to this class, because they are intended to be

Page 5

solved and the solution should look like a specific pic-ture, which can usually be looked up at the end of thebook.

For a Nonogram with a solution C, we define thecorresponding solution image as the geometric regionformed by the union of the black-coloured faces inC. If a Nonogram has multiple solutions it also hasmultiple solution images: one for each of its solutions.

Simple Nonograms In [7] the authors define afourth class of classic Nonograms called simple. TheseNonograms are a subset of the Nonograms that havea unique solution. While the complete definition ofa simple Nonogram can be found in the original pa-per, we provide a succinct definition of the used termsadapted to our definition of the classical Nonogramand extend its meaning to Curved Nonograms.

Let there be a Fix operation that operates on asequence. This operation assigns a colour to thosefaces in the sequence that must have a specific colour,based on the description and the faces for which thecolour has already been assigned. According to thedefinition in [7], a classic Nonogram is called simpleif it can be solved by the following algorithm:

1. apply the Fix operation to the sequence of eachrow

2. apply the Fix operation to the sequence of eachcolumn

3. if the Nonogram is not solved, go to step 1

There are several interesting aspects to simpleNonograms. Firstly, solving a simple Nonogram onlytakes polynomial time. The Fix operation can be exe-cuted in polynomial time with the dynamic algorithmgiven in [4]. For a simple Nonogram, the algorithmgiven above must determine the colour of at least onenew face in each iteration, or it would be stuck. Thismeans that a simple classic Nonogram with n rowsand m columns is solved by this algorithm in at mostmn iterations.

Secondly, the given algorithm is similar to thestrategy that most humans use to solve a Nonogram.This makes simple Nonograms relatively easy to solveby hand. Because of this property, most Nonogramsfound in puzzle books are simple.

Simple Curved Nonograms We give our defini-tion of a simple Curved Nonogram, and show that itis also solvable in polynomial time. Re-using the Fixoperation defined in the last paragraph, we define aCurved Nonogram to be simple if it can be solved bythe following algorithm:

1. apply the Fix operation to each of the sequencesof the Nonogram

2. if the Nonogram is not solved, go to step 1

For a classic Nonogram, this definition is equiva-lent to the one in the previous paragraph. In orderfor a Curved Nonogram to be solved by this algorithmwithout getting stuck, the colour of at least one newface must be determined in each loop. This meansthat it can take at most 2n loops to finish for a sim-ple Curved Nonogram with n puzzle curves, i.e. oneloop per half-curve. Because we already know thatthe Fix operation can be performed in polynomialtime, we can conclude that the entire algorithm takespolynomial time as well.

3.4 Aesthetics measurements

In Section 1.3 we gave six conditions that make for agood Curved Nonogram. Two of these conditions arerelated to the aesthetics of a Curved Nonogram: thatit should be visually unambiguous and that it shouldbe aesthetically pleasing. In this section, we deriveseveral concrete measurements that can be used to de-termine if a Curved Nonogram satisfies these two con-ditions. The concrete measurements in this sectionwere derived from a list of informal criteria. Thesecriteria describe the properties that an unambiguousand aesthetically pleasing Curved Nonogram shouldor should not have, and is based on a combination ofintuition and explorative research. The list of infor-mal aesthetic criteria that we use is as follows. An un-ambiguous and aesthetically pleasing puzzle should:

1. not have large, open areas where not much hap-pens

2. not have convoluted, confusing areas where a lotof curves and crossings lie close together

3. not have faces with very narrow parts

4. not have curves that intersect at a very smallangle

Each of these informal criteria is represented by atleast one of the concrete measurements defined below.

(a) Minimum vertex distance The first concretemeasurement is the minimum distance between anytwo vertices in the arrangement of the Curved Nono-gram. If this measurement has a high value, thereare no ambiguous areas with close-together vertices.Thus, this measurement can be used to assess aes-thetic criterion 2.

Page 6

Let d(v, w) be the Euclidean distance between ver-tices v and w in the plane. The minimum vertexdistance (ma) is defined as follows:

ma = minv,w∈V |v 6=w

d(v, w),

where V is the set that contains the vertices of thepuzzle arrangement.

(b) Largest inscribed circle of any face Thesecond measurement is based on the inscribed circlesof the faces of the arrangement. The inscribed circleof a face is the largest circle that fits in the inte-rior of the face. If we take the inscribed circle of allfaces of the arrangement, this measurement is equalto the area of the largest circle. A high value of thismeasurement means that the Curved Nonogram hasat least one face with a large, boring open area init, which makes it less interesting from an aestheticperspective. Thus it is an ideal measurement for cri-terion 1. We define the largest inscribed circle (mb)as follows:

mb = maxf∈F

A(I(f)),

where F is the set that contains the faces of thepuzzle arrangement, I(f) is the inscribed circle of facef , and A(c) is the area of circle c.

(c) Local geometric dilation The third and finalmeasurement is based on the concept of geometricdilation. For this measurement, we interpret the ar-rangement of the puzzle as a graph G embedded inthe plane. Let p and q be any two points on theedges of G, with p 6= q. Let d(p, q) again denote theEuclidean distance between p and q in the plane. Fi-nally, let dG(p, q) be total arc length of the shortestpath in G between p and q. The geometric dilationbetween p and q is then defined as follows:

δ(p, q) =dG(p, q)

d(p, q).

Geometric dilation can also be measured for theentire graph. The geometric dilation of G is definedas

δ(G) = supp,q∈G

δ(p, q).

A low geometric dilation can be considered a posi-tive aspect for a puzzle. One reason is that geometricdilation functions as a lower bound for the intersec-tion angles of the curves of the puzzle, which can beused to check criterion 4. To be precise, all angles αin the puzzle are bound by the following formula:

α ≥ 2 sin−1(1

δ(G)).

Because we have not placed any restrictions on theshape of the curves that form a Curved Nonogram,they can in principle contain many sharp angles andother small-scale irregularities that can make the puz-zle ambiguous. A low geometric dilation also lim-its how extreme these irregularities can be. Finally,the geometric dilation is the only measurement of thethree that relates to the narrowness of faces, whichrelates to criterion 3. If the arrangement has a facethat is at one part very narrow, or in other words,there are two curves that come very close at somepoint without touching, this causes the geometric di-lation of the arrangement to be high.

While geometric dilation is a useful measurement,it should be noted that a lower value is not alwaysbetter. An arrangement with minimum geometric di-lation has very regular faces that start to approximatecircles. What we really want is an arrangement withangles that are not too small, and faces that are notso narrow that they become ambiguous. Therefore,we introduce the concept of local geometric dilation(mc), an alternative form of geometric dilation thatis only affected by points that are within a certaindistance of one another.

mc = supp,q∈G|d(p,q)≤ddil

dG(p, q)

d(p, q),

where ddil is a constant that determines how closea pair of points should be to contribute to the mea-surement.

4 Algorithm

This section describes the algorithm that we use togenerate Curved Nonograms. The algorithm takesa coloured line drawing as input, and generates aCurved Nonogram that has a solution image that isthe same as the input. Because this is the first puz-zle generation algorithm for Curved Nonograms, thegoal is to generate Curved Nonograms that are notambiguous and could thus be used in a puzzle book.Thus we do not focus on low computation times oraesthetic aspects that are unrelated to the ambiguity.The solvability of the puzzle is also not taken into ac-count. However, because the puzzles are created froma solution image (the input), they are guaranteed tohave at least one solution.

First, the input for the algorithm is detailed in Sec-tion 4.1. Following that, Section 4.2 describes the

Page 7

structure of the solutions that our algorithm pro-duces. A parameterisation of the solution space isgiven in Section 4.3. Next, the implementation de-tails are given that are relevant to this description ofthe algorithm, in Section 4.4. The evaluation functionto judge solutions with is then defined in Section 4.5.Finally, in Section 4.6 we give the pseudo code of theoptimization algorithm that uses the aforementionedelements to find a good solution.

4.1 Input

The input to the algorithm proposed in this section isa planar arrangement induced by a set of input curvesand a rectangle R. The faces of this arrangement arecoloured either black or white.

An input curve is a chain of Bezier curves, whichcan be either first, second or third order. These typesof chains are widely used in the SVG format, a vector-graphics format, which has the advantage that anySVG drawing tool can be used as a graphical interfaceto provide input for the algorithm. Another reasonto use these Bezier curves is that they are not verycomplex.

The input curves are enclosed by R. This rectan-gle will be used as the puzzle enclosure of the out-put puzzle. While the puzzle enclosure of a CurvedNonogram can take any shape (as long as it is closed),we limit the shape to a rectangle in the input of thealgorithm. This simplifies some later steps in the al-gorithm, like point-inclusion checks and the globaloptimization. An example of the input is shown inFigure 4.

We use the SVG editor Inkscape to create the SVGfiles from which our program reads the input. Whileother formats could be used to store the input, thisformat offers the advantage of the availability of freelyavailable graphical editing tools. Of course, the inputcould be provided in text format or any other format,as long as the interpretation is the same.

4.2 Solution space

In principle, the only restriction placed on the outputof the algorithm is that it is a valid Curved Nonogramwith puzzle enclosure R. This is a very complex solu-tion space, with an unbounded dimensionality causedby the unbounded number of curves of unboundedcomplexity that any solution may contain. We chooseto limit the solution space for our algorithm by addingadditional constraints on the structure of the output.

Firstly, we let the input curves be preserved in theoutput. This means that we can keep the black andwhite coloured areas of the output exactly the same

Figure 4: An example of the input of the algorithmwith four (closed) input curves and one black face.

as those in the input. This allows the algorithm toperfectly achieve the previously stated goal that thesolution image of the output should be similar to theinput.

To be able to preserve the input curves in the out-put, those curves must be connected to the puzzle en-closure at both ends, as per the definition of a CurvedNonogram. The input curves are chains of Beziercurves, some of which may be closed. We choose tocut the input curves, at each point where two consecu-tive Bezier curves in the chain are not C1-continuous[16]. An example of this cutting is shown in Fig-ure 5. This procedure cuts open most of the closedchains, allowing them to be connected to the puzzleenclosure. It also serves to make the chains less long,which we expect to lead to a better puzzle. We callthese new, smaller Bezier chains the puzzle curves,because each of them will become a puzzle curve inthe output Curved Nonogram.

Note that closed input curves that are C1-continuous will not be cut open by this method. Wedo not connect these curves to the puzzle enclosure,so they will become background curves in the output.

The second constraint describes how each of thepuzzle curves is connected to the puzzle enclosure.A puzzle curve may have zero, one or two ends thatalready lie on the puzzle enclosure. We call the endsthat do not lie on the puzzle enclosure the extendable

Page 8

Figure 5: An example of how the input curves arecut. The four input curves are cut into nine puzzlecurves.

curve ends (ECEs). These curve ends are extendedto the puzzle enclosure by a single cubic Bezier curve,that fits to the chain C1-continuously. An example isshown in Figure 6.

The advantage of this constraint is that it limits thecomplexity of the output to a constant factor timesthe number of puzzle curves. The choice for exten-sion by a single Bezier curve was motivated by earlyexperimentation revealing that a generated solutionwould already become quite complex with this num-ber of curves added.

4.3 Parameterisation

The output structure described in the previous sec-tion consists of a set of Bezier chains with extendablecurve ends (ECEs) that are extended to the border bya single cubic Bezier curve that fits C1-continuouslyto the chain. In this section, the corresponding solu-tion space is parameterised, to allow the problem tobe solved using parameter optimization techniques.

The parameterisation is on a per-curve-end basis.Take an ECE of a puzzle curve c. This curve end willbe extended by a cubic Bezier curve that fits to itwith C1 continuity. Let Q be the Bezier curve thatis the last in the chain on this end of c. Q is an n-th order Bezier curve (n can be 1, 2 or 3), describedby its control points q0, . . . , qn. It will be connected

Figure 6: An example of how the puzzle curves are ex-tended to the puzzle enclosure by a C1-continuouslyfitting cubic Bezier curve.

to the puzzle enclosure by cubic Bezier curve B, de-scribed by b0, b1, b2 and b3. We will show that B hasthree degrees of freedom, which will be expressed bythe parameters p0, p1, p2. Firstly, B must start whereQ ends. Assuming Q is directed so that qn is theendpoint of the chain:

b0 = qn.

Secondly, the first derivatives of Q and B must beequal where they join to make them C1-continuous.Let Q(t) and B(t) be parameterisations of the twocurves with t ∈ [0, 1), for which the curves meet atQ(1) = qn = b0 = B(0). This requirement can thenbe written as

Q′(1) = B′(0),

which can be written in terms of the control pointsas

b1 = qn + (qn − qn−1).

The third control point of B has two degrees offreedom, which we parameterise as the x- and y-valueof this point:

b2 =

[p0p1

].

Page 9

Finally, the fourth control point has one degree offreedom. This point must lie on the puzzle enclosure,because it is the endpoint of B. Let R(t) be an arclength parameterisation [17] of the rectangular puzzleenclosure curve with t ∈ [0, 1). We parameterise thefourth control point as follows:

b3 = R(p2), p2 ∈ [0, 1).

Puzzle enclosure constraint According to theconstraints for a valid Curved Nonogram, we mustmake sure that B is contained in the puzzle enclo-sure. An efficient way to do this uses the ConvexHull property [18] of Bezier curves, which states thata Bezier curve is enclosed by its convex hull. We willforce the convex hull of the control points of B to beenclosed by the puzzle enclosure. Because the puz-zle enclosure is convex (a rectangle), this constraintis satisfied if all of the control points of B lie withinthe puzzle enclosure.

This constraint is already satisfied for b0, whichlies on an input curve, and b3 which lies on the puz-zle enclosure itself. Point b1 can lie outside the puzzleenclosure according to the definition given above, de-pending on the position of q2 and q3. In case thishappens, we instead use the following formula:

b1 = qn + a(qn − qn−1),

where a is chosen so that b1 lies on the puzzle en-closure. In this case, B is no longer a C1-continuousextension of Q, but is still a G1-continuous [16] exten-sion, because the directions of the derivatives of thecurves in their meeting point b0 are still the same.

Finally, to ensure that b2 lies within the puzzle en-closure, we limit p0 and p1 in the following ways:

p0 ∈ [Rxmin, Rxmax],

p1 ∈ [Rymin, Rymax],

where Rxmin, denotes the x value of the left side ofthe puzzle enclosure, etc.

Total number of parameters As shown above,the cubic Bezier curve extension of one curve endis described by three parameters. Since each puzzlecurve can have between zero and two ECEs, a solu-tion with n puzzle curves is described by at most 6nparameters.

4.4 Implementation details

This section contains implementation details that arenot part of the design of the algorithm, but may be

important to know for anyone wishing to implementthe algorithm for themselves.

Polyline approximation The most important im-plementation detail is that the arrangements of Beziercurves described in the previous sections are in factapproximated by arrangements of polylines. The rea-son for this approximation is that the code for the ex-act representation of Bezier curve arrangements wastoo unreliable, and also slower than the polyline ap-proximation. Note that the curves read from the in-put and written to the output are still Bezier curves:they are only approximated when an arrangement isneeded to be scored by the evaluation function.

The approximation that we use for a Bezier curveworks as follows. First, we make sure that the Beziercurve is parameterised in the standard way. For aBezier curve B with control points b0 . . . , bn, this isdefined as:

B(t) =

n∑i=0

(n

i

)(1− t)n−itibi, t ∈ [0, 1].

We then sample B(t) at values of t uniformlyspread between 0 and 1. These sampled points are thepoints of the approximating polyline, and we call thenumber of samples min approx points. The advan-tage of the given parameterisation B(t) is that theseuniformly sampled points lie closer together wherethe curvature of B is high, which is where we needthe most samples to be accurate.

In the last step of the approximation, we en-sure that the distance between two subsequent sam-pled points is never larger than the pre-definedmax segment length. If two subsequent sampledpoints B(sk) and B(sk+1) do not meet this criterion,we add the sample point B( sk+sk+1

2 ) between them.This is done recursively until each pair of subsequentpoints fulfils the criterion. The reason for this secondstep is that we want to ensure a minimum densityof sample points, even at low-curvature parts of B,because they are used by the dilation penalty calcu-lation in the evaluation function, as detailed in thenext section.

4.5 Evaluation function

In this section, we define an evaluation function whichtakes a Curved Nonogram puzzle as input, and out-puts a score that represents how good the puzzle is.This function is based on the observations from Sec-tion 3.4, in which we gave informal criteria and threemeasurements that can be used to judge the non-ambiguity and aesthetic merit of a Curved Nonogram.

Page 10

For the evaluation function, the focus is more on thenon-ambiguity of the puzzle than on the aestheticmerit. The evaluation function is based on threepenalties: the vertex penalty, the dilation penalty,and the face penalty. For each of these penalties, ahigher value indicates a worse puzzle, and a value ofzero is the best result possible.

Vertex penalty The first measurement given inSection 3.4 is the minimum vertex distance of thepuzzle’s arrangement. This is a simple way to mea-sure certain ambiguities. However, preliminary testsshowed that this measurement was too global, whichdoes not fit well with our optimization algorithm thatmostly makes local changes. Since the minimum ver-tex distance is typically determined by one pair ofvertices, it does not give any useful information formaking changes to a curve on which neither of thesetwo vertices lie.

For this reason we use a different measurement forvertex distance, the vertex penalty ρvert. All pairsof vertices that lie too close together contribute tothis measurement. The amount that a vertex paircontributes is based on the distance between them.To be precise:

ρvert =∑

v,w∈V |v 6=w∧d(v,w)<dvert

dvert − d(v, w),

where dvert is a constant that denotes the minimumdistance that we wish to have between vertices.

Dilation penalty In Section 3.4 we also introducedthe local geometric dilation measurement. This mea-surement is useful because there are two importanttypes of ambiguous situations that cause the dilationof the arrangement to be high. Firstly, curves thatintersect at a shallow angle, and secondly, curves thatalmost touch without actually intersecting. Becausethis is also a global measurement, like with the mini-mum vertex distance, we use a variation on this mea-surement that we call the dilation penalty ρdil.

The calculation of ρdil is based on the polyline ap-proximation described in Section 4.4. Let Z be thecollection of sampled points, i.e. the vertices of thepolylines that approximate the Bezier curves. We de-fine the dilation penalty as:

ρdil =∑

p,q∈Z|p6=q∧d(p,q)<ddil∧δ(p,q)>δmax

ρdil(p, q),

where ddil is the minimum distance that we wouldlike to have between points that have a dilation higher

than δmax, and ρdil(p, q) is the dilation penalty con-tribution of p and q. This contribution is defined as:

ρdil(p, q) = w(p)w(q)[ddil − d(p, q)].

The terms w(p) and w(q) are weights that have todo with the granularity of the polyline approximationaround p and q. If p has neighbouring vertices pn1 andpn2 on the polyline, then its weight is defined as:

w(p) =d(p, pn1) + d(p, pn2)

2.

These weights prevent that the total dilationpenalty scales with the granularity of the polyline ap-proximation.

The given measurement ρdil differs from the localgeometric dilation measurement in two ways. Firstly,all problematic point pairs contribute to the measure-ment, instead of only the pair with the highest dila-tion. This helps the algorithm make local changesthat affect the score. Secondly, the penalty for a pairof points with high dilation that lie close together isbased on the (Euclidean) distance between the points,instead of their dilation. For such point pairs, thelower the Euclidean distance, the more it seems likethe respective curves are touching - an ambiguous sit-uation, while a higher curve distance (the other termof the dilation) would not increase the ambiguity atall.

Face penalty The third and final measurementthat contributes to the score is the face penalty ρface.This measurement does not relate to any of the spe-cific measurements given in Section 3.4, but it doessolve the ambiguity caused by very small faces, whichis not always solved by the two measurements givenabove.

Specifically, during early testing the algorithm pro-duced many very small faces of which the verticeswere just far enough apart to not contribute to theclose-together vertex penalty. Even though thesefaces are very narrow in the absolute sense, they donot contribute to the high dilation penalty unless theyhave a very elongated shape. Thus, the small facepenalty is used to measure the presence of these typesof faces.

The measurement is calculated similarly to theclose-together vertex measurement. All faces that aretoo small contribute to it according to their size:

ρface =∑

f∈F |A(f)<Amin

Amin −A(f),

Page 11

where A(f) is the area of face f , and Amin is aconstant that denotes the minimum area that a faceshould have to be considered unambiguous.

Evaluation function The evaluation function thatwe use for our algorithm outputs a score that is aweighted sum of the three penalties:

score = wvertρvert + wdilρdil + wfaceρface,

wvert, wdil, wface ≥ 0,

where wvert, wdil, and wface are the weights forthe three corresponding penalties. Note that a higherscore indicates a worse solution. The best achievablescore is zero.

4.6 Optimization algorithm

Now that we have a parameterisation of the solutionspace and an evaluation function for solutions, we canintroduce our optimization algorithm. The algorithmconsists of a pre-processing stage, followed by multi-ple global and local optimization rounds.

4.6.1 Algorithm structure

The structure of the complete algorithm is shown inAlgorithm 1. In the pre-processing step, the inputcurves are cut and the extension curves E are ini-tialized. The parameters p0 and p1 of each extensioncurve are set to a uniformly randomly chosen pointin R and a uniformly randomly chosen point on theboundary of R, respectively.

The extension curves are then optimized in mul-tiple global and local optimization rounds. The al-gorithm can be said to use a Hill climbing strategy,which means that the evaluation score either staysthe same or improves after each global or local round.Finally, the output puzzle is created and returned.

In all the pseudo code in this section, constants likeglobal rounds are shown in cursive with underscorespacing.

4.6.2 Global optimization

The optimization starts with multiple rounds ofglobal Hill climbing, as detailed in Algorithm 2. Theoptimization happens for one parameter at a time.The parameter is set to several different values spreadover its range, while the rest of the parameters staylocked. The value that yields the solution that gets

Algorithm 1 Complete algorithm

function algorithm(inputCurves, R)puzzleCurves ← cutCurves(inputCurves)E ← extendRandomly(puzzleCurves)for i ← 1 to global rounds do

GlobalOpt(E)end forfor i ← 1 to local rounds do

LocalOpt(E)end forpuzzle ← BuildPuzzle(puzzleCurves, E, R)return puzzle

end function

Algorithm 2 Global optimization round

function globalOpt(E)score ← evaluate(E)for all e ∈ E do

for all p ∈ {0, 1} dobest = e.getParam(p)for all v ∈ globalVals(p) do

e.setParam(p, v)if evaluate(E) > score then

score ← evaluate(E)best ← v

end ifend fore.setParam(p, best)

end forend for

end function

Page 12

the best score from the evaluation function is chosen,and the algorithm moves on to the next parameter.

The values that are tested for each parameter aregenerated by the function shown in Algorithm 3. Inthis pseudo code, a function random(a, b) is usedthat returns a floating point value from the uniformrandom distribution over the interval [a, b).

Algorithm 3 Choose global parameter values

function globalVals(p)vals ← ∅if p == 0 then

steps ← steps p0rangeX ← Rxmax −RxminrangeY ← Rymax −RyminoffsetX ← random(0, rangeX / steps)offsetY ← random(0, rangeY / steps)for i← 0 to steps − 1 do

x ← offsetX + i / stepsfor j ← 0 to steps − 1 do

y ← offsetY + j / steps

vals.append(

[xy

])

end forend for

else if p == 1 thensteps ← steps p1offset ← random(0, 1 / steps)for i← 0 to steps − 1 do

v ← offset + i / stepsvals.append(v)

end forend ifreturn vals

end function

The goal of the global optimization rounds is toquickly move the solution away from the randomizedinitial state, which is generally very chaotic and am-biguous. Because the tested values are chosen glob-ally, independent of the current values, the influenceof the randomness of the initial state is reduced. Therandom offsets are used to prevent all the third andfourth control points of the extension curves from ly-ing on the same grid. It also increases the usefulnessof running multiple global optimization rounds.

4.6.3 Local optimization

The local optimization rounds are similar to theglobal rounds, in that the parameters are optimizedone by one and in that the best scoring value is al-ways chosen. The difference is that the tested values

are not chosen globally, but from the neighbourhoodof the current value. In case of a p0 point parame-ter, the tested values are points that are uniformlyspread over a circle centered on the current value ofp0. For a p1 parameter two values are tested, whichare gained by moving a certain distance clockwise orcounter-clockwise along the puzzle enclosure R fromthe current value of p1. The pseudo code of a localoptimization round is shown in Algorithm 4, and thedetails of how the values are chosen are shown Algo-rithm 5.

Algorithm 4 Local optimization round

function localOpt(E)score ← evaluate(E)for all e ∈ E do

for all p ∈ {0, 1} dobest = e.getParam(p)for all v ∈ localVals(p, best) do

e.setParam(p, v)if evaluate(E) > score then

score ← evaluate(E)best ← v

end ifend fore.setParam(p, best)

end forend for

end function

The goal of the local optimization rounds is to it-eratively make small adjustments to the extensioncurves that move them away from ambiguous situ-ations. Because only small local changes are madeeach round, the local optimization round should berepeated a larger number of times than the globaloptimization.

5 Experiment and results

This section describes an experiment in which theproposed algorithm is used to make Curved Nono-grams from a test set of input images. We do thisfor several different parameter values that determinehow the penalties of the evaluation function are cal-culated. The goal of the experiment is to answer thefollowing two questions:

1. Can simple images be made into unambiguousand visually pleasing Curved Nonograms?

2. Can simple images be made into Curved Nono-grams that have a unique solution?

Page 13

Algorithm 5 Choose local parameter values

function localVals(p, v)vals ← ∅if p == 0 then

steps ← circle pointsr ← step size p0offset ← random(0, 2π / steps)for i← 0 to steps − 1 do

point ← + i / stepsfor j ← 0 to steps − 1 do

α ← offset + 2πi / stepspoint ← pointOnCircle(v, r, α)vals.append(point)

end forend for

else if p == 1 thensize ← step size p1step ← size / boundaryLengthvals.append(v − step)vals.append(v + step)

end ifreturn vals

end function

The set-up of the experiment is described in de-tail in Section 5.1. Following that, the results of allruns of the algorithm are shown in Section 5.2. Thescores of the output are then broken up into theirthree component parts in Section 5.3. This is fol-lowed by Section 5.4 which contains an analysis ofthe influence of the complexity of the input on theresults. We then show a few selected output puzzlesin Section 5.5 and discuss the positive and negativeaspects of them. Finally, the solvability of the gener-ated Curved Nonograms is discussed in Section 5.6.

5.1 Experiment set-up

Test set To test the performance of the algorithm,we ran a series of experiments on a test set of 16 in-put images - see Table 1. The images can be found inAppendix A. The number of extendable curve endsfor each image is also shown, as this determines thenumber of parameters that the algorithm has to opti-mize. The images were created by us, and are meantto be representative for simple black-and-white svgdrawings. The drawings are varied in shape and com-plexity.

Experiment settings For each image from thetest set, the algorithm was run with five different set-tings, which can be found in Table 2. The difference

Image ECEspenguin 10coffee cup 12lamb 12butterfly 14fish 16monitor 16rocket 16mask 18airplane 22flower 22car 24hut 24catface 26church 32pumpkin 36space 38

Table 1: The images that were used as a test set, withthe number of ECEs of each image.

between the settings is the strictness of the evalua-tion function. Setting (a) is the least strict, whichmeans that only extremely ambiguous situations aretaken into account by the evaluation function. Thestrictness increases from setting (a) through setting(e), which is the most strict.

Subjectively, setting (b) marks the bottom line forwhat we find to be visually unambiguous puzzles.Setting (c) corresponds to puzzles that are comfort-ably unambiguous, they do not really need to be moreclear than this. Setting (d) is more strict than nec-essary for removing ambiguity, but this may improvethe aesthetics of the puzzle by forcing a more evendistribution of the curves and vertices over the space.Setting (a) and (e) are more extreme variants of (b)and (d) respectively, and serve to test the limits ofthe algorithm.

dvert Amin ddil δmax αminSetting (a) 5 25 3.75 11.5 10◦

Setting (b) 8 40 6 7.7 15◦

Setting (c) 11 55 8.25 5.8 20◦

Setting (d) 14 70 10.5 4.6 25◦

Setting (e) 17 85 12.75 3.9 30◦

Table 2: The five different settings with which thealgorithm was run. αmin is derived from δmax: it isthe minimum angle at which a curve intersection doesnot receive a dilation penalty.

Page 14

Static parameters The parameters of the algo-rithm that are not included in Table 2 are the samefor each of the five settings. Their values are shownin Table 3.

Parameter Valuemin approx points 100max segment length 2wvert 1wdil 0.04wface 0.3global rounds 3local rounds 20steps p0 8steps p1 32step size p0 10step size p1 10

Table 3: The parameters of the algorithm that arethe same across the five settings.

Performance While performance was not a goal ofthe algorithm’s implementation or design, and it wasthus not thoroughly tested, it may still be valuableto give a rough idea of the runtime. Most tests wererun on a PC with an Intel R©CoreTMi5-3570 CPU, withfour cores @ 3.40 GHz, and 12 GB RAM. With set set-up, the algorithm on Setting (c) took 13 hours and 23minutes to process the car test set image, using a sin-gle core. The majority of the processing time is spenton the calculation of the dilation penalty. With someoptimization of the implementation, we expect it tobe possible to cut this time in half, without changingthe design of the algorithm.

5.2 Results per experiment setting

Table 4 shows the evaluation function score of theoutputs of the test set images for each setting of theexperiment. The calculation of the score is explainedin Section 4.5. In short: the perfect score is zero,and each ambiguity in the puzzle increases the score.Please note that the scores are not directly compara-ble between the settings. For example, two verticesthat are at 3 distance from one another would in-cur a close-together vertex penalty of 2 according tosetting (a), where dvert = 5; the same two verticeswould incur a larger close-together vertex penalty of5 for setting (b), where dvert = 8. In general, thesame image always receives a higher score accordingto the evaluation function of a stricter setting, unlessthe score is zero.

From the score table, we can first of all observethat all images achieved a perfect or very low score(below 10) on setting (a). The same was achievedfor all but two images on setting (b), and for 10/16images on setting (c). For setting (d), there are still9/16 images that achieved a very low score, but boththe number of perfect scores and the average scoreare much higher than for setting (c). Only 3/16 im-ages have a very low score on setting (e), and manyimages have a very high score of more than 100 onthis setting.

We can conclude that for most images, setting (a)and setting (b) are probably not strict enough, be-cause they achieve a perfect score on these settings,meaning that they could have been optimized fur-ther. This is fortunate, because we judge setting (b)to be the bottom line where puzzles become visuallyunambiguous (if they achieve a perfect score).

A final observation is that there is a large differ-ence between the images in how they score across allsettings. This difference seems to be related to thenumber of extendable curve ends of each image. Forexample, all images with 18 or less ECEs had a verylow score on settings (a) through (d). At the otherend of the spectrum, the only images that did nothave a very low score on setting (c) were those with22 or more ECEs. The correlation between the num-ber of ECEs of the input and the score of the outputpuzzle is further analysed in Section 5.4.

Score progression Table 4 only shows the finalscores: the scores of the experiment which correspondto the output puzzles. In order to get a sense of howthe score progresses through the rounds of the al-gorithm, see Figure 7. This figure shows the scoreprogression of four images that are representative forthe test set. The graph shows that all scores havemore or less converged by the end of the algorithm.In fact, most have converged about halfway through.This lends weight to the scores in Table 4, becauseit means that they cannot be trivially improved byrunning the algorithm for a longer period of time.

5.3 Breakdown of the score compo-nents

In Section 4.5 we introduced the three componentsthat together make up the evaluation function: theface penalty, the vertex penalty and the dilationpenalty. In this section we analyse how much eachof these components contributed to the final scores inTable 4. The distribution of the final score over thecomponents is shown in Figure 8.

Page 15

0

10 -2

10 -1

10 0

10 1

10 2

10 3

10 4 rocket .svg

Set t ingu(a) Set t ingu(b) Set t ingu(c) Set t ingu(d) Set t ingu(e)

10 -2

10 -1

10 0

10 1

10 2

10 3

10 4 m ask.svg

10 -2

10 -1

10 0

10 1

10 2

10 3

10 4

10 5 airplane.svg

10 -1

10 0

10 1

10 2

10 3

10 4

10 5 space.svg

Algorithm urounds

Log

10

uofu

sco

re

0

0

Figure 7: The progression of the score through the rounds of the algorithm of several test set images, for each ofthe experiment settings. The black vertical lines indicate the start of a global optimization round, and the greyvertical lines indicate the start of a local optimization round.

Page 16

(a) (b) (c) (d) (e)05

10152025303540

penguin.svg

(a) (b) (c) (d) (e)02468

10121416

coffee_cup.svg

face_score vertex_score dilation_score

(a) (b) (c) (d) (e)0.00

0.01

0.02

0.03

0.04

0.05

0.06lamb.svg

(a) (b) (c) (d) (e)0

5

10

15

20

25butterfly.svg

(a) (b) (c) (d) (e)02468

10121416

fish.svg

(a) (b) (c) (d) (e)0.00

0.01

0.02

0.03

0.04

0.05

0.06monitor.svg

(a) (b) (c) (d) (e)0

1

2

3

4

5

6

7rocket.svg

(a) (b) (c) (d) (e)0

5

10

15

20

25

30

35mask.svg

(a) (b) (c) (d) (e)05

10152025303540

airplane.svg

(a) (b) (c) (d) (e)0

50

100

150

200

250

300flower.svg

(a) (b) (c) (d) (e)0

10

20

30

40

50

60

70car.svg

(a) (b) (c) (d) (e)0

10

20

30

40

50

60

70hut.svg

(a) (b) (c) (d) (e)0

20406080

100120140160

catface.svg

(a) (b) (c) (d) (e)0

20

40

60

80

100

120church.svg

(a) (b) (c) (d) (e)0

20

40

60

80

100

120

140pumpkin.svg

(a) (b) (c) (d) (e)0

20

40

60

80

100

120space.svg

Setting

Sco

re c

ontr

ibuti

on

Figure 8: Breakdown of the puzzle’s scores into their three components, after they have been weighed. The sumof these components is the total score of the puzzle.

Page 17

Image ECEs Setting (a) Setting (b) Setting (c) Setting (d) Setting (e)penguin 10 0.0 0.0 0.0 6.31973 61.7625coffee cup 12 0.0 0.0 0.0 0.87834 19.2688lamb 12 0.0 0.0 0.0 0.0 0.0butterfly 14 0.0 0.0 0.0 0.0 24.4548fish 16 0.0 0.0 0.0 2.37818 26.7813monitor 16 0.0 0.0 0.0 0.0 0.0rocket 16 0.0 0.0 0.130708 1.1384 6.76382mask 18 0.0 0.0 0.0 0.0 35.1045airplane 22 0.0 0.0 0.0 20.1173 54.1086flower 22 0.0 0.0 16.1588 129.103 373.143car 24 0.0 4.98756 17.5392 74.9453 94.2376hut 24 0.100717 0.0 29.3772 5.78301 123.141catface 26 0.0 0.0 26.4015 66.2723 262.231church 32 0.0 22.1669 0.0 56.5003 155.865pumpkin 36 0.0 32.0071 91.3201 219.049 258.504space 38 4.96683 0.716458 48.6102 116.422 194.859

Table 4: Final score for each test set image for each experiment setting. The number of ECEs is also shown foreach image.

The figure shows that the contribution of the scoretypes is relatively well balanced overall. Of the threecomponents, the face penalty is most often absentor relatively low. It can also be observed that mostoutputs that have a face penalty, also have a ver-tex penalty that is equal to it or higher. This canbe explained by the fact that the vertices incident toa small face typically lie close together, resulting ina vertex penalty. The result is that the function ofthe face penalty and the vertex penalty partly over-lap. The face penalty is mostly useful to prevent thespecific case of small, narrow faces that are only inci-dent to two edges and two vertices. These cases arevery prevalent if the algorithm is run without a facepenalty, which is why we feel that it is not superflu-ous.

5.4 Relationship between image com-plexity and score

In this section, we will explore if there is a correlationbetween the score of the output image and the com-plexity of the input image. We also give the numberof faces in each output puzzle, as a measure of thecomplexity of the output.

To start off with, Figure 9 shows a point plot foreach setting, in which each point represents an outputpuzzle. The horizontal placement of the point is de-termined by the number of extendable curve ends ofits input image, and the vertical placement shows thescore. Note that there are many overlapping pointsaround a score of zero, which unfortunately cannot

be seen clearly in the plots.

We expect the number of ECEs of an image to bea reasonable indication of the complexity of an inputimage, in terms of how difficult it is to handle forthe algorithm. It determines the number of extensionBeziers that are added to the puzzle, which, assumingthat the input image is (almost) unambiguous, are themain cause of ambiguities that the algorithm needsto resolve. Because of this, we expect that there is apositive correlation between the number of ECEs ofthe input image and the score of the output image.

Correlation Figure 9 also shows the correlation co-efficient and regression line for each setting. Becausethe number of data points in each plot is much toolow for statistic significance, we can only pose an hy-pothesis that may be researched in future work.

Our hypothesis is that there is a positive, but notlinear, correlation between the score of the outputpuzzle and the number of ECEs of the input image.The data seems to support this, as all correlation co-efficients are positive, but not very close to 1, and thelines do not fit very well.

The reason that we do not expect a linear correla-tion is that the score formula is non-linear, and can-not go below zero.

Observations Looking at Figure 9, the test set canbe split into three categories. The images with lessthan 20 ECEs all have very low scores compared tothe other images. The only setting for which some

Page 18

10 15 20 25 30 35 401

0

1

2

3

4

5Setting (a) (r= 0. 529)

10 15 20 25 30 35 40505

101520253035

Setting (b) (r= 0. 619)

10 15 20 25 30 35 4020

0

20

40

60

80

100Setting (c) (r= 0. 76)

10 15 20 25 30 35 4050

0

50

100

150

200

250Setting (d) (r= 0. 784)

10 15 20 25 30 35 400

50100150200250300350400

Setting (e) (r= 0. 674)

Number of ECEs

Sco

re

Figure 9: Point plots of the final scores from Table 4 versus the number of ECEs of the input images. The plotof each setting also shows the regression line as well as the correlation coefficient r for the points in that plot.

Page 19

of them ended with a score significantly above zero issetting (e).

The group of images that have between 20 and 30ECEs start showing some significant score at setting(c). This group also contains some images that havean exceptionally high score for their number of ECEs,most notably flower and catface. These images aredifficult to handle for the algorithm, for other reasonsthan their number of ECEs. Both of these imageshave many ECEs that lie close to one another, e.g.,where the petals and leaves of the flower meet in thecenter, and at the multitude of facial features in cat-face. These cases show a weakness of the method ofextending each ECE to the boundary. When ECEslie close together, and oriented towards each other, itmay be a better idea to connect them to each otherinstead.

Finally, the group of images with more than 30ECEs has cases where even setting (b) causes a sig-nificant score. It also has a lot of images with highscores overall, and a very low percentage of (near)perfect scores.

The differences between these groups suggest thatthe algorithm is limited in the number of ECEs in aninput image that it can successfully handle, depend-ing on the setting. Note that the input images inthe third group, although they are the most complexones of the test set, still look relatively simple. Thisstrongly suggests that a different method is requiredto handle complex images. Again, a similar methodthat connects some of the ECEs to each other in-stead of to the puzzle enclosure is a possible solutionto handle images with many ECEs.

Output complexity Table 5 shows the number offaces of each of the generated output puzzles. Thenumbers range from 40 to 239, with an average of112 faces (rounded). We can compare these numbersto the number of faces that we typically see in classicNonograms. In our experience, the classic Nonogramsfound in puzzle books typically have a 10x10 grid (100faces), a 15x15 grid (225 faces) or a 20x20 grid (400faces). We can observe that most of the generatedCurved Nonograms have a number of faces that isclose to that of a 10x10 or 15x15 classic Nonogram.Of course, these numbers are not directly comparable,but they suggest that the numbers of faces are in theright range for the puzzle to be fun.

Image ECEs (a) (b) (c) (d) (e)penguin 10 52 43 40 48 48coffee cup 12 74 80 62 58 52lamb 12 59 60 63 54 49butterfly 14 69 63 59 52 47fish 16 137 104 88 93 67monitor 16 114 95 109 78 84rocket 16 112 108 100 92 80mask 18 137 103 96 78 80airplane 22 127 119 107 106 88flower 22 167 136 157 118 130car 24 145 124 113 103 110hut 24 128 108 114 94 95catface 26 153 131 147 117 130church 32 184 159 137 127 121pumpkin 36 237 210 212 209 172space 38 239 206 190 221 194

Table 5: The number of faces in each of the generatedoutput puzzles, for settings (a) through (e).

5.5 Qualitative analysis ofoutput puzzles

This section contains several output puzzles from theexperiment. Because there is not enough room toshow all 16 times 5 output images, we focus on asubset of output images that most clearly show theworkings and shortcomings of the algorithm. To geta better impression of the full set of images, pleaserefer to Appendix B.

rocket The first image that we will discuss is rocket(see Figure 12, of which the output for all five settingsis shown in Figure 10 and 11). This image had a verylow score for all settings, making it a good exampleof an image that the algorithm can handle well. Infact, the only penalty that the image receives is a di-lation penalty on settings (d) and (e) which cannot behelped by the algorithm. These penalties are appliedbecause an input curve intersects the puzzle bound-ary at an angle that is too small for settings (d) and(e). This shows a potential drawback of our approachto not modify the input curves: ambiguous situationsthat only involve input or boundary curves cannot beimproved.

Another consequence of this approach is that thealgorithm cannot cut G1-continuous curves, not evenat points of high curvature. For an example of this,see the tail of the rocket’s middle fire trail, in thebottom-center area of the output images. At such apoint, it might be desirable to change the input curveslightly and create a point that is not G1-continuous,

Page 20

9

9

9

925

52

8

8

6

6

9

9

8736

63786188

8186

35121

12153

12123

31221

12

12

5

5

7

7

8

8

17

1711 5

5 11

2 19

1927112

2 11 7

14

14

22

22

77

7718

18

(a) Setting (a)

10

10

13

31

16

61

6

6

5

5

12

12

10538

8351081710

10178

73121

12137

12117

71121

51

15

5

5

10

10

7

7

11

115 5

55

2 16

1625102

2 10 5

11

11

31

13

34

4312

12

(b) Setting (b)

10

10

8

86

6

7

7

6

6

11

11

1637

7361

7171

1177

4 3 1 1 1

1 1 1 3 41 1 9 4

4 9 1 1

12

12

6

6

8

8

7

7

17 1

1 171 10 4

4 10 1

1 8

81

611

1 1 6

12

12

4

4

66

6616

16

(c) Setting (c)

8

8

5

57

7

4

4

5

5

1 8

81

3537

7353

7163

3167

3 2 1 1 1

1 1 1 2 31 1 8 3

3 8 1 1

8

8

3

3

51

1 5

8

8

15 1

1 151 7 6

6 7 1

5

521

12

9

9

5

5

5 1 1

115

111

11 1

(d) Setting (d)

Figure 10: Output puzzle of test set image rocket for settings (a), (b), (c) and (d).

Page 21

6

6

7

79

9

5

5

5

5

13

13

3 4 2

243

143

3 14

1223

3221

381

183

11

11

2

2

10

10

7

7

15

159 5

59

4 412

21

7

7

6

6

115

511

81

18

Figure 11: Output puzzle of test set image rocket forsetting (e).

Figure 12: Input image rocket.

(a) Setting (d)

(b) Setting(e)

Figure 13: Close-ups of Figure 10d and Figure 11focused on the window of the rocket. The red circleshighlight ambiguous bends in the curve very close toan intersection.

Page 22

so that it can be cut.

The output puzzles of rocket are also a good exam-ple of how the outputs are shaped by the different set-tings. While the initial curve-extension parameterswere randomized, we used the same seed for each ofthe settings. This means that any difference betweenthe output images is purely the result of the strictnessof each setting. To start off with, Figure 10a looksquite ugly, and contains some very small faces thatare hard to see.

Figure 10b has some curves that are in exactly thesame place as the first figure, but those in the mostambiguous regions have been moved by the algorithm.This is most visible at the lower center region of theimage, where there are a lot fewer intersections closetogether.

In Figure 10c a lot of intersections and small faceshave disappeared, which helps both the aestheticquality and the ambiguity. This makes it the firstof the images that looks like an enjoyable puzzle. Atthe upper center region of the image, where the win-dow of the rocket is, are three strange looking facesthat consist of one curve pulled slightly over anothercurve. As explained in Section 5.3, these are a goodexample of the face penalty at work, which is the rea-son that they have not been made too small by thealgorithm. However, from an aesthetic perspective,the puzzle would look better if it had fewer of thesetypes of faces. These kinds of faces are quite prevalentin the output, because it is hard for the Hill climbingalgorithm in the local phase to pull the two curvesacross one another and make the face disappear. Asmall step in this direction will at some point incur aface penalty, and possibly a vertex or dilation penaltyas well, causing it to not be chosen by the algorithm.

Figure 10d seems to be a relatively smaller step up.The last faces that might be considered too small aregone, and the curves are spread more equally over theimage. Finally, Figure 11 is the output of the strictestsetting. Compared to the other outputs, it looks veryclean and aesthetically pleasing.

On the subject of aesthetics, one of the weak pointsof these five output puzzles is that they all containa number of large faces that are not very interest-ing. This is most visible in the top right corner ofeach of the output puzzles. The measurement mb

that we introduced in Section 3.4 - a measurementfor the largest inscribed circle of any face - can beused to this aesthetic weak point. This measurementis not included in the algorithm because of the fo-cus on removing ambiguity over improving aesthetics.However, it could be included in future work that fo-

cuses more on the aesthetic aspects, to penalise large,empty faces such as these.

Curve bends near intersections The final obser-vation that we wish to make about the rocket puzzlesis shown in Figure 13. This figure shows a close-up ofthe area around the rocket’s window for the outputpuzzles of setting (d) and setting (e). In these close-ups, a total of three more or less ambiguous areas areshown. The problem with each of these is that thereis high curvature (a bend in the curve) very close toan intersection. This type of ambiguity appears inmultiple output puzzles, and is important to high-light because it is the only type of ambiguity that didnot receive a penalty from the evaluation function.

First of all, this shows a weak point in the evalua-tion function, since it should penalize all ambiguitiesthat can appear. Secondly, there are two reasons whythis type of ambiguity is created relatively often byour algorithm. The first of these reasons is that itis encouraged by the dilation penalty. Especially onthe stricter settings, curves need to intersect at largeangle, which can be achieved by bending the curvejust before the intersection. The second reason isthat our algorithm has no control over the placementof the second control point of an extension Beziercurve, because we enforce C1-continuity. This leadsto many cases where the second control point lies veryclose to the intersection point (which is the first con-trol point), which can result in very high curvaturearound these points. A solution would be to onlyenforce G1-continuity, which gives the second controlpoint one degree of freedom. This degree of freedomcan either be used to simply place the second controlpoint a set distance away from the first control point,or it could be given as an extra degree of freedom forthe algorithm to optimize.

hut Figure 14a shows the output puzzle of the hutimage on setting (c). This is a good example of apuzzle with a non-zero score that is easy to correctby hand. The puzzle contains two ambiguous areas,one on the lower left and one just right of the center,both of which are the result of the algorithm gettingstuck in a local minimum.

We focus on the lower left area, shown enlarged inFigure 14b. This area contains one face that is toosmall, coloured pink; two vertices that lie too close toeach other, marked with yellow circles; and a numberof sampled point pairs that lie too close while havingdilation that is too high, marked by blue lines. Thissituation is a local minimum in the evaluation func-tion. First of all, some of these curves are part of the

Page 23

8

85

5

6

6 0

0

2 5 7

75272

2 7

7

71

1

4 2 6

624

24

4 2

3 2 6 1

1623

1123

3 2 1 1

95

5 95 1 5

515

451

1 5 41 5 1 1

1151

8

84 1

14

18

1888

88

117

71191

19

12

2110

10

(a) whole puzzle (b) zoomed in (c) possible fix

Figure 14: Output puzzle of test set image hut on setting (c). The image on the right shows the ambiguous areaaround the bottom left at a larger scale. Ambiguities that contribute to the score have been marked.

input drawing (see Appendix A), which means thatthey cannot be manipulated. The curve that goes to-wards the top left cannot easily be moved out of theway, because this would increase its dilation with thebottom curve. It would be ideal to make it go straightdown and cross the bottom curve, but any incremen-tal changes in that direction have the same problemof increasing the dilation. Making the small face moreelongated to separate the vertices increases the dila-tion within the face. Enlarging the face towards theleft would introduce a dilation penalty with the curveleft of it.

This situation is easy to fix by hand, for exam-ple see Figure 14c. This fix was created simply bychanging the position of the third control point of twoextending Bezier curves, which solved all of the ambi-guities highlighted in Figure 14b without introducingthem anywhere else in the puzzle. The conclusionthat can be drawn from this is that these situationscan be fixed by our method of extending curves andmanipulating the control points of those extensions,but not by the used search algorithm.

Finally, an observation that can be made whenlooking at Figure 14a as a whole is that the in-put image is relatively easy to spot. The hut im-age consists mostly of straight lines, which makes thehigh-variation curves added by the algorithm standout. This could be prevented by basing the extensioncurves on the input curves of the image. For exam-ple, the extension curves could be built by copyingand scaling (parts of) the input curves.

5.6 Solvability results

In Section 3.3 we defined the four classes of solvabilityfor (Curved) Nonograms. Of the 80 output puzzlesthat were produced by the experiment, 75 fall in thesimple class of solvability. The other 5 puzzles (fishon settings (c), (d) and (e) and car on settings (b)and (d)) had multiple solutions. These five puzzlesall have multiple solutions because they have a closedbackground curve that is not intersected by any puz-zle curves. In this situation, the face inside the back-ground curve is not associated with any descriptions,so in any solution of the puzzle it can be either blackor white. Fortunately, this special case can be fixedby adding an additional rule to the Curved Nono-gram, that a face should be left white if there is noinformation available for it. In other words, the useris asked to find a minimal solution, where as few facesas possible are coloured black. Re-running the exper-iment with this added rule resulted in 80/80 puzzlesthat were simple.

The fact that all output puzzles are simple, whilethe algorithm does not take the solvability into ac-count, is a good sign. It suggests that the solvabilitydoes not need to be a primary concern when gener-ating these kinds of puzzles, which makes it easier tofocus on the aesthetic aspects.

A possible concern is that the produced puzzlesmay be too easy. Some of the harder puzzle bookNonograms require the puzzler to think a few stepsahead, meaning that they are not simple solvable,

Page 24

but close to it. Fortunately, making a puzzle harderis easy to achieve. One way to do this is by removingdescriptions from the puzzle until it has the desireddifficulty. Of course, this must be done in such away that the puzzle keeps a unique solution, insteadof multiple solutions. Combining a pure-aestheticsalgorithm like the one we proposed with such a post-processing step may be a good way to generate goodpuzzles in the future.

6 Conclusion

In this paper we have introduced a new type ofpencil-and-paper puzzle called the Curved Nonogram,which is a variation on the existing Nonogram puz-zle. Where a classical Nonogram consists of a squaregrid of rows and columns, a Curved Nonogram isbuilt from arbitrary curves which make it free to takeany shape. This property gives the puzzle designermore artistic freedom in shaping the solution imageas they want. In addition, much more variety is pos-sible in the shape of the unsolved puzzle, while clas-sic Nonograms only differ in the number of rows andcolumns. This gives a unique feel to each CurvedNonogram even before it is solved. Both of these dif-ferences make Curved Nonograms more visually ap-pealing and interesting. New puzzle mechanics arealso introduced. For example, a new situation ariseswhen one side of a curve comes across the same facemore than once. This gives a new form of informa-tion about the face, which the puzzler will sometimeshave to use in order to solve the puzzle.

After introducing the Curved Nonogram itself, wealso introduced ways to classify and judge these puz-zles. First, we defined the four classes of solvabilityto which a Curved Nonogram can belong: no solu-tion, multiple solutions, unique solution and simple.Secondly, we gave three measurements to judge itsaesthetics and level of ambiguity: the minimum ver-tex distance, the minimum edge length, the largestinscribed circle of any face, and the local geometricdilation.

Next, we proposed an algorithm that takes acoloured line drawing as input, and makes a CurvedNonogram whose solution image matches the draw-ing. The algorithm cuts the input curves wherethey are not smooth and extends the resulting curvepieces to the puzzle enclosure, which results in a validCurved Nonogram. The goal of the algorithm is tomake these extensions in a way that minimizes theambiguity in the puzzle. This is achieved by intro-ducing an evaluation function that measures ambigu-ity in the puzzle. The evaluation function has three

terms, based on some of the three measurements de-fined earlier: the vertex penalty, the dilation penaltyand the face penalty. A simple optimization algo-rithm is used to find a puzzle with a low evaluationscore (lower is better), ideally zero.

Finally, we conducted an experiment in which weran the algorithm on a test set of 16 simple images, forfive different settings of the algorithm. The settingsimpacted the level of ambiguity that the algorithmwould tolerate, ranging from very forgiving (setting(a)) to very strict (setting (e)). We found that a verylow score (under 10) was achieved for 14/16 of theimages on setting (b), for 10/16 images on setting (c)and for 9/16 images on setting (d). This means thata nice puzzle was created for at more than half ofthe images, and that only two images did not achievea good result on setting (b), which results in onlybarely unambiguous outputs. We also found that thescores of the output puzzles seemed to have a positivecorrelation with the number of extendable curve endsof the input images, although our data sample wasreally too small for proper analysis.

In the last part of the experiment we classified theoutput puzzles by solvability, and found that 75/80puzzles were simple. The other five are a specialcase of multiple solutions: they contain faces thatare only adjacent to background curves, which meansthat they have no descriptions associated with them,so they could be either colour. While solvability wasnot a concern of the algorithm, it is good to knowthat the puzzles produced this way are likely to besimple. This is the desired solvability class for all butthe hardest puzzle book puzzles, and making themharder to solve can be as easy as removing some ofthe descriptions.

In conclusion, by introducing Curved Nonogramstogether with a generation algorithm for these puz-zles, we were able to produce a set of puzzles thatmay serve as an example of what this kind of puz-zle looks like. While the generated puzzles are notperfect, they are at least mostly unambiguous andsimply solvable. In our opinion, the produced puz-zles look interesting enough that we hope that theCurved Nonogram concept is expanded upon in thefuture. With this goal in mind, we have presentedmany ways in which our algorithm can be adjustedand improved, which are summarized and expandedupon in Section 7.

7 Future work

Since the Curved Nonogram is a new type of puzzle,and the proposed algorithm is the first algorithm that

Page 25

generates this type of puzzle, there are many improve-ments and alternatives that could be part of futurework. In this section we will repeat the suggestionsthat are already mentioned in the thesis, and add newideas as well.

Pre-processing the input First of all, there aremany other ways to do the pre-processing of the inputimage. If the algorithm is allowed to change the inputcurves, it would be possible to cut the curves at G1

continuous points that have high curvature. This maylook more natural, because those points often do notlook smooth at all. In this case, the curve should bechanged slightly to create a cutting point that is notG1-continuous, so that the separated curves intersectat an angle greater than zero.

Another possibility would be to give the algorithmthe power to change all the input curves, using forcedirected placement like [13] uses for Graph drawing toencourage but not enforce that the curves stay similarto the input.

An option that is closer to our algorithm is to letthe optimization algorithm change the input curvesuntil they have a score of zero, before extending them.This would solve the problem that we found whereambiguous input curves made it impossible to achievea perfect score.

The initial extension curves An area in whichbig improvements can be made is how the curves areinitially extended, i.e. the start state of the optimiza-tion algorithm. Our method of spreading the controlpoints of the extension Beziers uniformly random cre-ates extremely chaotic start states, which makes itharder to optimize. The first step to improving theextension curves would be to enforce a minimum dis-tance between the first and second control point. Theexperiment outputs show many cases where ambigu-ity is caused by high curvature at the start of theextension curve. A bigger change to avoid the chaosis to extend each curve to the puzzle enclosure witha straight line. A variation on this is to use a ran-domized Bezier curve that is allowed to deviate a setamount from that line.

Another option that we discussed is to copy piecesof the input curves, and use those to build the ex-tending curves instead of always using a cubic Beziercurve. Although this is a more complicated process,the advantage is that the solution image should bebetter hidden in the puzzle, because the properties ofthe generated curves will be similar to those of the in-put curves (e.g., many straight lines and 90◦ angles).

The algorithm can also be changed to handle inputs

with many extendable curve ends better, by connect-ing some of those curve ends to each other insteadof to the boundary. In this variation, an extra stepcould be added to the algorithm in which it deter-mines the best pairs of ECEs to connect, based onsmoothness, distance or the resulting ambiguities.

Evaluation and optimization If the previouslygiven suggestion of cutting high-curvature points isimplemented, it would also be interesting to add apenalty term for curve pieces with high curvature tothe evaluation function. Many of the problems thatremained in the output puzzles of the experiment in-volved such high-curvature pieces.

Another measurement that can be added is theLargest inscribed circle of any face, introduced in Sec-tion 3.4. We expect that adding (a variant of) thismeasurement to the evaluation function would greatlyimprove the aesthetics of the puzzle, by forcing a moreequal distribution of the curves over the puzzle. Thisshould especially help for the faces adjacent to thepuzzle enclosure, which were somewhat large, emptyand uninteresting for many of the generated outputpuzzles.

For the optimization of the evaluation function, weused a simple Hill climbing algorithm. This oftenresulted in the algorithm getting stuck in a local op-timum. It would be interesting to see how well thealgorithm performs with a more sophisticated searchheuristic, such as Simulated Annealing. What wouldbe even more interesting to research is if (parts of) theevaluation function could be formulated as a math-ematical function and solved analytically. Althoughthis might require a different evaluation function, itwould certainly give us more insight into the opti-mization problem than any search heuristic.

Solvability and difficulty In Section 5.6 we foundthat the only output puzzles that were not in thesimple class of solvability, were those with one ormore faces that were not associated with any descrip-tion. The result is that it can have either colour,which gives the puzzle multiple solutions if it hasany. For cases like these, a rule can be added tothe Curved Nonogram definition that only a mini-mal solution is considered correct. That is to say, asolution should have the minimum amount of black-coloured faces, and a maximum amount of white-coloured faces. With this rule added, such problem-atic disconnected faces must be coloured white andthe problem is solved.

An interesting study into the solvability of CurvedNonograms would be to see how many descriptions

Page 26

can be removed from the puzzle curves before thepuzzle is no longer simple. This could be used asa tool to bring the puzzle to a desired level of diffi-culty. This process could even continue if the puzzlehas become uniquely solvable but no longer simple,in which case the difficulty could be defined as themaximum search depth (i.e. the maximum numberof concurrent assumptions that must be made on theface colours) that is required to find the solution. Forsuch a study, it would be interesting to see how manydescriptions can be removed before the puzzle startsto have multiple solutions.

User studies Finally, while we have not touchedon the subject of user studies in this paper, there aremultiple interesting user studies that can be done onCurved Nonograms. Before Curved Nonograms canbe included in actual puzzle books, user studies needto be done on whether the rules and puzzle conceptare clear enough for the general public. Similarly, astudy could be done to determine the desired level ofcomplexity of a Curved Nonogram in a puzzle book,e.g. in terms of the number of curves, or the maxi-mum number of faces adjacent to one side of a puzzlecurve.

A different topic that is also interesting for a userstudy is how well the solution images of (generated)Curved Nonograms are hidden in the puzzle. Someof the test set images surely seem to be too clearlyvisible in the output puzzle. It would be especiallyinteresting to find a way to measure how well-hiddenthe solution image is for a given puzzle. A user studywould be needed to find such a measurement andprove that it works.

References

[1] N. Ueda and T. Nagao, “NP-completeness re-sults for NONOGRAM via parsimonious reduc-tions,” preprint, 1996.

[2] K. Batenburg and W. Kosters, “A discrete to-mography approach to Japanese puzzles,” inProceedings of the 16th Belgium-NetherlandsConference on Artificial Intelligence (BNAIC),pp. 243–250, 2004.

[3] C.-H. Yu, H.-L. Lee, and L.-H. Chen, “An effi-cient algorithm for solving nonograms,” AppliedIntelligence, vol. 35, no. 1, pp. 18–31, 2011.

[4] K. J. Batenburg and W. A. Kosters, “SolvingNonograms by combining relaxations,” PatternRecognition, vol. 42, no. 8, pp. 1672–1683, 2009.

[5] S. Salcedo-Sanz, E. G. Ortız-Garcıa, A. M.Perez-Bellido, A. Portilla-Figueras, and X. Yao,“Solving Japanese puzzles with heuristics,” inComputational Intelligence and Games, 2007.CIG 2007. IEEE Symposium on, pp. 224–231,IEEE, 2007.

[6] J. Wolter, “The ’pbnsolve’ paint-by-number puzzle solver.” Available athttp://webpbn.com/pbnsolve.html.

[7] K. J. Batenburg, S. Henstra, W. A. Kosters, andW. J. Palenstijn, “Constructing simple Nono-grams of varying difficulty,” Pure Mathematicsand Applications (Pu. MA), vol. 20, pp. 1–15,2009.

[8] E. G. Ortiz-Garcıa, S. Salcedo-Sanz, J. M. Leiva-Murillo, A. M. Perez-Bellido, and J. A. Portilla-Figueras, “Automated generation and visual-ization of picture-logic puzzles,” Computers &Graphics, vol. 31, no. 5, pp. 750–760, 2007.

[9] M. Hunt, C. Pong, and G. Tucker, “Difficulty-driven sudoku puzzle generation,” Journal ofUndergraduate Mathematics and Its Applica-tions (UMAPJournal), p. 343, 2007.

[10] A. M. Smith, E. Butler, and Z. Popovic, “Quan-tifying over play: Constraining undesirable solu-tions in puzzle design.,” in Foundations of Digi-tal Games conference, 2013, pp. 221–228, 2013.

[11] M. Loffler, M. Kaiser, T. van Kapel, G. Klappe,M. van Kreveld, and F. Staals, “The connect-the-dots family of puzzles: design and automaticgeneration,” ACM Transactions on Graphics(TOG), vol. 33, no. 4, p. 72, 2014.

[12] G. D. Battista, P. Eades, R. Tamassia, and I. G.Tollis, Graph drawing: algorithms for the visual-ization of graphs. Prentice Hall PTR, 1998.

[13] T. M. Fruchterman and E. M. Reingold, “Graphdrawing by force-directed placement,” Soft-ware: Practice and Experience, vol. 21, no. 11,pp. 1129–1164, 1991.

[14] H. C. Purchase, R. F. Cohen, and M. James,“Validating graph drawing aesthetics,” in GraphDrawing, pp. 435–446, Springer, 1996.

[15] H. C. Purchase, “Metrics for graph drawing aes-thetics,” Journal of Visual Languages & Com-puting, vol. 13, no. 5, pp. 501–516, 2002.

Page 27

[16] B. A. Barsky and T. D. DeRose, Geometric con-tinuity of parametric curves. Computer ScienceDivision, University of California, 1984.

[17] R. T. Farouki, “Arclength parameterization,”Pythagorean-Hodograph Curves: Algebra andGeometry Inseparable, pp. 369–380, 2008.

[18] G. Farin, Curves and surfaces for computer-aided geometric design: a practical guide, p. 35.Elsevier, 2014.

A Test set

This appendix contains the 16 test set images thatwere used in the experiment, shown in Figure 15through Figure 30.

Figure 15: Test set image penguin.

Figure 16: Test set image coffe cup.

Figure 17: Test set image lamb.

Page 28

Figure 18: Test set image butterfly.

Figure 19: Test set image fish.

Figure 20: Test set image monitor.

Figure 21: Test set image rocket.

Page 29

Figure 22: Test set image mask.

Figure 23: Test set image airplane.

Figure 24: Test set image flower.

Figure 25: Test set image car.

Page 30

Figure 26: Test set image hut.

Figure 27: Test set image catface.

Figure 28: Test set image church.

Figure 29: Test set image pumpkin.

Page 31

Figure 30: Test set image space.

Page 32

B Experiment outputon setting (c)

This appendix contains the output puzzles thatwere produced in the experiment by running thealgorithm on the test set on setting (c). Of eachoutput puzzle, the empty puzzle is shown next tothe solution. These outputs are shown in Figure 37through Figure 46.

Page 33

9 90 0 66 00

161

1611311

1131

8 8 16

61

61

1 61 2 2

221

21

126

6

8 1 2

218

2114

4 1 1 2

9 90 0 66 00

161

1611311

1131

8 8 16

61

61

1 61 2 2

221

21

126

6

8 1 2

218

2114

4 1 1 2

Figure 31: Output puzzle of penguin on setting (c).

74

47

1

1

77 56

65645

546

218

812

1012

2110

26

62

1414

3

3

17 1

117139

9 3 1

11 2

211

226

6 2 2

74

47

1

1

77 56

65645

546

218

812

1012

2110

26

62

1414

3

3

17 1

117139

9 3 1

11 2

211

226

6 2 2

Figure 32: Output puzzle of coffee cup on setting (c).

Page 34

3 34 4 7

7

1

1

22

151

151

131

113

1161

1611

167

761

412

214

44

44

5

5

4

4

33

3 311

11

66 3 34 4 7

7

1

1

22

151

151

131

113

1161

1611

167

761

412

214

44

44

5

5

4

4

33

3 311

11

66

Figure 33: Output puzzle of lamb on setting (c).

612

12621

21

7 7

7720

20

96

6916

16

72

2 713

13

6 7

7 616

16

5 4

4510

10

612

12621

21

7 7

7720

20

96

6916

16

72

2 713

13

6 7

7 616

16

5 4

4510

10

Figure 34: Output puzzle of butterfly on setting (c).

Page 35

254

452

42

24

24741

14742

1172

2171

142

214256

652

94

49415

514

422

224

213

132

222

222

52

25

2 1 2

212

27

7 2

414

414

47

74

254

452

42

24

24741

14742

1172

2171

142

214256

652

94

49415

514

422

224

213

132

222

222

52

25

2 1 2

212

27

7 2

414

414

47

74

Figure 35: Output puzzle of fish on setting (c).

4

427

72

1 6

61

1 3 5

531

42

24

2317

71321132

2311

10

10

731

1 3 7

1111

1111

1251

1521

21

1 2

135731

1 3 7 5 3 11 3 1 2 5 3 1

1352131

2 3 11 5

51132

51332

2 3 3 1 5

5 10 1

11051515

5 1 5 1

420

204254

452

4

427

72

1 6

61

1 3 5

531

42

24

2317

71321132

2311

10

10

731

1 3 7

1111

1111

1251

1521

21

1 2

135731

1 3 7 5 3 11 3 1 2 5 3 1

1352131

2 3 11 5

51132

51332

2 3 3 1 5

5 10 1

11051515

5 1 5 1

420

204254

452

Figure 36: Output puzzle of monitor on setting (c).

Page 36

10

10

8

86

6

7

7

6

6

11

11

1637

7361

7171

1177

4 3 1 1 1

1 1 1 3 41 1 9 4

4 9 1 1

12

12

6

6

8

8

7

7

17 1

1 171 10 4

4 10 1

1 8

81

611

1 1 6

12

12

4

4

66

6616

16

10

10

8

86

6

7

7

6

6

11

11

1637

7361

7171

1177

4 3 1 1 1

1 1 1 3 41 1 9 4

4 9 1 1

12

12

6

6

8

8

7

7

17 1

1 171 10 4

4 10 1

1 8

81

611

1 1 6

12

12

4

4

66

6616

16

Figure 37: Output puzzle of rocket on setting (c).

112

211261

162

14

1455

55

1212 14

41

3153

31533263

3623

8 2

2 82 5 1

1 5 2

3 9 3

393

3343

3 4 3 3

0

05

5

3411

1143

17143

34171

322

2 2 32 2 3 9

9322

112

211261

162

14

1455

55

1212 14

41

3153

31533263

3623

8 2

2 82 5 1

1 5 2

3 9 3

393

3343

3 4 3 3

0

05

5

3411

1143

17143

34171

322

2 2 32 2 3 9

9322

Figure 38: Output puzzle of mask on setting (c).

Page 37

211

1122

2

14

146

64

43

3 4

40

0

9

99

9

4

4 4

4

2

2 2

2

5

55

5

4115

5 1 1 48 1 4

418

33

3 39

9 413

314

37

73

211

1122

2

14

146

64

43

3 4

40

0

9

99

9

4

4 4

4

2

2 2

2

5

55

5

4115

5 1 1 48 1 4

418

33

3 39

9 413

314

37

73

Figure 39: Output puzzle of airplane on setting (c).

12

12

71

17

29

92132

231

2171

1712

179

971

34451

1 5 4 4 31 5 17 3

31751

241

1 4 21 11

111

2857

7582

7202

2207

1 8 19

1981

12581

1 8 5 12

17

71

12

12

81212

21218

21618

81612

217

7 1 27 1 6

617

7611

1 1 6 71 1 1 7

7111

6

61

1

12

12

71

17

29

92132

231

2171

1712

179

971

34451

1 5 4 4 31 5 17 3

31751

241

1 4 21 11

111

2857

7582

7202

2207

1 8 19

1981

12581

1 8 5 12

17

71

12

12

81212

21218

21618

81612

217

7 1 27 1 6

617

7611

1 1 6 71 1 1 7

7111

6

61

1

Figure 40: Output puzzle of flower on setting (c).

Page 38

12

1223

32

4

41

1

11621

12161

1261

1621

1171

171111211

11211

12

1252

25

1 881

131

1 3 1

18

81161

161

91

19115

511

410

104524

425

9

934

43

12 1

112152

2 5 1

12

1223

32

4

41

1

11621

12161

1261

1621

1171

171111211

11211

12

1252

25

1 881

131

1 3 1

18

81161

161

91

19115

511

410

104524

425

9

934

43

12 1

112152

2 5 1

Figure 41: Output puzzle of car on setting (c).

8

85

5

6

6 0

0

2 5 7

75272

2 7

7

71

1

4 2 6

624

24

4 2

3 2 6 1

1623

1123

3 2 1 1

95

5 95 1 5

515

451

1 5 41 5 1 1

1151

8

84 1

14

18

1888

88

117

71191

19

12

2110

10

8

85

5

6

6 0

0

2 5 7

75272

2 7

7

71

1

4 2 6

624

24

4 2

3 2 6 1

1623

1123

3 2 1 1

95

5 95 1 5

515

451

1 5 41 5 1 1

1151

8

84 1

14

18

1888

88

117

71191

19

12

2110

10

Figure 42: Output puzzle of hut on setting (c).

Page 39

16

1693

39

185

518

577

775

4 10

104

414

4 1 4

110

101411

114

12

12 73

3 7

11

11 73

3 7

12

12 73

37

914

14926

26

42254

45224

45294

42954

137

73175

57

636

636106

610

11

11

4 4

5 1 5

5 1 55 8

8 5

16

1693

39

185

518

577

775

4 10

104

414

4 1 4

110

101411

114

12

12 73

3 7

11

11 73

3 7

12

12 73

37

914

14926

26

42254

45224

45294

42954

137

73175

57

636

636106

610

11

11

4 4

5 1 5

5 1 55 8

8 5

Figure 43: Output puzzle of catface on setting (c).

6

6

4

4

5

5

3

3 10

10

4

441

14

2

23

3

17

71

10

10

3

34

4 1 1

11111

1 1 1

223 3

515

515

57

75

7

7 9

9

1

111

1 1

6

611

11

1 1

1 16 1

1 6

4 22 44 4

4 4

6

6

6

66 4

46

9 1

19149

9 4 1

6

6

4

4

5

5

3

3 10

10

4

441

14

2

23

3

17

71

10

10

3

34

4 1 1

11111

1 1 1

223 3

515

515

57

75

7

7 9

9

1

111

1 1

6

611

11

1 1

1 16 1

1 6

4 22 44 4

4 4

6

6

6

66 4

46

9 1

19149

9 4 1

Figure 44: Output puzzle of church on setting (c).

Page 40

15

15

4

4

2

2

121

112

115

151

001

1

3

38

816

16

13 1

113154

4 5 1

2 21

2121152

2 5 11

124

412

446

644

213

132832

238

233

3 233 10 9

9103

91

19114

411

954

459458

854

2 2 5 6

6522

652

2 5 6

4 6 5

564

544

4 4 5

6 15

156

356

6 5 3319

19 35 2 3

3254 16

164774

4 7 7

162

216

2312

1232

1511

1151

11121

12111

15

15

4

4

2

2

121

112

115

151

001

1

3

38

816

16

13 1

113154

4 5 1

2 21

2121152

2 5 11

124

412

446

644

213

132832

238

233

3 233 10 9

9103

91

19114

411

954

459458

854

2 2 5 6

6522

652

2 5 6

4 6 5

564

544

4 4 5

6 15

156

356

6 5 3319

19 35 2 3

3254 16

164774

4 7 7

162

216

2312

1232

1511

1151

11121

12111

Figure 45: Output puzzle of pumpkin on setting (c).

115

151121

112

57

75

74

47

210

10272

27

7 10

107

105

5 10

832

238

218

812

124

4 124 8

84

76

6727

72

34

4341

14

4371

17341724

4271

5521

12551253

3521

6932

2396

88

88

442

2 4 42 4

42

1 7 3

3 7 13 1

1 36 8 6

686

66

6 6

88

88

236

632

321

12313

31

7423

3247

327

723

511

1 1 51 5

51

311

113

13

31

115

151121

112

57

75

74

47

210

10272

27

7 10

107

105

5 10

832

238

218

812

124

4 124 8

84

76

6727

72

34

4341

14

4371

17341724

4271

5521

12551253

3521

6932

2396

88

88

442

2 4 42 4

42

1 7 3

3 7 13 1

1 36 8 6

686

66

6 6

88

88

236

632

321

12313

31

7423

3247

327

723

511

1 1 51 5

51

311

113

13

31

Figure 46: Output puzzle of space on setting (c).

Page 41