I

I

Gestalt Principles Applied To Software EngineeringDiagrams: An Initial Study

Krystle Lemon 1, Edward B. Allen 1, Jeffrey Carver 1, and Gary Bradshaw2

10epartment of Computer Science and Engineering300 Butler Half, Box 9637

Mississippi State, MS 39762+1 662-325-2756

{kdI18, carver}@cse.msstate.edu;edward.allen@computer.org

ABSTRACT

Discovering root-causes of comprehension errors in softwaredesign is important to prevent their presence in software systems.This research synthesizes software engineering and Gestaltprinciples of similarity, proximity, and continuity for the purposeof discovering whether certain visual attributes of diagrams(dashed arrows, severe complexity, etc.) can affect the accuracyand efficiency of understanding correct relationships amongst theentities in the diagram. Twenty-seven subjects viewed diagramsof different types and answered questions about them. Theexperiment tested whether two dependent variables, accuracy andresponse time, were significantly affected by independentvariables, diagram type (simple 1, simple2, complex), Gestaltprinciples (good vs. bad), and forward/backward (question order).The results of this study indicated that the Gestalt principles didaffect the comprehension in the complex diagrams.

KeywordsGestalt principles, diagram comprehension, empirical softwareengineering, software architecture, cognitive science

1. INTRODUCTIONThe long-term goal of this work is to find ways to prevent

errors early in the software development lifecycle that may bedifficult and expensive to correct later. Software engineersroutinely use diagrams that depict component relationships. Suchdiagrams often represent the architecture of a system, modelingthe structure, behavior, relationships, and constraints amongcomponents while ignoring implementation details [3]. Becausesoftware system implementation is based on the softwarearchitecture, misinterpretation of software architecture diagramscould easily result in an incorrect system implementation.Therefore, it is important that software engineers understandsoftware diagrams to prevent errors from propagating to laterphases of the software-engineering life cycle.

With the aid of cognitive science, which seeks to understandthe human mind and how it learns, this work seeks to identifyfactors that can impair comprehension of software-engineeringdiagrams [4]. In cognitive science, Gestalt principles ofperceptual organization deal with features that combine to formoverall perception, such as the relation of figure to ground andrelationships among visual features [1]. Cognitive-scienceresearch results in areas such as visual search and tracing mayhelp uncover root-cause errors in comprehension of diagrams.

20epartment of PsychologyBox 6161

Mississippi State, MS 39762+1 662-325-0550

glb2@ra.msstate.eduI~~.i

This paper presents the results of a study coupling cognitivescience and software engineering to investigate whether certaindiagram characteristics affect comprehension by softwareengineers.

This paper is organized as follows. Section 2 provides anoverview of diagram comprehension and related work. Section 3describes the study. Results, discussion, and threats to validity arepresented in Section 4. Lastly, Section 5 discusses the conclusionsand future work.

2. DIAGRAMCOMPREHENSIONSoftware-architecture diagrams often show the flow of

information between components in a software system. Suchdiagrams consist of lines with arrowheads pointing in thedirection of information flow, boxes representing systemcomponents, and annotations. Software engineers mustcomprehend all of this information and process it within theirmental workspace [3]. Hungerford, et al. showed that diagramsimpose less cognitive load compared to text [2]. Not surprisingly,software engineers routinely make extensive use of a variety ofdiagrams.

The Gestalt principles address why individuals can perceivewhole elements out of incomplete elements. They address howobjects are viewed in relation to figure and ground, similarity,proximity, continuity, closure, area, and symmetry [I].

This study focuses on the Gestalt principles of similarity,proximity, and continuity. The principle of similarity states thatobjects with similar characteristics belong together. The principleof proximity states that objects that are close together areperceived as belonging together. Finally, the principle ofcontinuity states that continuous figures are perceived more oftenthan non-continuous figures. An example of a continuous figure iswhen a line is perceived to pass through an object instead ofviewing it as two separate lines on the object: one entering andanother leaving. Therefore, some of our diagrams were drawnwith these principles in mind, with solid lines with similar boxesgrouped together, and with minimum spacing between them. Thisinitial study did not try to separate out the effects of each of theseprinciples; rather they were all taken together. This studyinvestigated two research questions, in the context of diagramcomprehension.

Hypothesis 1: Diagrams that follow Gestalt principles ofsimilarity. proximity. and continuity offer better accuracythan diagrams that do not.

5th ACM-IEEE International Symposium on Empirical Software Engineering - Vol II: Short Papers and Posters 48

Hypothesis 2: Diagrams that follow Gestalt principles ofsimilarity, proximity, and continuity afferfaster response timethan diagrams that do not.

3. THE STUDYThe study was conducted at Mississippi State University

(MSU) in the Fall of 2005, using the Adobe Authorware softwareto administer the experiment The 27 subjects came trom theSoftware Architecture course (IS subjects) and the Introduction toSoftware Engineering course (12 subjects). These subjects wereupper-level undergraduates and graduate students. The softwarearchitecture course covers basic software architecture concepts, aswell as, creation and use of architecture diagrams. TheIntroduction to Software Engineering focuses on softwareengineering processes, including the diagrams that are usedduring those processes.

To simulate real-world diagrams, three diagrams, created bysoftware-engineering students as part of a homework exerciseduring a previous semester of the Software Architecture course,were selected and modified. Each diagram contained named

boxes (system components) and lines with arrowheads(representing information flow). The three diagrams were labeledsimple], simple2, and camplex, where simple] and simple2diagrams where similar in number of boxes and lines in thediagrams and the complex diagram had approximately three timesas many lines and twice as many boxes in the diagram. For eachof the three diagrams, two topologically equivalent versions wereproduced. The gaod version of the diagram was created using the

.Gestalt principles (described in Section 2), where grouping,proximity, and continuity where adhered to in the drawing. Thehad version of each diagram was simply the original diagram withthe same box names and connections as the good version withoutconsideration of the Gestalt principles. Figures I and 2 show thegood and bad versions of the complex diagrams.

Each subject viewed simple I diagram followed by simple2diagram followed by complex diagram. Subjects were given 20

~-' I~'=~~

I

_._~ i L~"";"'1 _.~_.--

]--, :;:;:...J.-~- J

, ~ r-: .~_ll- "'"l_~~__r-L? ~~.,.!~-1' ..~--

§. ~>~",::::-_.,.",-.",=~:J

-::E-'~ Il

! '-. .

"-;-1'-"" ..,,/' -'- / ..,.~..~-- ~ i

.

;,.:::;:::.,~.;- - i--~ ':.:- '. -,-_J JI,-'" l' ;::rJ ,'.1:;---}::r--:l I 11

-- L-~-J~~~~J~JIFigure 1: Complex diagram good version

questions per diagram. All students answered the same questions.However, to assess the potential effect of question order, someanswered questions I through 60 (forward) (I through 20 for eachdiagram) and some answered questions 60 through I (backward)(20 through I for each diagram). The questions asked the subjectsto determine whether messages were sent between objects in thediagram (i.e. whether there was a line connecting the boxes). Anexample question is "Does Location Identity send messages toTowing Garage?" The subjects were split into four balancedgroups as shown in Table 1.

" ."" .." " --_m- - ,.- _m "m',',',-'

Figure 2: Complex diagram bad version

The subjects were given as much time as needed to answer

the questions. At their own pace, they proceeded to answer thequestions one by one while the associated diagram was stillvisible. They were not allowed to skip any questions.

4. RESULTSand DISCUSSIONThis section describes the statistical analysis performed on the

quantitative data in order to determine whether a significantdifference in diagram comprehension exists between differenttypes of diagrams. As this was an initial study, an alpha value of0.1 was chosen for significance tests.

4.1 Summary of Quantitative DataThis study had two dependent variables and three independent

variables. The dependent variables investigated in this study werequestion accuracy and response time. The independent variableswere: Gestalt principles used (good vs. bad), type af diagram(simple], simple2, or complex), and question arder (forward orbackward). The question order was used as a control variable todetermine whether the presentation order of questions had anyeffect on subject response.

The goal of the statistical analysis was to determine if any ofthe independent variables had a significant effect on either of thedependent variables. To perform an outlier analysis, we made ascatter plot with the subjects' response time on one axis and thenumber of correct responses on the other axis. A visual inspectionof this chart showed that the accuracy for one subject was morethan two standard deviation below the mean. Therefore, thissubject's data was removed as an outlier and was not included inany further analysis.

For the dependent variable question accuracy. most subjectswere very accurate with low variability across all diagrams,Further analysis of this variable was unlikely to show interestingresults. Therefore, the remainder of this section focuses on therespanse time variable,

51hACM-IEEE International Symposium on Empirical Software Engineering - Vol II: Short Papers and Posters 49

Table 1: Experiment Design

Group Simple I Simple2 Complex FwdlBackwd

I Good Bad Good Forward

2 Good Bad Good Backward

3 Bad Good Bad Forward

4 Bad Good Bad Backward

Three separate ANOVA tests were run for the simplel,simple2, and complex diagrams to isolate the effects of theindependent variables for each diagram type. The ANOV A testfor response time for simplel diagrams was not significant. TheANOVA test did show an interaction between the variables

forwardlbackward and goodlbad with F (27.1)=5.896 and p=0.023.Further analysis was done using independent sets (-test, isolatingthe effects of the forward/backward variable and the good/badvariable. The forwardlbackward variable was not significant. Thegoodlbad variable was not significant alone.

The simple2 diagram ANOV A statistical tests yieldedinsignificant results for all variables and further I-tests confinnedthis. The ANOV A tests used both the accuracy and response timeas dependent variables to model the significance of the diagramtype variable, goodlbad variable, and the forward/backwardcontrol variable. A similar analysis was completed for thecomplex diagrams and the results showed that the goodlbadvariable was very significant where F (27, I) = 0.00 I and p=O.004.

4.2 Discussion of ResultsThe results obtained suggest that the range of complexity

among the diagrams presented in the experiment was notsufficient to obtain a useful variation in accuracy results.Consequently, Hypothesis I remains unevaluated.

Figure 3 depicts the distribution of response times. Thestatistical tests showed that the Gestalt principles for the complexdiagram significantly affected the response time variable.Therefore, we conclude that the comprehension time of gooddiagrams was faster than that of bad diagrams providing supportfor Hypothesis 2. This conclusion encourages futureexperimentation to detennine which Gestalt principles can beapplied in software engineering to make more comprehensiblediagrams and what features should be avoided.

4.3 Threats to ValidityBy using two different question orderings a potential threat

caused by the ordering of the questions was assessed. Theanalysis showed that question order had no effect on eitheraccuracy or time. By balancing the presentation of good and baddiagrams, the influence of individual abilities was balanced.

Although we had only three pairs of diagrams, decreased

350000-1

JOOOOO-II

Good/Bad

or 0 Good

. 2SOOOO

I

§\! Bad

~;!'- I

I !Simpl..1 Sim~I..2 I jComplex

Type.fExperiment

Figure 3: Distribution of response time in milliseconds

,

response times from simple] to simple2 diagrams suggest that alearning effect occurred between the subjects viewing the first andsecond diagrams. This threat to validity was not addressed. Theincrease of time from the simple2 to complex diagram couldcorrespond to subjects having a higher mental workload becausethe complexity increases, as expected. This study did not evaluatethe effect of complexity. Due to the fact that the subjects werestudents, a threat to external validity is present. However, thestudents' tasks were similar to professional subjects' tasks. .

5. CONCLUSIONSThis initial experiment investigated the basic question of

whether certain factors affect diagram comprehension. The resultsof this study showed that the Gestalt principles did affect thecomprehension in the complex diagram. Gestalt principles ofperceptual organization show promise in easing the task ofcomprehending software-engineering diagrams. This researchaims at helping software engineers to detennine what type ofartistic approaches to consider when designing the softwarearchitecture diagrams to help avoid errors in the later stages of thesoftware-engineering lifecycle.

Future work will include studies that analyze various types ofdiagram features and the effect of each Gestalt principle.

J

f

6. ACKNOWLEDGMENTSOur thanks go to Ginger Cross, student subjects, MSU ESE

research group, Byron Williams, and F. Chevonne Thomas. This

work was supported in part by NSF grant CCR-0132673.

7. REFERENCES[I] J. A. Anderson, Cognitive Psychology and Its Implications,

W. H. Freeman and Company, New York, New York, 1990,66-68.

[2] B. C. Hungerford, A. R. Hevner, and R. W. Collins,"Reviewing Software Diagrams: A Cognitive Study," IEEETransactions on Software Engineering, vol. 30, no. 2, Feb2004, 84-95.

[3] T. Klemola and J. Rilling, "Modeling Comprehension

Processes in Software Development," Proceedings: First

IEEE International Conference on Cognitive Informatics,

Aug 2002, 329-336.

[4] 1. Reason, Human Error, Cambridge University Press,

Cambridge, United Kingdom, 1990.

[5] P. Thagard, Mind: Introduction to Cognitive Science, MIT

Press, Cambridge, Massachusetts, 2005.

5th ACM-IEEE International Symposium on Empirical Software Engineering - Vol II: Short Papers and Posters 50