Embedding Critics in Design EnvironmentsGerhard Fischer 1

Kumiyo Nakakoji 1 ,3

Jonathan Ostwald 1,4Gerry Stahl 1,2

Tamara Sumner 1

University of ColoradoBoulder, Colorado 80309-0430, USA

e-mail: gerhard@cs.colorado.edu

2) School of Environmental DesignUniversity of Colorado

Boulder, Colorado 80309, USA

3) Software Engineering LaboratorySoftware Research Associates, Inc.

1-1-1 Hirakawa-cho, Chiyoda-ku, Tokyo 102, Japan

4) Nynex Science and Technology CenterWhite Plains, New York, USA

AcknowledgmentsThe authors thank the members of the Human-Computer Communication group at the University ofColorado, who contributed to the conceptual framework and the systems discussed in this paper. Theresearch was supported by the National Science Foundation under grants No. IRI-8722792 and IRI-9015441, by the Colorado Advanced Software Institute, by US WEST Advanced Technologies, by theNYNEX Science and Technology Center, and by Software Research Associates, Inc. (Tokyo, Japan). Weespecially wish to thank Barbara Gibson at Kitchen Connection in Boulder, Colorado, for sharing herexpertise in kitchen design.

2

Acknowledgments______________________________________________________________1Abstract______________________________________________________________________31. Introduction ______________________________________________________________32. The Critiquing Approach____________________________________________________3

2.1. Importance of Human Critiquing __________________________________________42.2. Applying Computer-Based Critiquing to Design_______________________________5

3. Embedding Critics in Integrated Design Environments____________________________73.1. Analyses of Early Critiquing Systems _______________________________________83.2. HYDRA: A Multifaceted Architecture for Design Environments _________________103.3. Scenario Illustrating Generic, Specific, and Interpretive Critics_____________________13

4. Three Embedded Critiquing Mechanisms______________________________________144.1. Generic Critics________________________________________________________164.2. Specific Critics ________________________________________________________184.3. Interpretive Critics_____________________________________________________20

5. Benefits of Embedding: Increasing the Shared Context __________________________225.1. Increasing the Designer’s Understanding of Design Situations _________________________________ 23

5.2. Pointing Out Significant Design Situations _____________________________________235.3. Locating Relevant Information In Large Information Spaces ______________________24

6. The Dynamic Nature of Critiquing Knowledge__________________________________266.1 Supporting Designers in Adapting the Critiquing System __________________________266.2 “Seeding” the Critiquing System with Domain Knowledge_________________________266.3 Accumulating Design Knowledge Through Critics _______________________________26

7. Conclusions _____________________________________________________________278. References_______________________________________________________________28

3

AbstractHuman understanding in design evolves through a process of critiquing existing knowledge andconsequently expanding the store of design knowledge. Critiquing is a dialog in which the interjection of areasoned opinion about a product or action triggers further reflection on or changes to the artifact beingdesigned. Our work has focused on applying this successful human critiquing paradigm to human-computer interaction. We argue that computer-based critiquing systems are most effective when they areembedded in domain-oriented design environments, which are knowledge-based computer systems thatsupport designers in specifying a problem and constructing a solution. Embedded critics play a number ofimportant roles in such design environments: (1) they increase the designer’s understanding of designsituations by pointing out problematic situations early in the design process, (2) they support theintegration of problem framing and problem solving by providing a linkage between the designspecification and the design construction, and (3) they help designers access relevant information in thelarge information spaces provided by the design environment. Three embedded critiquing mechanisms –generic, specific, and interpretive critics – are presented, and their complementary roles within the designenvironment architecture are described.

Keywords: Cooperative Problem-Solving Systems, Critics, Critiquing, Design, Design Analysis, DesignEnvironments, Design Rationale, Explanation, Generic Critics, Specific Critics, Specification, InterpretiveCritics, Domain-orientation

1. IntroductionHuman understanding in design evolves through a process of critiquing [Fischer, 1991 #74] existingknowledge and consequently expanding and refining the state of knowledge. Our work has focused onapplying this human critiquing paradigm to human-computer interaction. Our experience with thisapproach is based on several years of system prototyping, the integration of cognitive and designtheories, and empirical evaluation of these systems. Based on these experiences, we conclude thatcomputational critiquing systems are most effective at supporting human designers when embedded indomain-oriented design environments [Fischer, 1992 #238].

In Section 2, we explain why the critiquing paradigm is essential for supporting the complex activity ofdesign. Using illustrations from critiquing systems we have built, we demonstrate in Section 3 howembedding in design environments enhances the computational critiquing process. Examples of ourembedded critiquing system are drawn from HYDRA-KITCHEN, a residential kitchen design environmentwe have built. Section 4 explains three embedded critiquing mechanisms we have designed,implemented, and studied, called generic, specific, and interpretive critics. Finally, in Section 5 we assesssome of the benefits of these embedded critiquing mechanisms.

2. The Critiquing ApproachCritiquing is a dialog in which the interjection of a reasoned opinion about a product or action triggersfurther reflection on or changes to the artifact being designed. For example, a kitchen designer might

4

critique a kitchen floor plan in terms of building code violations, efficiency, safety concerns, or eventualresale value. An agent – human or machine – capable of critiquing in this sense is a critic. Computer-basedcritics are made up of sets of rules or procedures for evaluating different aspects of a product; sometimeseach individual rule or procedure is referred to as a critic [Fischer, 1991 #74].

2.1. Importance of Human CritiquingHuman critiquing plays an important role in design both in the growth of human knowledge and in terms oferror elimination. By “human critiquing” we mean subjecting our designs and products to the scrutiny ofother people, be they peers, domain specialists, or society in general.

Complex design activities prohibit an individual from knowing everything that is relevant; in addition,expertise is frequently controversial. Complex design situations can therefore be characterized by a“symmetry of ignorance’’ [Rittel, 1984 #243], and the knowledge needed to solve a design problem isdistributed among designers and their clients [Rittel, 1984 #71]. Critiquing is an important method forworking within such a framework of distributed knowledge because it fosters a maximum of participation inorder to activate as much of the distributed design knowledge as possible. In kitchen design, the designerand the homeowner take turns proposing ideas and criticizing each other’s suggestions. In this way, theoften tacit knowledge [Polanyi, 1966 #208] that each party has can come into play and complement theother’s partial grasp of the design problem.

Critiquing is ubiquitous. It is, for example, at the heart of the scientific method. Popper [Popper, 1965#62] theorized that science advances through a cycle of conjectures and refutations. Scientists formulatehypotheses and put forth these conjectures for scrutiny and refutation by the scientific community.Besides contributing to the growth of knowledge, this critiquing cycle of conjectures and refutations isessential for creating a shared understanding within the scientific community and providing a stable basefor future growth in scientific knowledge.

Critics play an important role in making designers aware of breakdown situations [Fischer, 1993 #239].Petroski [Petroski, 1985 #63] noted the importance of failure in the growth of engineering knowledge.For instance, when an airplane crashes, the Federal Aviation Administration sends a team of specialists tothe site to determine the cause of the accident. In essence, these specialists are critiquing the plane’sdesign and construction and current aviation practices. Over the years, this practice has contributed muchto the growth of aviation knowledge in terms of both airplane design and improved safety regulations[Chambers, 1985 #237]. In turn, this growth in knowledge contributes toward future error elimination; thatis, planes with the same defect are repaired and aviation regulations are improved to prevent similarcrashes.

The activity of critiquing plays an important role in engineering, science, and design in general. It producesmany benefits, including the growth of knowledge, error elimination, and the promotion of mutualunderstanding by all participants. Through the critiquing process, designers gain a better understandingof the design problem by hearing the different points of view of other design participants. In our work, wehave taken this successful human critiquing paradigm and shown how it can be effectively applied to

5

enhance human-computer interaction. In the remainder of this paper, the term “critiquing” will refer tocomputer-based critiquing systems.

2.2. Applying Computer-Based Critiquing to DesignOur design environments are cooperative problem-solving systems [Fischer, 1990 #14] in which thecomputer system helps users design solutions themselves as opposed to having an expert systemdesign solutions for them. As illustrated in Figure 1, critiquing is integral to cooperative problem-solvingsystems. The core task of critics is to recognize and communicate debatable issues concerning a product.Critics point out problematic situations that might otherwise remain unnoticed. Many critics also adviseusers on how to improve the product and explain their reasoning. Critics thus help designers avoidproblems and learn different views and opinions. Critiquing systems augment the ability of humandesigners to evaluate their solutions; decisions concerning whether or not to follow the critic suggestionsare left up to the designers.

DW

Design a kitchen fora left-handed cook

Solution

Problem

Critic Rule: The dishwasher should be on the left sideof the sink if the cook is left-handed.

ComputerCritic

CreateModify

Analyze

CritiqueReflect

Figure 1. A cooperative problem-solving system has two agents – a human designer anda computer-based critic. Both agents contribute what they know about the domain tosolving some problem. For the critiquing systems discussed in this paper, the human’sprimary role is to generate and modify solutions; the computer’s role is to analyze thesesolutions and produce a critique for the human to consider in the next iteration of thisprocess.

Critiquing systems are well suited for design tasks in complex problem domains in which the traditionalexpert systems or automated design approaches have proven inadequate. Such design tasks have thefollowing characteristics: (a) knowledge about the design domain is incomplete and evolving , (b) theproblem requirements can be specified only partially, and (c) necessary design knowledge is distributedamong many design participants.

6

a. Knowledge about the design domain is incomplete and evolving. Some domains, such as userinterface design [Lemke, 1990 #91] and lunar habitat design [Stahl, 1993 #171], are not sufficientlyunderstood; that is, creating a complete set of principles that exhaustively captures their domainknowledge is impossible. Complex problem domains are continually changing as new design knowledgeis gained and old design knowledge becomes obsolete. For example, user interface design principleshave certainly changed to accommodate the shift from primarily character-based user interfaces tosophisticated graphical user interfaces. Any system supporting design in complex domains must be ableto evolve with the domain.

Expert systems and automated design approaches are infeasible in these complex situations in which allthe potential relevant background knowledge cannot be articulated [Winograd, 1986 #213]. Becauseautonomous expert systems leave the human out of the decision process and all “intelligent” decisionsare made by the computer, these systems require a priori a comprehensive knowledge base covering allaspects of the tasks being performed. Most expert systems also fail to adequately support the evolution ofdomain knowledge. First, expert systems typically do not support the addition of knowledge by domainexperts and instead rely on knowledge engineers to acquire this knowledge from domain experts andsubsequently codify it for the specific system. Second, expert systems have shown themselves to bebrittle [Rittel, 1984 #71]; that is, a small shift in the problem domain can render an expert system’sknowledge base obsolete and inoperative [Buchanan, 1984 #224].

An important aspect of embedded critiquing systems is their incremental nature; they do not need a largeor comprehensive rule-base to be effective. Because critics are structured to be independent entities,adding or modifying a critic does not affect the behavior of the remaining critics. Parts of the critiquingsystem can remain operational and continue to support the design process while other parts undergoevolutionary change. In the HYDRA-KITCHEN system we have prototyped a “generic” critiquing mechanismthat is knowledgeable about commonly accepted design principles and standard design practices. Theseprinciples are found in textbooks and training programs and are recognized by professional kitchendesigners as being important aspects of producing a “good” floor plan. Although this general knowledgebase is insufficient for automating the design of kitchen floor plans or for making a detailed analysis of theappropriateness of the design for a particular client, the generic critiquing system provides designers withvaluable feedback concerning their floor plan designs. One study involving both amateur and expertkitchen designers showed that HYDRA’S generic critics helped both categories of designers even thoughits rule-base contained only 24 critic rules [Fischer, 1989 #152].

b. The problem requirements can be specified only partially. Design problems are ill-defined: they cannotbe precisely specified before attempting a solution [Rittel, 1984 #71]. Problem specifications reflect thedesigner’s understanding of the problem framing and the problem solution. Researchers in situatedcognition [Lave, 1988 #95] and design [Schoen, 1983 #129] have shown that designers arrive atsolutions by iteratively reframing the problem – adjusting and refining their understanding of the problemframing and problem solution to reflect decisions made, means that may be chosen, materials available,and other changes in the context. Thus, problem specifications are not only incomplete, they are alsodynamic in nature.

7

The expert system approach is based on the assumption that the problem to be solved can be fullyarticulated to the system a priori. The system can return a solution only if given a complete and accurateproblem specification. Furthermore, changes in the problem specification can completely invalidate theexpert system’s proposed solution. Thus, expert systems are inadequate in ill-defined domains with partialand evolving problem specifications.

We have constructed a critiquing mechanism that supports design as a process of problem reframing. This“specific” critiquing mechanism enables only those critics pertinent to the current partial specification, andas such embodies domain knowledge concerning situation-specific design characteristics that not everydesign will share. In kitchen design, professional designers elicit this situation-specific knowledge fromtheir customers using predefined questionnaires; the answers to these questionnaires form part of thekitchen specification. In HYDRA-KITCHEN, as the designer changes the problem specification, the“specific” critiquing mechanism brings different sets of critics to bear upon the design. This mechanismsupports the coevolution of problem framing and problem solving by making explicit the relationshipbetween the partial problem specification and the current design solution.

c. Necessary design knowledge is distributed among many design participants. Design domains such asnetwork design are so large and complicated and have so many subdomains that no single person canknow all there is to know [Fischer, 1991 #72]. In such complex domains, the necessary design knowledgeis distributed among many participants and most design work is done by teams whose members havediffering areas of expertise [Hackman, 1974 #198]; [Johansen, 1988 #202]. When designing in ill-defineddomains, there are no “optimal” solutions [Simon, 1981 #4]. Conflicts in opinion about how to proceedoften arise due to differences in the designers’ areas of expertise, their personal styles, and their particularproblem framing. Often, such conflicts are resolved and design proceeds after designers presentreasoned arguments supporting their opinions for discussion and negotiation.

Our critiquing systems support design as a deliberative and interpretive process. Critiquing systemscontain a collection of critics that embody different areas of domain expertise, different design styles, andoften diverging opinions. Our “interpretive” critiquing mechanism supports designers with varyinginterests and differing areas of expertise to work together by allowing design knowledge to be definedand bundled into personal or topical groupings. Using this mechanism, designers can examine theirdesign from many different perspectives in which each perspective brings different design knowledgeand critics to bear upon the current design.

All of our critiquing mechanisms – generic, specific, and interpretive – support design as a deliberativeprocess. Besides simply pointing out a potential flaw in the design, these critics offer a reasoned opinionas to why their suggestion should or should not be followed. This interaction style typifies cooperativeproblem-solving systems: it is the role of the critiquing system to bring relevant design knowledge to thedesigner’s attention; it is the role of the designer to evaluate the trade-offs and make the final decisions.

3. Embedding Critics in Integrated Design EnvironmentsOur early research focused on building and evaluating general purpose (i.e., not-domain-oriented)critiquing mechanisms [Fischer, 1991 #74]. During later work, we became interested in building domain-

8

oriented design environments [Fischer, 1992 #238]. In the last few years, we have merged these tworesearch interests by embedding critiquing mechanisms into domain-oriented design environments. Thisembedding enhances both the richness of the critiquing process and the ability of our designenvironments to support the complex activity of design. This section discusses early critiquing systemswe have built and how they contributed to the development of the multifaceted architecture, HYDRA, fordesign environments. A scenario using HYDRA-KITCHEN illustrates how the embedded critiquingmechanisms integrate the various components in the design environment.

3.1. Analyses of Early Critiquing SystemsCritical analyses of our early stand-alone critiquing systems [Fischer, 1991 #74] and systems built byothers [Burton, 1982 #214]; [Silverman, 1992 #211], combined with empirical evaluations, led us torealize that the challenge in building critiquing systems is not simply to provide feedback: the challenge isto say the right thing at the right time. Our analyses identified several shortcomings in early critiquingsystems that hindered their ability to say the “right” thing at the “right” time:

a. lack of domain orientation;b. insufficient facilities for justifying critic suggestions;c. lack of an explicit representation of the user’s goals;d. no support for different individual perspectives;e. timing problems with critic intervention strategies.

a. Lack of domain orientation. LISP-CRITIC [Fischer, 1987 #146] allows programmers to requestsuggestions on how to improve their code. The system proposes transformations that make the codemore cognitively efficient (i.e., easier to read and maintain) or more machine efficient (i.e., faster orsmaller). However, the lack of domain orientation limits the depth of critical analysis the critiquing systemcan provide. Without domain knowledge, critic rules cannot be tied to higher level concepts; LISP-CRITICcan answer questions such as whether the Lisp code can be written more efficiently, but it cannot assist auser in deciding whether the code can solve a specific problem.

b. Insufficient facility for justifying critic suggestions. FRAMER [Lemke, 1990 #91] enables designers todevelop window-based user interfaces on Symbolics Lisp machines. FRAMER’s knowledge base containsdesign rules for evaluating the completeness and syntactic correctness of the design as well as itsconsistency with interface style guidelines. Evaluations of FRAMER showed (1) that many users did notunderstand the consequences of following the critic’s advice or why the advice was beneficial to solvingtheir problem, and (2) that when users do not understand why a suggestion is made, they tend to blindlyfollow the critic’s advice whether or not it is appropriate to their situation. FRAMER provided shortexplanations to address this problem. However, in design there are not always simple answers; access toargumentative discussions detailing the pros and cons of a particular suggestion are necessary [Rittel,1984 #71].

c. Lack of an explicit representation of the user’s goals. JANUS [Fischer, 1989 #152] is a step towardaddressing the previous shortcomings. JANUS allows designers to construct kitchen architectural floorplans. It contains two integrated subsystems: a domain-oriented kitchen construction kit and an issue-based hypermedia system containing design rationale. Critics respond to problems in the construction

9

situation by displaying a message and providing access to appropriate rationale in the hypermedia system.However, these critics often give spurious or irrelevant advice resulting from the lack of an explicitrepresentation of the user’s task. The only task goal built into JANUS is one of building a “good” kitchen;that is, a kitchen that conforms to commonly accepted standards and design practices. With an explicitmodel of the designer’s intentions for a particular design, critics can be selectively enabled based on thismodel and provide less intrusive and more relevant advice.

d. No support for different individual perspectives. It is not possible to anticipate all the knowledgenecessary for a critiquing system to say the “right” thing in every design situation. Design domains arecontinually evolving as new knowledge is gained. JANUS-MODIFIER [Fischer, 1990 #240] was developedto respond to this problem by making the domain knowledge (including critics) end-user modifiable. Butbeing able to add new knowledge is not sufficient; different users must be able to organize and managedesign knowledge and critics to reflect their perspectives on design. Design environments need tosupport interpretation of a problem from many perspectives (technical, structural, functional, aesthetic,personal), and critique accordingly.

e. Timing problems with critic intervention strategies. A number of systems [Fischer, 1985 #144]; [Burton,1982 #214] investigated critic intervention strategies, which determine when and how a critic shouldsignal a potential problem. This research focused on studying active versus passive interventionstrategies. Active critics continually monitor user actions and make suggestions as soon as a problematicsituation is detected. Passive critics are explicitly invoked by users to evaluate their partial design.

A protocol analysis study [Lemke, 1990 #91] showed that passive critics were often not activated earlyenough in the design process to prevent designers from pursuing solutions known to be suboptimal.Often, subjects invoked the passive critiquing system only after they thought they had completed thedesign. By this time, the effort of repairing the situation was expensive. In a subsequent study using thesame design environment, an active critiquing strategy was shown to be more effective by detectingproblematic situations early in the design process.

However, our interactions with professional designers showed that active critics are not a perfect solutioneither: they can disrupt the designer’s concentration on the task at the wrong time and interfere withcreative processes. Interruption becomes even more intrusive if the critics signal breakdowns at a differentlevel of abstraction compared to the level in which the task users are currently engaged. For example, ifthe designer is currently concerned about where the refrigerator should be located in a kitchen floor plan,then a critic suggestion that a double-bowl sink is better than a single-bowl sink is probably inappropriateand distracting at this point in time.

What is needed is a critiquing system that: (1) alerts designers to problematic solutions, (2) avoidsunnecessary disruptions, and (3) allows users to control the critic’s intervention strategy. Embeddingcritics in design environments allows users to control critic intervention through interaction with theconstruction, specification, and perspective design components built into the design environment.

10

3.2. HYDRA: A Multifaceted Architecture for Design EnvironmentsDesign environments are computer programs that support designers in concurrently specifying a problemand constructing a solution. Design environments provide information repositories to store domainknowledge and allow designers to accumulate additional domain-knowledge through interaction with theenvironment.

HYDRA (Figure 2 represents its components schematically; Figure 3 provides a screen image) containsdesign creation tools in the form of a construction component and a specification component. Designinformation repositories are provided in the form of argumentation and catalog knowledge bases. Thearchitecture is multifaceted because these components provide multiple representations of both thecurrent design and underlying domain knowledge. The critiquing mechanisms integrate these facets inthe design environment architecture. The various representations are managed by the following fourcomponents:

— The construction component is the principal medium for modeling a design. It provides apalette of domain-oriented design units, which can be arranged in a work area using directmanipulation. Design units represent primitive elements in the construction of a design,such as sinks and stoves in the domain of kitchen design. Critics can be tied to thesedomain-oriented design units and to relationships between design units.

— The specification component allows designers to describe abstract characteristics of thedesign they have in mind. The specifications are expected to be modified and augmentedduring the design process, rather than to be fully articulated at the beginning. Thespecification provides the system with an explicit representation of the user’s goals. Thisinformation can be used to tailor both the critic suggestions put forth and the accompanyingexplanations to the user’s task at hand.

— The argumentative hypermedia component contains design rationale based on theprocedural hierarchy of issues (PHI) structure (see Figure 5) [McCall, 1987 #209]; [Conklin,1988 #153]. The PHI structure consists of issues, answers, and arguments about decisionsmade during the course of design. Users can annotate and add argumentation as it emergesduring the design process. Argumentation is a valuable component in a critic’s explanation; itidentifies the pros and cons of following a critic suggestion and helps the user to understandthe consequences of following a suggestion.

— The catalog component provides a collection of previously constructed designs. Theseillustrate examples within the space of possible designs in the domain and support reuse[Prieto-Diaz, 1987 #226] and case-based reasoning [Kolodner, 1991 #225]. Catalog entriesare also important components in a critic’s explanation. Often a critic does not suggest acourse of action but instead points out a deficiency in the current design; catalog entries canthen be used as specific examples illustrating sample solutions that address a deficiencynoted by a critic.

11

Specification Component

ConstructionComponent

Catalog Component

ArgumentationComponent

critic messages

argumentation

examples

Construction Analyzer

Argumentation Illustrator

Figure 2. The critiquing process within HYDRA. The links between the components– the CONSTRUCTION ANALYZER and the ARGUMENTATION ILLUSTRATOR – arecrucial for exploiting the synergy of the integration.

This architecture derives its power from the integration of its components. When used in combination,each component augments the value of the others in a synergistic manner. The components of thearchitecture are integrated by two linking mechanisms (see Figure 2). Together, these linking mechanismssupport the critiquing process by providing critic messages, explanatory argumentation, and illustrativeexamples:

— The CONSTRUCTION ANALYZER is the core critiquing component in HYDRA. This mechanismanalyzes the design construction for compliance with the currently enabled set of critic rules.When a lack of compliance is detected, the critic signals a breakdown and provides entry intothe exact place in the argumentative hypermedia component in which the appropriateexplanation is located.

— The ARGUMENTATION ILLUSTRATOR can retrieve both positive and negative catalog examplesto illustrate the problematic situation detected by the CONSTRUCTION ANALYZER. Providingspecific examples is essential because the explanation given in the form of argumentation isoften highly abstract and conceptual. Concrete design examples that match this explanationassist designers in understanding the potential problem, assessing the design situation,and devising a solution.

In addition to the construction and argumentation components of its predecessor, JANUS, HYDRAsupports a specification component [Fischer, 1991 #77] and a catalog of designs. The specificationformat is based on questionnaires used by professional kitchen designers to elicit their customers’requirements, such as the kitchen owner’s cooking habits and family size. Each component in HYDRAcontains design knowledge that can be used by an embedded critiquing mechanism to overcome thedeficiencies of the stand-alone systems previously described.

12

As mentioned in Section 2.2, we have studied three classes of embedded critiquing mechanisms:generic, specific, and interpretive critics. These mechanisms embody different types of designknowledge and correspond to three dimensions of embedding. Generic critics are embedded in theconstruction and use domain knowledge concerning desirable spatial relationships between design unitsto detect problematic situations in the partial design construction. Specific critics are embedded in thepartial specification and take advantage of additional knowledge in the partial specification to detectinconsistencies between the design construction and the design specification. Interpretive critics areembedded in a perspective mechanism that enables designers to create topical groupings of critics anddesign knowledge; such groupings support designers in examining their artifacts from differentviewpoints. The argumentation and catalog components provide rich sources of domain knowledge thatall three mechanisms use in their explanation process when communicating with the designer.

The following section provides a scenario depicting how kitchen designers work within the HYDRAenvironment. The scenario describes the three critiquing mechanisms and it illustrates the benefitsderived from embedding these mechanisms in the multifaceted architecture.

13

3.3. Scenario Illustrating Generic, Specific, and Interpretive CriticsImagine that Bob, a professional kitchen designer, has been asked to design a kitchen for the Smithfamily. The partial specification of the Smith’s kitchen is articulated using HYDRA, as shown in Figure 3.

Bob begins working on a floor plan in the construction area. He moves the dishwasher next to the cabinet.Bob’s action triggers a generic critic, and the message, “The dishwasher is too far from the sink,” isdisplayed. Generic critics reflect knowledge that applies to all designs, such as accepted standards,

Figure 3. Screen image of HYDRA-KITCHEN. The “Current Specification” window shows asummary of currently selected answers using the specification component. An indicatorattached to each of the selected answers allows users to assign weights of importance tothe specified item in order to set priorities. The “Catalog” window shows previous kitchendesigns that can be examined or reused. The “Current Construction” window shows apartial construction being built using components provided in a palette of kitchen designunits (not shown). The “Messages” window is used to present critic notificationmessages. The number attached to the critic message is a weighted measure indicatingthe relevance of the fired critic.

14

building codes, and domain knowledge based on physical principles. Often, this generic knowledge canbe found in textbooks, training curricula, or by interviewing domain practitioners. Bob highlights the critic’smessage and elects to see its associated argumentation. The argumentation explains that plumbingguidelines require the dishwasher to be within one meter of the sink. Bob follows the critic’s suggestionand moves the dishwasher next to the right side of the sink (for details, see Fischer, et al. [Fischer, 1991#74]).

This action triggers a specific critic with the rule, “If you are left-handed, the dishwasher should be on theleft side of the sink.” Specific critics reflect design knowledge that is tied to situation-specific physicalcharacteristics and domain-specific concepts that not every design will share. These critics areconstructed dynamically from the partial specification to reflect current design goals. This particular criticrule was activated because Bob specified that the primary cook is left-handed (see Figure 3). Bobexamines the supporting argumentation, “Having the dishwasher to the left of the sink creates an efficientwork flow for a left-handed person.” Bob decides this is an important concern and puts the dishwasher onthe left side of the sink.

Then Bob remembers that the Smiths are remodeling mainly to increase their property value in anticipationof selling in two years. So Bob decides to examine his design from a resale-value perspective. When Bobswitches to the resale-value perspective, an interpretive critic is triggered with the rule, “The dishwashershould be on the right side of the sink.” Interpretive critics support design as a interpretive process byallowing designers to interpret the design situation from different perspectives according to theirinterests. In this perspective, the critic about the dishwasher and sink has been redefined and itsassociated rationale has been modified. Now the argumentation says, “Optimizing your kitchen for left-handed cooks can adversely affect the house’s resale value since most kitchen users are right-handed.”Bob decides that enhancing the Smiths’ resale value is the more important consideration and moves thedishwasher. As long as he remains in the resale-value perspective, Bob will be informed by the criticswhenever they detect a feature negatively affecting resale value. Additionally, the critics will provide Bobaccess to argumentation concerning designing for resale.

4. Three Embedded Critiquing MechanismsThis section describes in detail three embedded critiquing mechanisms – generic, specific, andinterpretive critics. Examples of how these three critic styles are deployed was illustrated in the previousscenario. In all three mechanisms, critic knowledge is captured by rules with condition and action parts.The condition clause checks whether a certain situation exists in the current design construction. Theaction clause notifies the designer that a particular situation has been detected. Figure 4 illustrates acondition-action critic rule in which the condition checks if the stove is away from the window; the actionpart notifies the designer that “the stove is not away from the window.”

For all three mechanisms, the basic critiquing process consists of the following phases: (1) the set ofappropriate critic rules to be enabled is identified; (2) the design construction is then analyzed forcompliance with the currently enabled set of critic rules; (3) when a lack of compliance is detected, thecritic signals a possible problem and provides entry into the argumentative hypermedia component inwhich the appropriate explanation is located; and (4) concrete catalog examples that illustrate the

15

explanation given in the form of argumentation can optionally be delivered [Fischer, 1991 #74]. Asillustrated in Table 1, the three critic mechanisms differ mainly in terms of how they enable critic rules and inthe types of design knowledge embodied in their rules.

Table 1. The three critic mechanisms – generic, specific, and interpretive - differ in howthey enable critic rules, the rules’ scope of applicability, and the types of designknowledge each mechanism is best suited to represent.

Generic critics [Fischer, 1991 #74] are enabled by the placement of design units into the constructionarea. These critics apply to all designs containing the design unit to which the critics are attached. Genericcritics reflect knowledge that is applicable to all designs, such as accepted standards or regulations ordomain knowledge based on physical principles (see Table 1).

Specific critics [Nakakoji, 1993 #242] are constructed dynamically to reflect the designer’s goals as theyare stated explicitly in the specification component. These critics apply only to the design situationcurrently under consideration. Specific critics reflect design knowledge that is tied to situation-specificphysical characteristics and domain-specific concepts that not every design will share.

Interpretive critics [Stahl, 1993 #171] provide a mechanism for supporting design as an interpretiveprocess; that is, they are a response to the recognition that domain concepts such as “cabinet height” and“efficiency” can have more than one definition or interpretation depending upon the current situation andthe designer. Interpretive critics allow designers to view their work from multiple perspectives by creating,managing, and selectively activating different sets of design knowledge.

Specific examples illustrating each of these critic mechanisms will be discussed below. Generic critics willbe used to discuss the basic critiquing process described at the beginning of this section. The threemechanisms for embedded critics differ from one another primarily in how they determine which set of

How Enabled Applicability Design Knowledge Example

Generic Enabled by placing All designs Standards Cabinets should be 150 cm. above floor.design units into the

construction area Physical Principles Heat ignites flammable objects.

Specific Enabled by the Specific design

Situation Characteristics Cook is left-handed and 150cm. in height.partial specification

Abstract Domain Concepts Efficiency; safety.

Interpretive Enabled by the Specific Multiple interpretations Cabinet height: convenient for cook.currently active perspective of domain concepts

design perspective Cabinet height: desirable for resale value.

16

critic rules should be enabled. The discussion of specific critics and interpretive critics will focus on howthese mechanisms determine which critics are currently enabled.

4.1. Generic CriticsGeneric critics reflect knowledge that applies to all designs, such as accepted standards, building codes,and domain knowledge based on physical principles. Often, this generic knowledge can be found intextbooks, training curricula, or by interviewing domain practitioners. A generic critic representing anaccepted kitchen design standard is the cabinet height critic. Kitchen designers agree that unless morespecific information regarding the primary cook is known, the top cabinets should be placed 150 cm.above the floor. A generic critic reflecting domain knowledge based on safety principles is the “stoveshould be away from the window” rule shown in Figure 4. This rule reflects the principle that objects thatgenerate heat (e.g., the stove) should not be placed under flammable objects (e.g., the curtains on thewindow).

Generic critics in HYDRA are implemented as object-oriented methods of appliances and other designunits in the design construction. When the design construction is altered, all design units implicated bythe changes evaluate their critic methods. These methods are defined and parameterized by theinformation in property sheets such as those shown in Figure 4. For example, the rule box shown definesa generic critic for stoves. This method checks that the stove is “away from” all windows in the constructionarea.

Figure 4. The “stove should be away from the window” critic rule and the definition of the“away-from” spatial relation.

The condition away-from is defined in the relation property sheet as taking two objects and evaluatingwhether or not the minimum distance between them is greater than 12 inches. The correspondingmessage for display if this condition is not met is the critique: the first object “is not away from” the secondobject.

17

The critic defined in the rule sheet applies this relation to the stove as the first parameter and sequentiallyto each window in the construction as the second parameter. The definition specifies that this rule shall beapplied to all windows (Apply to: All) because stoves should be away from all windows to prevent fires.Other critic rules specify only that there should exist at least one object in the construction (Apply to: One)that matches the condition relation with the first parameter -- for example, the dishwasher should be nearat least one sink.

Further requirements can be specified for the applicability of the critic rule. These applicabilityrequirements make use of domain concepts like “generates heat,” “has curtains,” and “is flammable.” Inthe example rule, a stove has to be away from a window only if the stove generates heat (e.g., it is not amicrowave), if the window has curtains, and if the curtains are flammable. Finally, the definition of the criticlists a topic in the argumentation issue-base that will be displayed if this critic fires and the user selects thecritic message.

All generic critics in HYDRA are defined through property sheets like these for rules and relations. Usingthese property sheets, designers are able to modify the definitions of existing critics and to createadditional critics.

Critics inform designers of potentially problematic situations by using a three-tiered approach that involvessimple notification, supporting argumentation, and specific examples. First, the critic signals the designerof a potentially problematic situation with a simple initial notification message. The form of this initialnotification message is defined by the critique phrase in the spatial relation definition. The critic shown inFigure 4 would display the message “Stove-1 is not away from Window-1.” Variables in the notificationstring are resolved into specific design units by the critic rule using the spatial relation. Associatingnotification messages with the spatial relations allows these messages to be shared by many critic rules.The downside of this approach is that the notification message signals only that a spatial relation wasdetected and does not report why this is significant.

As discussed in Section 3.1, our work has shown that such “one-shot” notifications, which merely identifya situation, are inadequate. Critics that support design as an argumentative process [Rittel, 1984 #71]should be capable of presenting different alternatives and opinions and each alternative’s correspondingadvantages and disadvantages. The critiquing systems use the argumentation component of HYDRA toprovide the second tier of explanation, thereby “making argumentation serve design’’ [Fischer, 1991#75].

Each critic rule has an associated link into the argumentation component where issues pertaining to thesituation identified by the critic are discussed. For the critic in Figure 4, the associated link is found in theslot “Argumentation Topic: answer (stove, window).” The designer can view the critic’s associated designrationale by selecting the initial notification message displayed in the Message area (Figure 3). Becausedesign rationale contains design issues accompanied by positive and negative argumentation, criticexplanations in this form help the designer understand why the current design situation may be significantor problematic.

18

Sometimes designers may not understand the arguments made in the design rationale or they mayunderstand the arguments but not know what action to take. In these situations, providing designers withspecific examples can be helpful. The third tier of critic explanation delivers specific examples uponrequest that illustrate the issue being discussed. Designers can select an issue in the argumentation andrequest to see a positive example or a counter example. As illustrated in Figure 4, critic conditions areassociated with argumentation issues. When the designer requests to see an example of a specific issue,the ARGUMENTATION ILLUSTRATOR (see Figure 5) takes the critic condition associated with the selectedargumentation issue and searches the catalog component for examples that fulfill the condition.

Figure 5. Argumentation consists of issues, answers, and arguments supporting orrefuting answers. The designer can view the stove-away-from-window critic’s associateddesign rationale by selecting the initial notification message displayed in the Messagearea (e.g. “Stove-1 is not away from Window-1”) of Figure 3. The arguments shownexplain why many kitchen designers believe windows and stoves should not be adjacent.Choosing the menu item “Show Example” causes example designs that illustrate theanswer advocated in the argumentation to be delivered to the designer.

4.2. Specific CriticsIn HYDRA, specification knowledge is related to: (1) situation-specific physical characteristics such as thesize and shape of the kitchen or the owner’s height, (2) specified requirements such as “a dishwashershould be included,” and (3) abstract domain concepts such as safety and efficiency. The specificationissues were derived from questionnaires used by professional kitchen designers [Nakakoji, 1993 #242].

19

Specific critics evaluate the construction situation for compliance with the partial specification. Theyreduce the intrusiveness of a critiquing system by narrowing the enabled critics to those that are relevantto the task at hand as determined from the partial specification. Specification-linking rules [Fischer, 1991#77] are used to dynamically identify the set of specific critics to be enabled.

The specification consists of issue/answer pairs (see Figures 3 and 6). A specification linking rulerepresents a dependency between an issue/answer pair in the specification and associated pro and conarguments in the argumentation component. As shown in Figure 6, a specification linking rule connectsthe argumentation issue “Where should the stove be located?” with the specification item “Is safetyimportant to you?” The shared domain distinction “safety” is used to establish a dependency between thisparticular specification item and the argumentation issue.

A critic condition is associated with each answer in the specification, and a domain distinction is associatedwith each argument. Domain distinctions are a vocabulary for expressing domain concepts, such as safetyor efficiency. Whenever the designer modifies the specification, the critiquing system recompiles thespecification-linking rules to reflect the newly relevant domain distinctions. In this way, critiquing criteria aretied to a representation of the partially articulated goals of a specific design project.

Where should the stove be located?Issue:

Answer: The stove should be away from all doors.

safetyDomain-distinction:

Arguments:[pros]

Arguments:[cons]

(away-from stove door)Critic Condition:

If stove is not away from a door, it is a fire-hazard.

If stove is away from a door, it is not convenient for serving meals.

efficiencyDomain-distinction:

Argumentation-Base

Is safety important to you?Issue:Answer: Yes

safetyDomain-distinction:

Specification

DW

Construction

safety -> (away-from stove door)Specification-linking rule

Figure 6. Derivation of the Specification-Linking Rules. The domain distinction associatedwith a specification item is paired with a matching pro or con argument in the hypermediaissue base. The critic condition associated with an answer is linked with the domaindistinction to form a specific critic rule.

The operation of the specification-linking rules can best be conveyed with an example. Assume thedesigner knows that the kitchen owners have young children and he specifies that having a safe (child-proof) kitchen is very important (Figure 6). The domain distinction associated with this specification item is“safety.” In the argumentation, answers (e.g., “the stove should be away from all doors”) are associatedwith critic conditions (e.g., “away-from stove door”). Pro and con arguments are associated with domain

20

distinctions. In Figure 6, the domain distinction “safety” is associated with the pro argument and thedomain distinction “efficiency” is associated with the con argument.

Specification-linking rules link the domain distinctions activated in the specification with the appropriatecritic condition. First, the argumentation is analyzed until the domain distinction activated in thespecification (safety) is found. If the domain distinction is associated with a pro argument, then aspecification-linking rule is created with the form: domain distinction implies critic condition. If the domaindistinction is associated with a con argument, then a specification-linking rule is created with the form:domain distinction implies not critic condition. The specification-linking rules “safety implies stove away-from door” and “efficiency implies stove not away-from door” can be derived from the example in Figure 7.Whenever the designer modifies the specification, the critiquing system recomputes the specification-linking rules. For the partial specification shown in Figure 7, specification-linking rules supporting thenotion of safety will be constructed. The right side of the specification rules are the enabled criticconditions used to evaluate the design construction for adherence to the current specification.

Often, conflicts between specific critics arise. The designer could have specified that he was concernedwith both safety and efficiency. For example, having the stove to the left of the refrigerator may beefficient, but it may also be less safe if this places the stove next to a door. Using the specificationcomponent, the designer can not only state which concepts are of interest, he can also articulate his levelof interest by weighting specification items. The critiquing system uses these weights to help prioritizecritic activity. When a critic fires, it displays an importance weight next to the initial notification message thatreflects the weights assigned to the specification items that enabled the particular critic rule (see Figure 3).The designer can then take these relative weights into account when deciding to respond to the criticmessages.

4.3. Interpretive CriticsDesign can be viewed as an interpretive process [Stahl, 1993 #171]. Designers and their clients interpretthe design situation according to personal backgrounds, experiences, and concerns. This means thatthere cannot be a unique set of domain knowledge that is adequate for all people and all interests. Wehave prototyped a design environment [Stahl, 1992 #169] with perspectives [Bobrow, 1980 #236] toprovide alternative views or approaches to given design situations. The perspectives mechanismorganizes all the design knowledge in the system. It allows items of knowledge to be bundled intopersonal or topical groupings or versions. For instance, a resale-value perspective might include criticsand design rationale pertinent to homeowners concerned about their home’s resale appeal. A kitchendesign environment might have perspectives for evaluating kitchens from the perspective of anelectrician, a plumber, an interior designer, a realtor, a mortgage writer, or a city inspector. Perspectivescould also be defined for individuals who have special preferences or for specific kitchens. A perspectivefor the Smith’s kitchen would include design rationale for its unique set of design decisions, so that anyfuture modifications could be checked for consistency with those decisions.

The organization of knowledge by perspectives encourages users to view the knowledge in terms ofstructured, meaningful categories, which they can create and modify. It provides a structure of contexts

21

that can correspond to categories meaningful in the design domain. This can ease the cognitive burden ofmanipulating large numbers of alternative versions of critics and other design knowledge.

Interpretive critics are the result of interactions between the perspectives structure and the criticmechanisms (Figure 7). Critics are associated with design perspectives. The perspectives provide amechanism for creating, managing, and selectively activating different sets of critics along with their relateddesign knowledge, such as spatial relations, domain distinctions, palette items, and argumentation. Aperspective can incorporate critics from other perspectives, including generic and specific critics from thedefault perspective (see Figure 7). Additionally, a perspective may modify any inherited critics and definenew ones.

Critic Rule: Top Cabinets should be placed not lower than 140 cm or higher than 160 cm.

Default Residential KitchenGeneric Critic Rule: Top Cabinets should be placed 150cm above floor.

Cook's Height = 150 cm.

Critic Rule: Top Cabinets should be placed at 80% of cook's height.

Smith's Residential Kitchen

Resale Residential Kitchen

Figure 7. Perspectives are arranged in an inheritance network. Three perspectives – a “default kitchen,”“Smith’s kitchen,” and a “resale kitchen” – are shown. The preferred placement of the top cabinetsdepends on the perspective selected. The critic rule analyzing the placement of the top cabinets isredefined within each of the three perspectives.

Designers switch perspectives to examine a design from different viewpoints. Switching perspectiveschanges the currently effective definitions of critics, the terms used in these definitions, and other domainknowledge. As a result, the critics adapt to the different perspectives -- hence the term “interpretive”critics. The designer always works within a particular perspective. At any time, the designer can select adifferent perspective by name. New perspectives can also be created by assigning a name and selectingexisting perspectives to be inherited. Bob, the designer working with the Smiths in the previous scenario,could create a Smith’s kitchen perspective and select the resale perspective to be inherited by it.

Perspectives are connected in an inheritance network; a perspective can modify any knowledge inheritedfrom its parents or it can add new knowledge. Consider the inheritance network shown in Figure 7.Suppose that in the default perspective there is a rule that checks “if the top cabinets are 150 cm abovethe floor.” In the Smith’s kitchen perspective the rule that determines cabinet height is based on thecook’s height. This same critic rule will be evaluated differently in the three different perspectives becauseit is defined in terms of the spatial relationship whose definition varies. Similarly, either the rule or thespatial relationship in the rule could be defined indirectly in terms of something in the argumentationissue-base, such as the answer to an issue requesting the primary cook’s height. Critics and the design

22

knowledge on which they are based can be adapted to interpret designs differently in many ways: byinheritance, by modification of inherited objects, or by addition of new objects into a perspective.

Interpretive critics based on perspectives provide a mechanism for refining the critiquing process that isorthogonal to the specific critics. Specific critics fine-tune the generic critics that embody general domainknowledge, relating them to the design choices specified for a given project. Whereas the set of genericand specific critics may be extensible in the sense that new critics can be added from time to time, theperspectives mechanism provides for multiple definitions of these sets to exist simultaneously so thatindividual designers can fluidly adopt varying viewpoints on designs. This provides a means for structuringnew critics and other knowledge representations as they emerge during use of the design environmentand systematically retaining this knowledge for use in future projects.

5. Benefits of Embedding: Increasing the Shared ContextComputational media offer great capacity for storing large volumes of information and support formanaging dynamic information spaces [Norman, 1993 #175]. Computational media can integrate diverseinformation sources such as reference materials, solutions to previous design problems, and collectionsof design rationale. However, access to large information spaces creates a new problem for designers:information overload. In situations of information overload, the critical resource for designers is notinformation, but rather the attention with which to process information. Simon [Simon, 1981 #4] arguedwith convincing examples that a design representation suitable for a world in which the scarce factor isinformation may be exactly the wrong one for a world in which the scarce factor is attention. Whenpresenting people with information, the primary concern is to present items that are relevant to the task athand [Fischer, 1991 #77]. Critics embedded in design environments exploit a rich notion of thedesigner’s task at hand, or context, to provide relevant information to designers.

Design environments support a cooperative problem-solving process in which the designer determinesthe context of design by manipulating interface objects (such as graphical objects and form-basedobjects) in the construction, specification, and perspective components. Objects in the constructioncomponent define a construction context that provides generic critics with a representation for the task athand. Values and priorities for specification objects define a specific context that allows specific critics tocompute relevant information for the particular task as specified by the designer. The perspectivemechanism determines an interpretive context that enables collections of critics and their associatedargumentation.

The context defined by the construction, specification, and perspective situations allows the system toprovide information relevant to a dynamic representation of the task at hand that is shared by the designerand the design environment. This shared context enables precise intervention by critics, reducesannoying interruptions, and increases the relevance of information delivered to designers. Criticsembedded in design environments benefit the design process by increasing the designer’sunderstanding of design situations, by pointing out significant design situations that might have beenoverlooked, and by locating relevant information in very large information spaces.

23

5.1. Increasing the Designer’s Understanding of Design SituationsThe solution of a design problem necessarily involves coming to a deeper understanding of the problemthrough attempts to solve it. Design problems cannot be clearly defined “up front,” before any attempt at asolution is made. New requirements emerge during the design process [Schoen, 1983 #129]; [Rittel,1984 #243]; [Fischer, 1992 #83] that cannot be identified until portions of the artifact have beendesigned or implemented. These aspects of design create the following dilemma: (1) one cannot gatherinformation meaningfully unless the problem is understood, (2) one cannot understand the problemwithout having a concept of the solution in mind, and (3) one cannot understand the problem withoutinformation about it.

Problem framing and problem solving are mutually enabling design processes because each informs theother. Design methodologists such as Schoen [Schoen, 1983 #129] and Rittel [Rittel, 1984 #243] stressthe strong interrelationship between problem framing and problem solving. They characterize designproblems by the need for designers to impose a discipline, or framing, on the problem in order to reducethe complexity of the situation to a manageable level. Problem framing is the process of determining theboundaries (or framework) of a problem, such as determining the “givens” of the problem, theassumptions under which the designer operates, and the criteria for evaluating a solution. Each movetoward a design solution tests the problem framing, potentially exposing conflicting or unrealistic goals.Critics embedded in design environments support designers in creating and modifying the problemframing throughout the design process – not just in the beginning. Critics support a design processwhere “understanding the problem is the problem.”

In this view of design, in which problem framings and problem solutions coevolve, each action by thedesigner has the potential to alter the understanding of the problem, which in turn can influencesubsequent actions. Our goal is to support design as a cooperative problem-solving dialog between thedesigner and the evolving design situation.

5.2. Pointing Out Significant Design SituationsBy seeing design as a “reflective conversation with the situation” [Schoen, 1983 #129], action isgoverned by nonreflective thought processes and proceeds until it breaks down. A breakdown [Fischer,1993 #239] occurs when the designer realizes that nonreflective action has resulted in unanticipatedconsequences – either good or bad. Schoen described this realization as ``the situation talks back.’’Reflection is used to repair the breakdown, and then (nonreflective) situated action continues. Thehallmark of reflection-in-action is that it takes place within the action present – within the time period duringwhich the decision to act has been made but the final decision about how to act has not. This is the timeperiod during which reflection can still make a difference in what action is taken.

Schoen’s theory of design is based on designers interacting with traditional media, and the back-talk fromthe situation is determined solely by the designer’s skill, experience, and attention. Computationaltechnology, such as critics embedded in design environments, affords a new type of back-talk from thedesign situation. Computational design situations can actively point out breakdowns to designers. Thisactive design support enables designers to hear the situation talk back in situations that might haveremained mute in passive media.

24

Reflection-in-action, as supported by embedded critics, is an ongoing cycle of action, breakdown, andreflection. Designers act when they shape the design situation. They establish a shared context with thedesign environment by manipulating interface objects in the construction, specification, or perspectivecomponents. Breakdowns are triggered by critics embedded in the design environment that detectsituations that indicate the designer might need to reflect. Based on the shared context, critics supportreflection by delivering information relevant to the breakdown situation. Argumentative information helpsdesigners understand the breakdown situation, and the catalog contains design solutions that provideexamples of how other designers have resolved similar problems.

The scenario illustrates how embedded critics support design as a reflective conversation with thesituation. In the scenario, critics triggered two consecutive breakdowns. In the first, the constructionsituation talked back to Bob when his actions violated a generic kitchen design principle that “thedishwasher should not be too far from the sink.” After some reflection, he moved the dishwasher nearer tothe sink to comply with the critic. However, this action created a new breakdown situation. A specific criticsignaled a breakdown to remind Bob that his actions were inconsistent with his partial specification; that is,his placement of the sink might not be optimal for left-handed cooks. This breakdown led him to reflect onhis goals; instead of altering the design construction, Bob reformulated his partial specification.

5.3. Locating Relevant Information In Large Information SpacesMaking information relevant to the task at hand poses many challenges for the design of interactivecomputer systems, particularly for problems in which the need for information is critical and yet preciseinformation needs cannot be known in advance of attempts to solve the problem. Our designenvironments that support design in complex domains are high-functionality computer systems; that is,they provide a large amount of functionality and are built on large information bases. Such systemsprovide more information and functionality than a single person can master [Draper, 1984 #85]. Twofactors contribute to this behavior: (1) the effort of finding information often outweighs the perceivedbenefits of doing so, and (2) users are not aware that the information even exists. Both factors can berelated to the discrepancy between the designer’s perception of an information space and the actualinformation contained in a high-functionality system (see Figure 8).

Designers are often unwilling to disrupt the design process to search for information in large informationspaces, even if they know the information exists. In addition, designers may not know when they needinformation. Embedded critics save designers the trouble of explicitly querying the system for information.Critics notify designers of situations indicating the need to reflect (breakdowns) and provide access toinformation fueling reflection. The context of the breakdown situation serves as an implicit query thatenables embedded critics to deliver relevant information. Designers benefit from needed informationwithout having to explicitly ask for it.

Embedded critics can also deliver relevant information [Nakakoji, 1993 #242] about which designers wereunaware (see Figure 8). Critics provide the designer with a pointer into part of the system’s informationspace with which the designer needs to become aware. The designer can further browse the unfamiliarportion of the information space starting from the entry point provided by the critic.

25

Critics afford learning on demand [Fischer, 1991 #72] by letting designers access new knowledge in thecontext of actual problem situations; users are informed (1) when they are getting into trouble, (2) whenthey are missing important information, and (3) when they come up with problematic solutions. Learningon demand is a promising approach for the following reasons: (1) it contextualizes learning by integrated itinto work rather than relegating it to a separate design phase; (2) it lets designers see for themselves theusefulness of new knowledge for actual problem situations, thereby increasing the designers’understanding of their situations; and (3) it makes new information relevant to the task at hand, therebyleading to better decision making, better products, and better performance.

Information Contained inDesign Environment butUnknown to Designer

Information Knownto be Contained inDesign Environment

Information MistakenlyBelieved to Exist inDesign Environment

Information Contained inDesign Environment

Information Space asPerceived by Designer

Figure 8. Large information spaces contain more information than a single person canknow exists. The oval represents the information a designer perceives to be in the designenvironment. The square represents the information actually contained in the designenvironment. This figure illustrates that the designer’s perception includes informationthat does not exist in the design environment, and does not include some informationthat actually exists in the design environment.

Critics exploit the shared context of breakdown situations to compute what information is relevant to thetask at hand. In the scenario, each critic’s notification message was linked to information in theargumentation component. For the “dishwasher not too far from the sink” issue, the designer wasreminded of plumbing requirements he might have known about but did not remember in the context ofthe design situation. The “left-handed” specific critic identified information the designer had previouslybeen unaware of: that the recommended positions of the sink and dishwasher are dependent on whetherthe cook is right- or left-handed. The interpretive critic (enabled by adapting a Resale Perspective)informed Bob of additional information about which he had previously been unaware. Now that he is awareof this “resale” value concern, Bob could explore further implications of a resale perspective by browsingrelated information or by continuing his design process, where he will be informed on demand.

26

6. The Dynamic Nature of Critiquing Knowledge6.1 Supporting Designers in Adapting the Critiquing SystemTo be successful, embedded critiquing systems must adapt to reflect changes in the design domain. Twoquestions arise when considering system adaptation: will designers be able to adapt the system asrequired and will designers be motivated to adapt the system? End-user modifiability components anddesign environment “seeds” are important steps toward answering these questions.

Adapting the critiquing system involves modifying or adding critic rules, design units, design unit relations,and critic explanations in the form of argumentation and catalog examples. Sometimes, adapting thesystem is as simple as changing parameters or filling out specialized forms. Girgensohn [Girgensohn,1992 #172] explored end-user modifiability in domain-oriented design environments. His work showedthat end-users without any formal training in computer science need considerable environmental supportin the form of explanatory help, critics that support modification processes, task decomposition agendas,and computer-supported object classification to effect significant system changes. Even with thisextensive environmental support, none of the subjects in his user studies were able to complete theadaptations without intervention from the study supervisor. Girgensohn’s research has demonstrated thatenabling designers to adapt their systems is a very difficult problem which requires further research in theareas of demonstration components, domain-oriented knowledge representations, and adaptive usermodeling components. The HERMES project is exploring a different approach toward achieving end-usermodifiability by building into the design environment an English-like end-user programming language[Stahl, 1992 #31].

6.2 “Seeding” the Critiquing System with Domain KnowledgeWhereas ongoing adaptation of embedded critiquing systems is in the hands of designers solving designproblems, system builders must create the original conditions that enable and motivate this evolutionprocess to occur. Specifically, system builders must provide initial environments in the form of a seed.

We cannot offer an easy-to-follow prescription for successful seed building. Seed building requires adeep understanding not only of the application domain, but also of the practice [Ehn, 1989 #2] of thepeople who will use the system. System builders cannot hope to attain such an understanding without, atleast to some extent, becoming domain experts themselves. But this is generally infeasible. For usefulseeds to be built, system-building must be based on a process of mutual education [Greenbaum, 1991#241] between system builders, who know about building software design environments, and domaindesigners, who understand the practice of design in the target application domain. The goal of this mutualeducation process is to establish a shared understanding of what domain knowledge a seed shouldcontain so that it will immediately support the practice of designers within that domain.

6.3 Accumulating Design Knowledge Through CriticsEmbedded critics play the crucial role of “knowledge attractors” in domain-oriented design environments.Design knowledge surfaces during reflection-in-action, when designers reflect upon the source ofbreakdowns and devise courses of action for resolving the breakdowns. User observations in usingspecific critics revealed that when designers were fired a critic rule, they often argued for or against the

27

associated argument and were motivated to describe the reason by articulating pro or counter argumentsto the argumentation [Nakakoji, 1993 #242]. The incomplete nature of design knowledge guarantees theargumentation is never complete. Designers who arrive at an innovative resolution to a breakdown mayadd their arguments to the existing rationale, enriching the information space contained in the designenvironment.

7. ConclusionsAlthough this paper focuses primarily on a single design environment built for residential kitchen design,the HYDRA-KITCHEN system, other ongoing research in our group has demonstrated that embeddedcritiquing systems have broad applicability to a variety of domains and that embedded critiquing systemscan be applied to complex, new domains with few accepted design rules and practices, and non-spatially-oriented domains.

The interpretive critiquing mechanism is being explored in the domain of lunar habitat design [Stahl, 1993#171]. Unlike kitchen design, lunar habitat design is a completely new domain with few design rules andno standardized vocabulary. In domains with few standards, negotiation, argumentation, andinterpretation are increasingly important aspects of design. This aspect of the lunar habitat design domainled us to extend our critiquing systems to include interpretive mechanisms.

The Voice Dialog Design Environment tests the applicability of critiquing systems to non-spatial domains.The system supports the design and simulation of applications with phone-based interfaces [Repenning,1992 #33]. In this domain, design units include audio prompts, voice menus, and telephone touch-toneinput. Relations between design units are temporal in nature; that is, design units occur before or aftercertain events in the execution sequence. This design environment is part of a joint research projectbetween the University of Colorado and voice dialog application designers at US WEST AdvancedTechnologies [Sumner, 1991 #32].

We have demonstrated how embedding critic mechanisms in design environments overcomes manydeficiencies found in stand-alone critiquing systems. The generic, specific, and interpretive critics wehave explored correspond to three dimensions of embedding. Generic critics are embedded in theconstruction context because they are enabled by the placement of design units in the work area.Specific critics are embedded in the partial specification by being dynamically constructed from domaindistinctions tied to specification items; they reduce the intrusiveness of generic critics by narrowing theenabled critics to those that are relevant to the partially specified task at hand. Interpretive critics areembedded in the network of perspectives that supports the evolution of alternative viewpoints ondesigns; using these critics, designers are able to consider their designs critically from multipleperspectives. The beneficial role of human critiquing in science, design, and engineering had beensocially recognized long before the advent of computational critiquing systems. Our approach ofembedding critics into integrated design environments is an important step toward applying the critiquingparadigm to create more useful and usable knowledge-based computer systems .

28

8. ReferencesD. G. Bobrow, I. Goldstein, Representing Design Alternatives, In the Proceedings of AISB Conference,AISB, Amsterdam, 1980.

B. Buchanan, E. Shortliffe, Human Engineering of Medical Expert Systems, in B. Buchanan and E.Shortliffe (Eds.), Rule-Based Expert Systems: The MYCIN Experiments of the Stanford HeuristicProgramming Project, Addison-Wesley, Reading, MA, 1984, pp. 599-612.

R. Burton, J. S. Brown, An Investigation of Computer Coaching for Informal Learning Activites, in D.Sleeman and J. S. Brown (Eds.), Intelligent Tutoring Systems, London, Academic Press, 1982, pp. 79-98.

A. B. Chambers, D. C. Nagel, Pilots of the Future: Human or Computer?, Communications of the ACM,Vol. 28, No. 11, 1985, pp. 1187-1199.

J. Conklin, M. Begeman, gIBIS: A Hypertext Tool for Exploratory Policy Discussion, Transactions of OfficeInformation Systems, Vol. 6, No. 4, 1988, pp. 303-331.

S. W. Draper, The Nature of Expertise in UNIX, In the Proceedings of Proceedings of INTERACT'84, IFIPConference on Human-Computer Interaction (Amsterdam), Elsevier Science Publishers, 1984, pp. 182-186.

P. Ehn, Work-Oriented Design of Computer Artifacts , (2nd ed.), Arbetslivscentrum, Stockholm, 1989.

G. Fischer, A Critic for LISP, In the Proceedings of Proceedings of the 10th International Joint Conferenceon Artificial Intelligence (Milan, Italy), Morgan Kaufmann Publishers, 1987, pp. 177-184.

G. Fischer, Communication Requirements for Cooperative Problem Solving Systems, InformationsSystems, Vol. 15, No. 1, 1990, pp. 21-36.

G. Fischer, Supporting Learning on Demand with Design Environments, In the Proceedings ofProceedings of the International Conference on the Learning Sciences 1991 (Evanston, IL), 1991, pp.165-172.

G. Fischer, Domain-Oriented Design Environments, In the Proceedings of 7th Annual Knowledge-BasedSoftware Engineering (KBSE-92) Conference (MCLean, VA.), IEEE Computer Society Press, LosAlamitos, CA., 1992, pp. 204-213.

G. Fischer, Turning Breakdowns into Opportunities for Creativity, in E. Edmonds (Eds.), Creativity inCognition, Penrose Press, (in press), 1993.

G. Fischer, A. Girgensohn, End-User Modifiability in Design Environments, In the Proceedings of CHI'90(Seatle, WA.), ACM Press, 1990, pp. 183-191.

G. Fischer, J. Grudin, A. C. Lemke, R. McCall, J. Ostwald, B. N. Reeves, F. Shipman, Supporting Indirect,Collaborative Design with Integrated Knowledge-Based Design Environments, Human ComputerInteraction (Special Issue on Computer Supported Cooperative Work), Vol. 7, No. 3, 1992.

29

G. Fischer, A. C. Lemke, T. Mastaglio, A. Morch, The Role of Critiquing in Cooperative Problem Solving,ACM Transactions on Information Systems, Vol. 9, No. 2, 1991, pp. 123-151.

G. Fischer, A. C. Lemke, R. McCall, A. Morch, Making Argumentation Serve Design, Human ComputerInteraction, Vol. 6, No. 3-4, 1991, pp. 393-419.

G. Fischer, A. C. Lemke, T. Schwab, Knowledge-Based Help Systems, In the Proceedings of HumanFactors in Computing Systems, CHI'85 Conference Proceedings (San Francisco, CA) (New York), 1985,pp. 161-167.

G. Fischer, R. McCall, A. Morch, Design Environments for Constructive and Argumentative Design, In theProceedings of CHI '89 (Austin, Texas), ACM Press, New York, 1989, pp. 269-275.

G. Fischer, K. Nakakoji, Making Design Objects Relevant to the Task at Hand, In the Proceedings of AAAI-91, Ninth National Conference on Artificial Intelligence (Cambridge, MA), AAAI Press/The MIT Press,1991, pp. 67-73.

A. Girgensohn, End-User Modifiability in Knowledge-Based Design Environments, Unpublished Ph.D.Dissertation, Department of Computer Science, University of Colorado at Boulder, 1992, Also available asTechReport CU-CS-595-92.

J. Greenbaum, M. Kyng, Design at Work: Cooperative Design of Computer Systems , Lawrence ErlbaumAssociates, Hillsdale, NJ, 1991.

J. R. Hackman, R. E. Kaplan, Interventions into group process: An approach to improving theeffectiveness of groups, Vol. 5, 1974, pp. 459-480.

R. Johansen, Groupware: Computer Support for Business Teams , Free Press, New York, 1988.

J. Kolodner, Improving Human Decision Making through Case-Based Decision Aiding, Vol. 12, No. 2,1991, pp. 52-68.

J. Lave, Cognition in Practice , Cambridge University Press, Cambridge, UK, 1988.

A. C. Lemke, G. Fischer, A Cooperative Problem Solving System for User Interface Design, In theProceedings of Proceedings of AAAI-90, Eighth National Conference on Artificial Intelligence, AAAIPress/The MIT Press, Cambridge, MA, 1990, pp. 479-484.

R. McCall, PHIBIS: Procedurally Hierarchical Issue-Based Information Systems, In the Proceedings ofProceedings of the Conference on Architecture at the International Congress on Planning and DesignTheory (New York), American Society of Mechanical Engineers, 1987.

K. Nakakoji, Increasing Shared Knowledge of Design Tasks Between Humans and Design Environments:The Role of a Specification Component, PhD Dissertation Thesis, Department of Computer Science,University of Colorado at Boulder, Boulder, CO, 1993.

D. A. Norman, Things That Make Us Smart , Addison-Wesley Publishing Company, Reading, MA, 1993.

30

H. Petroski, To Engineer Is Human: The Role of Failure in Successful Design , St. Martin's Press, NewYork, 1985.

M. Polanyi, The Tacit Dimension , Doubleday, Garden City, NY, 1966.

K. R. Popper, Conjectures and Refutations , Harper & Row, New York, 1965.

R. Prieto-Diaz, P. Freeman, Classifying Software for Reusability, Vol. 4, No. 1, 1987, pp. 6-16.

A. Repenning, T. Sumner, Using Agentsheets to Create a Voice Dialog Design Environment, In theProceedings of Symposium on Applied Computing (SAC '92) (Kansas City, MO), ACM Press, 1992, pp.1199-1207.

H. Rittel, Second Generation Design Methods, in N. Cross (Eds.), Developments in Design Methodology,John Wiley & Sons, New York, 1984, pp. 317-327.

H. Rittel, M. M. Webber, Planning Problems are Wicked Problems, in N. Cross (Eds.), Developments inDesign Methodology, John Wiley & Sons, New York, 1984, pp. 135-144.

D. A. Schoen, The Reflective Practitioner: How Professionals Think in Action , Basic Books, New York,1983.

B. Silverman, Survey of Expert Critiquing Systems: Practical and Theoretical Frontiers, CACM, Vol. 35,No. 4 (April 92), 1992, pp. 106-127.

H. A. Simon, The Sciences of the Artificial , (second ed.), The MIT Press, Cambridge, MA, 1981.

G. Stahl, Toward a Theory of Hermeneutic Software Design , No. CU-CS-589-92, Computer ScienceDepartment, University of Colorado at Boulder, 1992.

G. Stahl, Supporting Interpretation in Design, Journal of Architecture and Planning Research (SpecialIssue on Computational Representations of Knowledge), 1993 (Forthcoming).

G. Stahl, R. McCall, G. Peper, A Hypermedia Inference Language as an Alternative to Rule-based ExpertSystems, In the Proceedings of submitted to Expert Systems ITL Conference, 1992.

T. Sumner, S. Davies, A. C. Lemke, P. G. Polson, Iterative Design of a Voice Dialog Design Environment ,Technical Report No. CU-CS-546-91, Department of Computer Science, University of Colorado atBoulder, 1991.

T. Winograd, F. Flores, Understanding Computers and Cognition: A New Foundation for Design ,Addison-Wesley, Menlo Park, CA, 1986.