knowledge engineering and psychology

7/28/2019 Knowledge Engineering and Psychology

1/16

Int. J. Human-Computer Studies 64 (2006) 12141229

Knowledge engineering and psychology: Towards a closer relationship

Nick Milton a, , David Clarke b , Nigel Shadbolt c

a Epistemics, Strelley Hall, Nottingham, NG8 6PE, UK b School of Psychology, University of Nottingham, University Park, Nottingham, NG7 2RD, UK

cDepartment of Electronics and Computer Science, University of Southampton, Southampton, SO17 1BJ, UK

Received 13 April 2005; received in revised form 19 August 2006; accepted 21 August 2006Communicated by E. Motta

Available online 2 October 2006

Abstract

Knowledge engineering projects deal with a wide range of domains within organizational and academic contexts. A number of elicitation techniques are used to acquire knowledge from experts. Most of these techniques originated within psychology but have beendeveloped by knowledge engineers to become more structured, efcient and systematic. Until now, nobody has tried to re-apply thesemodied techniques back into psychology. This paper describes work that addresses this matter. It focuses on the psychologicalknowledge possessed by all people that enables them to deal with everyday problems and make life decisions. We refer to this as personalknowledge. To take a knowledge engineering approach to personal knowledge, we investigated the use of knowledge elicitationtechniques to capture personal knowledge. We describe an empirical study involving ten participants and 80 knowledge acquisitionsessions that assessed eight elicitation techniques in this context. The results revealed that each of the techniques showed promise atefciently capturing and structuring aspects of an individuals personal knowledge. A content analysis of the acquired knowledge led tothe construction of a meta-model (a primitive ontology) of personal knowledge and to the design for a new methodology forpsychological research. From the perspective of psychology, the paper shows that knowledge engineering methods can be of value topsychologists. From the perspective of knowledge engineering and the wider computer science community, the paper shows thatempirical methods used by psychologists can benet the development and evaluation of ontologies and elicitation techniques.r 2006 Elsevier Ltd. All rights reserved.

Keywords: Knowledge elicitation; Knowledge acquisition; Psychology; Psychotherapy; Meta-model; Qualitative methods; PCPACK

1. Introduction

Knowledge engineering is that part of AI concerned withthe principles, methods and tools for acquiring knowledgeand developing knowledge-based systems ( Studer et al.,

1998 ; Schreiber et al., 2000 ). A primary research areaduring the 1980s and 1990s addressed the elicitation of knowledge from subject matter experts ( Hoffman et al.,1995 ). Although research in this area has dwindled over thepast few years, a number of organizations are benetingfrom the practical application of these research ndings,

not only for AI but also for the wider eld of KnowledgeManagement ( Milton et al., 1999 ; Hammersley et al.,1999 ).

A key feature when assessing knowledge elicitationtechniques has been the use of empirical studies, particu-

larly controlled experiments ( Shadbolt and Burton, 1995 ).Over recent years, a shift in emphasis in knowledgeengineering from acquisition techniques to ontologies andcomputer languages has taken place. This shift has resultedin a lack of emphasis on experiments and other empiricalmethods of assessment. One of the aims of the workpresented in this paper is to demonstrate that a properlydesigned empirical study can be of benet to the develop-ment and evaluation of ontologies as well as for elicitationtechniques. Another aim, from the perspective of psychol-ogy, is to show that techniques used by knowledge

ARTICLE IN PRESS

www.elsevier.com/locate/ijhcs

1071-5819/$- see front matter r 2006 Elsevier Ltd. All rights reserved.doi: 10.1016/j.ijhcs.2006.08.001

Corresponding author. Tel.: +44 1159135815; fax: +441159061304.E-mail addresses: [email protected] (N. Milton) ,

[email protected] (D. Clarke) , [email protected](N. Shadbolt) .
http://www.elsevier.com/locater/ijhcshttp://localhost/var/www/apps/conversion/tmp/scratch_2/dx.doi.org/10.1016/j.ijhcs.2006.08.001mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]://localhost/var/www/apps/conversion/tmp/scratch_2/dx.doi.org/10.1016/j.ijhcs.2006.08.001http://www.elsevier.com/locater/ijhcs


2/16

engineers to acquire and model knowledge can be of benetfor psychological research and perhaps as a therapeutictool.

1.1. Personal knowledge

Knowledge engineering projects typically involve do-mains associated with organizational or academic knowl-edge. In this paper, we explore a different type of knowledge: the psychological knowledge possessed by allpeople that enables them to deal with everyday problemsand make life decisions. This includes the knowledgeindividuals have of their life history, their behaviours, theirmoods, their attitudes, their relationships, their abilities,their aspirations, and so on. We refer to such knowledge aspersonal knowledge.

The study of personal knowledge is usually the preserveof psychologists, psychotherapists, psychiatrists and coun-sellors. We aim to show in this paper that a knowledgeengineering approach to personal knowledge can providemethods that add to those already used by these profes-sionals. We believe that these methods can help in thedevelopment of new psychological theories and be of benet to ordinary people by increasing their self-under-standing. This follows a trend in psychology, and the socialsciences, to widen the use of qualitative techniques ( Smithet al., 1995 ).

In exploring these issues, we follow an interdisciplinaryapproach by treating personal knowledge as if it were atype of expert knowledge. In other words, we take the

principles and techniques used by knowledge engineers indomains such as medicine, electronics and geology andapply them to the domain of personal knowledge. Oursubject matter experts are ordinary people each of whompossesses the expertise to deal with life situations withincomplex social environments.

1.2. Knowledge elicitation techniques

It is important to note that many knowledge elicitationtechniques originated within psychology. However, workundertaken by the knowledge engineering community overthe past 20 years has transformed these techniques so thatthey have to become more structured, efcient andsystematic. One development has been the use of structuredmethodologies such as CommonKADS ( Schreiber et al.,2000 ) and MOKA ( Stokes, 2001 ). Another has been the useof software support tools such as Prote ge (Prote ge website )and PCPACK ( Epistemics website ). Another has been theuse of generic knowledgebases such as CYC ( CYCWebsite ). In spite of these advances, there has been noattempt to export the adapted techniques back intopsychology and assess their merits for psychologicalpurposes. The research presented here aims to rectify thismatter.

1.3. Structure

The rest of the paper is organized as follows. Section 2describes the background and rationale for pursuing aknowledge engineering approach to personal knowledge.Section 3 describes an empirical study that examined the

use of knowledge elicitation techniques to acquire personalknowledge. Section 4 presents a meta-model of personalknowledge derived from the empirical results. Section 5describes a new methodology for psychological research.Section 6 contains a summary and concluding remarks.

2. A knowledge engineering approach to personal knowledge

2.1. Approach

When dening the approach to be used in the study, werst sought to determine the signicant differences betweenthe domain of personal knowledge and conventionalknowledge engineering domains. We discerned four keydifferences. First, unlike most knowledge domains, perso-nal knowledge is not based on a core set of theories andpractices taught to practitioners at schools and universities.Second, the history of psychology shows that personalknowledge is vastly complex, with numerous perspectivesand points of view. Indicative of this is the variety of different psychotherapies and psychological research ap-proaches that are used. Third, the domain of personalknowledge has no obvious experts. Although some may layclaim to be experts, we feel that every person in the worldcan be considered to be an expert on their own life. Fourth,

knowledge engineering domains tend to be problem-focused and often involve structured knowledge, clear-cutgoals and decisions. In contrast, an individuals personallife tends to be less driven by the need to solve clear-cutproblems and may involve less structured knowledge, suchas knowledge associated with emotions and feelings.

The differences outlined above led us away from aconventional knowledge engineering approach; that is, toacquire and model the knowledge of domain experts tocreate an intelligent software system. The development of such a system in the context of personal knowledge wouldbe too ambitious and too problematic. Instead, we lookedto the use of personal knowledge in psychology. Psychol-ogists capture and analyse personal knowledge for twomain reasons: (i) to provide psychotherapeutic help and (ii)to develop theories. Thus, we have two distinct possibleaims: to help in the provision of psychotherapeutic help, orto help in the development of new psychological theories.We chose to explore both of these aims, but with anemphasis on the latter.

To address these aims, we asked two questions: (i) Howis personal knowledge used to provide psychotherapeuticbenets? (ii) How is personal knowledge treated withinpsychological research? The following sections brieyaddress these questions as a background to the studydescribed in Section 3.

ARTICLE IN PRESS

N. Milton et al. / Int. J. Human-Computer Studies 64 (2006) 12141229 1215


3/16

2.2. Personal knowledge and psychotherapy

There are many approaches and methods used bypsychotherapists, clinical psychologists and counsellors.Indeed, there are over 250 different forms of psychotherapy(Meichenbaum, 1997 ). Taken as a whole, the literature on

psychotherapy is equivocal in the evidence for and againsteach of the therapies. What can be discerned is that mostforms of psychotherapy involve personal knowledge.Indeed, many important and well-known therapies usetechniques that involve eliciting, revealing, analysing andchallenging personal knowledge. To illustrate this, thefollowing paragraphs give a brief account of the mainpsychotherapeutic approaches and the some of thetechniques that are used.

2.2.1. The psychodynamic approachThe psychodynamic approach has its roots in the

psychoanalytic tradition of Freud and his followers.Therapy is based essentially on interview sessions, withanalysis and interpretation by the therapist. Hence, thistype of therapy has been described as a talking cure(Rycroft, 1995 ). The key aim is to uncover and revealunconscious memories, wishes, motivations and fears.These are seen as the cause of emotional disturbance butare not able to be explicitly verbalized due to a copingmechanism of internal repression. Therapy involves tech-niques such as free association, analysis of dreams,association tests and interpretation of meaningless orambiguous images.

2.2.2. The humanistic approachThe humanistic approach is inspired by the work of Carl

Rogers (1951) and places great importance on the support,empathy and unconditional positive regard that thetherapist displays to the client. The purpose of mosthumanistic therapies is to help the client to become whathe/she is capable of becoming ( Maslow, 1968 ), rather thandiagnose a problem and cure it. The therapist aims toprovide opportunities, ideas and experiences within thetherapeutic session. The techniques that are used can beseen as ways of being with a client rather than proceduresthat operate to make things happen in the client ( Bohartand Tallman, 1996 ).

2.2.3. Cognitive-behaviour therapiesCognitive-behaviour therapies include a family of

approaches based on empirical psychological research.The main therapeutic goal is the modication of behaviourand/or cognitions. The two main approaches are behaviourtherapy and cognitive therapy. Behaviour therapy is basedon the behaviourist movement of experimental psychology,primarily with the work of Pavlov and Skinner. Therapyaims to break maladaptive associations that were learnedvia conditioning. Techniques such as systematic desensiti-zation and token economies are used ( Murdoch andBarker, 1991 ). Cognitive therapy is based on the idea that

symptoms (e.g. of depression) and negative automaticthoughts about the self (e.g. Im worthless) are locked ina vicious cycle ( Beck et al., 1980 ). Therapy involves anumber of techniques such as monitoring daily activities,disrupting negative automatic thoughts, challenging under-lying assumptions and preparing for future life changes.

2.2.4. Personal construct psychologyPersonal construct psychology ( Kelly, 1955 ) is a

constructivist system based on the idea that an individualspsychological processes are driven by the way in whichevents are interpreted. Prediction of future events is basedon a persons constructs, i.e. the ways in which entities areconstrued as being similar or different from each other.Maladaptive thoughts and behaviours are seen to stemfrom inappropriate constructs. The therapeutic approachencourages the client to develop alternative construct

systems. The main therapeutic technique is the roleconstruct repertory test, now referred to as the repertorygrid ( Fransella and Bannister, 1977 ). This method involvesthe construction of a grid consisting of numerical ratings of constructs for a set of elements (e.g. signicant people).Evidence indicates that revelation of underlying constructsusing this technique has benecial effects on mood anddecision making ( Beail, 1985 ; Smith, 1990 ). Of particularlyrelevance for our research is the work of Mildred Shaw andBrian Gaines who have used the repertory grid techniquefor both psychological and knowledge engineering pur-poses ( Shaw, 1980 ; Gaines and Shaw, 1993 ; PCP Website ).This work is an important precedent for our approach andsuggests that the techniques of knowledge engineering maynot need substantial modication when applied to personalknowledge. However, in the case of techniques other thanthe repertory grid, this remains an important research issuerequiring empirical examination.

2.2.5. Self-help and self-development approachesSelf-help and self-development approaches do not rely

on a therapist to relate directly to the client. Instead, theseapproaches make use of self-help books, diaries andcomputer-assisted therapies. Self-help books, in particular,are very popular; for example, surveys have shown thatover 30% of American adults having purchased a self-helpbook ( Wood, 1988 ). These approaches split into those thataddress a particular problem and those that aim tofacilitate more self-understanding and engender personaldevelopment. Of the problem-focussed approaches, manystudies have shown that self-help therapies are just aseffective as those involving a therapist for many types of psychological problems ( Bloom, 1992 ; Gould and Clum,1993 ; Marrs, 1995 ). Knowledge-based systems developedfor therapeutic uses have been shown to engage the userand provide a cost-effective means of therapeutic interven-tion ( Binik et al., 1988 ; Velicer et al., 1993 ).

ARTICLE IN PRESS

N. Milton et al. / Int. J. Human-Computer Studies 64 (2006) 121412291216


4/16

2.2.6. ImplicationsEven this brief survey of psychotherapies shows the

variety of different methods and approaches that are used.Which of these should we adopt? We decided not to adoptany particular psychotherapeutic approach. Instead, wecircumvented the issue, by choosing to involve people with

no specic psychological problem. In this way, we coulduse a holistic approach in the manner of self-help and self-development approaches. Thus, we hypothesized that thesimple act of eliciting an individuals personal knowledgeand providing him/her with the resulting structuredrepresentations (e.g. concept maps) would be benecial;for example, in realizing new ideas, seeing things from anew perspective, making new connections, etc.

2.3. Personal knowledge and research psychology

Rather than aim to help people directly, many researchpsychologists focus on the development of new theories inareas such as perception, cognition, learning, decisionmaking and social interaction. How do such researcherstreat personal knowledge? This depends on the researchmethod being used. The methods generally fall into twomain groups: quantitative and qualitative. There has been along debate in psychology and the social sciences about therelative merits of these two approaches and some havesought to unite the two ( Robson, 2002 ; Todd et al., 2004 ).However, the vast majority of research still uses oneapproach or the other.

For the present purposes, there are three reasons forexamining the different methods that the quantitative and

qualitative approaches adopt: (i) the pragmatic reason of deciding which methods to use in our empirical study; (ii)the longer-term aim of providing new techniques forpsychological research; (iii) to promote methods used inthe social sciences for use within knowledge engineeringand the wider computer science community. For thesereasons, we take a brief look at the two approaches.

2.3.1. The quantitative approachThe quantitative approach takes its practices from the

natural sciences, favouring the testing of hypotheses toexamine small aspects of behaviour and experience. Theuse of controlled experiments, predened metrics andmultivariate statistics is commonplace. Personal knowledgeis acquired using observation and self-reporting techniques.Opponents of the quantitative approach argue that theresults lack validity due to the articial nature of theexperiments and that various biases are introduced by theexperimenter and experimental environment resulting inconclusions of questionable value ( Smith et al., 1995 ).

2.3.2. The qualitative approachThe qualitative approach takes its practices from

linguistics, sociology and the humanities ( Weinberg,2002 ). Researchers acknowledge the multi-faceted realityof experience, treating people as complex, self-reective

entities, inextricably bound to their social and culturalclimate. Personal knowledge is primarily acquired usinginterviews, although other methods are used such asrepertory grids ( Collett, 1979 ), concept ranking ( StaintonRogers, 1995 ) and role play ( Yardley-Matwiejcuzuk, 1997 ).Although qualitative approaches capture a rich form of

knowledge, opponents criticize the exible, non-rigorousmethods, the subjective interpretation of data and thehighly contextualized nature of the studies. In addition,they note that studies are very expensive in resources,particularly the time needed from highly skilled psychol-ogists and highly motivated participants ( Miles, 1979 ).

2.3.3. ImplicationsA knowledge engineering approach to personal knowl-

edge has many similarities with the qualitative approach topsychological research. Perhaps it is the case that theprinciples and techniques used in knowledge engineeringmay mitigate some of the problems with the qualitativeapproach. An attractive notion is to use techniques thatgather a rich form of structured knowledge from a largenumber of people. This may provide a crossover between aquantitative and qualitative approach. Use of elicitationtechniques that allow the participants to structure andcodify their own knowledge may diminish the problems of expensive resources and subjective interpretation of datalevelled at the qualitative approach. In addition, the use of an ontology of personal knowledge (sadly lacking in thepsychological literature) might be useful when acquiringknowledge and developing new theories.

2.4. Research questions

Having surveyed the use of personal knowledge withinpsychology, we are now in a position to identify the specicquestions to be addressed in the empirical study. Canknowledge elicitation techniques be used to capturepersonal knowledge? What sort of knowledge will becaptured? Can the process of eliciting personal knowledgefrom an individual help that individual to understand moreabout themselves? Can that increased understanding be of benet to the individual, psychotherapeutically or in someother way? Which techniques will be best at capturingpersonal knowledge and at facilitating more understand-ing? In what ways might knowledge elicitation techniquesbe used for psychological research and psychotherapy?What would a meta-model of personal knowledge look likeand how might it be used? To tackle these researchquestions, we undertook a large-scale empirical study.

3. Empirical study

3.1. Design considerations

The empirical study aimed to assess various knowledgeelicitation techniques with a number of participants actingas experts. The domain of their expertise is their own

ARTICLE IN PRESS



5/16

personal knowledge, including knowledge of their beha-viours, emotions, aspirations and life story.

We adopted a tried-and-trusted format of knowledgeacquisition session in which a knowledge engineer interactswith a single expert. The techniques to be assessed were amixture of paper-based methods and those that are

supported by knowledge acquisition software. For thelatter, we chose the PCPACK suite of knowledge tools,which has been used on many knowledge acquisitionprojects and in domains outside knowledge engineering(Shadbolt and Milton, 1999 ).

3.2. Selection of elicitation techniques

A variety of techniques have been developed for eliciting,mapping and validating knowledge from subject matterexperts. A number of authors have provided reviews andcomparisons (e.g. Boose, 1989 ; Cooke, 1994 ; Hoffman etal., 1995 ; Shadbolt and Burton, 1995 ). For example,Shadbolt and Burton (1995) provide a taxonomy of techniques with two high-level classes: non-contrived (akanatural) techniques and contrived techniques. Non-con-trived techniques include interviews (structured, semi-structured and unstructured) and protocol analysis techni-ques ( Belkin et al., 1987 ). Contrived techniques includemapping techniques, such as concept mapping ( Zaff et al.,1993 ) and repertory grid ( Gaines and Shaw, 1993 ), andgoal decomposition techniques, such as laddering ( Cor-bridge et al., 1994 ) and limited information tasks ( Grover,1983 ).

For the current study, it was decided that eight

techniques would be assessed. Seven of these were selectedfrom a range of elicitation techniques assessed during apilot study. An eighth technique was added as a controltechnique to act as a benchmark against which to comparethe others.

The decision of which elicitation techniques to select wasbased on a number of criteria. First, each technique shouldbe capable of acquiring knowledge, preferably structuredknowledge. Second, each technique should be able to befully explained and implemented with a participant in asession lasting approximately 1 hour. Third, there shouldbe at least one technique that can acquire knowledge fromthe different types of personal knowledge, e.g. knowledgeof social entities, attitudes, behaviours, emotions, goals,events and attributes. Fourth, each technique should haveperformed well during the pilot study (that involved 20knowledge acquisition sessions in which various techniqueswere tested and developed). The following paragraphsdescribe the techniques that were selected based on thesecriteria.

A semi-structured interview technique was selectedbecause it is representative of a questioning techniqueand can acquire a broad range of knowledge. The semi-structured interview is a standard technique used in manyknowledge engineering projects. It makes use of a pre-designed set of questions but allows unplanned supple-

mentary questions to be asked during the session. Thedesign of the interview included 12 questions partitionedinto three sections based on the structure of a psychologicalinterview developed by De Waele and Harre (1979) .

A repertory grid technique, named the people grid , wasselected because of its widespread use for acquiring

knowledge and its uses in psychology. The people gridtechnique is based on the construction of a repertory gridin which the elements are aspects of the participant andsignicant others. The constructs (attributes) are person-ality traits, which were selected from those described by theparticipant during the semi-structured interview. Thetechnique involves rating the elements for each of theconstructs on a numerical scale. These ratings are subjectedto a cluster analysis using special software to produce afocus grid. The nal stage involves reviewing and amendingthis grid.

A second repertory grid technique, named the events/ periods grid , was selected to capture knowledge of theparticipants life history by comparing different events andtime periods. Thus, the elements in the grid were events andtime periods in the participants life. The constructswere attributes of these events and time periods.These constructs were elicited using a triadic elicitationmethod.

A diagram-construction technique, named the statediagram technique , was selected because it is a structured,model-based technique that can elicit behaviours, moodsand emotions. This technique is based on the constructionof a state transition network; a diagram showing states asnodes linked by arrows representing transitions. The

participant selects states they regularly experience (e.g.emotions) which are represented as nodes. The transitionsare the ways in which the participant moves from state tostate (usually actions performed or external events). Thetechnique involves the construction of such a diagram fromscratch on a large sheet of paper. The technique concludeswith the participant assessing what percentage of time he/she spends in each state.

A combined laddering and concept-mapping technique,named the aspirations technique , was selected because it is astructured technique that can capture knowledge of decisions, goals and motivations. This technique has twostages. First, a decision tree is constructed based on a keydecision that the participant is considering or has recentlymade. The nodes on the tree are the possible courses of action, and the sub-nodes are advantages and disadvan-tages. There then follows a questioning procedure in whichthe participant is asked repeated why questions for thereasons behind the advantages and disadvantages. Asecond diagram, the aspirations diagram is then con-structed using the abstracted answers from the whyquestions as nodes on a concept map. Extra nodes areadded to show all aspirations and associatedconcepts. Links are added to the diagram to represent therelationships between the aspirations and associatedconcepts.

ARTICLE IN PRESS



6/16

A process mapping technique, named the eventdiagram technique , was selected because it is a structured,model-based technique that can elicit and capture event-based knowledge. The technique is based on the construc-tion of a set of diagrams similar to the process maps usedby knowledge engineers. There are three main types of

nodes on the diagrams: events, inputs and outputs. Theinputs represent the state of affairs before an event, andthe outputs show what the event has caused to change. Thetechnique involves the construction of a hierarchical set of these diagrams using special software.

An interview review technique was selected based on theuse of diaries for therapeutic means. It makes use of thetranscripts from the semi-structured interviews. Eachparticipant is supplied with a transcript of the semi-structured interview in which they took part some monthspreviously. The participant reads and reviews the tran-script. Their reactions are captured as the transcript is readby adding symbols and remarks to the text.

The Keirsey Temperament Sorter (Keirsey and Bates,1984 ) was selected as the control technique since it is astandard personality technique that captures knowledge of an individuals personality and provides information thataims to increase self-awareness. This technique is based onJungian psychoanalysis and is derived from the Myers Briggs type indicator, the most widely used technique forpersonal development in organizations ( Quenk, 2000 ).Answers to a set of 70 questions provide scores on fourscales (extrovertintrovert, judgementperception, thin-kingfeeling and sensingintuition). These scores allowthe participant to be categorized into one of 16 tempera-

ments that group into four temperament classes. Standar-dized descriptions of each temperament and class areprovided for the participant to read and reect upon.

3.3. Participants

The number of participants to involve was determinedby two opposing factors. First, having enough participantsto ensure a good spread of demographics and enoughassessment data for quantitative analyses to be applied.Second, having a small enough number to ensure that therewas enough time to assess each technique with eachparticipant and enough time for a full analysis of theresulting data. On this basis, it was decided to involve 18people: eight in the pilot study and ten in the main study.The participants in the pilot study were all post-graduateuniversity students. The participants in the main studywere all employees of a large engineering organization.These people were selected using a quasi-random proce-dure (i.e. random within the constraints imposed by thestudy) so that they represented a range of ages, educationalbackgrounds, work areas and job positions. In recruitingthe participants, we followed the normal procedures of apsychological study to ensure ethical standards wereobserved.

3.4. Assessment methods

There are many problems when evaluating knowledgeelicitation techniques ( Shadbolt et al., 1999 ). There is thedifculty of involving a large enough sample of experts togive statistical signicance. There is the difculty of

assessing and validating the knowledge acquired giventhe lack of gold standards to compare against. There isthe need to discover the range of tasks and domains forwhich the techniques are useful. There is also the difcultyof isolating what added value a technique gives when eachtechnique may need to work alongside other techniques.

A further difculty arises from the fact that knowledge isnot easy to quantify. Various measures can be used, such asthe number of rules, the number of frames and the numberof nodes in semantic networks. Since the techniques beingassessed here are inherently different in nature, suchmeasures cannot be used. Rather than concoct a knowl-edge-based measure that may be biased and invalid, it wasdecided to follow the assessment procedures used in manypsychological experiments, and allow the participants toprovide subjective judgements of the techniques. Hence,questionnaires and feedback interviews were used. Fourmeasures were designed: (i) a questionnaire containingopen-ended questions, (ii) a questionnaire containingrating scale items, (iii) a semi-structured feedback interviewand (iv) a ranking technique. These different methodsoverlap in what opinions they capture, hence provide a wayto assess the reliability and validity of the measures.

3.5. Method

All ten participants in the main study were seenseparately on eight occasions, over a period of about 8months, i.e. 80 separate knowledge acquisition sessionstook place. Each session lasted about 1 hour. The rstsession for all of the participants involved the semi-structured interview technique. Excluding the interviewreview technique, the other six techniques were randomlyassigned so that each participant assessed them in adifferent sequence. At the end of each session, theparticipant was given a questionnaire to complete. Thequestionnaire captured the participants opinions of thesession using open-ended questions and rating scales.

Following the seventh session with each participant, theinterview review technique was used. The eighth and nalsession with each participant (the feedback session)comprised a review of all the techniques and of the studyas a whole. This session used a semi-structured interviewformat and included a ranking of the techniques using acard sorting technique.

All the sessions were tape-recorded. The rst session (thesemi-structured interview) and the nal session (the feed-back session) were transcribed in full. The other sessionswere selectively transcribed.

Analysis involved both qualitative and quantitativemethods. Qualitative methods were used to analyse the

ARTICLE IN PRESS



7/16

content of the knowledge captured, the responses to theopen-ended questionnaire and the responses made at thefeedback session. Quantitative (statistical) methods wereused to analyse basic measures of each knowledge model(such as the number of nodes), ratings made on thequestionnaire and rankings made at the feedback session.

3.6. Results

This section contains the main results of the empiricalstudy. A full description of the results, and the study as awhole, can be found in Milton (2003) .

3.6.1. Assessment of each techniqueA brief summary of the results obtained for each of the

techniques is as follows.Semi structured interview : Responses to the interview

questions provided a large and rich source of personalknowledge, covering areas such as life events, self-descrip-tions and opinions of personal knowledge. The content of the responses also provided an efcient means of leadinginto the other techniques being assessed. A number of participants felt that it was useful to have the opportunityto talk about and examine things that they do not normallytalk about. The main problem identied by the participantswas a lack of clarity in some questions, which may beovercome by a more structured answering system andadvice on what knowledge can be captured.

People grid : A content analysis identied four classes of constructs: (i) ambition and strength, (ii) emotions andniceness, (iii) social skills and (iv) thinking skills. Partici-pants who liked the people grid did so because it wasthought-provoking and revealed similarities between peo-ple they had not previously realized. A statistical analysis

of the results of the questionnaire showed that the peoplegrid technique was rated more highly by women and byyounger participants. Generally, participants felt that themain problem with this technique was difculty inperforming the ratings.

Events/periods grid : A content analysis identied eightclasses of constructs that were used, the most commonbeing associated with feelings/emotions, moods/states anddecisions/outcomes. The technique was found to be theleast favoured by the participants as a whole. The mainreported reason for this was the failure to reveal anythingnew to many of the participants. A statistical analysis of the results of the questionnaire showed that the events/periods grid technique was rated more highly by womenand by participants without a university degree. The mainproblems identied by the participants were in the choiceof which events/periods to use and the generation of theconstructs. Using less important life decisions and usinganother technique to elicit the constructs may overcomethese problems.

State diagrams : Each participants diagram contained,on average, eight states and 22 links between states. Anexample of part of a state diagram is shown on Fig. 1 . The

ARTICLE IN PRESS

Analytical

Bored

Have success or Change in

circumstances

Somethingstimulating happens Come to terms with

success or change

No challenge or Become Ill

Bad eventhappens

Bad event goes awayor Something good

happens

Too much work or People wasting my

time

Challenge Occurs or Become well again

Have rest or Have a holiday

Overwork continues

Timepasses

Happy

Stressed

Tired

Sad

Too muchrest

Bad eventhappens

Think too much

Fig. 1. Part of a typical state diagram constructed with a participant.



8/16

most common states used by participants were happy,stressed and worried. The state diagram technique wasfound to be the most popular technique by the participants(as shown later). There were indications that thispopularity may result from the enlightening natureof this technique. Suggestions for possible improvements

included the use of standard lists of states and transitionsto select from, improving the presentation, and havingmore time.

Aspirations technique : Each participants decision treecontained, on average, four courses of action, each of which had about three advantages and three disadvantages.Each aspirations diagram contained about 27 nodes and 37links between the nodes. An example of part of a typicalaspirations diagram is shown in Fig. 2 . The most commontypes of nodes on the aspirations diagram were associatedwith interests, friends/others and job. A statistical analysisof the results of the questionnaire showed that theaspirations technique was rated more highly by men, olderparticipants and university graduates. Suggestions forpossible improvements included the need for more thanone decision to be mapped, and having more time.

Event diagrams : Each participant produced about fourdiagrams each containing, on average, six events and nineinputs/outputs. An example of one of these diagrams isshown in Fig. 3 . The most common types of events were

associated with starting a job, the break up of a marriageor serious relationship and birth/deaths. The most commontypes of inputs/outputs were associated with feelings,situation/context and interests. A statistical analysis of the results of the questionnaire showed that the eventsdiagram technique was rated more highly by older

participants. A number of participants found that thetechnique revealed new thoughts such as identifyingpatterns and connections. Suggestions for possible im-provements included the use of standard prompt questionsfrom the software and having more time.

Interview review : Analysis of the comments made by theparticipants on the interview transcript showed that about80% of statements were agreed with, and about 8% weresurprising to the participant. There was a polarization of opinions with some participants liking this technique andothers not. Those who liked the technique did so becausereading the transcript revealed how much they hadchanged in the intervening time and in what ways theyhad changed. A statistical analysis of the results of thequestionnaire showed that the interview review techniquewas rated more highly by participants with a universitydegree. The main problems identied by the participantswere the need for a longer period in between the interviewand the review, and for an improvement to the markingscheme, possibly making use of software.

ARTICLE IN PRESS

aspiration

happiness andcontentment

doing good forothers

self worth enjoyment

avoiding stress

and worries

self confidence holidays

promotion tobetter job

moneykeeping familycommitments

less timeRed Cross

charity work

Oxfam

is a

leads to

require leads to

leads to

part of

hinders is a

leads to

part of

hindersis a

leads to

increaseshinders

helps helps

helps

reduces

helps

Fig. 2. Part of a typical aspirations diagram constructed with a participant.



9/16

Keirsey technique : Most participants felt that thestandardized descriptions were surprisingly accurate. How-

ever, they felt that the technique was neither thought-provoking nor enlightening. Those who liked the techniquedid so because it was enjoyable and interesting. A statisticalanalysis of the questionnaire results showed that theKeirsey technique was rated more highly by youngerparticipants. The main problem identied by the partici-pants was difculty in answering the questions.

3.6.2. Comparison of the techniquesHow do the techniques directly compare to one another?

To establish this, two assessment measures were used. Bothwere based on the opinions of the participants as to theperceived quality of the technique to capture their knowl-edge and be of psychological benet. The rst measureused the results from a rating-scale questionnaire presenteddirectly after each technique. The key items on thequestionnaires were ratings of interesting , thought-provok-ing, enlightening and recommendation . During the analysis,these were combined to provide an overall measure. Thesecond measure was taken from rankings of the techniquesmade by each participant during the feedback session. Theanalysis demonstrated a strong correlation in the results of these two measures. This correlation was found to bestatistically signicant using a Pearsons product-momenttest ( r 0.902, N 8). Note, this correlation coefcientis negative because high ratings correspond with low

rankings. An illustration of the strength of this correlation

is shown in Fig. 4 , which is a scatter diagram of the averagerankings from the feedback session plotted against theaverage rankings of the ratings from the post-sessionquestionnaires. Note, that the axes are both rankings;hence techniques that are closer to the origin (i.e. bottomleft) are more preferred.

Further statistical tests (pair-wise comparisons using t-tests) provided a verication of the information presentedvisually in the graph above, i.e. that there are fourdivisions of techniques: (1) most favoured technique isthe state diagram technique; (2) the second most favouredtechniques are the interview, people grid, aspirationstechnique and event diagrams; (3) in the third division isthe interview review; (iv) the least favoured are the events/periods grid and Keirsey technique. These results werefurther supported by participants comments on the open-ended questionnaires and during the nal feedbacksessions.

A statistical analysis of the factors that lead to theparticipants rating and ranking of each technique pro-vided no conclusive results. It seems that a combination of factors is involved. There was some indication that thosetechniques that acquired a substantial, and important, partof a persons knowledge were those techniques that werepreferred as a whole. Attempts to assess what percentage of

knowledge each technique could acquire proved incon-clusive.

3.6.3. Benets of capturing personal knowledgeDoes the process of capturing an individuals personal

knowledge increase his/her personal knowledge? Feedbackfrom the participants indicated that some increase inpersonal knowledge does occur during an elicitationsession. The majority of the comments concerning newthoughts were of two types. First, many comments wereassociated with the thought-provoking nature of thetechniques, with phrases such as made me thinkabout y and made me consider y being commonly

ARTICLE IN PRESS

left school at 16 ('82)

joined apprenticeshipscheme ('82)

took ONC (84-86)

decided to go touniversity ('86)

wanted money had enough of school not ambitious

wanted to do sales more ambitious for education new circle of friends

got grades for university more confident

Fig. 3. Typical example of an event diagram constructed with aparticipant.

Fig. 4. Graph of the average rankings from the feedback session and thepost-session questionnaires.



10/16

used by the participants. Second, many comments wereassociated with an increase in personal knowledge, withphrases such as made me realize that y and made meunderstand y . Often a participant would see links orassociations between factors (such as people or events) thatthey had not previously realized.

3.6.4. Benets of increasing personal knowledgeThe results indicated that the techniques have the ability

to help people gain more understanding and have newideas. These can be seen in terms of the persons past,present and future.

In terms of the past, this can be (i) a better under-standing of how and why life events have occurred andhow they impacted on other events or circumstances; (ii)a better understanding of how one has changed anddeveloped over time; (iii) an improved feeling concern-ing past decisions and actions.In terms of the present, this can be (i) a betterunderstanding of the emotional states one gets intoand how one might deal with less desired states; (ii) abetter understanding of social relationships and howpeople are affected and behave with different people;(iii) a new, or renewed, perspective on life and re-assessment of what is important.In terms of the future, this can be (i) a betterunderstanding of ones wishes and aspirations; (ii) abetter understanding of the factors that help or hindergaining what one wants; (iii) a new, or renewed,motivation to take certain actions.

During the nal session, each participant was askedwhether any changes in thinking or behaviour had resultedfrom each of the previous sessions. Results showed thatabout one third of the sessions had provoked some changesin thinking or behaviour. These changes included thinkingabout issues raised during the session, changes in attitudesor understanding, increased understanding about pastevents, increased motivation for change and modicationof behaviours.

3.6.5. Benets of using knowledge representationsDo the results of capturing and representing personal

knowledge in an explicit format (e.g. transcripts, diagrams,grids) help the person? During the nal session, eachparticipant was asked for his/her ideas on possible uses forthe transcripts, diagrams and grids created during thesessions. Four main uses were identied:

1. Diary uses, such as reviewing the way one had thoughtand felt when creating the representations, seeing howmuch one has changed, and capturing ones lifestory.

2. Uses in decision making, such as capturing the decisionprocess (for later reection) and making better decisionsby learning from the past.

3. Uses in gaining a perspective on where one is (such asones emotional state), how one got there, and how onecould proceed.

4. Increased self-understanding, such as providing moreself-awareness, altering priorities, and identifying areasfor growth.

3.6.6. Summary of empirical studyThe results demonstrated that most of the techniques

showed promise at being used for psychological purposes.The overwhelming impression from the participants wasthat the techniques were interesting and were thought-provoking. Although it may have been expected that thesemi-structured interview and repertory grid techniquesshould do well, it was encouraging to see that the threediagram-based techniques, particularly the state diagramtechnique, all performed favourably. Each of the partici-pants felt that a number of the techniques were enlighten-ing and were capable of helping in various ways. Incomparison to the control technique, all but one of thetechniques gained better rankings and ratings. When theparticipants made comparisons to other techniques of asimilar nature, the techniques under assessment fared verywell. It was particularly encouraging to have two partici-pants who are HR (Human Resource) professionals statethat a number of the techniques could provide usefulsupplements to existing employee development tools.Future work could build upon this by involving profes-sional and practitioner psychologists as assessors of the

techniques and the gathered knowledge.

4. Meta-model of personal knowledge

This section describes a meta-model based on a contentanalysis of the knowledge captured during the empiricalstudy. A meta-model is a structured representation of knowledge for a domain that falls between a taxonomy andan ontology. As the name suggests, it is a model thatdescribes other models. In our case, it is a model thatdescribes three factors: (i) a set of high-level classes of knowledge objects; (ii) a taxonomy for each of the high-level classes and (iii) the relationships between the high-level classes of knowledge objects. It was the case a fewyears ago, that a meta-model (and indeed a simpletaxonomy) could be thought of as an ontology (e.g. Noyand Hafner, 1997 ). With a shift in work on ontologies toweb languages such as RDF and OWL, ontologies are nowgenerally viewed as computer codications of knowledge.

4.1. Purpose of the meta-model

There are a number of reasons for creating a meta-modelof personal knowledge. First, no comprehensive meta-model of personal knowledge currently exists in thepsychological literature. Second, the meta-model can help

ARTICLE IN PRESS



11/16

to make the acquisition of personal knowledge moreefcient, e.g. by providing templates, prompts and genericmodels. Third, the meta-model can provide a commonunderlying language for describing personal knowledge.Fourth, the meta-model acts as a step towards thedevelopment of an ontology of personal knowledge. Fifth,

the meta-model and subsequent ontology would providethe means for manipulating and analysing personalknowledge for research purposes.

4.2. Development of the meta-model

The meta-model was developed in three main steps. First, aninitial set of high-level knowledge objects was selected. Second,relationships were created between pairs of these knowledgeobjects. Both of these steps drew upon three sources of information: (i) an initial set of elements underlying the choiceof the techniques being assessed, (ii) a content analysis of the

knowledge captured during the study and (iii) ontologicalelements described in the background literature. The third stepadded lower-level classes of knowledge objects using materialfrom the study. In performing this step, the initial higher-levelclasses were modied to maintain consistency.

4.3. Description of the meta-model

4.3.1. Concept map of the meta-model Fig. 5 is a concept map showing the relationships

between the high-level classes in the meta-model (although

for clarity, some have been excluded). Note, that theknowledge objects shown in the diagram represent anindividuals personal knowledge, not the concepts them-selves. For example, the object event represents the personsknowledge of events, not the events themselves.

4.3.2. Classes of personal knowledgeThe meta-model also incorporates a taxonomy for each

of the high-level classes. Lack of space does not permit afull description of these taxonomies, so the followingparagraphs make some brief notes on the 20 high-levelclasses with some examples of lower-level classes andinstances.

Person : Knowledge of the characters in a persons life.Examples include the self, signicant others, admiredothers, and hypothetical others (e.g. the ideal me, me asothers see me, me in 5 years time), and stereotypicalothers (e.g. a good boss, a poor teacher, an ideal

parent).Life history : Knowledge of the biographical history of a

persons life. A life history is segmented into time periodsand made up of events.

Time period : Knowledge of the main phases thatconstitute a persons life history. Time periods are oftendened by two main factors: (1) the places resided in,educated in and worked in and (2) the jobs/roles occupied.

Event : Knowledge of the happenings that involveor affect one or more people. Types of events includesingle events, complex events (e.g. mum had lots of

ARTICLE IN PRESS

Time period

Decision

Aspiration

RelationshipCharacteristic

State

Behaviour

Job/Role

Resource

Ability

Place

Interest

Belief

IncidentPerson

Event Property

Event

Life History

Action

Transition

affects

happens to

is a

has

definesaffects

resides in

has

defines

makes

has

part of

affects

has

has affects

is a

has

part of affects

is ahas

performs

has

is a

has

affects

affects

is adefines

affects

has

affects

affects

alters

affects

Fig. 5. Concept map of the meta-model of personal knowledge.



12/16

miscarriages), generic events and hypothetical events.Types of events include incidents and actions.

Incident : Knowledge of the types of event that arebeyond the control of a person, such as accidents, illnesses,natural disasters, deaths, redundancies and socio-politicaloccurrences (e.g. wars, economic depressions).

Action : Knowledge of the types of events in which aperson is a main actor. Actions can be tangible (involvingsomething of the physical world) or intangible (e.g. acognitive action, such as reections and realizations).

Event property : Knowledge of the properties of events.These include the valence of the event, such as positive (e.g.I went on holiday) and negative (e.g. I was not allowed togo on holiday) and emotiveness (e.g. it was good, it wasdreadful).

State : Knowledge of the temporary characteristics that aperson can exist in. Examples include emotions (e.g. happy,sad, and excited), cognitive states (e.g. ambitious, creative),and physical states (e.g. tired, ill).

Transition : Knowledge of the ways in which a personsstate is altered. The three main types of transitions areactions, incidents and the passing of time.

Characteristic : Knowledge of the stable properties of aperson. Examples include a persons traits (e.g. sociability,intelligence, niceness) and physical characteristics.

Behaviour : Knowledge of the actions of a person that arerepeated and form patterns. Examples include particularresponses to particular situations (e.g. they did a lot forthe church, Im always doing things like that).

Decision : Knowledge of the choices made by a personwhen there is more than one course of action available.

Aspiration : Knowledge of the ambitions, goals, motiva-tions, wishes and desires of a person. Examples includebeing happy, doing good and looking after myfamily.

Resource : Knowledge of the economic, physical andcognitive capabilities possessed by, or available to, aperson. Examples include money, possessions, tools andtechniques.

Ability : Knowledge of the physical and cognitiveaptitudes and resources possessed by a person. Examplesinclude the capability of the person to perform certainactions in certain ways.

Belief : Knowledge of the ways in which a person thinksabout various aspects of their life. Examples includethoughts, ideas, opinions, attitudes (likes and dislikes),priorities, perspectives, expectations and second-orderbeliefs (i.e. beliefs about beliefs).

Interest : Knowledge of the domains and subjects withwhich a person is involved. Interests can be at work (e.g.occupation, subject areas, expertise) and outside work (e.g.pastimes, hobbies, arts, crafts, voluntary work, religion).

Place : Knowledge of the locations in which a personresides or visits for work or social occasions. Places are animportant factor in dening time periods.

Job/Role : Knowledge of the occupations and/or posi-tions held by a person. Jobs/roles exist both within work

and outside work in various organizations and socialsituations.

Relationship : Knowledge of the interactions of a personwith signicant others, such as family, friends and workcolleagues. Relationships include having beliefs concerningone another and inter-personal actions.

4.4. Testing the meta-model

As a test of the completeness of the meta-model, it wascompared to the elements that constitute various psycho-logical theories and models.

4.4.1. Decision making and behavioural modelsTable 1 shows a comparison of the classes in the meta-

model and those used in decision making and behaviouraltheories. The theories included in the table are subjective

expected utility theory ( Wright, 1984 ), conict theory of decision making ( Janis and Mann, 1977 ), theory of plannedbehaviour ( Ajzen and Madden, 1986 ), health belief model(Becker, 1979 ) and transtheoretical model of change(Prochaska and DiClemente, 1992 ).

4.4.2. Attitude, attribution and schema modelsTable 2 shows a comparison of the classes in the meta-

model and those used in some inuential theories andmodels of social psychology. The theories and modelsincluded in the table are the three-component model of attitudes ( Rosenberg and Hovland, 1960 ), an inuential

theory of attributions called correspondent inferencetheory ( Jones and Davis, 1965 ) and the main schemadeveloped for use in social cognition ( Fiske and Taylor,1991 ).

4.4.3. Psychological models comprizing many elementsAs a nal comparison, the meta-model was tested

against two approaches that contain a substantial numberof elements (see Table 3 ). The rst of these is theintentional action formulation used in descriptive psychol-ogy ( Ossorio, 1981 ). This is a parametric analysis of behaviour intended to identify and discriminate between

behaviours along ten dimensions. The second approach isthe Biographical Inventory of the autobiographical re-search method ( De Waele and Harre , 1979 ), which is a verylarge questionnaire used to capture knowledge of manyaspects of a persons life.

4.4.4. Validity of the meta-model As shown in Tables 13 , a high degree of congruence was

found between the meta-model and the elements frommany psychological theories and models. This indicatesthat the meta-model has a certain validity and complete-ness in modelling the high-level classes of personal knowl-edge that we all possess and use in our everyday life.

ARTICLE IN PRESS



13/16

5. The personal knowledge methodology

The empirical study and meta-model led us to design anarchitecture for a web-enabled system that provides a newmethodology for psychological research. We call this thepersonal knowledge methodology.

The personal knowledge methodology has two aims. Theprimary aim is as a research method to gather structuredpersonal knowledge for analysis and theory construction.A second aim, indicated by the results of our study, is as anintervention method providing direct psychological benetto its users. At the core of the methodology are knowledgeengineering techniques that involve the acquisition, repre-sentation, presentation, modelling and analysis of personalknowledge. These activities are underpinned by anontology of personal knowledge. The methodology is anevolving system that captures knowledge from a large

number of people then uses this to improve the techniquesbeing used. The basic architecture is shown in Fig. 6 .

The user interacts with an interface that includes a suiteof web-enabled elicitation techniques, similar to thoseassessed during the empirical study presented earlier. Thisinterface is supported by a generic ontology that allowsknowledge re-use and acts as a common language forcommunication between users. As knowledge is acquiredfrom the user, it adds into a domain ontology of personalknowledge.

The domain ontology, and other information gatheredfrom the user (such as their use and ratings of particulartechniques) are analysed either by psychologists directly orby web services that automatically analyse the knowledge.

The results of the analysis phase contribute to thedevelopment of new or revised theories. The increasedtheoretical understanding is used to update the body of

ARTICLE IN PRESS

Table 1Comparison of decision-making theories with classes from the meta-model

Theory Elements used in the theory Classes from the meta-model

Subjective expected utility theory Actions ActionsEvents IncidentsAssessment of the probability BeliefsExpected utility Beliefs

Conict theory of decision making Actions ActionsCoping patterns BehavioursPros/cons Beliefs

Theory of planned behaviour Beliefs BeliefsAttitudes BeliefsIntentions AspirationsBehaviour Behaviours

Health belief model Actions ActionsAssessments Beliefs

Transtheoretical model of change Stages of behavioural change Behaviours, actions and decisionsProcesses of change Actions

Table 2Comparison of attitude and attribution theories/models with the meta-model classes

Theory/model Elements used in the theory/model Classes from the meta-model

Three-component model of attitudes Stimuli Events and relationshipsAttitudes BeliefsAffect States, actions and behavioursCognition Actions and behavioursBehaviour Actions and behaviours

Correspondent inference theory Dispositions CharacteristicsIntention AspirationsKnowledge Beliefs and abilitiesAbility AbilitiesAction Actions

Schema from social cognition Person schema PeopleSelf-schema PeopleRole schema Jobs/rolesEvent schema EventsCausal schema Beliefs



14/16

applicable theories that drive the methodology. The bodyof applicable theories is a resource for the design and re-design of the interface tools and ontology. This nal stagein the cycle enables a tight loop of theorizing, intervention

and evaluation, such that new ideas and techniques can bequickly tested and assessed. The nal element of themethodology requires that the body of applicable theoriesconstantly draws from and updates the body of possibletheories that involve personal knowledge.

This methodology is a qualitative approach to psycho-

logical knowledge, but seeks to mitigate many of theproblems associated with qualitative methods in a numberof ways:

By involving large numbers of people.By acquiring structured knowledge.By applying rigorous theory-driven methods.By using semi-automated analysis methods.By adopting a tight loop of theorizing and assessment.

The methodology can achieve all this by combining themethods and principles of knowledge engineering with the

latest advances in ontological engineering. It is envisagedthat the methodology could be implemented using emer-ging semantic web technologies and services (such as RDF/OWL) and the forthcoming web-enabled version of PCPACK.

6. Summary and conclusions

6.1. Summary

The paper has presented work that examined a knowl-edge engineering approach to personal knowledge. To thisend, we undertook an empirical study that involved 18participants acting as experts of their personal knowledge.80 knowledge acquisition sessions took place in the mainstudy to assess eight knowledge elicitation techniques: asemi-structured interview, an interview review, a repertorygrid using people as elements, a repertory grid using events/periods as elements, a state diagram technique, an

ARTICLE IN PRESS

Table 3Comparison of classes in the meta-model with two substantial psycholo-gical methods

Theory/approach Elements used in thetheory/approach

Classes from themeta-model

Intentional actionformulation

Behaviour Behavioursintentional action ActionsIdentity PeopleWant AspirationsKnow Beliefs and statesKnow how Ability and statesPerformance Actions and

behavioursAchievement Actions and

resourcesIndividual difference CharacteristicsSignicance Event properties

Concepts from theautobiographicalresearch

The time course Life historySignicant others PeopleWishes AspirationsAttitudes BeliefsArtefacts ResourcesSocio-economicfactors

Incidents andresources

Norms BeliefsExpectations BeliefsRoles Jobs/rolesInstitutions Places

Interpretations BeliefsOccupations Jobs/rolesInterests InterestsLeisure activities InterestsGoals AspirationsAspirations AspirationsConicts Beliefs

Interface(Elicitation tools

and Ontology)

Body of possibletheories

Body of applicable

theories

Web pages

DomainOntology

New/Revisedtheories

User

Re-Designinterface & pages

AnalyseDomain

Ontology

Self-help

information Search

Cycle

Fig. 6. Basic architecture of the personal knowledge methodology.



15/16

aspirations mapping technique, a life-event mappingtechnique, and a personality technique. The latter wasincluded as a control technique with which to assess theother techniques for their psychotherapeutic value. Theparticipants assessed each of the techniques andprovided feedback of their opinions using a number of

assessment methods. The study produced the following keyndings:

A range of elicitation techniques taken from knowledgeengineering proved useful in capturing personal knowl-edge. Most of the techniques also prompted theparticipants to see new things about themselves. Mostof the techniques were considered to be better than thecontrol technique, a well-known personality testingtechnique. The more unusual elicitation techniques,based on the construction of network diagrams, werehighly rated.A technique based on the construction of a statediagram was clearly the most favoured by the partici-pants. Such a technique has its roots in AI and computerscience but has not previously been used in psycholo-gical research or practice. We believe that such atechnique would prove a valuable addition to themethods used by psychologists.While most people responded well to the state diagramtechnique, other techniques (notably the interviewreview) were liked by some participants but not byothers. Although there was some indication that thismay be due to demographics (such as gender, age and

personality type), more research is required to examinethis.The use of multiple assessment methods (rating-scalesquestionnaires, open-ended questionnaires, feedbackinterview and ranking) proved successful in gainingopinions from the participants. Consistency was foundbetween the different measures, suggesting both relia-bility and validity. It is unusual to employ suchmeasures when assessing knowledge elicitation techni-ques, but this proved valuable in this context and maydo so more generally.A meta-model was developed based on a contentanalysis of the knowledge elicited from the participants.This meta-model showed consistency with a number of leading psychological approaches, indicating a degree of completeness and discrimination.

The empirical study and meta-model led to the design of an architecture for a web-enabled system that provides anew methodology for psychological research. By incorpor-ating the latest principles and technologies of knowledgeengineering, this methodology could provide a tool forpsychologists that might overcome some of the problemswith existing techniques, from a theoretical and practicalperspective.

7. Conclusions

We conclude from our study that knowledge engineeringhas much to offer the world of psychology. We alsoconclude that the empirical methods used by psychologistsand other social scientists have much to offer to the world

of knowledge engineering. We believe that the name of this journal, that includes the words human and computerseparated by a mere hyphen, should not only mean thatboth be considered when they meet directly (e.g. work onhuman factors) but also in more distant elds. We suggestthat there should be more of the human involved incomputer research (e.g. in the development of ontologies)and that there should be more of the computer involved inhuman research (e.g. in the development of psychologicaltheories).

Acknowledgements

We give many thanks to the participants involved in thestudies, for their honesty, openness and ideas. We also givethanks to employees of Rolls-Royce plc, especially PaulRiley, who helped organize the main study. We thank thedirectors of Epistemics ( www.epistemics.co.uk ) for theirpermission to use PCPACK. The personal knowledgemethodology was rst outlined at the InternationalConference on Critical and Qualitative Approaches toHealth Psychology, St Johns, Newfoundland, Canada,1999.

References

Ajzen, I., Madden, T.J., 1986. Prediction of goal directed behavior:attitudes, intentions and perceived behavioral control. Journal of Experimental Social Psychology 22t, 453474.

Beail, M., 1985. Repertory Grid Technique and Personal Concepts:Clinical and Educational Settings. Croom Helm, London.

Beck, A., Rush, J., Shaw, B., Emery, G., 1980. Cognitive Therapy of Depression. Guilford, New York.

Becker, M.H., 1979. Understanding patient compliance. In: Cohen, S.J.(Ed.), New Directions in Patient Compliance. Heath, Lexington, MA.

Belkin, N.J., Brooks, H.M., Daniels, P.J., 1987. Knowledge elicitationusing discourse analysis. International Journal of ManMachineStudies 27 (2), 127144.

Binik, Y., Servanschreiber, D., Freiwald, S., Hall, K., 1988. Intelligentcomputer-based assessment and psychotherapyan expert system for

sexual dysfunction. Journal of Nervous and Mental Disease 176,387400.

Bloom, B.L., 1992. Computer-assisted psychological interventionareview and commentary. Clinical Psychology Review 12 (2), 169197.

Bohart, A.C., Tallman, K., 1996. The active clienttherapy as self-help.Journal of Humanistic Psychology 36 (3), 730.

Boose, J.H., 1989. A survey of knowledge acquisition techniques andtools. Knowledge Acquisition 1, 3958.

Collett, P., 1979. The repertory grid in psychological research. In:Ginsberg, G. (Ed.), Emerging Strategies in Social PsychologicalResearch. Wiley, Chichester.

Cooke, N.J., 1994. Varieties of knowledge elicitation techniques. Interna-tional Journal of HumanComputer Studies 41 (6), 801849.

Corbridge, C., Rugg, G., Major, N., Shadbolt, N., Burton, A.M., 1994.Laddering: technique and tool use in knowledge acquisition. Journal of Knowledge Acquisition 6, 315341.

ARTICLE IN PRESS

http://www.epistemics.co.uk/http://www.epistemics.co.uk/


16/16

CYC Website: / http://www.cyc.com S (valid URL in August 2006).De Waele, J.P., Harre , R., 1979. Autobiography as a psychological

method. In: Ginsberg, G. (Ed.), Emerging Strategies in SocialPsychological Research. Wiley, Chichester.

Epistemics website: / http://www.epistemics.co.uk S (valid URL inAugust 2006).

Fiske, S.T., Taylor, S.E., 1991. Social Cognition, second ed. McGraw-

Hill, New York.Fransella, F., Bannister, D., 1977. A Manual for Repertory GridTechnique. Academic Press, London.

Gaines, B.R., Shaw, M.L.G., 1993. Basing knowledge acquisition tools inpersonal construct psychology. Knowledge Engineering Review 8 (1),4985.

Gould, R.A., Clum, G.A., 1993. A meta-analysis of self-help treatmentapproaches. Clinical Psychology Review 13, 169186.

Grover, M.D., 1983. A pragmatic knowledge acquisition methodology. In:Bundy, A. (Ed.), IJCAI-83: Proceedings of the Eighth InternationalJoint Conference on Articial Intelligence. Kaufmann, Los Altos, CA.

Hammersley, M., Shadbolt, N.R., Golightly, D., Cottam, H., Riley, P.,1999. Knowledge engineering for web based knowledge management:a case study from Rolls Royce Plc. Knowledge Management 12(October 1999).

Hoffman, R., Shadbolt, N.R., Burton, A.M., Klein, G., 1995. Elicitingknowledge from experts: a methodological analysis. OrganizationalBehavior and Decision Processes 62, 129158.

Janis, I.L., Mann, L., 1977. Decision Making. Free Press, New York.Jones, E.E., Davis, K.E., 1965. From acts to dispositions: to attribution

process in person perception. In: Berkowitz, L. (Ed.), Advances inExperimental Social Psychology, vol. 2. Academic Press, New York.

Keirsey, D., Bates, M., 1984. Please Understand Me: Character andTemperament Types. Prometheus Nemesis, Del Mar, CA.

Kelly, G.A., 1955. The Psychology of Personal Constructs. Norton, NewYork.

Marrs, R.W., 1995. A metaanalysis of bibliotherapy studies. AmericanJournal of Community Psychology 23 (6), 843870.

Maslow, A.H., 1968. Towards a Psychology of Being. Van Nostrand,Princeton, NJ.

Meichenbaum, D., 1997. Cognitive behaviour therapy. In: Baum, A., et al.(Eds.), Cambridge Handbook of Psychology, Health and Medicine.Cambridge University Press, Cambridge.

Miles, M.B., 1979. Qualitative data as an attractive nuisance: the problemof analysis. Administrative Science Quarterly 24, 590601.

Milton, N., 2003. Personal Knowledge Techniques. Ph.D. Thesis. Schoolof Psychology, University of Nottingham, UK.

Milton, N., Shadbolt, N., Cottam, H., Hammersley, M., 1999. Towards aknowledge technology for knowledge management. InternationalJournal of HumanComputer Studies 51 (3), 615641.

Murdoch, D., Barker, P., 1991. Basic Behaviour Therapy. BlackwellScientic, Oxford.

Noy, N.F., Hafner, C.D., 1997. The state of the art in ontology. AIMagazine 18 (3), 5374.

Ossorio, P.G., 1981. Outline of descriptive psychology for personality

theory and clinical applications. In: Davis, K.E. (Ed.), Advances inDescriptive Psychology. JAI Press, Greenwich, CT.

PCP Website: / http://www.repgrid.com/pcp S (valid URL in August2006).

Prochaska, J.O., DiClemente, C.C., 1992. Stages of change in themodication of problem behavior. In: Hersen, M., Eisler, R., Miller,P.M. (Eds.), Progress in Behavior Modication, vol. 28. SycamorePublishing Company, Sycamore, IL.

Prote ge Website: / http://protege.stanford.edu/ S (valid URL in August2006).

Quenk, N.L., 2000. Essentials of MyersBriggs type indicator assessment.Wiley, New York.

Robson, C., 2002. Real World Research: A Resource for Social Scientistsand Practitioner-researchers, second ed. Blackwell, Oxford.

Rogers, C.R., 1951. Client-Centred Therapy. Constable, London.

Rosenberg, M.J., Hovland, C.I., 1960. Cognitive, affective and behavioralcomponents of attitudes. In: Hovland, C.I., Rosenberg, M.J. (Eds.),Attitude Organization and Change. Yale University Press, NewHaven.

Rycroft, C., 1995. A Critical Dictionary of Psychoanalysis, second ed.Penguin Books, New York.

Schreiber, A.T.H., Akkermans, J.M., Anjewierden, A., De Hoog, R.,Shadbolt, N., Van de Velde, W., Wielinga, B., 2000. KnowledgeEngineering and Management: The CommonKADS Methodology.MIT Press, Cambridge, MA.

Shadbolt, N.R., Burton, A.M., 1995. Knowledge elicitation: a systematicapproach. In: Wilson, J.R., Corlett, E. (Eds.), Evaluation of HumanWork: A Practical Ergonomics Methodology, second and revised ed.Taylor & Francis, London, p. 2nd.

Shadbolt, N.R., Milton, N.R., 1999. From knowledge engineering to

knowledge management. British Journal of Management 10,309322.Shadbolt, N.R., OHara, K., Crow, L.R., 1999. The experimental

evaluation of knowledge acquisition techniques and methods: history,problems and new directions. International Journal of HumanCom-puter Studies 51 (4), 729755.

Shaw, M.L.G., 1980. On Becoming a Personal Scientist: InteractiveComputer Elicitation of Personal Models of the World. AcademicPress, London.

Smith, J.A., 1990. Transforming identities: a repertory grid case-study.British Journal of Medical Psychology 63, 239253.

Smith, J.A., Harre , R., Van Langenhove, L. (Eds.), 1995. RethinkingMethods in Psychology. Sage, London.

Stainton Rogers, R., 1995. Q Methodology. In: Smith, J.A., Harre , R.,Langenhove, L.V. (Eds.), Rethinking Methods in Psychology. Sage,

London.Stokes, M. (Ed.), 2001. Managing Engineering Knowledge: MOKA

Methodology for Knowledge-Based Engineering Applications. ASMEPress, New York.

Studer, R., Benjamins, V.R., Fensel, D., 1998. Knowledge engineering,principles and methods. Data and Knowledge Engineering 25,161197.

Todd, Z., Nerlich, B., Mckeown, S., Clarke, D.D. (Eds.), 2004. MixingMethods in Psychology: The Integration of Qualitative andQuantitative Methods in Theory and Practice. Routledge,London.

Velicer, W., Prochaska, J., Bellis, J., DiClemente, C., Rossi, J., Fava, J.,Steiger, J., 1993. An expert system intervention for smoking cessation.Addictive Behaviors 18, 269290.

Weinberg, D., 2002. Qualitative Research Methods. Blackwell, Oxford.

Wood, L., 1988. Self-help buying trends. Publishers Weekly, p. 33.Wright, G., 1984. Behavioural Decision Theory. Penguin Books,

Harmondsworth.Yardley-Matwiejcuzuk, K.M., 1997. Role PlayTheory and Practice.

Sage, London.Zaff, B., Mcneese, M., Snyder, D., 1993. Capturing multiple perspectives:

a user-centred approach to knowledge and design acquisition.Knowledge Acquisition 5, 79116.

ARTICLE IN PRESS

http://www.cyc.com/http://www.epistemics.co.uk/http://www.repgrid.com/pcphttp://protege.stanford.edu/http://protege.stanford.edu/http://www.repgrid.com/pcphttp://www.epistemics.co.uk/http://www.cyc.com/

knowledge engineering and psychology

Documents