1995_solo_assess learning in higher edu

8/3/2019 1995_SOLO_Assess Learning in Higher Edu

1/13

This article was downloaded by: [National Institute of Education]On: 11 November 2011, At: 07:48Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Higher Education Research &DevelopmentPublication details, including instructions for authors and

subscription information:

http://www.tandfonline.com/loi/cher20

The SOLO Taxonomy as a Means of

Shaping and Assessing Learning in

Higher EducationGillian M. BoultonLewis

a

aQueensland University of Technology

Available online: 01 Nov 2006

To cite this article: Gillian M. BoultonLewis (1995): The SOLO Taxonomy as a Means

of Shaping and Assessing Learning in Higher Education, Higher Education Research &Development, 14:2, 143-154

To link to this article: http://dx.doi.org/10.1080/0729436950140201

PLEASE SCROLL DOWN FOR ARTICLE

Full terms and conditions of use: http://www.tandfonline.com/page/terms-and-

conditions

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expresslyforbidden.

The publisher does not give any warranty express or implied or make anyrepresentation that the contents will be complete or accurate or up to date. Theaccuracy of any instructions, formulae, and drug doses should be independently

verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand, or costs or damages whatsoever orhowsoever caused arising directly or indirectly in connection with or arising out ofthe use of this material.
http://www.tandfonline.com/page/terms-and-conditionshttp://www.tandfonline.com/page/terms-and-conditionshttp://dx.doi.org/10.1080/0729436950140201http://www.tandfonline.com/loi/cher20


2/13

Higher Education Research and Developm ent, Vol 14, No 2, 1995 143

The SOLO Taxonomy as a Means of Shaping andAssessing Learning in Higher Education

Gillian M. Boulton-LewisQueensland University of TechnologyA B S T R A C T - This paper is a discussion of the use of the SOLO (Structure of ObservedLearning Outcomes) Taxonomy (Biggs & Collis, 1982, 1989; Biggs, 1991, 1992a, 1992b;Boulton-Lewis, 1992, 1994) as a means of developing and assessing higher order thinking inHigher E ducation. It includes a summ ary of the research into its use to date as an instrument tofind out what students know and believe about their own learning, to assess enteringknowledge in a discipline, to present examples of structural organization of knowledge in adiscipline, to provide m odels of levels of desired learning outcome s, and in particular to assesslearning outcomes. A proposal is made for further research.

LEA R N IN G A N D TEA C H IN G IN H IG H ER E D U C A TIO NBarnett (1990) argues that:

an educational process can be termed higher education when the student is carried on to levelsof reasoning which make possible cri t ical reflection on his or her experiences, whetherconsisting of proposit ional knowledge or of knowledge through action. These levels ofreasoning and reflection are "higher", because they enable the student to take a view (fromabove, as it were) of what has been learned. Simply, "higher education" resides in the higher-order states of mind. (p202).

The belief about higher education supporting this paper is that it should lead to higher orderthinking for most graduates. The discussion rests on the assumption that lecturers in highereducation should intend that students develop deep knowledge in their chosen disciplines, ifnecessary change their conceptions and their world view, and learn to think critically. Thisimplies that lecturers should teach to facilitate such outcomes (cf Samuelowicz & Bain, 1992;Prosser, Trigwell & Taylor, 1994). The use of adaptations of the SOLO Taxonomy as oneway of influencing and assessing learning outcomes to facilitate higher order thinking isdescribed.Higher order thinkingOne of the goals of higher education is to teach students so that they develop a soundkno wled ge of the content and structure of their chosen disciplines and can app ly it effectivelyin further development of knowledge and in vocational situations. This should producegraduates who can be involved in rational decision making and leadership roles in society.Such behaviour requires critical thinking as Norris (1989) defined it. Critical thinkers are

Downloadedby[NationalInstituteofEducation]at07:4811No

vember2011


3/13

144 The SOL O Taxonomy as a Means of Shaping and AssessingLearning in H igher Education

those who seek reason s, attempt to be well informed, use and acknow ledge credible sources,consider alternatives and other points of view, withhold judgement until they have sufficientevidence and seek to be as precise as possible. In order to think in such a manner it isnecessary to possess requisite declarative, procedural and theoretical know ledge, at least, asdescribed below and to value critical thinking. Possession also of conditional andmetatheoretical knowledge would enhance the process.

Declarative knowledge consists of factual knowledge of a discipline, represented insymbols, and the way in which it is structured for retrieval. Procedural knowledge is thatwhich allows the purposeful manipulation of declarative knowledge to undertake a task, solveproblem s, make decisions, understand and so o n. It also includes som e generalizable cognitiveskills (Lehman, Lem pert & Nisbett, 1988). Conditional knowledge allows one to know whento use certain procedures and access appropriate content. It is 'know ing how and w hy' (Biggs,1992a). Biggs (1992a) also describes theoretical and m etatheoretical know ledge, which are a thigher levels of abstraction than declarative know ledge. He asserts that in higher education wehave aimed at teaching theoretical and metatheoretical know ledge and h ave found difficulty inteaching procedural and conditional know ledge effectively.

Critical thinking as described rests on deep rather than surface learning o utcom es and hencedeep approaches to learning (cf. Marton & Saljo, 1976a, 1976b; Marton & Saljo, 1984). Adeep approach to learning is one in which the student intends to understand the material, torelate parts to a whole, to integrate it with existing knowledge and to apply it in real worldsituations. Critical thinking also assumes qualitative conceptions of learning (cf. Marton,Da ll'Alba & Beaty, 1993) which involve understanding, seeing something in a different wayand ultimately perhaps chang ing as a p erson. These three qualitative conceptions subsum e andbuild on quantitative conceptions of learning which Marton et al. described 'as increasingon e's kno wledg e, memorising and reproducing and ap plyin g'. In order to think critically in adiscipline it is necessary to acquire knowledge and be able to memorize and apply it. One thenneeds to understand the material and to evaluate, challenge and perhaps extend it to a new area.Determining the conceptions that students hold about learning is important because theyshape the approach that a student takes to learning in most situations (cf. Prosser & Millar,1989) provided that the context of learning is congruent w ith a particular conception (cf. Dart,1994).There is a great deal of evidence that most students graduate from Universities with littlebut surface declarative knowledge of their disciplines and that they do not learn to think likeexperts in their areas of study (Ramsden 1988). In short, their conceptions of learning areprobably quan titative and so are the learning outc om es. They acqu ire information fromlectures, and texts and may be ab le to apply it routinely, but do n ot necessarily understand itnor become critical thinkers. Such outcomes may be not only a result of students' lack ofknowledge about learning or lack of motivation. They could also be the result of lecturers'know ledge of learning (or lack of it), expectations, strategies, and course organization.

Download

edby[NationalInstituteofEducation]at07:4811No

vember2011


4/13

Higher Education Research and Development, Vol 14, No 2, 1995 145

T H E S O L O T A X O N O M YThe SOLO Taxonomy developed by Biggs and Collis (Biggs & Collis, 1982; 1989) is ataxonomy of the structure of learning outcomes for which SOLO is an acronym. It is notnecessarily content specific and provides descriptions of the structural organization ofkno wled ge at increasingly mo re comp lex levels across mod es of learning. The y describe fivelevels of the structure of observed learning outcomes. These range, on the basis of thestructural organization of the knowledge in question, from incompetence to expertise inhierarchical order as follows; prestructural (incom petence, nothing is know n ab out the area),unistructural (one relevant aspect is known); multistructural (several relevant independentaspects are kno wn ), relational (aspects of know ledge are integrated into a structure), extendedabstract (knowledge is generalised to a new domain). As a person develops, and learns moreabou t a discipline, levels of structural organization of know ledge recur in a cy clical fashion,for increasingly more formal modes of learning, from sensori-motor through iconic andconcrete-symbolic to formal-1 and then formal-2 (later described as postformal) modes ofknow ing. Biggs & Collis (1989) and Biggs (1991) propose that the concrete symbolic m odewa s typical of most secondary school learning, that the formal-1 (formal) m ode is typical ofearly university learning, and that the formal-2 (or postformal) mode should be achieved inpostgradu ate study. The formal m ode of learning entails declarative, theoretical and proceduralknowledge and the postformal mode, in addition, entails metatheoretical and conditionalkno wledg e. These formal m odes of learning, particularly those at the postformal level, shouldunderpin the ability to think critically in a discipline area.

Biggs and C ollis (1982) provide examples of SOLO levels for a range of secondary schoolsubject areas. There are also models of levels of outcomes in some discipline areas for tertiaryeducation and these are described later in this paper. Further examples can be generated orderived from stude nts' w ork. In summ ary, students in und ergraduate study at the tertiary levelshould develop knowledge in their discipline areas that is organized structurally at therelational or extended abstract level in the formal mode and in postgraduate study at thepostformal m ode (Biggs & Co llis, 1989; Bigg s, 199 1; Big gs, 1992a).THE 3P MODEL OF LEARNINGBiggs (1993) presents a 3P m odel of learning (developed in earlier publications). Th e pu rposeof discussing it here is to relate the use of the SO LO T axonom y to the mo del. It is suggestedthat the SO LO T axono my as a tool can b e utilized in each of the 3Ps of the m odel, that is inPresag e, Process and P roduct. In the 3P m odel Presage includes student factors such as p riorknowledge, abilities, ways of learning, value and expectations, and teaching factors such ascurriculum, teaching m ethod, climate and assessment (B iggs, 1 993). W ith relation to Presagethe SOL O Taxon omy can be useful as a tool in determining stude nts' prior know ledge of theirown learning and of content and its structural organization in a discipline. It can also b e usedas part of the teaching context to mod el desired learning outcom es. In the Process part of themodel, use of SOLO as described above, explicitly attempts to shape the Product shouldfacilitate effective approaches to learning. Most importantly the Taxonomy can be usedeffectively in the Product part of the mo del for assessment, where the intention is to measu re


vember2011


5/13

146 The SOLO Taxonomy as a Means of Shaping and AssessingLearning in H igher Education

both understanding and organization of content. In summary it can be used to influence allparts of the 3P model and in particular to assess the structure of the outcomes of teaching andlearning. This paper is an examination of its application to the Presage and Product aspects ofthe model.TEACHING, LEARNING AND THE SOLO TAXONOMYIf it is intended that students acquire qualitative conceptions of learning and adopt approachesthat will lead to critical thinking then teaching and assessment must be directed to that end. Inorder to do this it is important to determine student presage factors. In particular it isimportant to understand what students know and believe about their own learning, the level ofknowledge in a discipline that students possess when they enter a course and to take arelational view of learning (Ramsden, 1988). If the discipline is new to students one canassume that they will know very little in a formal sense and will need to need to becomefamiliar first with jargon and factual details or, in other word s, requisite declarative k now ledgeand basic procedural and theoretical knowledge before they can engage in higher order learningrequiring conditional and metatheoretical knowledge.

M cKeach ie, Pintrich, Lin & Smith (1990) cite research which indicates that if m aterial ispresented in a structured way it helps students to organize course content. Entwistle andEntw istle (1992) found that students appreciate structure in material presented by lecturers butsuggest that, paradoxically, too much structure may relieve students of the need to derive theirown personally intelligible forms of a discipline. Probably the less students know about adiscipline the more structure they need initially. Examples of learning outcomes at the desiredlevel, that is in the relational or extended levels in the formal m ode could be used to p rovidestudents with some structure and as a guide to the level of organization that they should aimto achieve.

A search of the literature provides examples of the use of SOL O in tertiary teaching for arange of Presage purposes. They include: a means of shaping learning; a way of determiningwhat students know and believe about their own learning; and a way of determining studen ts'knowledge of a discipline at the time they enter.Boulton-Lewis (1992; 1994; Boulton-Lewis, Wilss & Mutch, in press) proposed, as amodification and extension to the SOLO Taxono my, examp les of levels, in the formal mo de,that could be used effectively as models to assess and describe students entering, and changing ,knowledge of their own learning as they progress through courses in higher education. Anillustrative example of a SOLO response at each level is given in Table 1 (Boulton-Lewis,1994). Responses at each level ranged from weak to strong, depending on the inclusion ofrelevant information about the learning process, as well as on the structure. Except for theexample at the prestructural level, which may reflect lack of interest, the examples givenrepresent strong responses at each level in the formal mode .


vember2011


6/13


Tab le 1: Exam ples of Resp onses at Each Level of the SOL O Taxo nom yfor Knowledge of Learning.

PrestructuralNo idea.UnistructuralReal learning is what you remember, that is important values and lessons, even information you remember fromschooling in the years after it. That is how I have found it. You remember it and in your later years it is amazingthe data you can recall.MultistructuralLearning is to have real understanding about a particular subject whether through actual experience or throughother sources such as text books etc.

This belief was probably acquired through the conservative thinking of the education system I w as brought upwith. I know that I can learn easily and quickly if there is a 'teacher who teaches in a methodological plannedway; where I can see the plan and know where I am heading. I usually go about learning by reading, memorising,by applying to my actual experiences or knowing about other peoples' experiences. By applying it to reality itbecomes simpler. Other peoples' espoused opinions and beliefs and values influence how I learn, and som etimesmake it difficult to learn. By this I mean , I sometimes want to hold my own 'opinion but other people try to sway meto their belief. I know that I have learnt something when I feel satisfied. I also know when I use the knowledgeyears later.RelationalLearning involves the sharing of knowledge to facilitate personal growth and a greater understanding of the worldaround me. This perception of learning is based entirely on my own subjective value judgements and could not beconsidered a view that I have acquired within the education system which in the most part, focuses on a relativelyutilitarian approach, i.e., producing skilled people for the work force. Learning for me is facilitated by my beingable to gain further insight into why I am here, how I fit into the society I live in, and why certain attitudes andbelief systems exist in that society. I know I have leamt something when I reach a greater understanding of myselfas a person and the complex inter-relationships that mould the society in which I live.Extended A bstractFirst of all, learning for me is a body of information that's there to be acquired. But I don't think that body ofinformation should be taken in and just regurgitated to others. So often in our society people think learning is abouthow "well" or "good" you can regurgitate it. I believe in the synthesis of information and one's own lifeexperience, that is, applying your own experience to the information that you learn, and looking for the sense orreasoning contained within that melting pot of "experience and information". This is the only way we make senseof our w orld. In order to feel confident in the world w e live in we need to have understanding and know ledge ofourselves, others, and the things around us. If we know how something works and understand it, it gives usconfidence which I believe is a virtue. However, it only becomes really virtuous w hen w e use that know ledge andunderstanding for the good of mankind. And this is where moral implications come into play. Knowledge andunderstanding of something involved with law and order, or more specifically "justice", is what constitutes "goodlearning".

Download


vember2011


7/13

148 The SOLO Taxonomy as a M eans of Shaping and AssessingLearning in Higher E ducation

Analysis of SOLO levels by motives and strategies as measured by the Study ProcessQuestionnaire (SPQ) (Biggs, 1987) showed that as the structural organization of knowledge oflearning increased then the concern with surface m otives declined and deep m otives and de epstrategies assumed more importance (Boulton-Lewis, 1994).Boulton-Lewis, Wilss & M utch (in press) asked qualified, experienced teachers wh o w ereundertaking an in-service subject in adult learning to write statements abou t learning. Eightypercent of these students gave responses at the m ultistructural level. This indicated that theyhad knowledge of many aspects of learning but could not explain it in an integrated andcoherent way that would allow them to apply the knowledge flexibly to their own and oth ers'learning. It was notable from a content analysis of these responses that very little mentionwas m ade of factors typical of adult learning. This suggested that these teachers needed to learnmo re about their own learning in order to becom e the independent self-directed learners thatstatements about adult learning would lead one to expect. The conclusion was that teachingneeded to be organized to facilitate such behav iour rather than to assum e it existed.

Boulton-Lewis (1992; Boulton-Lewis & Dart, 1994) used discussion of SOLO levels andexam ples, as part of the teaching process, in an attempt to shape stud ents' learning outcomesin courses in adult learning. Students, who in each case were experienced educators wereintroduced to the SO LO Taxo nom y, discussed it as part of the content of the course and w eretold that their work wo uld be assessed for conten t and structure. Th e results in terms ofimproving students' structural organization of knowledge were a little disappointing in bothstudies. The majority of students were, and remained, multistructural at the formal levelalthough they did improve in terms of knowledge within levels in most cases. These resultsprobably did not match expectations because the approach was confined to one subject in a 13week semester. Dart (1994) subsequently reported results from a study where significantchanges in SOLO levels, inter alia, were brought about by improved learning activities.A S S E S S M E N T A N D S O L OOne important purpose of assessment is to determine whether teaching is accomplishing itsgoals for student learning in terms of the objectives and principles of instruction. If theintention is that students become critical thinkers it is necessary to assess the level ofstructural organization as well as content of the discipline knowledg e to measure such learningoutcomes. Nickerson (1989) asserts that a major criticism of tests constructed for the purposesof determining student grades is that they tend to emphasize recall of declarative or proceduralknow ledge and prov ide little indication of either the level at which students understand subjectmatter or of the quality of their thinking (Fleming & Chambers, 1983; Morgenstern &Renner 1984) (p3). He goes on to say that if the major goal of education is higher-ordercognitive functioning then it is regrettable that we lack adequate tools for assessing suchfunctioning. Because of increasingly large numbers of students in core subjects in universitycourses resort is often ma de to assessing, at least in part, their know ledge of con tent throughshort answer or multiple choice tests. It is possible for multiple choice questions to bedesigned so that they measure some aspects of higher order thinking (Nickerson, 1989; Norris,1989; Collis & Romberg, 1992) but usually they only measure recalled declarative and

Download


vember2011


8/13


theoretical information. Most tests demand surface rather than deep knowledge (Marton &Saljo, 1976a, 197 6b). Objective tests and examinations generally em phasise low level skills,facts, and memorization of procedures, and these are not the aspects of knowledge thatconstitute the higher order thinking needed for generating arguments and solving problems(Frederickson & Collins, 1989). In order to test for higher order thinking assessmentprocedures are required that measure theoretical and m etatheoretical content and procedural andconditional knowledge.Entwistle and Entwistle (1992) propose that when students study they frame theirpreparation strategies in line with the "form" of understanding they perceive will meetrequirements. They describe five such forms, four of which are assessment driven (fromrepeating lecture notes to personal restructuring of course material), with only one formaddressing the developm ent of an individual conception of the discipline. They co nclude thatthe form of understanding developed in an academic course often represents an uneasy

comprom ise between desire for understanding and the constraints of the cou rse and assessmentprocedures. Nickerson (1989) suggests that because there is a causal function betweenassessment and instruction we should make a virtue of a perceived vice, direct tests at theoutcomes of learning that we want for students and teach for them. Dart (1994) found 'that theway in which learners interpret the learning environment determines to an important degreethe goal orientation they pursue which, in turn, influences the learning strategies theyemploy' (p 467). He also found support for the possibility that students could hold bothlearning and performance goals simultaneously. That is they both want to understand thesubject and to produce work that is assessed highly. Assessment procedures therefore, as partof the learning environment, need to be carefully designed, made explicit, and implemented sothat they encourage and recognize the development of those aspects of learning that lead tohigher order thinking, if that is the intended outcome.

McKeachie et al. (1990) describe a range of procedures for assessing the cognitive structureof a student's knowledge. These include both direct and indirect approaches derived fromstudies of knowledge structures at both the high school and college level. They state that datagathering (hence testing) procedures for these approaches should include a definition of keyconcepts, the appropriateness of responses to be elicited, clear instructions for the subject andan adequate scoring procedure. The direct approaches, which seem to be more relevant toassessing knowledge of structure and course content, include such tasks as tree construction,concept mapping, concept structuring analysis technique (a combination of free sorting andinterview), networking (which can be applied to text analysis) and other sim ilar alternatives.They conclude that all these techniques for assessing and inferring representations of cognitivestructure have limitations when considered for practical applications for the following reasonsin representations of cognitive structure are limited in advance w hen key concepts are defined,most of the representations access static propositional declarative knowledge, the cognitivestructure as described by the tester may be the consensus of experts but may differ from theindividual's cognitive structure, there are difficulties with transforming cognitive structureinto diagrams and finally, and perhaps most importantly, there is the impracticality of usingsuch comp licated techniques for general assessment purposes. It is proposed here therefore thatprobably the most effective means of assessing deep learning outcomes and critical thinking isthrough essay s tyle, long or short, answers that require students to provide exam ples of their

Download


vember2011


9/13

150 The SOLO Taxonomy as a Means of Shaping and AssessingLearning in Higher E ducation

level of understanding, based on the organizational structure and extent of their contentknow ledge, and, if relevant, its application to a problem or situation. Such a procedure has theadvantage that it does not limit the concepts that the students m ight present, the responses donot need to be static, they are potentially individual and, if properly moderated, challengestudents to present conditional and metatheoretical knowledge if they possess it.

Grading procedures for essays have been criticized on the basis of subjectivity howeverchecklists for m arking, such as that of Caw ley (198 9), cited in Ramsden (1992 ), can obviatesuch problem s. Biggs (1992b ) proposes that such responses can be assessed effectively forcontent and structure using the SOLO T axonomy (cf. B iggs, 1992b). He describes a procedu refor using the SOLO Taxonomy for assessment in higher education and argues that it wouldconvey appropriate messages about learning by addressing higher level cognitive outcomesand be seen by students to be doing so. In essence he proposes a category system based onletter grades A, B, C, D and F, for example, each of which would describe a qualitativelydifferent kind of performance. Th ese performances would be ordered along a scale of increasingacceptability with F being unacceptable and A outstanding. The grade of F would bemultidimensional and account for failure to learn or moral or administrative problems. Theother grades all relate to a formal level of SOL O as follows; D , unistructural, the student h asonly understood one or a few aspects of the course; C, multistructural, student has understoodor used several aspects of the course; B, relational, aspects of the student's response form acoherent whole (he suggests that most questions or assignment topics would require thisstructure); A, extended abstract, high level of abstract thinking, generalizations to newcontexts or original conclusions. Biggs also suggests that there be three levels within eachcategory which indicate that the student has met the level minimally, adequately, or very well.He de scribes procedures for com bining grades with the caveat that these are qualitative scoresand ideally should be treated as such.

A search of the literature provides som e examples of the use of SO LO in tertiary teachingfor assessing learning outcomes, assessing beliefs about a course and assessing learning onspecific tasks.Boulton-Lewis (1992) and Boulton-Lewis and Dart (1994) used the SOLO Taxonomy tograde students' work in postgraduate courses in teaching and learning. A similar system to

that described by B iggs (1992b) was evolved to assess stude nts' assignm ents based on levelsand content of responses. Students' work was assessed by the lecturer as EA (Extendedabstract), R (Relational), M (Multistructural), and U (Unistructural) at those levels or at '+ ' or'- ' those levels depending on mastery and kind of content at each leve l. Grades were returned tostudents as, for example, HD (High D istinction), HD+ or HD -, if they were EA , EA + or EA -,in order to conform with the University's grading system. These were treated as qualitativegrades in determining the final comp osite grade. Regrettably it was also necessary to give thefinal grade a nominal percentage to conform with the university grading system. Boulton-Lewis & Dart (1994) also compared written responses, categorised according to SOLO, withconcept m aps as an alternative way for students to demonstrate their know ledge and structuralorganization of course content. Concept m aps were assessed quantitatively and also at S OL Olevels. There wa s 75 % agreement between SOL O levels for concept m aps and written materialat the end of the subject. Concept m aps have a minor disadvantage, when com pared to textual

Download


vember2011


10/13


material, in that their use must be taught separately as a technique.Tang and Biggs (1995) describe an innovative procedure, the letter to a friend technique(LTF), influenced b y the w ork of Trigwell and Prosser (1990 ). Th e LT F allowed analysis ofconceptual change, as part of an assessment portfolio which included letters written early andlate in a unit of a course. The objectives in the 'Nature of Teaching and Learn ing' course forin-service primary and secondary teachers were formulated in terms of the target levels ofcognitive complexity for instruction. The categories of objectives could also be used forgrading purposes in a manner similar to that suggested by Biggs (1992). The LTFs wereassessed for content and SOLO structure. The grades were taken as criteria of change and'roughly one third produced evidence...that reflective decision making had occurred'. TheSOLO structure of the second LTF was more likely to be relational. The task of writing aletter would seem to provide a better stimulus to describe changed conceptions than thestraight statements about learning and teaching requested by Boulton-Lewis.

Galenza (1993) describes the teaching and assessment of three courses in introductorypsychology where the desirability of achieving high level responses was addressed explicitlyand facilitated by practice and discussion. Questions, for example, for the experimental groupin one study, were designed at each SOLO level for four exams during the semester and thestudents were informed that the final exam would be marked according to SOLO. Thisconsistent approach produced better results only for students who were described as middlelevel learners (and hence probably needed more assistance with structure).Trigwell & Prosser (1991) and Prosser & Trigwell (1991) used the SOLO Taxonomy toassess qualitative differences in learning outcomes in a course, in first year nursing

communications, in response to a question asking students to describe what the course wasabou t. They found that responses could be classified as either relational or multistructural andan example at each of these levels was provided. The results suggest a positive relationshipbetween the Deep and R elating Ideas approaches to learning (derived from an adaptation of theApproaches to Study Inventory, Entwistle and Ramsden, 1983) and higher level qualitativeoutcomes as assessed by SOL O at the course level.Van Rossum & Schenk (1984) used the SOLO Taxonomy to assess the learningoutcomes, as expressed in answers to open questions, for a sample of 69 first year psychologystudents who were asked to study a text. They categorised most of the learning outcomes at

either the multistructural or relational levels. They found a strong relation between a surfaceapproach to learning and multistructural responses, and between a deep approach to learningand responses at the relational (and 3 at the extended abstract) levels. W atkins (1983 ) used theSOLO Taxonomy to assess the level of learning outcomes as determined by the quality ofstudents' explanations of a learning task they had worked on recently in class. The sampleconsisted of 60 students, chosen to represent the 10 highest scores on the meaning orientationscale and the 10 highest scores on the reproducing orientation scale of Entwistle's (Entwistle& Ramsden, 1983) Approaches to Study Inventory, from each of a faculty arts, science andeconomics. Watkins found it hard to distinguish between relational and extended abstractresponses but obtained high inter-judge agreement w hen these responses were categorized a shigh and responses at the multistructural level and below as low. There was a strongrelationship between high SOLO ratings and deep processing and low SOLO ratings andsurface processing. Exam ples of responses at different levels in the discipline areas are given.


vember2011


11/13

152 The SOLO Taxonomy as a Means of Shaping and AssessingLearning in Higher E ducation

In summary the use of the SOLO Taxonomy for assessment indicates that it is an effectivemeans for distinguishing between relational and multistructural responses at least. The finerdifferences between responses, and hence gradings, probably need to be determined on the kindand extent of knowledge in the responses.C O N C L U S I O NThe research described above sug gests that the SOLO Taxo nomy is a potentially useful toolin higher education both to shape and assess learning. There are some examples of howmodifications have been used already to find out wh at students know a bout their learning, toassess entering knowledge in a discipline, to present examples and models of structuralorganization of content in a discipline area, to illustrate acceptable learning outcom es and toshape learning through assessment Others could be generated.It also appears that there is a relationship, though not necessarily cau sal, between SOL Olevels and deep approaches to learning. This is indicated by results that show that as thestructural organization of know ledge of learning improves, as demonstrated by higher SOL Olevels, then the concern with surface motives declines and deep motives and deep strategiesassum e more imp ortance (Boulton-Lew is, 1994; Trigwell & Prosser, 1 99 1; Van Rossum &Schenk, 1984; Watkins, 1983).

It is acknowledged that use of the SOLO Taxonomy is only one way of assessing theextent of a student's content knowledge in a discipline and of its structural organization.McKeachie et al. (1990 ), describe a range of alternative methods as discussed above. However,the SOLO Taxonomy has the advantage that it can also be used as a model to challengestudents to engage in deep learning and to organize and present their knowledg e in their ownway to demonstrate understanding. It is not difficult or too time consum ing to u se and thereare suggested m odels for grading responses. It requires some creative thinking on the part oflecturers, how ever, to generate models in their own disciplines and to then co llect a range ofexamples from students.

It seems that further research in the use of SOLO to improve the quality of learning intertiary education is warranted. The next step for the present author will be to use SOLOexamp les in the ways described abov e, in teaching graduate and post graduate students abo utlearning, and to further research and document the outcomes. SOLO will be used inconjunction with other strategies which challenge students to becom e critical thinke rs. Thiswill include situations that increasingly require students to take control of their own learningincluding reading, summarizing, presenting and discussing material with their peers, takingresponsibility for searching the literature themselves and drawing out implications forpractice. They w ill also be encouraged to write, discuss and rewrite m aterial, individually andin groups, until their descriptions reach at least the relational level in the formal mode.Finally it will be made clear to students, in descriptions of objectives and assessmentprocedures, that their work w ill be assessed according to the level of its structural organizationas well as the nature and depth of the know ledge presented.

Download


vember2011


12/13

Higher Education Research and Developmen t, Vol 14, No 2, 1995 153

Address for correspondance: Associate Professor Gillian Boulton- Lewis, School ofLearning and Development, Faculty of Education, Queensland University of Technology,Locked Bag No 2 , Red H ill, Qld 4059.R E F E R E N C E SBarnett, R. (199 0). The Idea of Higher Education. Buckingham: The Society for Research intoHigher Education and Open University Press.Biggs, J.B. (1987). The Study Process Questionnaire (SPQ) Users' M anual. Hawthorn, Vic:ACER.Biggs, J.B. (ed.), (1991). Teaching for Learning: the View from Cognitive Psychology.

Hawthorn, Vic: ACER.Biggs, J.B. (1992a). Modes of learning, forms of knowing, and ways of schooling. In

Demetriou, A., Shayer, M. and Efklides, A. (eds.), Neo-Piagetian Theories ofCognitive Development. London: R outledge, 31-51.Bigg s, J.B . (1992b). A qualitative approach to grading students, HERDSA News, 14 (3), 3-6.Biggs, J.B. (1993). What do inventories of students* learning processes really measure? Atheoretical review and clarification. British Journal of Educational Psychology, 63, 3-19.Biggs, J.B. & Collis, K.F. (1982). Evaluating the Quality of Learning: The SOL OTaxonomy. New York: Academic Press.Bigg s, J.B. & C ollis, K.F . (1989). Towards a model of school-based curriculum developmentand assessment: Using the SOLO Taxonomy, Australian Jo urnal of Educa tion 33 ,149-161.Boulton-Lewis, G.M. (1992). The SOLO taxonomy and levels of knowledge of learning.Research an d Developmen t in Higher Edu cation, 15, 482-489.Boulton-Lewis G. M . (1994) Tertiary stu dents' know ledge of their own learning and a SOLOTaxonomy, Higher Education, 28, 387-402.Boulton-Lewis, G.M. & Dart, B.C. (1994). Assessing students' knowledge of learning: Acomparison of data collection methods. In Gibbs, G. (Ed.) (1994) Improving StudentLearning: Theory an d Practice. OCSD: Oxford.

Boulton-Lewis, G.M., Wilss, L. & Mutch, S. (in press). Teachers as adult learners: theirknow ledge of their own learning, Higher Education.Collis, K.F. & Romberg, T.A. (1992). Collis-Romberg mathematical problem solvingprofiles. Hawthorn: Australian Council for Educational Research,Dart, B.C. (1994). Teaching for improved learning in small classes in higher education. Paperpresented at the annual conference of the Australian Association for Research inEducation, November 1994, University of New casde, A ustralia.Entwistle, N. & Ramsden, P. (1983). Understanding Student Learning. London: CroomHelm.

Entwistle, A. & Entwistle, N. (1992). Experiences of understanding in revising for degreeexaminations, Learning and Instruction, 2, 1-22.Fredrickson, J .R. & Collins, A. (1989). A systems approach to educational testing.EducationalResearcher, 18 (9), 27-31.


vember2011


13/13

154 The SOLO Taxonomy as a Means of Shapingand AssessingLearning in H igher Education

Galen za, B. (1993). Th e SOL O Taxon om y applied to undergraduate instruction. Un iversity ofAlberta: Unp ublished PhD thesis.Leh m an, D .R., Lemp ert, R .O . & Nisbett, R .E. (19 88). The- effects of gradua te training onreasoning: Formal discipline and thinking about every-day events . AmericanPsychologist, 43, 431-442.Marton, F. & Saljo, R. (1984). Approaches to learning. In Marton, F., Hounsell, D. &Entwistle (Eds.). The Experience of Learning. Edinburgh: Scottish Academic Press.Marton, F. & Saljo, R. (1976a). On qualitative differences in learning. I - outcome andprocess, British Journal of EducationalPsychology 46, 4-11.Marton, F. and Saljo, R. (1976b). On qualitative differences in learning. II - outcome as afunction of the learner's conception of the task, British Journal of EducationalPsychology 46, 115-127.

Marton, F. , Dall ' Alba, G . & Beaty, E. (199 3). Conceptions cf learning. International Journalof EducationalResearch, 19 (3), 277-300.M cKeachie, W.J., Pintrich, P.R, Lin Y .G. & S mith, D.A.F. (2nd Ed.), (1990). Teaching andlearning in the college classroom. University of M ichigan: NCRIPT AL .Nickerson, R.S. (1989). New directions in educational assessment. Educational Researcher,18,(9), 3-7.No rris, S P . (1989). Can w e test validly for critical thinking? Educational Researcher, 18 (9),21-26.Prosser , M. T. & Millar , R. (1989). The 'How and What ' of learning physics: aphenomenographic study. European Journal of Psychology of E ducation, 4, 513-528.Prosser, M., Trigwell, K. & Taylor, P. (1994). A phenom engraphic study of academics'conceptions of science learning, Learning and Instruction, 4 (3), 217-231.Prosser, M. & Trigwell, K. (1991). Student evaluations of teaching and courses: Studentlearning approaches and outcomes as criteria of validity, Contemporary EducationalPsychology, 16, 269-301.Ram sden, P. (1988 ). Studying learning: improv ing teaching in Ram sden P. (ed.), ImprovingLearning: New Perspectives. London: Kogan Page.Ramsden, P . (1992). Learning to teach in higher education. London: R outledge.

Samuelowicz, K. & Bain, J.D. (1992). Conceptions of teaching held by teachers, HigherEducation, 24, 93-112.Tang, C. & Biggs, J. (1995), Letter to a friend: assessing conceptual change in professionaldevelopment Research an d Development in H igher Education, 18, 698-703.Trigwell , K. & Prosser , M. (1990). Using student learning outcome measures in theevaluation of teaching. Research and D evelopment in Higher Education, 13, 390-397.Trigwell, K. & Prosser, M. (1991). Relating approaches to study and quality of learningoutcomes at the course level, British Journal of EducationalPsychology, 61, 265-275.Van Rossum, E.J. & Schenk, S.M. (1984). The relationship between learning conception,study strategy and learning outcome, British Journal of Edu cational Psychology, 54,73-83.Watkins, D. (1983). Depth of processing and the quality of learning outcomes, InstructionalScience, 12, 49-58.

Downloadedby[NationalInstituteo

fEducation]at07:4811November2011

1995_solo_assess learning in higher edu

Documents