an overarching conceptual framework for assessing key ... · ence for assessing and developing key...

An overarching conceptual framework for assessing keycompetences in an international context

Lessons from an interdisciplinary andpolicy-oriented approach

Dominique Simone Rychen

In:

Descy, P.; Tessaring, M. (eds)

The foundations of evaluation and impact researchThird report on vocational training research in Europe: background report.

Luxembourg: Office for Official Publications of the European Communities, 2004(Cedefop Reference series, 58)

Reproduction is authorised provided the source is acknowledged

Additional information on Cedefop’s research reports can be found on:http://www.trainingvillage.gr/etv/Projects_Networks/ResearchLab/

For your information:

• the background report to the third report on vocational training research in Europe contains originalcontributions from researchers. They are regrouped in three volumes published separately in English only.A list of contents is on the next page.

• A synthesis report based on these contributions and with additional research findings is being published inEnglish, French and German.

Bibliographical reference of the English version:Descy, P.; Tessaring, M. Evaluation and impact of education and training: the value of learning. Thirdreport on vocational training research in Europe: synthesis report. Luxembourg: Office for OfficialPublications of the European Communities (Cedefop Reference series)

• In addition, an executive summary in all EU languages will be available.

The background and synthesis reports will be available from national EU sales offices or from Cedefop.

For further information contact:

Cedefop, PO Box 22427, GR-55102 ThessalonikiTel.: (30)2310 490 111Fax: (30)2310 490 102E-mail: [email protected]: www.cedefop.eu.intInteractive website: www.trainingvillage.gr

http://www.trainingvillage.gr/etv/Projects_Networks/ResearchLab/

www.cedefop.eu.int

www.trainingvillage.gr

Contributions to the background report of the third research report

Impact of education and training

Preface

The impact of human capital on economic growth: areviewRob A. Wilson, Geoff Briscoe

Empirical analysis of human capital development andeconomic growth in European regionsHiro Izushi, Robert Huggins

Non-material benefits of education, training and skillsat a macro levelAndy Green, John Preston, Lars-Erik Malmberg

Macroeconometric evaluation of active labour-marketpolicy – a case study for GermanyReinhard Hujer, Marco Caliendo, Christopher Zeiss

Active policies and measures: impact on integrationand reintegration in the labour market and social lifeKenneth Walsh and David J. Parsons

The impact of human capital and human capitalinvestments on company performance Evidence fromliterature and European survey resultsBo Hansson, Ulf Johanson, Karl-Heinz Leitner

The benefits of education, training and skills from anindividual life-course perspective with a particularfocus on life-course and biographical researchMaren Heise, Wolfgang Meyer

The foundations of evaluation andimpact research

Preface

Philosophies and types of evaluation researchElliot Stern

Developing standards to evaluate vocational educationand training programmesWolfgang Beywl; Sandra Speer

Methods and limitations of evaluation and impactresearchReinhard Hujer, Marco Caliendo, Dubravko Radic

From project to policy evaluation in vocationaleducation and training – possible concepts and tools.Evidence from countries in transition.Evelyn Viertel, Søren P. Nielsen, David L. Parkes,Søren Poulsen

Look, listen and learn: an international evaluation ofadult learningBeatriz Pont and Patrick Werquin

Measurement and evaluation of competenceGerald A. Straka

An overarching conceptual framework for assessingkey competences. Lessons from an interdisciplinaryand policy-oriented approachDominique Simone Rychen

Evaluation of systems andprogrammes

Preface

Evaluating the impact of reforms of vocationaleducation and training: examples of practiceMike Coles

Evaluating systems’ reform in vocational educationand training. Learning from Danish and Dutch casesLoek Nieuwenhuis, Hanne Shapiro

Evaluation of EU and international programmes andinitiatives promoting mobility – selected case studiesWolfgang Hellwig, Uwe Lauterbach,Hermann-Günter Hesse, Sabine Fabriz

Consultancy for free? Evaluation practice in theEuropean Union and central and eastern EuropeFindings from selected EU programmesBernd Baumgartl, Olga Strietska-Ilina,Gerhard Schaumberger

Quasi-market reforms in employment and trainingservices: first experiences and evaluation resultsLudo Struyven, Geert Steurs

Evaluation activities in the European CommissionJosep Molsosa

An overarching conceptual framework for assessing key competences

in an international context

Lessons from an interdisciplinary and policy-oriented approach

Dominique Simone Rychen

Abstract The OECD project DeSeCo (Definition and selection of competences: theoretical and conceptual foun-dations) was undertaken in response to this increasing interest in education outcomes and their effects.DeSeCo developed a common, overarching conceptual frame of reference for identifying and assessingkey competences. Key competences are individually based competences considered necessary or desir-able for effective participation in democratic societies and for coping with global demands, particularlythose related to the so-called knowledge economy or information society. DeSeCo was initiated in the OECD context at the end of 1997 and carried out under the leadership ofthe Swiss Federal Statistical Office. It is embedded in OECD’s long-term programme on education indi-cators (INES) which aims to provide measures on the functioning, development and impact of education. The work of DeSeCo was designed to complement past and current international empirical studies, inparticular the International Adult Literacy Survey (IALS), the Programme for International StudentAssessment (PISA) and the Adult Literacy and Life Skills (ALL) survey. The analysis and reflection inDeSeCo is not restricted to what can be learned and taught in schools or to what is currently or readilymeasurable in large-scale assessments. DeSeCo has not addressed its task by an inductive method,starting from factual situations, but rather by starting at a more general level, laying out conceptual andtheoretical considerations.DeSeCo’s approach is international, interdisciplinary and policy-oriented. Inputs to DeSeCo haveincluded discipline-oriented academic contributions, commentaries from leading experts working invarious fields, as well as country-specific reports. Two international symposia and various expert meet-ings provided opportunities for gaining interdisciplinary insight and a better understanding of the needsand concerns of policy-makers and experts from education, business, labour, health and other relevantsectors. These varied inputs have led to the construction of an overarching conceptual frame of refer-ence for assessing and developing key competencies.

Table of contents

1. Introduction 315

2. Background information 316

2.1. Policy context 316

2.2. Research context and focus of the study 316

3. Necessary steps in the definition and selection of key competences 318

3.1. Analysis of international comparative studies on learning outcomes 318

3.2. Towards a clarification of the concept of competence 318

3.3. Expert opinions from multiple disciplines and sectors 319

3.4. Country contribution process 319

3.5. Debates and exchange: from multiple perspectives towards interdisciplinary insight 319

4. Elements of DeSeCo’s overarching frame of reference for key competences 321

4.1. Criteria for competences and key competences 321

4.1.1. The concept of competence 321

4.1.2. The concept of key competence 321

4.2. Three categories of key competences 322

4.2.1. Acting autonomously 322

4.2.2. Using tools interactively 323

4.2.3. Interacting in socially heterogeneous groups 323

4.3. Interacting of key competences 323

5. Implications for a coherent assessment strategy 324

5.1. Dealing with the complex nature of key competences 324

5.2. Towards a coherent assessment strategy 325

6. Conclusions 326

List of abbreviations 327

Bibliography 328

The DeSeCo project (Definition and selection ofcompetences: theoretical and conceptual foun-dations) is a response to several trends that aroseindependently in different sectors that are nowbeginning to converge, with major policy implica-tions. One trend is a concern for people’s abilitiesto cope with the many rapid changes that aretaking place in the world, including the emer-gence of the so-called knowledge economy andinformation society. Although there is littleconsensus on specific competences and skillsthat individuals will need, there is considerablymore consensus on more general requirements,such as creativity, initiative taking and adapt-ability. Another trend is the shift from input- tooutcome-based public policies, particularly ineducation and training and the resulting demandfor relevant statistical data and indicators. Simul-taneously, large-scale educational assessmentsare expanding from measuring academicachievement alone to considering individuals’ability to apply knowledge and skills innon-school contexts. This expansion brings withit a number of new terms and concepts, but littleconsistency in their usage.

In light of these trends, an attempt to relate thedemands of the modern world to abilities, skillsand other qualities individuals need in order to

provide useful information for policy and planningis both appealing and logical. In the context ofthe OECD, the natural starting point was itssystem of education-related indicators. Althoughthese indicators have undergone significantdevelopment and have always been policy-rele-vant, a broader perspective on what should beassessed and an overarching theoretical frame-work for such assessments were deemed neces-sary to help guide future indicator developmentand long-term assessment strategies.

Throughout its brief history, DeSeCo has soughtto include a broad range of perspectives not onlyfrom the academic world, but from the worlds ofpublic policy, labour, education and business aswell. Attention has also been given to the varyingcontexts found throughout the participating coun-tries. Each input has contributed a unique combi-nation of perspective, experience, expertise andinterest that has helped shape the goals, defini-tions, limitations and potential of DeSeCo. Bytaking an approach that was both conceptual andpragmatic, it is hoped that DeSeCo will enhancethe relevance of international assessments. Basedon the outcomes of this study we are nowprepared to establish and affirm several importantguiding principles and discuss the implications ofDeSeCo for further work.

1. Introduction

2.1. Policy context

In light of common challenges and transforma-tions taking place in economies and societiesthroughout the world, education is widely viewedas an important investment and asset for boththe individual and the community. Whileeconomic growth remains a main objective forgovernments and societies, there are increasingpolicy concerns about its impact on the naturaland social environments (for instance OECD,2001a). Lifelong learning and competence or skilldevelopment have become key notions in inter-national efforts to enhance economic and socialdevelopment (for instance the World Bank, 2002;OECD, 2001b). With this heightened importanceplaced on education and lifelong learning,tomorrow’s curriculum and educational goalshave become a relevant topic in politicaldiscourses all over the world (within OECD seefor instance Trier, 2003). There is a growingconcern among governments, employers and thegeneral public about the adequacy and quality ofeducation and training. This concern is, forexample, reflected in the European Commissioncommunication, Making a European area of life-long learning a reality (2001, p. 3): ‘Traditionalpolicies and institutions are increasinglyill-equipped to empower citizens for activelydealing with the consequences of globalisation,demographic change, digital technology andenvironmental damage. Yet people, their knowl-edge and competences are the key to Europe’sfuture’.

With a shift from input- to outcome-orientedpolicies, the OECD and other international institu-tions have invested considerable effort in devel-oping comparable outcome indicators in theeducation field to provide information relevant forgovernment policy (Salganik et al., 1999;Salganik, 2001). In general, these indicatorsreflect traditional notions of academic achieve-ment and skill development, such as reading andmathematics skills, either in curriculum-based orreal-world-based contexts. This focus partiallyresults from practical considerations, but also

results from the fact that these skills are recog-nised as foundation skills, crucial to success inthe so-called knowledge and information societyof today. Though traditional competences suchas reading literacy have been measuredthroughout OECD countries and beyond, there isa broad consensus that we are far from assessinga truly comprehensive set of competences rele-vant to human and social development and polit-ical and economic governance. OECDconstituencies, recognising the need for aconceptual basis for defining and selecting rele-vant competences, thus welcomed the workprogramme of the DeSeCo project. DeSeCo’stheory-based and policy-oriented workprogramme found broad support and interestwithin and outside of the OECD.

2.2. Research context and focusof the study

The DeSeCo project originated in a governmentalcontext, the OECD, at the end of 1997 and wascarried out under the leadership of the SwissFederal Statistical Office. It is embedded inOECD’s long-term programme on education indi-cators (INES), aimed at providing measures onthe functioning, development and impact ofeducation from early childhood, through formaleducation and to learning and training throughoutlife. The work of DeSeCo was designed tocomplement past and current international empir-ical studies, in particular the International AdultLiteracy Survey (IALS), the Programme for Inter-national Student Assessment (PISA) and theAdult Literacy and Life Skills (ALL) survey.Throughout the work programme, close ties withthese and other international surveys have beenmaintained in recognition that success in thiscomplex field depends not only on theoreticaland empirical work, but also on the iterativeprocess entailed by constant dialogue andexchange among the various specialists andstakeholders.

2. Background information

The main goal of DeSeCo was the construction ofan overarching conceptual frame of reference rele-vant both for the development of key competencesin a lifelong learning perspective and for the assess-ment of these competences in an internationalsetting. This study does not address the topic ofeducation outcomes by proceeding with an induc-tive method, starting from factual situations, butrather by starting at a more general level, laying outconceptual and theoretical considerations. DeSeCoconsiders the topic of important, necessary or desir-able competences from a broad, holistic and inter-disciplinary perspective. Thus, the reflection on andanalysis of key competences is not limited to theschool context, student achievement or workers’skills and the demands of the labour market.

While recognising the value and importance ofdomain- or job-specific competences and tech-nical skills, DeSeCo focuses on competencesthat apply across multiple areas of life (forinstance, in the family, at work, in the politicalsphere, in the health sector, etc.) and thatcontribute to personal, economic and socialwell-being. The leading theme is: in light of thedemands of democratic societies and the globalchallenges of modern life including those relatedto the knowledge-based economy or informationsociety, what competences are important ornecessary for everyone; and, what are the norma-tive assumptions and conceptual criteria throughwhich these competences are constructed?

Considerations of whether these competencescan be assessed or measured in large-scale orqualitative studies – and if so, how – were not atthe forefront. The determination of potentially‘new’ competences is based, instead, on severalissues that have not played prominent roles in thepast, including:(a) theoretical models and concepts – what is a

competence? What makes a competence‘key’?

(b) cultural context, biographical variability – towhat extent are the competences identifiedrelevant across cultures and to individualsoccupying different positions in society andthe economy?

(c) political negotiation, consensus formation –which competences do policy-makers valueand which do they have interest in assessing?

(d) visions of society and individuals – to whatextent do the identified competences dependon a particular set of assumptions about theway the world should be and the way peopleshould behave?

DeSeCo’s approach of asking these types ofquestions, of stepping back from the immediatedevelopment of assessment instruments and ofproviding a platform for reflection and exchangeamong scholars, policy-makers and experts fromvarious fields proved a useful model for furtheringthe conceptual and theoretical foundations ofrelevant competences.

An overarching conceptual framework for assessing key competences in an international context – Lessons from an interdisciplinary and policy-oriented approach 317

The conclusions and recommendations drawnfrom this interdisciplinary and policy-drivenresearch are the results of a collaborative effortamong scholars from different disciplines; expertsfrom education, business, labour, health andother relevant sectors; as well as various stake-holders at the national and international level.

The work programme consists of four mainstudies and two international symposia.Throughout the project, attention has been givento ensure inclusion of both theoretical considera-tions, as represented in many of the expertpapers (Rychen and Salganik, 2001) and practicaland policy-oriented needs, as described in thereports stemming from the country contributionprocess (Trier, 2003). Since these activitiesconstitute the key sources and reference mate-rials for the construction of an overarching frameof reference and for the conceptualisation of keycompetences, the following sections will highlightthe rationale and main findings for each activity.

3.1. Analysis of internationalcomparative studies onlearning outcomes

To understand the state of the field, uncover anygaps in knowledge and work out the mostpromising strategy at the international level,DeSeCo began in 1998 with an analysis of threemajor studies on competences that had beenconducted within the OECD context: theCross-Curricular Competences Project, an explo-rative study; the International Adult LiteracySurvey (IALS) and the Human Capital IndicatorsProject (Salganik et al., 1999). Several centralthemes were found to transcend these threestudies, including: desired outcomes of educationare broader than the acquisition of thesubject-related knowledge typically taught inschool; competences develop beyond the schoolcontext throughout life; and education is not anend in itself, but a means for making studentscompetent and thereby prepared for life.

The authors also found that, despite commongoals and a connection to the OECD, there werefew explicit links between the three studies, interms of both an overarching theory-groundedframework and a larger coordinated researcheffort. Since the studies arose from differentpurposes and focused on different types of popu-lation groups, this was neither surprising norconsidered a shortcoming of their designs at thattime. In all of these studies, pragmatic considera-tions dominated and, ultimately, priority wasgiven to empirical testing of concepts by applyingexisting methodologies, instruments and, whenpossible, existing data. Due to the emphasis onthe feasibility of measurement, theoretical andconceptual considerations such as an under-standing of the nature of competences and theinterrelations among various competences werenot an issue. These findings confirmed the needto further advance the theoretical underpinning ofcompetences, in particular by developing anexplicit, overarching conceptual framework toguide future work.

More recent and partly still continuing projects,such as the PISA, the ALL survey and the IEACivic Education Study (CivED), were further anal-ysed in 2000. Two of the main findings of thisreview, specifically that there has been consider-able investment in ‘conceptual work and [the]development of measures grounded in well-established theory’ (Salganik, 2001, p. 30) andthat lessons learned from previous studies weremore fully incorporated in the new studies, indi-cate potential interest in and application forDeSeCo’s work.

3.2. Towards a clarification of theconcept of competence

In light of the terminological and conceptualconfusion associated with notions such ascompetence, skills, qualifications, standards,literacy and so on, an expert paper was commis-sioned to provide a preliminary clarification of the

3. Necessary steps in the definition and selection ofkey competences

concepts related to competence and key compe-tence (Weinert, 2001). With its systematic analysesof existing terms and the theoretical and concep-tual approaches within the social sciences to theconcept of competence, the paper served a neces-sary and important function. In line with Weinert’srecommendations as confirmed by subsequentdiscussions on conceptual issues, DeSeCo, asdiscussed below, opted for a functional (demand-oriented) approach to competences.

3.3. Expert opinions frommultiple disciplines andsectors

In 1999, renowned scholars from differentacademic disciplines (anthropology, psychology,economics, sociology and philosophy) wereasked to construct a set of relevant key compe-tences from their own theoretical background anddisciplinary perspective. The authors wereexpected to justify their selections theoretically,taking into account any available state-of-the-artresearch-based evidence.

The philosophers Canto-Sperber and Dupuy(2001) drew on their expertise in moral and socialphilosophy and the philosophy of mind not only toaddress the question of key competences but alsoto consider what constitutes ‘the good life’. Haste(2001) emphasised from her sociopsychologicalviewpoint the need to look at individuals in acultural, social and linguistic context. For Perre-noud (2001), a sociologist, the central question tobe addressed was what competences are neededby everyone to freely exercise his or her autonomyin multiple social fields. He associated asuccessful life with not being abused, alienated,dominated or exploited. Levy and Murnane (2001),as well as Goody (2001), differed from the otherauthors in so far as they used an empiricalapproach to the issue of competence. Levy andMurnane used relevant economic theories andempirical results to address the question of thecompetences workers need to succeed in thelabour market. Goody, the anthropologist, rejectedengaging in a decontextualised discussion of keycompetences on the grounds that theory mustalways be considered in relation to practice.

Subsequently a commenting process was initi-ated (Rychen and Salganik, 2001), wherein the

scholarly papers were distributed among theauthors, other academics and leading representa-tives from various social sectors. It was a first stepto explore main convergences and divergencesamong the different disciplinary perspectives andto gain insight on priority areas from practical andpolicy-oriented perspectives. The results of thiscommenting process constituted a starting pointfor further exploring common or complementaryconcepts and models relevant from an interdisci-plinary and international perspective.

3.4. Country contribution process

Recognising that defining key competences is aprocess that draws not only on a scientific foun-dation but also on negotiations among variousstakeholders in the political and economicarenas, a country consultation process waslaunched to gain insight on the public debate andthe education needs and priorities of variousnational contexts and sectors both within andacross countries. A summary report wasprepared based on the national contributionssubmitted from 12 OECD countries (Trier, 2003).Convergence rather than divergence was foundacross countries and between the economic andeducation sectors within countries. Similarcompetence areas with almost identical contentare emphasised, even if they are worded differ-ently. Indeed, there is a broad consensus on theimportance of social competences, communica-tion, literacies, lifelong learning, personal compe-tences and competences necessary for participa-tion in political life. Value orientation is anotheraspect of particular concern in several countries.

3.5. Debates and exchange: frommultiple perspectivestowards interdisciplinaryinsight

Each of the four activities described above raisedspecific challenges that DeSeCo had to overcomeand emphasised the need to bring together theknowledge and insights gained through theseactivities. Two international symposia and variousexpert meetings provided opportunities for moving


from the specific activities and multiple perspec-tives towards increased understanding and aware-ness of the issues at stake and eventually to aconsensus on an integrated frame of reference.

The first international symposium in 1999 wasa forum bringing together scholars and experts,including the authors of the papers (Rychen andSalganik, 2001). It succeeded in creating an inter-national network of interested academics andexperts, increasing the awareness of the issuesinherent in dealing with questions of compe-tences and their assessment and clarifying andprioritising avenues for further work in this area.

The second international symposium (Rychenet al., 2003), with approximately 170 participants,including policy-makers, policy analysts, assess-ment specialists, academics and representativesfrom national and international institutions,provided a forum for further consolidatingconcepts and ideas on key competences andworking towards consensus around the questionof what competences are relevant for today’ssocieties and economies. It succeeded inoutlining options and strategies for the eventualdevelopment of a coherent international assess-ment programme (OECD, 2002).

The foundations of evaluation and impact research320

Given the initial finding of a lack of a commonconceptual framework for understanding compe-tences (Salganik et al., 1999; Weinert, 2001), theresearch conducted within the scope of DeSeCoresulted in a comprehensive frame of reference tosituate current and future key competencies in alarger conceptual context. It includes, as coreelements, the concepts of competence and keycompetence and a threefold categorisation forkey competences considered as critical for asuccessful life and a well-functioning society.

4.1. Criteria for competences andkey competences

4.1.1. The concept of competenceA competence is defined as the ability to meet acomplex demand. Each competence correspondsto a combination of interrelated cognitive andpractical skills, knowledge and personal qualitiessuch as motivation, values and ethics, attitudesand emotions. These components are mobilisedtogether for effective action in a particular context.This definition represents a demand-oriented orfunctional approach, placing at the forefront themanifold demands individuals encounter in thecontext of work and everyday life. It is holistic, inthe sense that it integrates and relates demands,individual attributes and context as integralelements of competent performance.

To facilitate the further discussion of compe-tence, it is important to emphasise that in theDeSeCo project, the terms ‘competence’ and‘skill’ are not used as synonyms. The term‘competence’ (a holistic concept) designates acomplex action system encompassing knowl-edge, cognitive skills, attitudes and othernon-cognitive components, while the term ‘skill’is used to designate an ability to perform motorand/or cognitive acts.

4.1.2. The concept of key competenceBased on the concept of competence describedabove, one could imagine any number of compe-

tences. DeSeCo’s policy orientation, however,requires further focus on key competences in thesense of ‘competences of prime importance for asuccessful life and effective participation indifferent fields of life – including economic, polit-ical, social and family domains; public and privateinterpersonal relations; and individual personaldevelopment’ (SFSO, 2001b). This approach isconsistent with the general commitment of OECDcountries to expand opportunities for individualsin various spheres of life, to improve overall livingconditions in society and to invest in the develop-ment of competences for all (OECD, 2001b).Based upon a review of existing work in the areaof competence and on the input of experts from avariety of backgrounds, two important criteria forkey competences are put forth; namely, that keycompetences: (a) contribute to broad desired outcomes of an

overall successful life and a well-functioningsociety;

(b) enable all individuals to cope successfullywith complex demands and challengesacross a wide spectrum of relevant contextsand domains.

The first criterion raises the important ques-tions ‘What constitutes a successful life?’ and,‘What are desirable goals for societies?’. Indi-vidual and societal ideals and goals are as variedas individuals and societies themselves.However, numerous international conventionsand agreements, such as the Universal Declara-tion of Human Rights, the Rio Declaration onEnvironment and the World Declaration onEducation for All, exhibit consensus aroundbroad desirable goals for all individuals and coun-tries that can serve as the basis for a commonnormative framework. For example, the WorldDeclaration on Education for All (1990) describedseveral goals of education that are relevant for allindividuals in all societies: ‘To survive, to developtheir full capacities, to live and work in dignity, toparticipate fully in development, to improve thequality of their lives, to make informed decisionsand to continue learning’ (Article 1). Thus, basicprinciples of human rights, democratic values and

4. Elements of DeSeCo’s overarching frame ofreference for key competences

the broad goals of sustainable developmentprovide a normative anchoring point for thediscourse on key competences and their selectionand development. While more specific normativeframeworks could be developed for specific indi-viduals and societies, this framework is consideredapplicable to all and is specific enough to ensurethat key competences relate only to demands thatare meaningful and desirable.

The second criterion narrows selection to onlythose competences that are transversal in thesense that they are necessary for understandingand acting effectively in and across different rele-vant domains or areas of life such as the work-place, personal life, the health field, the politicaldomain, etc. Thus, competences that aredomain-specific, that is, applicable in only onesector of the economy or society, are not deemedkey competences. DeSeCo’s focus on keycompetences does not question the relevance ofsuch domain specific competences. These arenecessary and often constitute importantresources for coping with the demands of partic-ular contexts and situations.

In addition, conceptualising key competencesrequires consideration of demands and chal-lenges faced by individuals and societies. Whenspeaking of the complexity and challenges oftoday’s world, there is broad consensus thatbasic skills, although important, are not sufficient.Many scholars and experts agree that effectiveand responsible participation in today’s worldcalls for the development of a higher mental levelof complexity (Kegan, 2001) that implies criticalthinking and a reflective, holistic approach to lifeon the part of the individual. This notion of reflec-tivity (though a variety of terms are used) is acommon thread throughout the various contribu-tions from scholars and experts prepared forDeSeCo (in particular, Kegan, 2001;Canto-Sperber and Dupuy, 2001; Perrenoud,2001). This higher level of mental complexity (or,using other terms, a critical stance and a reflec-tive/integrated/holistic approach to life) repre-sents a central transversal feature of the identi-fied key competences.

To summarise, key competences as adopted inthe OECD are competences (each of whichconsists of a cluster of cognitive skills andnon-cognitive dimensions such as attitudes,motivation, values and emotion, the structure of

which are defined by the demand in question)that help individuals cope with complex demandsand thus require critical thinking skills and areflective/holistic approach to life. Key compe-tences apply to various relevant areas of life and,as such, can be considered transversal compe-tences. They contribute to highly valued indi-vidual and societal outcomes (i.e. an overallsuccessful life for individuals and a well-func-tioning society) consistent with the universalgoals and values identified by internationalconventions.

4.2. Three categories of keycompetences

Based on the inputs and interdisciplinary perspec-tives received in the course of the DeSeCo project,we have constructed a threefold categorisation ofkey competences for interpreting and furtherconceptualising key competences. The threetheory-grounded, broad categories are actingautonomously, using tools interactively and inter-acting in socially heterogeneous groups.

Within the three categories we have high-lighted a number of key competences that areparticularly relevant for coping with many of thecomplex demands individuals and societies face.The identified competences are the result of ananalysis of the various lists of key competencespresented in the country reports and the expertcontributions in light of the established defini-tional, conceptual and normative criteriadescribed above. In the terminology of humancapital, they contribute to important outcomessuch as economic resources/employment, polit-ical participation, economic growth, social cohe-sion and the realisation of human rights.

The three categories and the key competencesidentified within each category, briefly describedbelow, are detailed in DeSeCo’s final report(Rychen and Salganik, 2003).

4.2.1. Acting autonomouslyTo act autonomously incorporates two centralinterrelated ideas, the development of personalidentity and the exercise of relative autonomy inthe sense of deciding, choosing and acting indifferent social fields. Acting autonomously doesnot mean functioning in social isolation. Instead,


it implies that individuals are enabled to managetheir lives in meaningful ways by exercisingcontrol over their living and working conditionsand by playing an active part in shaping their ownlives. It requires an orientation towards the futureand an understanding of one’s environment, itsfunctioning and the roles one plays within thatenvironment (Perrenoud, 2001). The following keycompetences are relevant when it comes toacting autonomously:(a) the ability to defend and assert one’s rights,

interests, limits and needs;(b) the ability to form and conduct life plans and

personal projects; (c) the ability to act within the big picture/the

larger context (i.e. the larger historical,cultural or environmental context of actionsand decisions and their long-term and indi-rect consequences).

4.2.2. Using tools interactively The word ‘tool’ is used in the broadest sense of theterm to include language, information and knowl-edge in addition to physical tools. To use a toolinteractively assumes not only a familiarity with thetool itself but also an understanding of how the toolchanges the way one can interact with the world. Atool is not just a passive mediator but is instru-mental as part of an active dialogue between theindividual and his or her environment (Haste, 2001).Underlying this is the idea that we encounter ourworld through our cognitive, social and physicaltools. These encounters shape how we makesense of the world and become competent in inter-action. The following key competences have beenidentified as relevant when it comes to using toolsinteractively:(a) the ability to use language, symbols and text

interactively;(b) the ability to use knowledge and information

interactively (to manage and use it as a basisfor understanding options, forming opinions,making decisions and for taking informedactions);

(c) the ability to use (new) technology interac-tively (including the ability to see potentialnew uses of it).

4.2.3. Interacting in socially heterogeneousgroups

In this category the focus is on the interactionwith the ‘other’, the different other. Human

beings are dependent on ties to others for mate-rial and psychological survival, for a sense ofself, identity and social meaning (Ridgeway,2001). Living and participating in multiculturalsocieties and coping with increasing individualand social diversity require the ability to interactand coexist with people who do not necessarilyspeak the same language (literally or metaphori-cally) or share the same memory or history. Bybridging these divides, these interpersonal orsocial competences help create social capital.The following key competences are relevantwhen it comes to interacting effectively withother people:(a) the ability to relate well to others (involving,

initiating and maintaining relationships withfamily members, friends, neighbour, boss,colleague, etc.);

(b) the ability to cooperate (to work togethertoward a common goal);

(c) the ability to manage and resolve conflict (in aconstructive manner).

4.3. Interacting of keycompetences

The hypothesis is that for individuals to meetthe complex demands of modern life in asuccessful way and to contribute to the qualityof society, they need to mobilise these kinds ofcompetences in particular contexts. Thecompetences do not, however, operate inde-pendently of each other. They are interrelated.The resulting patterns of these interrelatedcompetences can be conceived as constella-tions, whose forms or interplay – the particularrelevance of each competence within thisconstellation – depend on the desired outcomesat the individual and societal level and thecontext in which they apply.

One way to illustrate the relative importanceof key competences with regard to their contri-bution to desired outcomes in different contextsis by conceptualising a multi-dimensional spacewhose axis are defined by the key compe-tences. The various contexts (a country or asocial field) can then be located in the spacedepending on the relative importance of the keycompetences for meeting the demands or goalsin question.


Developing assessments and indicators for keycompetences is not in DeSeCo’s scope of work,although such work is clearly implicated by thenotions discussed above. At this point, with theestablishment of a conceptual frame of referencefor key competences, it is possible and appro-priate to highlight some of those implications forthe development of assessments and indicators.This section presents some issues that are rele-vant to the elaboration of a long-term datastrategy for evaluating learning outcomes in theOECD.

5.1. Dealing with the complexnature of key competences

The three broad categories of key competences,acting autonomously, using tools interactivelyand interacting in socially heterogeneous groups,have been recognised in the OECD as a valuableconceptual tool for mapping and further concep-tualising the competences for which statisticalindicators are developed. DeSeCo’s work alsoconfirms the theoretical relevance of what hasbeen measured or proposed to be measured ininternational comparative assessment projectssuch as IALS, PISA and ALL. With the exceptionof initial efforts related to acting autonomously(self-regulated learning in PISA) and interacting insocially heterogeneous groups (exploration ofteamwork for ALL), these projects have focusedon competences which are related to using toolsinteractively (mainly related to using language,symbols and texts). This is an area for whichtheory provides a strong basis for measurement.Future research efforts in the field of key compe-tence should be directed to the development andvalidation of measures related to key compe-tences identified in the other two categories,acting autonomously and interacting in sociallyheterogeneous groups. However, key compe-tences in these areas are much more socially andculturally embedded, a fact that must be takeninto account in the development of any measures

and particularly internationally comparablemeasures (Murray, 2003). Focusing on newcompetence domains does not, of course,preclude continued research and development ofthe competences included in the category usingtools interactively. In particular, research isrequired with respect to representing the level ofmental complexity that involves critical thinkingand a reflective, holistic and integrated approach.

In all categories of key competences, devel-oping appropriate and reliable measures andinterpreting the significance of any results neces-sitates open and innovative approaches toassessment methodologies, for several reasons.One is that assessments ideally reflect thecomplex nature of a competence, a combinationof different interrelated component elements,including knowledge, cognitive skills and socialand behavioural components (e.g. motivation andethical/moral attitudes). This requires either somemethod of assessing the competence itself, orassessing one or more of the componentelements and developing a model to relate theresults to statements about the level of compe-tence development.

Second, because key competences are inter-related and operate as constellations that varywith cultural and contextual factors, assessingkey competences in isolation will not do justice tothe multi-dimensional reference points associatedwith key competences. A variety of methodolo-gies and settings will need to be explored tomore accurately capture the complex nature of acompetence and effectively profile the constella-tions of key competences that contribute tohighly valued outcomes.

Third, similar to the more recent conceptualisa-tions of ‘literacy’ and unlike traditional notions of‘literate’ and ‘illiterate’, assessing competence isnot a matter of assessing whether an individualdoes or does not possess it, but rather of deter-mining where along a continuum of competencean individual’s performance falls. Such acontinuum must include a comprehensive rangeof levels to reflect the implications of the neces-sary level of mental complexity, in which individ-

5. Implications for a coherent assessment strategy

uals ground their decisions and actions in criticalthinking and a reflective/integrative/holisticapproach.

Hence, the findings from DeSeCo reinforce theimportance of further developing complementaryassessment methodologies to produce a systemfor providing policy-relevant information on keycompetences and to more accurately capture theexpression of key competences. The complexnature of key competences – including suchaspects as their demand orientation, interrelated-ness and relation to broadly desired outcomes –requires data from multiple sources, including,but not limited to, the collection of large-scaleassessment data through paper and pencil orcomputerised instruments. Interactive computersimulations hold particular promise, as they allowfor broadening assessment tasks and creatingdynamic assessment situations. Adaptive andinteractive testing simulates a more authenticcontext, where an individual’s actions may bejudged not only in response to a single staticprompt, but progressively, on multiple levels as asituation develops. In addition, performanceassessments and portfolios provide othercomplementary approaches for reviewing individ-uals’ development and progress in one or moreareas over time. And, last but not least,video-based data collection, as pioneered inTIMSS could provide more in-depth and qualita-tive information and add to a more completepicture of competences within various popula-tions.

It should be noted that assessments usingthese methods are not well established and arealmost certainly more expensive than traditionalassessments. This should not be viewed as aninsurmountable barrier to developing indicatorsfor key competences though, but rather as anindication of the challenges faced.

5.2. Towards a coherentassessment strategy

Good governance and effective policy-makingrely on information, data and statistics that arebased on sound foundations. The design of a

coherent international assessment strategy forcompetences therefore needs to be guided bypolicy priorities and by an overarching conceptualframe of reference such as that provided byDeSeCo. With specific regard to key compe-tences, policy-makers need to reach consensuson the value of choosing to measure some keycompetences or components of competencesover others (considering factors such as theextent to which they are amenable to policy inter-vention). At the same time, researchers need toprovide further conceptual input during thatselection process and effective guidance on thefeasibility of developing valid and reliablemeasures of new theory-grounded constructs. Afuture coherent assessment programme shouldtake advantage of the conceptual and empiricalframeworks that currently support internationallyvalid, reliable and comparable assessments ofcompetences, building on the empirical experi-ences of current large-scale studies, in particularPISA and ALL.

A forward-looking assessment strategy shouldinclude plans and methods not only for providingdata on the level and distribution of key compe-tences over time and in different contexts, butalso for using that information to learn moreabout the interrelatedness of key competencesand their influence on desired outcomes.Containing this type of information, compe-tence-related assessments could thus be used toanswer such questions as, ‘Do certain sub-popu-lations show higher levels of development incertain key categories?’ ‘How has the distributionof key competences changed over time and whatare possible explanations for these changes?’‘Are there correlations between levels of develop-ment of certain key competences?’ and ‘Do keycompetences in fact appear to lead to desiredsocietal outcomes?’. Answering such questionswould require a mechanism for collecting data onsocial contexts, a longitudinal approach, assess-ment of people from different backgrounds and atdifferent stages in life and a cyclical structure ofassessments. A coherent assessmentprogramme, in turn, calls for coordinated plan-ning and implementation at the international leveland sustained political and financial support overmany years.


The OECD project DeSeCo concluded with itsfinal report entitled Key competencies for asuccessful life and a well-functioning society(Rychen and Salganik, 2003).

Through its collaborative process, DeSeCosucceeded in its missions: to propose a frame-work for understanding key competences basedboth on current theory from a variety of academicdisciplines and on practical considerations. Thisoverarching frame of reference serves severalpurposes: it presents a basis for a common,coherent and international discourse on compe-tence development in a lifelong learning perspec-tive; it has the potential to focus and improvefuture initiatives related to key competences,most significantly the development of a compre-hensive strategy for data collection and analysis,including a determination of which competencesto assess, both in the mid and long term; and toguide the development of assessment instru-ments and policy-relevant indicators.

Any such strategy must be linked to workalready underway regarding domain-specificassessments. DeSeCo can help situate theseassessments – and other empirical findings on

the outcomes of learning and teaching – in abroader conceptual context and clarify what isactually being measured and what is not.

Beyond assessments and indicators, theresults of DeSeCo can provide valuable input tothe prioritisation of particular competences foreducation curricula and training programmesdevelopment and for discussions of policies andprogrammes aimed at enhancing key compe-tences among specific or general populations.

Although DeSeCo has provided some answers,it has raised new questions and provided a glimpseof the significant challenges that lie ahead. It isclear, for example, that further discussion andconsensus-building around the recommendationsof the project presented to the OECD are neces-sary (OECD, 2000). In particular, OECD Membercountries need to establish broad consensus onnew competence areas for assessment at the inter-national level and start mapping out a coherentstrategy for the assessment of key competences ofyouth and adults. A significant research effort isneeded to make substantial strides in the breadthof new competences that can be meaningfullyincluded in large-scale assessments.

6. Conclusions

List of abbreviations

ALL Adult literacy and life skills

DeSeCo Definition and selection of competences: theoretical and conceptual foundations

IALS International adult literacy survey

INES International indicators of education systems [OECD’s long-term programme on education indicators]

PISA Programme for international student assessment

Binkley,M. etal. An overarching framework for under-standing and assessing lifeskills. In: Frameworks:working drafts. Briefing materials for ALL SurveyNational Study Managers meeting held inLuxembourg, 23-24 September 1999.

Callieri, C. The knowledge economy: a businessperspective. In: Rychen, D. S.; Salganik, L. H.(eds) Defining and selecting key competencies:theoretical and conceptual foundations.Göttingen: Hogrefe and Huber, 2001,p. 228-231.

Canto-Sperber, M; Dupuy, J.P. Competencies forthe good life and the good society. In:Rychen, D. S.; Salganik, L. H. (eds) Defining andselecting key competencies: theoretical andconceptual foundations. Göttingen: Hogrefeand Huber, 2001, p. 67–92.

Carson, J. Defining and selecting competencies:historical reflections on the case of IQ. In:Rychen, D. S.; Salganik, L. H. (eds) Defining andselecting key competencies: theoretical andconceptual foundations. Göttingen: Hogrefeand Huber, 2001, p. 32–44.

Delors, J.; Draxler, A. From unity of purpose to diver-sity of expression and needs: a perspectivefrom Unesco. In: Rychen, D. S.; Salganik, L. H.(eds) Defining and selecting key competencies:theoretical and conceptual foundations.Göttingen: Hogrefe and Huber, 2001,p. 214–221.

DeSeCo. Available from Internet: www.deseco.admin.ch. [cited 30.8.2002]

Farrugia, J.P. Competency management as aninvestment: a business perspective. In:Rychen, D. S.; Salganik, L. H. (eds) Defining andselecting key competencies: theoretical andconceptual foundations. Göttingen: Hogrefeand Huber, 2001, p. 93–120.

Fratczak-Rudnicka, B.; Torney-Purta, J. Compe-tencies for civic and political life in a democracy.In: Rychen, D. S.; Salganik, L. H.;McLaughlin, M. E. (eds) Contribution to thesecond DeSeCo’s symposium. Neuchâtel,Switzerland: SFSO – Swiss Federal StatisticalOffice, 2003.

Gilomen, H.; Rychen, D. S.; Salganik, L. H.Concluding remarks. In: Rychen, D. S.;Salganik, L. H. (eds) Defining and selecting keycompetencies: theoretical and conceptualfoundations. Göttingen: Hogrefe and Huber,2001, p. 247–251.

Gonczi, A. Teaching and learning of the key compe-tencies. In: Rychen, D. S.; Salganik, L. H.;McLaughlin, M. E. (eds) Contribution to thesecond DeSeCo’s symposium. Neuchâtel,Switzerland: SFSO – Swiss Federal StatisticalOffice, 2003.

Goody, J. Competencies and education: contextualdiversity. In: Rychen, D. S.; Salganik, L. H. (eds)Defining and selecting key competencies: theo-retical and conceptual foundations. Göttingen:Hogrefe and Huber, 2001, p. 175–190.

Harris, B. Are all key competencies measurable? Aneducation perspective. In: Rychen, D. S.;Salganik, L. H. (eds) Defining and selecting keycompetencies: theoretical and conceptualfoundations. Göttingen: Hogrefe and Huber,2001 p. 222-227.

Haste, H. Ambiguity, autonomy and agency:psychological challenges to new competence.In: Rychen, D. S.; Salganik, L. H. (eds) Definingand selecting key competencies: theoreticaland conceptual foundations. Göttingen:Hogrefe and Huber, 2001, p. 93–120.

Keating, D. P. Definition and selection of competen-cies from a human development perspective.In: Rychen, D. S.; Salganik, L. H.;McLaughlin, M. E. (eds) Contribution to thesecond DeSeCo’s symposium. Neuchâtel,Switzerland: SFSO – Swiss Federal StatisticalOffice, 2003.

Kegan, R. Competencies as working epistemolo-gies: ways we want adults to know. In:Rychen, D. S.; Salganik, L. H. (eds) Defining andselecting key competencies: theoretical andconceptual foundations. Göttingen: Hogrefeand Huber, 2001, p. 192–204.

Levy, F.; Murnane, R. J. Key competencies criticalto economic success. In: Rychen, D. S.;Salganik, L. H. (eds) Defining and selecting keycompetencies: theoretical and conceptual

Bibliography

foundations. Göttingen: Hogrefe and Huber,2001, p. 151–174.

Making a European area of lifelong learning a reality/ European Commission. Luxembourg: Officefor Official Publications of the EuropeanCommunities, 2001 (COM(2001) 678 final).

Murray, T. S. Reflections on future internationalassessments. In: Rychen, D. S.; Salganik, L. H.(eds) Key competencies for a successful life anda well-functioning society. Göttingen: Hogrefeand Huber, 2003, p. 135-160.

Oates, T. Key skills/key competencies: avoiding thepitfalls of current initiatives. In: Rychen, D. S.;Salganik, L. H.; McLaughlin, M. E. (eds) Contri-bution to the second DeSeCo’s symposium.Neuchâtel, Switzerland: SFSO – Swiss FederalStatistical Office, 2003.

OECD. Definition and selection of competencies(DeSeCo): theoretical and conceptual founda-tions (Strategy Paper). Paris: OECD, 2002.Available from Internet: http://www. statistik.admin.ch/stat_ch/ber15/deseco/deseco_strategy_paper_final.pdf. [cited 6.6.2003]

OECD. The well-being of nations – the role of humanand social capital. Paris: OECD, 2001a.

OECD. Investing in competencies for all. Commu-niqué for the meeting of the OECD EducationMinisters, Paris, 3-4 April 2001. Paris: OECD,2001b. Available from Internet: http://www.oecd.org/pdf/M00008000/M00008906.pdf [cited6.6.2003]

OECD. Measuring student knowledge and skills: anew framework for assessment. Paris: OECD,1999.

Ouane, A. Defining and selecting key competenciesin lifelong learning. In: Rychen, D. S.; Salganik,L. H.; McLaughlin, M. E. (eds) Contribution tothe second DeSeCo’s symposium. Neuchâtel,Switzerland: SFSO – Swiss Federal StatisticalOffice, 2003.

Perrenoud, P. The key to social fields: competen-cies of an autonomous actor. In: Rychen, D. S.;Salganik, L. H. (eds) Defining and selecting keycompetencies: theoretical and conceptualfoundations. Göttingen: Hogrefe and Huber,2001, p. 121–150.

Ridgeway, C. Joining and functioning in groups,self-concept and emotion management. In:Rychen, D. S.; Salganik, L. H. (eds) Defining andselecting key competencies: theoretical and

conceptual foundations. Göttingen: Hogrefeand Huber, 2001, p. 205–211.

Ritchie, L. Key Competencies for whom? A laborperspective. In: Rychen, D. S.; Salganik, L. H.(eds) Defining and selecting key competencies:theoretical and conceptual foundations.Göttingen: Hogrefe and Huber, 2001,p. 236–240.

Rychen, D. S. Introduction. In: Rychen, D. S.;Salganik, L. H. (eds) Defining and selecting keycompetencies: theoretical and conceptualfoundations. Göttingen: Hogrefe and Huber,2001, p. 1-15.

Rychen, D. S.; Salganik, L. H. (eds) Defining andselecting key competencies: theoretical andconceptual foundations. Göttingen: Hogrefeand Huber, 2001.

Rychen, D. S.; Salganik, L. H. (eds) Key competen-cies for a successful life and a well-functioningsociety. Göttingen, Germany. Hogrefe andHuber, 2003.

Rychen, D. S.; Salganik, L. H.; McLaughlin, M.Contribution to the second DeSeCo’s sympo-sium. Neuchâtel, Switzerland: SFSO – SwissFederal Statistical Office, 2003.

Salganik, L. H. Competencies for life: a conceptualand empirical challenge. In: Rychen, D. S.;Salganik, L. H. (eds) Defining and selecting keycompetencies: theoretical and conceptualfoundations. Göttingen: Hogrefe and Huber,2001, p. 17–32.

Salganik, L. H. et al. Projects on competencies in theOECD context: analysis of theoretical andconceptual foundations. Neuchâtel, Switzerland:SFSO – Swiss Federal Statistical Office, 1999.

Stein, S. Equipped for the future content standards:what adults need to know and be able to do inthe 21st century. Washington, DC: NationalInstitute for Literacy, 2000.

SFSO – Swiss Federal Statistical Office. Countrycontribution process: summary and countryreports. Briefing materials prepared for DeSeCo’s2nd International Symposium, Geneva,11-12 February 2002. Neuchâtel, Switzerland:SFSO, 2001a. (CCP Reports) Available fromInternet: http://www.statistik.admin.ch/stat_ch/ber15/deseco/deseco_country.htm) [cited6.6.2003]

SFSO – Swiss Federal Statistical Office. DeSeCobackground paper. Neuchâtel, Switzerland:SFSO, 2001c. Available from Internet:


http://www.statistik.admin.ch/stat_ch/ber15/deseco/deseco-background-paper02.pdf[cited 6.6.2003]

The World Bank. Lifelong learning in the globalknowledge economy: challenges for devel-oping countries. 2002. Available from Internet:http://www1.worldbank.org/education/pdf/Lifelong%20 Learning_GKE.pdf [cited 6.6.2003]

Trier, U. P. Defining educational goals: a window tothe future. In: Rychen, D. S.; Salganik, L. H. (eds)Defining and selecting key competencies: theo-

retical and conceptual foundations. Göttingen:Hogrefe and Huber, 2001, p. 241-246.

Trier, U. P. Twelve countries contributing to DeSeCo: asummary report. In: Rychen, D. S.; Salganik, L. H.;McLaughlin,M.E. (eds) Contribution to the secondDeSeCo’s symposium. Neuchâtel, Switzerland:SFSO – Swiss Federal Statistical Office, 2003.

Weinert, F. E. Concept of competence: a conceptualclarification. In: Rychen, D. S.; Salganik, L. H.(eds) Defining and selecting key competencies:theoretical and conceptual foundations.Göttingen: Hogrefe and Huber, 2001, p. 45-66.


International symposia proceedings and papers

DeSeCo Symposium, Neuchâtel, October 13-15, 1999. Available from Internet:http://www.statistik.admin.ch/stat_ch/ber15/deseco/deseco_symp99.htm DeSeCo Symposium, Geneva, February 11-13, 2002. Available from Internet:http://www.statistik.admin.ch/stat_ch/ber15/deseco/deseco_int02.htm

Further information

DeSeCo SecretariatDominique Simone RychenProgram ManagerSwiss Federal Statistical OfficeEspace de l’Europe 10CH-2010 Neuchâtel

Phone: +41 (0)32 713 61 60 Email: [email protected]: www.deseco.admin.ch

an overarching conceptual framework for assessing key ... · ence for assessing and developing key...

Documents