DOCUMENT RESUME

ED 059 890 SE 013 338

TITLE Council ot Europe information Bulletin 3/1971.INSTITUTION Council of Europe, Strasbourg (France). Documentation

Cente, for Education in Europe.PUB DATE Dec 71NOTE 75p.

EDRS PRICE MF-$0.65 HC-$3.29DESCRIPTORS *Achievement Tests; *Aptitude Tests; Grading; Higher

Education; *Objective Tests; Research; *StudentEvaluation; Test Reliability; *Tests; TestValidity

IDENTIFIERS Council of Europe

ABSTRACTThe major part of this bulletin consists of two

studies on the place of examinations in the school system. In a loigpaper (28 pages) on "New Techniques for Assessment of Pupils' Work,"A.D.C. Peterson discusses: (1) efforts to improve reliability by theuse of objective tests, improved marking procedures, and betterstandardization; (2) efforts to improve validity by the use ofcontinuous assessment and taxonomies of objectives; (3) scholasticaptitude tests; and (4) oral examinations. The second study consistsof three shorter contributions relevant to the question of secondaryschool leaving examinations: "Examination Research: Results Thus Farand Outlook for the Future," by M. Reuchlin; "Objective Testing andEducational Assessment," by W. D. Halls; and "From Point-in-timeExamination to General Assessment," by J. Capelle. The remainder ofthis bulletin is made up of reports of meetings of the ConsultativeAssembly of the Council of Europe, the Council for CulturalCooperation, and the Committees on Higher Education anri Research,General and Technical Educatioa, Out-of-School Education and CulturalDevelopment, and Educational Documentation and Research. MO

December 1971

Contents

pageFIRST PART

Consultative Assembly = 1

Council for Cultural Co-operationHigher Education and ResearchGeneral and Technical Education ............ .......Out-of-School Education and Cultural Development ..Educational Documentation and Research

SECOND PART

.........

........... ..........

7

9

13

20

25

Council of Europe Studies

New Techniques for Assessment of Pupils' Work, A. D. C. PetersonGeneral introduction 27

Efforts to improve reliability 30

Efforts to improve validity 40

Scholastic aptitude tests 49

Oral examinations 52

Conclusion 54

The Egger report on Secondary School-leaving Examinations

Examination research : results thus far and outlook for the future,M. Reuchlin 56

Objective testing and educational assessment, W. D. Halls ........ 61

From "point-in-time" examination to general assessment, J. Capelle 66

PUBLICATIONS 71

The Information Bulletin which is distributed free of charge three times a year in anEnglish and a French edition, informs on the educational, cultural and scientific activitiesof the Council of Europe and reprints important policy documents of European interestin these fields.

First Part

Consultative Assembly

During its twenty-first session, which took place in Strasbourg from 4th to 9th October1971, the Consultative Assembly of the Council of Europe held an important debate onculture and:,edueation. The following reports were presented : "European Co-operation inthe field of Culture and Education" by A. Borel ; "Present trends in educational reformand further prospects with a view to permanent education" by J. Capelle ; "The settingup of a tele-university" by G. Vedovato. At the close of the debate the Assembly unani-mously adopted a number of resolutions and recommendations.

EUROPEAN CO-OPERATION IN THE FIEL_J OF CULTURE AND EDUCATION

In the framework of the Consultative Assembly's annual debate devoted to culture andeducation, Mr. A. Borel, Chairman of the Committee on Cultui and Education, presenteda report on the situation of European co-operation in this field. The report, which tradi-tionally deals with recent developments and future prospects, emphasizes this year theimportance or a more pragmatic approach to European co-operation and solutions liableto reactivate this co-operation in the short term. After recommending that a EuropeanOffice of Education should be set up [Rec. 567 (1969)1, the Assembly, wishing to cometo an arrangement with the Committee of Ministers, agreed that the CCC should performthe functions allocated to such an office. This could bring about certain transformations,which the Assembly feels are inescapable.

The Assembly unanimously adopted the Recommendation and the Resolution whichunderline the main ideas of the report. Extracts of them are given below.

RECOMMENDATION 649 (1971

on European co-operation in the field of culture and education

The Assembly,

Recalling its proposals for restructuring and reinforcing European cultural co-opera-tion, and in particular its Recommendation 567 (1969) on "Twenty years of Europearcultural co-operation" ;

_ Confirming that its proposals are designed to re-establish European educational andcultural co-operation on fresh foundations on the eve of the enlargement of the Euro-pean Communities ;

Conscious that with this in mind governments will find it necessary to review theterms of reference and functions of European intergovernmental organisations, andthat it is therefore more important than ever to insist on the Council of Europe'sspecial task in the field of culture and education, and in particular with regard to thedefinition and application of a European policy for permanent education and culturaldevelopment ;

Increasingly concerned by the fact that in the educational field Europe is laggingbehind what has been accomplished in the economic sphere because it has been unableto avail itself of a complete, integrated and coherent system of co-operation, and con-vinced that the desire for the widest possible cultural unity in Europe should lead to

.;*

the search for such systems not in a community restricted to a small number of coun-tries, but in the wider framework of the States parties to the European Cultural Con-vention ;

Observing that by their very nature problems concerned with education and culturecannot suitably be dealt with by a community as such, and emphasising on the otherhand the admirable flexibility of the system instituted in the Council of Europewhereby a certain number of governments are able to co-operate in the frameworkof so-called "partial or limited" agreements with a view to carrying out priorityprojects over a number of years, by means of which the governments concerned areenabled to intensify their co-operation in a given field in a way which enables all themember States to t--alef it from the results obtained ;Realising that, though the establishment of a European Office of Education as advo-cated in Recommendation 567 (1969) must be regarded as a long-term objective, thereis an immediate need to find a practical solotion by conferring at once on the CCCthe task-of- performing the functions of such an Office on an experimental basis ;Aware that in this case it would be essential to review if not the terms of referenceat least the composition of the delegations to the CCC as well as the vital problemconcerning the relations between that body and the Conference of European Ministersof Education ;

Considering in this context that the CCC should not limit itself to mere study andresearch, but assume certain political responsibilities which alone would enable it topass beyond the stage of mere international co-operation and reach that of commonredefinition of national policies ;

Believing that, in order to faciliLate such a development, it is necessary to provide thesystem of co-operation with "political leadership" and put the CCC under the techni-cal control of the Conference of Eizropean Ministers of Education and a similar Confe-rence of European Ministers of Culture the establishment of which is becomingincrersingly indispensable if we Pre to encourage a long-term policy of culturaldevelopment ;

Recommends the Committee of Ministers :to call upon the Conference. of European Ministers of Education :

to ensure to the fullest possible extent the co-ordination from the planning stageonwards of the activies of the varicus international organisations concernedwith the field of education ;to exercice, in accordance with Recommendation 567 (1969), a technical controlover the CCC with regard to the development of education ;

to establish a Conference of European Ministers of Culture whose principal taskwould be to lay down, for the guidance of the CCC, in association with represen-tatives of any other ministries which might be concerned, the priorities for a Euro-pean programme of cultural development ;to instruct the CCC to perform, for an experimental period of five years, the func-tions allocated to a European Office of Information in accordance with the letterand spirit of Recommendation 567 (1969), and for this purpose :

to revise the composition of the delegations to the CCC by ensuring the predomi-nance of the educational and cultural elements through the presence at the headof these delegations of officials from the immediate entourage of the EuropeanMinisters of Education and of the Ministers responsible for culture ;to establish a plan with a view to at least tripling over a period of five years thegovernmental contributions to the Cultu-:al Fund, so as to permit that body toprovide adequate finance for the harmorious expansion of a European programmefor permanent education and long-term cultural development in consonance withthe aims of the Council of Europe.

2

RESOLUTION 499 (1971)

on European co-operation in the field of culture and educaionThe Assembly,

Having regard to the work of the 7th Conference of European Ministers of Education,held in Brussels from 8 10 June 1971, and noting with satisfaction the resolutionsadopted by that Conference ;Noting with special satisfaction that, in accordance with the spirit of Assembly Re-commendation 567 (1969), the Conference adopted a permanent statute, thus fulfillingone of the conditions essential to enable it to play to the full its role at the head of asystem of European co-operation in urgent need of reform ;Fully approving the decisions taken by the Conference which, with a view to ensuringthe continuity of its work, extended the terms of reference of its Committee of SeniorOfficials whose task in future will consist not merely in the preparation of futureconferences, but in observing the development of the situation in Europe in the fieldof education, in maintaining closer contact with the international organisations con-cerned and in the practical implementation of resolutions of the Conference, thus infact making this Committee an organ capable of taking technical decisions ;Instructs its Committee on Culture and Education to establish close contact with theConference of European Ministers of Education, and in particular its Committee ofSenior Officials ;Calls on its members to take all necessary steps to ensure that the Ministers of Educa-tion will be present in person at forthLoming conferences or are represented by otherministers, and in all cases by political personages capable of undertaking responsi-bilities on behalf of their governments.

PRESENT TRENDS IN EDUCATIONAL REFORM AND FURTHER PROSPECTSWITH A VIEW TO PERMANENT EDUCATION

In presenting his report on "Present trends in educational reform and further prospectswith a view to permanent education", Rector J. Capelle, Vice-Chairman of the Committeeon Culture and Education, informed the Assembly about the programme and the resultsof the Symposium on Basic education held in Salerno (Italyy. in July 1971. He empha-sized that his report endeavours to put out, on the basis of the Symposium documents,the questions affecting new trends in education. Commenting on the four topics of theSalerno meeting, the rapporteur gave detailed information on each of these : relationsbetween parents, teachers and pupih, acquisition of means of expression at the variouslevels of basic education, study of the attitude towards knowledge and finally the placeof technical education at the basic levels of education.

In concluding, Rector Capelle observed that each of these themes brought out anotheraspect of the need to rethink the traditional educational system, and consequently tea-cher training. He expressed his regrets that very often in the past, people were contentedwith small reforms whieh superficial as they were, frequently served to disguise a fun-damentally conservative approach. The Resolution, which was unanimously adopted, isgiven below.

RESOLUTION 500 (1971)

on present trends in educational reform and further prospects with a view to per anenteducation

The Assembly,

Recalling its Recommendation 611 (1970) and Resolution 46 (1970) on permanen edu-cation in Europe ;Having regard to the report by its Com ittee on Culture and Education on present

trends in educational reform and future prospects wii li a view to permadent educa-tion, and noting especially the results of the Symposium on Basic Education held inaccordance with the aforementioned resolution on 28 and 29 June 1971 at Salerno(Italy) ;

Noting that the aim of this Symposium was to .-tudy :

in what way education, and more generally educational activities open to youngpeople from birth to 18 years of age, should be devised so as to meet the demandsof permanent education ;in support of this study, to put for ard a set of specific measures for considerationby governments ;

Noting in this context that the Symposium dealt with the following subjects, on whichthe problems of school reform in Europe are concentrated :

the three groups of participants in the educational process : parents, teachers, pupils ;the acquisition of means of expression at the various levels of basic education ;

the reappraisal of the attitude towards knowledge ;

the place of technical education at the basic levels of education ;

Invites the Conference of European Ministers of Education and the Council for Cul-tural Co-operation, in their work aimed at reforms in the school system from theviewpoint of permanent education to be pi ()posed to the States adhering to the Euro-pean Cultural Convention, to be guided by the principles and measures set outhereunder.

General principles and specific measuresIn regard to the three participating groups (pare ts, teachers, pupils)

There is a crisis in the relationships among parents, teachers and pupils. Participationshould be neither a dilution of responsibilities nor a misapprehension as to where com-petence lies, nor a rejection of authority. Thus it shall be organised on a realistic footing,by ;

preparing the pupils to assume a measure of responsibility accord ng to their degree ofmaturity, while satisfying their need for security and guidance ;

encouraging the "school for parents" by bringing in large-scale educational assistanceand integrating this into an educational insurance scheme on the lines of existingsocial insurance schemes as part of a coherent system of methods designed to trainchildren for their full responsibility as adults ;

in an endeavour to correct the inequality of social 3pportunities, developing nurseryschools and taking steps to bring them within closer reach of families, particularly insparsely populated rural areas ;

encouraging co-operation according to the age of pupils from the viewpoint cyc con u-ing education ;

studying the mutual responsibilities and the deontology of the teaching profession ;making the relations between parents and teachers more reciprocal and more func-tional ;

developing appropriate methods of compensation in favour of children whose familybackground is culturally inferior ;

developing a critical spirit among the under-eighteens and defining the limLts ofprotest action that represents a constructive preparation for maturity.

4

regard to the acquisition cf means of expression

One of the main objects of basic education is to equip the individual with means ofexpression so that he may then develop his personality fully and establish relations withhis surroundings.

This, then, is a matter of developing normative expression (mathematics, languages,conventional drawing) and at the same time spontaneous expression (artistic, poetic) toencourage individuality.

Measures proposed :

to stimclate artistic expresrHn from pre-scho l days, primarily through games andthe child's freedom of choice, and to continue such action by appropriate means at allstages so that it can be dovetailed with subsequent artistic activities when adult ageis reached ;

to introduce the practical and direct teaching of a foreign language at the pre-schoolstage, and to this end

to see that teachers are trained ;to define a teaching system geared to the objectives peculiar to each of the stagesin compulsory educationto plan and make the best use of the teaching aids made available by moderntechnology ;from the outset of compulsory schooling, to teach pupils to speak, particularly indiscussions or at symposia and in front of an audience, beginning before their owr,class.

in regard to the reappraisal of the attitude towards knowledge

In view of the rapid increase in knowledge, its changing pattern and frequently itsobsolescence, the basic question is how to approach and use knowledge. It is a matter ofreshaping education, giving the methodology of access to knowledge priority over theacquisition of knowledge.

There is need for a reappraisal of the concept of "discipline" in the sense of the multi-disciplinary approach to social and to scientific thought in the contemporary world. Basiceducation should prepare the pupil for integration into three environments :

the human environment, through the study of civilisation ;the natural environment, through knowledge of ecology ;the technological environment.

Measures proposed :

to introduce into basic education practical training in the "processing- of informationso as to prepare young people, through their own active participation, to "store" infor-mation or to "select" it. For this it is necessary to pass from the stage of the staticlibrary, designed for the scholarly, to the data-processing laboratory ;

to encourage docimological research which should be developed with the aim ofdefining what qualities of the pupil's personality require analysis, particularly byreference to an ideal "profile" where basic qualities would be precisely defined andcapable of quantitative assessment ;

-- to redefine the teacher's function, which should aim at developing in the pupil adynamic and responsible attitude towards knowledge and in face of the situations hewill meet, in particular the pressures of information.

5

7

a the place of technical education at the level of basic education

The aim is to give to technical training, an essential factor of economic and social pcress, the position and prestige consonant with its mission.

Specific measures :

To improve the rewards for "produc on duties, at present lagging behind "service"duties :

to help persons with trade qualifications craft workers, skilled workers, technicians)to gain earier access to jobs as representatives and to positions of responsibility ;

to ensure that operational skills and the corresponding diplomas rank as high in publicesteem as degrees testifying to academic excellence ;

to expand technical education in two directions : better interchangeability betweenspecialisations and better training in human terms, partieularly in the mastery ofexpression and knowledge of social patterns ;

to introduce technical subjects into the general education preceding initial technicaleducation, principally in the lower secondary stage ;

to link up the technical educational establishment and professional circles with a viewto permanent education, from several viewpoints :

by developing vocational training through combi ed efforts on the part of teachersand firms ;by arranging tra ning pe,rods during school studies ;

by arranging for supervision and sponsorship of adaptation periods for those whohave just left school ;by arranging for people already at work to undergo training for different jobs, tobring themselves up to date, or to qualify for promotion ;

to combine political action with suitable co-operation between technical education andindustry on behalf of young people who after their national service have not under-taken any further studies and have not yet secured a job.

EUROPEAN TELEVISION UNIVERSITY

The rapporteur, Mr. G. Vedovato, recalling the Order No. 308 (1970) adopted and theterms of reference given by the Assembly to work out a project of European TelevisionUniversity, stressed that the Institute would be acting as a medium to stimulate univer-sities to produce television teaching programmes or else it would produce these program-mes in co-ordination with them, or any other specialised institute. Its activity would beextended to all levels : pre-university, university, post-university, and would respond tothe ever-growing educational needs. The rapporteur submitted to the Assembly a Recom-mendation and a European Television University Project conceived as a European inter-university institute for the development of multi-media distant study systems.

Council for Cultural Co-operation

The twentieth session of the CCC was held in Stroshoure from 17th to 23rd September1971. It was attended by delegates f. a twenty member States, representatives of theConsultative Assembly, the Chairmen of the CCC's permanent Committees, as well asobservers from -UNESCO, OECD, the Commission of European Communities and theEuropean Cultural Foundation.

After having heard the customary progress reports of each of the chairmen of the threepermanent Committees and the statements by the representatives of the ConsultativeAssembly, the CCC examined the various items placed on its agenda and adopted itsprogramme/budget for 1972 in its new form.

Below is given a summary of the conclusions concerning three fields, namely : Europeancultural co-operation, permanent education, requirements of European higher educationfor satellite communication services and frequency band allocations.

Twenty years of cultural. co-operation European Office of Education

Pursuing the discussion on Recommendation 567 of the Consultative Assembly on TwentyYears of European Cultural Co-operation and in the light of proposals made and decisionstaken at the Seventh Conference of European Ministers of Education, the CCC adoptedan Opinion for the attention of the Committee of Ministers. Excerpts of it are givenbelow :

"Invited by the Committee of Ministers to 'study the long-term aspects of the Assembly'sproposal for the creation of a European Office of Education' and to report to it thereon,the CCC has noted that the Conference of European Ministers of Education, in deciding togive itself a permanent character while maintaining its independent status, has placedgreat stress on the development of collaboration between the international organisationsalready active in the field of education in Europe, and has envisaged for the CCC impor-tant tasks involving the promotion of new and intensified forms of co-operal Jn betweenthe countries of Europe. Following the thought underlying Resolution No. 3 of the BrusselsConference, the CCC shares the opinion already expressed by the Committee of Ministersthat it would be premature at this stage to establish a European Office of Education asa separate institution.

It has now decided to set up a Working party which will examine further the practicalmeans whereby the functions which had been envisaged for a European Office of Educa-tion can be progressively carried out within the CCC itself.

One of the first tasks for the Working Party will be to consider the possibility of settingon foot, in selected cases, projects called the 'priority subjects' which might be supportedand financed by those member governments most directly interested.

It is not intended that the Working Party should consider afresh the general operationsor programmes of the CCC in the educational field, since these are already the subject ofcontinuous study by the CCC itself. It is envisaged, however that the Working Party,subject to the results of its consideration of the principle of 'priority projects' as definedabove, should provisionally select one or two such projects with a view to examining theirimplications and potentialities in greater depth. Projects which would offer possibilities ofeconomies to individual member governments through increased European co-operationwould merit particular attention in this context.The Working Party will also consider, having regard to the decisions of the Brussels Con-ference, means of strengthening relations between the CCC and the Committee of SeniorOfficials responsible for the preparation and follow-up of the Conferences of EuropeanMinisters of Education."

7

9

Permanent cc/ /cation

The CCC was informed of the work and prcposals of two meetings, one held in Paris on7th 9th June 1971, the other in Strasbourg on 15th 16th September.

The September meeting attended by representatives of the three Permanent Committeesand by experts, approved the report preparec. under the responsibility of Mr. B. Schwartz,Project Director, on "Fundamentals for an integrated educational policy" as well as theworking plan suggested by the Second Round Table on Permanent Education which metin June.

After discussion, the CCC adopted the document on the fundamentals for an integratededucational policy and decided to set up a Steering Group, the functions and workingprocedures of which would be

to select, for study and evaluation, on the basis of criteria established by the CCCpilot experiments in progress in member States ;to act a body available to the three Permanent Committees for purposes of consul-tation ;to examine, once a year, with representatives chosen from each of these Committees,the Committee programmes in the light of the concept of permanent education and toreview its own work and the criteria for the selection of pilot experiments.

As regards these criteria, the participants were of the unanimous opinion that the pilotexperiments should have an important bearing on the work of at least two PermanentCommittees of the CCC and should exemplify the practical application of one or more ofthe main principles of permanent education. Among the most important of these are :

the promotion of the process of learning throughout life, whether for vocational ornon-vocational reasons ;the promotion of the means of continuous review of education systcms, with the activeparticipation of the teachers and with particular reference to curriculum reform ;the promotion of participation in the educational process by those taught.

The study and evaluation of experiments in progress in member States geared to theconcept of permanent education would be the second operational phase of work in thisfield.

Require ents of European higher education for satellite communication services andfrequency band allocations

The CCC was informed that the Committee for Higher Education and Research hadapproved the final report adopted by the Steering Group on "Requirements of Europeanhigher education for satellite communication services and frequency band allocations" byMr. H. L. Jankovich, consultant expert.

The aim of this project is threefold :Short-term objective : to make sure that the World Administrative Conference of theITU (International Telecommunications Union) reserved certain frequency bands foreducational purposes ;

Medium-term objective : to set out in detail, on the basis of Mr. Jankovich's scientificstudy, the nature of these needs and the technical means for meeting them ;Longer-term objective : progressively to implement the ideas and suggestions set outin the Jankovich report. This work will have to he carried out by national bodies,institutes and departments, as well as by international organisations, and in particularby the Working Party on Educational Technology.

In conclusion, the CCC decided to refer the J ankovich Report to :The Steering Group on educational technology for study and follow-up action

the Assembly Committee on Science and Technology for its opinion ;thc following organisations for information :

International Telecommunications Union (ITU)European Space Research Organisation (ESRO)

a European Space Vehicle Launcher Development Organisation (ELDO)European Conference on Satellite Communications (CETS)European Space Conference (ESC).

Documents : CCC (71) 36 ; DECS/1nf. (71 ) 8.

H gher Education and Research

Strasbourg 27th - 29th October 1971

Twenty-fourth meeting of the CommitteeThe autumn meeting of the Committee, attended by delegates of nineteen member States,by observers from UNESCO, the European Communities and the Consultative Assembly,was devoted, in particular, to the discussion of the future programme and workingmethods.

In examining the draft programme for 1973, the Committee decided to continue to con-centrate its future activities around some major fields.Two UNESCO projects were also discussed : one concerning the possible creation of aEuropean University, and the other the setting up of a European Centre for HigherEducation.

Moreover, the Committee gave its approval to the concept of creating a European Inter-University Institute for the Promotion of Distant Study Systems. It emphasised, however,that the use of the term "Tele-University" should be avoided. With regard to the foundingof a European "Open University", the participants were unanimously in agreement thatsuch a scheme was still premature.

Documen_s CCC/ESK 71) 87.

Florence 30th - 31st August 1971

The creation of a European tele-university(Ad hoc Sub-Committee)

Parliamentarians from the Consultative Assembly's Committee on Culture and Educationand university representatives from the CCC's Committee for Higher Education and Re-search examined together various aspects of a possible European "tele-university" to beset up in Florence. They discussed in particular the objectives and the functions, the terms

9

10

of reference, the status and the organisation, as well as the staff c- egories of such aninstitution. A summary of conclusions is printed below.

Ob)ectives and functions

The tele-university must be neither a ''super university" nor a "counter university". Itsmain aim would be to promote multi-media distant study systems in member States andto help the national universities to produce software packages and to set up such teachingand learning systems. Its activities will therefore cover not only television but also medialike films, video cassettes, video tapes, correspondence material, programmed text books.It would help to meet the problems of student influx, to make higher education accessibleto a wider public. Likewise, the teaching of outstanding specialists would be available tostudents of other universities. It would address students in higher education, graduateswishing to polish up their knowledge and adults who are neither students in higher edu-cation, nor graduates.

As regards inctions, the "tele-university" w Lad have the following tasks :

Collection of information on national experiments with multi-media distant studysystems and available software in higher education ; confrontation and evaluation ofthese experiences ;

Supply of technical assistance by creating an exchange of existing material so that thesoftware in one country would become available to universities in other countries ;Organisation of meetings of national administrators responsible for the possible intro-duction of multi-media systems in higher education ;Assistance in the preparation of multi-media software packages, e.g. by way of con-vening university teachers in selected disciplines in order to reach agreement on thepossible content of such material ;Research into all aspects of multi-media distant study systems ;Promotion of multi-media distant study systems ;

Organisation of training courses for university teachers to introduce them to the newmethods and techniques.

Ternls of reference

Apart from training courses for university teachers, the "tele-university" xrou1d not pro-vide direct teaching. It would also refrain from producing the necessary software packagesitself. With the exception of refresher courses, the content of the teaching would alwaysbe at higher education level and it would be so conceived that formal integration into astudy course in higher education would always be possible.

Statuts and organisa on

The "tele-university" would legally be an independent institution to be created under theauspices of the Council of Europe by way of convention open to signature to all memberStates of the CCC. Its bodies would be composed of a director (or a body of directors), ascientific council and an administrative council.

Staff

The staff would consist of the three following categoriespermanent and temporary academic staff ;professional staff experienced in the new media ;

-- technical staff.

ecpx. 12

During the meeting it was generally emphasised that the tele-university was not to beconsidered as a proper university, as it was not going to have students of its own, norwas it going to grant degress and diplomas. For this reason and at least by some partici-pants, it was recommended to avoid using the term "university". Once set up, the institu-tion, as proposed by some participants, might be called "European Inter-University Insti-tute for Tele-Teaching" or a variation of this title.

Document : CCC/ESR (71) 62.

itrasbourg 4th - 5th November 1971

Mobility, of higher education staff and research workers(meeting of experts)

Greater mobility of university staff and research workers is of vital importance for pro-gress in research arid for the restructuring of higher education at European level.

On the basis of reports and documents presented to the meeting, the participants fromnine member States examined the present situation and tried to distinguish the priorityneeds in this field.

The most important factors of a concerted policy on mobility were defined by Mr. H.Lesguillons, President of the Association "Europe Université". Linking closely the themeof mobility with structural reforms and current trends, he stressed both in his report andin his statement the harmonisation not only of initiatives but also of university regula-tions and career structures, as well as the removal of legal and statutory obstacles. Finally,he pointed out that four types of stimulus should be developed :

the liberalisation and systematic diffusion of information ;the extension of the right of teachers and research workers to permanent training ;the setting up of machinery for equivalences ;the development of facilities to promote integration into the host country of foreignacademic staff.

After discussion, participants agreed that the national policies, pursued in recent years,have gradually removed some of the main obstacles to mobility. However, it is still tooearly to talk about their complete removal in the near future.On the other hand, abolition of the legal requirement whereby teaching or research postsin higher education must be held by nationals of the country concerned could have littlepractical effect, if State regulations require national diplomas for access to the teachingprofession.

In the interest of mobility within Europe, it is also necessary to remove the differencesbetween the national structures as well as between staff structures of higher education andto break down compartmentalisation.

Furthermore, the meeting decided that highly specialised seminars in the natural sciences,such as the EUCHEM, EUREMECH, etc. Conferences should continue to be organised.

After having examined the present situation, the meeting dealt with its future work pro-gramme. It was aware of the fact that complete freedom of movement within Europefor university staff and research workers could not be reached merely through changesin the legal requirements but that it presupposed a long slow evolution. It was feltnecessary to proceed gradually by means of short and long-term stages.Lastly, concrete proposals were put forward for further action. The following priority

11

13

areas were chosen student mobility, short-term mobility of staff and improvement ofthe systematic diffusion of information.

As regards a European status for staff in higher education and research, as envisaged inResolution No. 2 adopted by the Seventh Conference of European Ministers of Education,the participants recommended the definition of certain basic principles. It was felt that aEuropean partial agreement between member States with comparable higher educationsystems might contribute to the formulation of such principles.

Docu ents : CCC/ESR (70) 18 ; 19 ; 28.CCC/ESR (71) 11 ; 47 rev. ; 84.

Strasbourg 9th 10th Nove ber 1972

Ethics of-science(Meeting of experts)

Experts from thirteen member States together with observers from UNESCO dis ussedvarious problems connected with the responsibility of scientists.Advances in science and the applications of science have given rise to a host of problems,with ethical and moral implications, in which scientists must feel specially involved.These matters are of concern to many other people and in particular to politicians, doc-tors, religious leaders and educators. The participants were aware that no particular groupis likely to find solutions alone, and see value in a common approach. There is a need forcontinuing interaction among these groups, interaction that . will involve new forms ofco-operation and some changes of attitude.The role of scientists may often be not to offer a solution but rather to provide "a warningof risks and proclamation of benefits and discussion of quandaries".Nevertheless, the present attitudes of scientists need reassessment, particularly the normand value commitments implicit in their activities. The group was impressed by the sug-gestion that a code of ethics should be accepted by scientists similar to that adopted bythe medical profession.The group then turned to problems of education a ad p: rticularly to the introduction inthe education of scientists of a better understandin'i- their role in society. This workwill extend over several disciplines including social and behavioural sciences and willcall for the co-operation of several groups of specialists such as sociologists. A problem-oriented treatment is likely to be the most effective. An attempt should be made to intro-duce studies of this kind into university curricula ; this will provide a new field foruniversity research.

The following recommendations were made :The Committee for Higher Education and Research shouldadvise its members to takesuch steps as are necessary to ensure that these proposals are considered by groups inparticipating countries. It would be important that recommendations from individualcountries should come back to the Committee for Higher Education and Research forfurther consideration by this group or its successor.Further study should be made of the many-sided problems that face politicians andparticularly of the mechanisms that should be developed to aid the co-operation ofscientists and politicians and others in tackling these problems. In some countries thiswill mean bringing together parliamentarians and scientists ; in others it will involvestrengthening and widening existing arrangements.Some group should be asked to do the preparatory work associated with the formula-tion of a code of ethics for scientists.

Documents : CCC/ESR (71) 53 ; 59 ; 64 ; 65 ; 66 ; 68 ; 69 ; 75 ; 77 ; 80 ; 81 ; 91.

12

General and Technical Education

Strasbourg 25th - 29th October 1971

Tenth meeting of the CommitteeThe Committee held its meeting under the chair anship of Mr. J. de Bruyn (Nether-lands). It was attended by delegates of twenty member States, representatives of theConsultative Assembly, together with observers from the Commission of European Com-munities and the European Schools' Day.

The Committee discussed the various items on its agenda, in particular : structure andorganisation of basic education ; teachers ; curricula as well as past, present and futureactivities and conclusions of important meetings which took place during the period of1970-71.

In examining its future activities and in approving the concentration of the programmearound a limited number of priority objectives, the Committee agreed in princ;ple on thegeneral approach of the programme, which is evenly based on five fields of equal impor-tance :

structure and organisation of educa ion ;teachers ;curricula ;media and methods ;assessment and guidance.

As regards the methods and the planning of the programme, the Committee decided toset up co-ordinating groups for the following sectors : pre-school education and primaryeducation ; secondary education ; technical and vocational education ; curricula ; assess-ment and guidance.

Furthermore, the Committee discussed the documentation to be presented to the nextConference of European Ministers of Education, which will be held in Switzerland in1973 and will have as the main theme : "The needs of the 16-19 age group, both in full-time and part-time education".

Document CCC/EGT (71) 47.

Vienna 21st - 25th June 1971

Road safety educa ion in schools(Conference)

The Second Conference of Governmental Experts on Road Safety Education in Schools,organised jointly by the Council of Europe and the European Conference of Ministers ofTransport (ECMT) in co-operation with the Austrian Federal Ministry of foreign Affairs,was also attended by observers from Austria ustrian Government. Delegates from twentyStates parties to the European Cultural Convention and twenty-seven member States ofthe European Conference of Ministers of Transport took part in the Conference, whichwas also attended by observers from Austria and representatives from OECD, the Euro-pean Communities, the United Nations, the IFSPO (International Federation of Senior

13

Police Officers the OTA (World Touring and Automobile Organisation), the PRI (Inter-national Prevention of Road Accidents), and the IFP (International Federation of Pedes-trians).

The Conference emphasised the importance of road safety eduction for children fromthe age of two. It pointed out that the present situation was highly disturbing : statisticson the number of children killed or injured in road accidents showed that the casualtyrate had risen fa..'Aer amongst young people than in the population as a whole. There wastherefore an urgent nt_-.d for action by goverments and local education authorities aswell es by parents and teachers.

The Conference requested the member S ates of the Council of Europe and the EuropeanConference of Ministers of Transport to increase their expenditure on road safety arran-gements and asked the two organisations to urge that the necessary political decisions betaken to this end.

The two main themes of the Conference were "The education of children in road safety"and "The training of teachers for road safety education". Reports were submitted on eachtheme.

Road safety education for children

Research into road safety education has shown that children behave very differently fromadults ; hence the need to adapt children's environment according to the various psycho-logical and physical factors which condition their road behaviour.Alongside the various measures which need to be taken by national authorities to dealwith the problems raised by children (town planning, layout of roads, revision of highwaycodes, attention to the design of vehicles and school buildings, etc.), parents and teachersmust make every effort to prwride children with road safety education that is moreeffective and better suited to the different stages of their development.The objectives of road safety education and its place in the curriculum were preciselydefined by the Conference. It was agreed that road safety education should be dispensedas a compulsory subject, systematically and continuously in kindergartens and in primaryand secondary schools. To obtain its full educational value, it should not be treated as anisolated element, but should be fully integrated with the curriculum, being linked up inparticular with technical subjects, natural science, ethics, social sciences, physical educa-tion and hygiene.

The aim of road safety education should be to make children behave responsibly as bothpedestrians and vehicle-users. At least twenty hours should be set aside for road safetyteaching every school year, the length of a lesson depending on the class.School crossing patrols are an excellent device for substantially improving the safety ofchildren, as well as of adults, on their way to and from school. In view of the very goodresults achieved in countries which have already instituted this arrangement, the Con-ference adopted a considerable number of basic principles on the subject and recommendedthat they be applied in all countries.

Training of teachers for road safety education

'Me Confereace unanimously agreed that parents were primarily responsible for thesafety of their children on roads but that teachers should be required to co-operate withparents, the police and others in a safety campaign.

Teachers should, it was felt, be given thorough training in road safety education, includingthe relevant aspects of child psychology. Teachers already in service should be providedwith introductory courses in the subject and kept in touch with the improvements thatare constantly being made to road safety promotion methods.

International seminars for teachers would also be highly desirable.

14

Palma deMallorca

Teachers should establish close co-operation with the various authorities and groupsconcerned with road safety, such as the police, motoring organisations and pedestrians'associations, both national and international.Also, the results of research into road safety education should be made available to tea-chers. At international level this research should be carried out in co-operation withOECD ; at the same level national ,.ts should be co-ordinated, information pooled andpriority subjects selected.

In conclusion, the Conference stressed the importance of international action to ensure thecontinuation of work on road safety education in schools. For this purpose, it called onthe Council of Europe and the European Conference of Ministers of Transport to set up anad hoc committee of educational and road safety experts.

Its terms of reference would be to follow up and co-ordinate the application of the Con-ference's proposals and recommendations, and it would also serve as an appropriate forumfor the exchange of experience on all sectors of road safety education, including research.

Documents : CCC/EGT (71) 13 ;EC/Conference (71) 14 and 15 + AddenduConference (71) 2, 3 and 4.

21st 26th June 1971

The contribution of audio-visual media to the trainingand further training of teachers(Symposium)

The Symposium, attended by delegates from seventeen member Sta es as well asobservers from national and international organisations, was organised by the SpanishGovernment.

Its aims wereto sum up the experiments carried out by the experts of the Committee for Generaland Technical Education on the contribution of audio-visual media to the training andfurther training of teachers ;

. to define the present arrangements for the production, distribution and use of suchaids for this purpose, and also the methods used for promoting their use and assessingtheir effectiveness ;to identify the main trends in this field and establish a programme for European co-operation.

In recent years, teacher trainers have come to realise that audio-visual techniques affordthem new information, learning and practice possibilities. Research has made it possibleto establish the broad lines of a methodology for the use of audio-visual aids in this field.The methods vary according to the type of training and the type of trainee. On the onehand, there is the need to improve the training of future teachers, and hence ofschool methods and techniques. On the other hand, provision has to be made for in-servicetraining, refresher training and re-training. In both cases audio-visual techniques haveproved effective, provided they are used for their proper purpose and in the propermanner.

Conclusions

The great advantage of audio-visual techniques in teacher training is that, intelligentlycombined, they can at the same time help to reach a mass audience and provide sophisti-

17

15

cated analysis instruments for group-work and self-teaching. The active encouragementgiven to the setting up of learning resources centres in many training establishments willgreatly assist in the rational incorporation of new methods and techniques both in classteaching in such establishments and in separate or complementary self-teaching systems.Careful and systematic combination of audio-visual aids can help further the cause ofeducation on three fronts : the proper training of future teachers, the accelerated trainingof teachers in sectors where there is a staff shortage because of the school populationexplosion and lastly in-service and refresher training. Furthermore, there is reason toexpect that teachers trained by audio-visual methods will naturally tend to use them intheir own teaching. Having been taught by them, they will have practical knowledge andpersonal experience of their strong and weak points.

With the aid of audio-visual media it is now possible to :

manipulate time by recording the sounds of all kinds of pedagogical situations,repeating,. them at will and watching or listening to them individually or in groups ;provide a pedagogical mirror in which the teacher can see himself at work and socriticise and correct his performance ;make diachronic comparisons in order to measure better the students' progre s andthe effectiveness of their training ;create test situations and teach the students how to deal with them.

Future teachers are capable of a better performance if they are so motivated and activatedas to be themselves involved in the educating process. Learning situations should be highlyindividualised as there are considerable differences in the experience, ability and know-ledge of the students.

Audio-visual media help to improve communications between teacher and pupil, as toge-ther they learn a new language, a new mode of expression designed to modify pupils'bel-aviour and overcome their feeling that school and life are two different things.As regards in-service and refresher training, it is important for courses to allow opportu-nities to criticise and to make changes. In the case of practising teachers, audio-visualmedia can be used for a variety of purposes : to inform, to arouse awareness, to changebehaviour patterns and attitudes, to bring teaching up to date, to encourage innovation.It seems to be essential not only that all refresher training techniques be used in combi-nation but also that they be co-ordinated in a comprehensive system (television teaching,attendance at courses, conferences, seminars, correspondence courses).

As for the use of audio-visual media at home, it should be noted that it is not very effective,unless followed up by group discussion. This group use is more effective, since discussionautomatically ensues, but it raises other problems : if organised during school hours, thepupils may suffer ; if organised outside school hours, the teachers have to work over-time.

As regards the production of material, four levels must be distinguished : local (trainingcentres), regional, national and international. At each of these levels production meetscertain types of need and displays a certain complexity and a certain degree of technicalperfection There is a place but a strictly limited one for school production : itsfunction is to satisfy the requirements of perhaps a single class, to personalise professionalproduction, as it were, with made-to-measure material or supplementary detail. The func-tion claim to compete with commercial mveil the mysteries of audio-visual techniquesand to train the students in non-verbal expression. In no circumstances can local produc-tion claim to compete with commercial material. Regional, national and internationalproduction must be adapted to the available production facilities, users' needs and generalobjectives of education. At these three levels production should concentrate more onmulti-media material rather than on isolated documents.

The participants in the Symposium agreed that international exchanges and co-product onwere essential.

16

Reconmertda_ions

On the basis of these conclusions, the delegates approved the following recommendations.The Council of Europe should :

set up a working party to investigate information, documentation and research in tea-cher raining and further training, and the production and distribution of suitableaudio-visual material ;compile a glossary of terms used in connection with the training and further trainingof teachers ;promote and co-ordmate the use of audio-visual media in that field ;study the problem of copyright in relation to the increasing possibilities of reproduc-tion ;take stock of the situation in member Sta with regard to the use of learning labora-tories ;encourage the various countries to instal closed circuits and organise a symposium ofspecialists to work out methods for the tr aining of users ;invite member States to study the problem of standardising audio-visual equipment,with particular reference to the international compatibility of video-tape recorders ;facilitate access by training establishments to the archives o f radio and televisionorganisations.

Documents : CCC/EGT (71) 9 ; 29 ;DECS/EGT (71) 29 ; 39 ; 44.

Vienna 22nd d Tune 1971

Films on road safety education in schools(Meeting of experts)

Every year the Committee for General and Technical Education arranges for a smallgroup of experts to view and select films and other audio-visual material in conjunctionwith one of the activities in the Council of Europe's programme. This year, in connectionwith the Second Conference of Governmental Experts on Road Safety Education inSchools, material relating to that subject was viewed and selected by a group of expertsfrom six member States.

Fourteen countries were represented by the material, which included forty-four films. Inmaking their selection, the experts had regard to the material's educational value, tech-nical quality and European nature (i.e. its suitability for international exchanges). Theyjudged the material as a whole and, rather than select best items, chose some examplesof current trends in the production of educational films. They accordingly distinguishedbetween films for the classroom (8 mm, usually short), motivational or introductory films(16 nun lasting 15-20 minutes) and films equally suitable for use in schools, on televisionand in adult education (35 or 16 mm lasting 30 minutes or so).

The following selections were made :

Category one : Material for classroom teaching

Films :

" Jeux et circulation (France)"Chercher Verreur n° 4" (France)

17

"Joupi n" 2" (Belgium)"FWU-Information : Pamfi Medienkombination" (Federal Republic of Germany).

Sets of slides :"Slides on traffic education" Spain)"Trafik GUveni- Turkey).

Category two : Motivational or introductory"Der Radfahrer" (Austria)"Zoo Logic" (Ireland).

Category three : Films for schools, telev s on and adult education"Mit voller Wucht" (Federal Republic of Germany)"Peut-il s.'arrêter pile" (France).

Technical details of this material are to be found in the catalogue, document DECS/EGT(71) 67. The material selected was also shown to the participants in the Second Conferenceof Governmental Experts on Road Safety Education in Schools.

There is a possibility of grants being provided to enable the films to be dubbed in theCouncil of Europe's official languages.

Docu ents : DECS/EGT (71) 67 ;CCC/EGT (71) 28.

BrusselS 4th 8th October 1971

Creativity and artistic activities in school(Symposium)

The Symposium was devoted to the discussion of questions relating to the developmentof creative powers among pupils in primary and secondary schools, in particular in thosesubjects which offer special possibilities for creative expression : drawing, painting, theplastic arts, music, dance, mime and drama. It was attended by delegates from eighteenmember States of the CCC, teachers and inspectors in all branches of the arts, teachertrainers as well as university lecturers and professors.

The programme of the Symposium included ntroductory speeches, lectures, direct contactswith pupils and teahcers, visits to schools and teacher training colleges. Participants werealso shown films and slides on creative work in schools.In a final general statement, the Symposium :

Emphasised that creativity was one of the most decisive elements in the process of thedevelopment of the individuaL It is essential to foster creativity in school and adultlife and to encourage every individual to develop his creative potential to the full in ademocratic social context ;

Stressed that it was indispensable to develop and perfect techniques for the promotionof creativity in all subjects in the curriculum ;

Recommended that all teachers should be made aware of the importance of creativity.They should be provided with the appropriate training and equipment for the success-ful promotion of creativity. Account should be taken of the necessity of incorporatingconditions favourable to creativity in the building of schools ;

18

Claimed that the arts (language, music, drama, movement, and the visual and plasticarts) had a unique contribution to make in the development of creativity and that,consequently, they should be given a more central place in the school curriculum thanin the past. They should be formally recognised as a fundamental element in education.

In their recommendations to the Council of Europe, the participants proposed that theOrganisation should :

Study in detail the full implications for the school of an education based on creativity.Particular attention should be paid to curriculum development, teacher training, me-thodology, evalution and assessment, resources for learning and the design of schoolbuildings ;

Examine the application to education of the techniques for promoting creativityalready in use in industry and scientific research ;Collect and distribute to curriculum development centres in the member States of theCCC, information and documentation on projects aimed at stimulating creativity i allsubjects in the curriculum ;

Organise a symposium on methods, content and trends in the basic training of teachersof the arts ;Prepare and distribute a survey of in-service training facilities for teachers of the artsin the member States of the CCC

Prepare and distribute a study on methods of developing creativity in artistic subjectsin the member States of the CCC.

Documents : 13ECS/EGT (71) 82 ; 90 ; 95 ; 10 ; 102 ; 120 ;CCC/EGT (71) 15 ; 45.

Venice IIth - 16th October 1971

Pre-school education Aims, methods and problems(Symposium)

This Symposium, organised by the Italian Government under the auspices of the Councilof Europe, was attended by delegates from member States and observers from UNESCOand the European Communities.

The purpose of the Symposium was to examine the aims, forms and content of pre-schooleducation.

Conclusions and recommendations

Pre-school education has three main func n : education, compensation and therapy, anddetection.

Its educational role involves not only the child but also its parents and its background asa whole. The psycho-analytical school has stressed the decisive importance of the firstyears of life for future psychological development, and recognition of the deep impact thatits child's first experiences are likely to have on its personality has been a vital factor indrawing attention to the importance of pre-school education from birth onwards.

Although the compensatory and therapeutic role of pre-school education has only beenunderstood for a few decades, many studies have already shown divergences in thedevelopment of children living in environments that differ economically and culturally.

19

21

At the pre-school ritage particularly, the well-trained teacher is able to play a major partin detecting backwardness in young children.

These general factors, which were discussed in detail by the working groups, formed thebasis for a number of recommendations, the most important of which are set out below :

Governments ought to recognise the importance of pre-school education both for theindividual development of each child and for the general good of the community. Allchildren, of no matter what social class, should be given the opportunity of attendingpre-school establishments, by the age of three at the latest, and consequently it willbe necessary to set up and expand such establishments.

Whenever local conditions make it possible, pre-school educatior should be broughtunder the authority of a single ministry grouping all educational, administrative andsocial services.

Pre-school education should be acecioted as an independent branch, without becominga preserve cut off from all other forms of education ; its autonomy should be acknow-ledged ',y setting up a team of specifically qualified inspectors.

Pre-schoolA atign should beaaesOeel--psery facilities available for adequate periodsoutside school hours so that parents may rest assured that their children are safe whenthey are unable, for valid reasons, to look after them themselves.

Pre-school teachers should have the same educational standard as that req red ofteachers at the elementary level and they should enjoy the same professional statusand salary conditions. Student teachers should be capable of helping to educate pa-rents and be introduced to group discussion and behaviour techniques and in the pro-blems of group dynamics so as to improve their relations with parents.

Member States should ensure that children are prepared for the transition from a pre-school establishment to the primary school during the last year of pre-school educa-tion (visits, meetings, ...).

Research should be carried out into specific aspects, e.g. vocabulary development, andit should have a multi-disciplinary character.

A special meeting should be arranged to discuss the possibilities, advisability and waysof preparing children to learn reading, writing and arithmetic at pre-school level, inthe light of modern scientific findings.

Docume- CCC/EGT (71) 46.

Out-o chool Education and Cultural Development

Strasbourg 18th,- 22nd October 1971

Second meeting of the Committee

The activities of the Committee composed of two main parts, the educational and thecultural sections, were discussed by delegates from eighteen member States. Observersfrom UNESCO, and the European Communities also attended the meeting chaired byMr. M. Hicter (Belgium).

20

22

In connection with concentrating the CCC programme around a limited number of mainthemes, the Committee noted that an important work had since been done in this field.In examining the activities of the period 1970-71 and the draft programme for 1973, itacknowledged the methodical development and the gradual completion of projects in thetwo main fields of its programme.

STRUCTURES AND ORGANISATION OF THE EDUCATION SYSTEM

Permanent education plan for co-ordinating and evaluation projects

The conclusions of the meeting of the representatives of the three Committees and thoseof the CCC were communicated to the Committee. In supporting these decisions, it con-sidered that the setting up of a Steering Group on permanent education and its futurework would _make a two-way feedback possi ble : revision of the concept based on theresults of pilot-experiments and better orientation of such experiments at national level.This could inaugurate a new form of European co-operation with the emphasis no longeron comparisons between national experiences but on a comparison of the experiments inthe light of the common concept.

Organisation and future structures of adult educationThe present situation and possible developments in adult education as well as the legisla-tion and planning in this field were discussed by the Committee. After having beeninformed about the conclusions of the Riischlikon Symposium and the proposals of thegroup of experts responsible for working out a European unit/credit system for theteaching of modern languages to adults, the Committee approved the proposed plan ofwork.

Educational technology means and methods

The Committee took note of the formation of the Steering Group on educational tech-nology, which is concerned not only with the training and retraining of adults but alsowith all the educational techniques that are innovating education. This Group is alsorequired to take over or reorientate the on-going work of the Steering Group on newtypes of Out-of-school education whose mandate wa9 cut at:the end of 1971.Activities concerning the documentation, the studies and the co-production projects in thisfield is in progress. The Committee showed particular interest in European co-productionprojects for multi-media programmes.

CULTURAL DEVELOPMENT

Management of cultural affairs

The need for governments to have a cultural policy next to economic and social policies wasexplained by Mr. A. Girard (France), Project Director. He also reported on the state ofthe work and indicated possible or desirable issues. The Committee gave its approval tothe various plans and propositions.As regard the experimental study of cultural development in European towns, the Com-mittee recommended the CCC to seek via the Committee of Ministers the support to theproject of the governments of the eleven countries in which the towns are situated. It wasregarded as very desirable that these towns have at their disposal the necessary financialresources to enable them fully to participate in this European project.

Cultural enrichment

Mr. A. J. Simpson (United Kingdom), Project Director, explained the general purposes andtrends of the programme which consists of two parts : research and experiment.

21

The varied and complex activities in this field bear on :Socio-cultural facilities (facilities and innovations ; animation methods ; Europeansystem for the exchange of information ; training and status of cultural managers) ;

New audio-visual media ;Contents of cultural advancement (programmes) ;Other cultural activities not comprised in the new programme (European art exhibi-tions in their previous form ; film week ; cultural identity card).

Sport for All

The work and the results of various meetings during the period 197041 were approved bythe Committee. It was suggested that the CCC give its support to the Consultative Assem-bly proposal to prepare a draft European "Sport for All- Charter and invite the Euro-pean Conference of Local Authorities to study ways and means of closer co-operation inthis field.

Youth

The Committee was informed about various activities in this field, namely research intoyouth questions, the European Youth Centre and the European Youth Foundation.

European Youth Centre

The statutes being adopted, the Advisory Committee having already met, the coursescould in principle be organised in the building of the Centre. The budget of the Centrewhich is now a subsidiary of the general budget of the Council of Europe is no longerdependent on that of the CCC. Its Governing Board will meet in December 1971.

Three intensive European language courses and eight information courses were proposedfor 1972.

Document : CCC/E 71 130.

Strasbourg 29th - 30th June 1971

European co-operation in Sport for All(Co-ordinating Group)

The aim of this meeting, attended by six governmental and non-governmeatal experts,was to study-the latest developments in Sport for All and to make proposals as regardsobjectives and contents of European Co-ope ration in this field.

After having examined the different suggestions put forward to set up a rational plan-ning system, the participants agreed on the following guidelines :

the needs and the proposals as expressed periodically by governments and NGOs shouldbe the starting point of a planning system ;the selected priorities should be integrated into a flexible overall long-term plan ;

within the framework of the long-term plan, short- and medium-term working plansshould be developed.

Conclusions

In summarising its discussions of the previous and the present meetings, the group out-

22

La Chaux-de-Fonds(Switzerland)

lined five principles for a European co-operation system, aiming to develop Sport for \11.These are :

Close co-operation between the public and the private section is necessary at European, national, regional and local level ;The initiative of the Consultative Assembly to draft a European Charter on Sport forAll is of i he greatest importance : any common action requires a consensus on certaincommon principles ;

A certain amount of common planning at European level is necessary in order to pro-mote the best possible development of Sport for All : the establishment of a long-termplan is therefore a top pri:rity ;A certain degree of division of labour between member countries is necessary in orderto implement a common European policy under a common plan ;

The Clearing-House is an important instrument both for the planning and implemen-tation of such a common policy.

Document CCC/EFS (71) 83.

30th September -1st October 1971

Experimental study of the cultural development of Europeantowns(Meeting of experts)

Representatives from the ten towns taking part in this study, whose main aim is to awakenlocal authorities to the difficulties facing municipalities in regard to cultural policy, wereable for the:first time to consider jointly the various aspects involved in implementationof this Council of Europe project. They discussed the most effective means of inculcatinga logical and forward-looking approach but were forced to concede that the definition ofthe ultimate aims and objecti ves of the project was far from easy.Each town made a statement on the basic features of its current cultural policy.The coherence and effectiveness of the project would largely depend on three factors :methods of observing cultural life would have to be rationalised, a co-ordinating body setup and the results of the experiment analysed.To publicise the project and obtain a more exact idea of its objectives, it was agreedthat the Secretariat should commission a representative from each town to draw up areport along the following guidelines :

An introduction summarising the general socio-economic features of the town, andpossibly of the urban area or region, its cultural policy to date and its current posi-tion with regard to the Council of Europe project.A definition of the programme, setting out the general aims of local cultural policy,its future policy and definition of possible guide-lines for the fields selected, at thethree levels specified in the Council of Europe memoranda : final objectives, interme-diate objectives and resources.An analysis of the content of the programme, in particular financing systems andsociological studies, with special emphasis on the cultural aspirations of the varioussocial categories, the relations between cultural life and the general activities of thepopulation and the results of the experiment.Another section of the report, on the implementation of the programme, was to deal

25

23

with the bodies responsible for putting the project into etlect, administrative, techni-cal and financial resources and the part (active/passive) to 1 played by the population.

At present the following towns have been selected for the project : Annecy (France),Apeldoorn (Netherlands), Bologna (Italy), Exeter (United Kingdom), Krems (Austria), LaChaux-de-Fonds (Switzerland), Namur (Belgium), Orebro (Sweden), Stavanger (Norway)and Turnhout (Belgium).

Documents : CCC/EES (71) 64 ; 107 ; 112 ; 124.

Strasbourg 30th September 1st October 1971

Adult language learning A European unit-credit system(Meeting of experts)

As a follow-up to the Symposium held in Rtischlikon, May 1971, the Secretariat convenedin Strasbourg a meeting of experts to prepa re on the basis of recommendations made inRiischlikon a phased plan for the implementation of a European unit/credit system forthe learning of modern languages in adult education. Experts from five member Statesand an observer from the British Broadcasting Corporation took part in the meeting.

The main tasks concerning the implementati n of i.his plan were defined by the partici-pants as follows :

to break down the global concept of language into units and sub-units based upon ananalysis of particular groups of adult learners, corresponding to their personal andtypical communication situations. This analysis should lead to a precise articulation ofthe notion of "common core", with specialist extensions at different proficiency levels ;

to set up on the basis of this analysis an operational specification for learnintl ob-jectives ;to formulate, in consultation with the Steering Group on educational technology, ametasystem defining the structure of multi-media learning systems to achieve theseobjectives in terms of the unit cridit concept.

As for the preparation of a plan of work, it was decided that in the first phase of adevelopment and research programme (1971-72), the following preliminary studies shouldbe carried out :

A methodological analysis of adult learners groups in terms of their communicationsituations, with a view to establishing a model for the definition of language needs ofadults learning a modern language.

An investigation into the linguistic and situational content of the "common core" ina unit/credit system.

A definition of a level of basic competence (threshold-levels) in each of the four skills(listening, speaking, reading and writing).

This work will help to map out, in a second phase (1972-73), an integrated units creditsscheme. The results of this research, intended to provide an adequate specification oflanguage content, should then, in a third phase (1973-74), be combined with media taxo-nomy, with a view to producing multi-media materials on the basis of the proposed units/credits system.

Documents : EES/Symposium 53, 12 ;CCC/EES (71) 55 ; 102 ; 120.

24

Educational Docurnentation and Research

Neuchfitel 21th 24th September 1971

Research into the acquisition of reading skills(Symposium)

A Symposium on research into the acquisition of reading skills was arranged under theauspices of the Council of Europe by the Institut Romand de Recherches et de Documen-tation pédagogiques (I.R.D.F.), whose Director, Professor S. Roller, acted as Chairman.This Symposium was in line with the CCC's efforts to encourage member States to pro-mote meetings between research workers and administrators in this new form so that thetwo groups might have the opportunity of studying their problems, exchanging expe-rience, co-ordinating both their projects and their requirements and, wherever possible,reaching conclusions with regard to new teaching methods and research on a Europeanscale.The Neuchatel Symposium was attended by experts from Belgium, Canada, France,Luxembourg, Switzerland and Sweden.The discsusions were primarily centred on two aspects :

Progress made in psychopedagogie work concerning the acquisition of reading skills,especially in the following five fields : perception, spoken and written language, lear-ning, affectivity, vehicular thinking and vocabulary.Progress made by educational research workers in the assessment of short, mediumand long-term methods of acquiring reading skills,

The Symposium included a number of talks on :the state of reading instruction in four wholly or partly French-speaking countries :Belgium, Canada, France, Switzerland ;the present state of research by psychologists, psycho-linguists and psycho-educatio-nalists into the learning of written language and particularly of reading skills.

In addition, three working groups considered problems related to the preparation, acqui-sition and consolidation of reading skills and drew up reports providing the basis for asummary of the discussions.This summary was intended to encourage the establishment of machinery for mutualexchange of information compiled by psychologists, linguists and psycho-educationalists,on the one hand, and by teachers, on the oth er.A report on the proceedings and findings of the Symposium will be drawn up by Profes-sor Roller and submitted to the Council of Europe for comment and approval.

Paris llth -12th October 1971

EUDISED project(Meeting of the Steering Group)

The second stage of the EUDISED project was con _hided at the Paris meeting of itsSteering Group. A draft report, prepared by the Group's rapporteur Mr. J. Viet, and eightstudies commissioned from experts of various member States were discussed in greatdetail. The final versions as they emerged from the meeting will now be published by theSecretariat and submitted for further deliberation and action to the next plenary meetingof the ad hoc Committee for Educational Documentation and Information.Whereas the first stage of the project led to the publication of a feasibility study sup-ported by national reports and technical studies (EUDISED Report, 3 vols., Strasbourg,

25

1969), the second stage concentrated on examining the technical agreements which have tobe reached to implement the project. How can computerised national and internationalprojects concerning educational documentation and information be co-ordinated ? Whatare the requirements which a network for information retrieval concerning educationalresearch and development, planning and policy, technological media, subject matterinstruction to ri:rne only these fields has to fulfil ? Ho an a multi-lingual thesau-rus, which is to be used by all centres and projects co-operaLnig within the system, bebuilt up ? What agreements on common formats and standards are necessary to enablea direct exchange of tapes or disks ? These are the questions which the second EUDISEDReport seeks to answer.Documents : DECS /Doc (71) 1 ; 6 ; 8 ; 9.

London 10th - llth November 1971

Colloquiuui of Directors of Educational Reserach OrganisationsForty-one directors from eighteen member States met for three days to discuss commonproblems. Observers from UNESCO, OECD, the European Commission, the European Cul-tural Foundation, the United Kingdom Social Science Research Council and the CanadianOntario Institute for Studies in Education attended the meeting which was organisedby the Educational Research Committee in collaboration with the National Foundationfor Educational Research in England and Wales. It was the first time that such a Collo-quium had been held. When at the end of the meeting participants unanimously recom-mended that such colloquia should be repeated at two-year intervals, it became clear thatalso in this field a European community has come into being, requiring its own chanr,Asof communication.The Colloquium was introduced by addresses from Mr. W. van Straubenzee, ParliamentaryUnder-Secretary of State, Department of Education and Science and Mr. N. Borch-Jacobsen,Director of Education and of Cultural and Scientific Affairs of the Council of Europe.The Chairman of the Colloquium, Professor W. Taylor, Bristol University, gave the intro-ductory lecture on "Prospects and problems in educational research co-operation inEurope" which was followed by a panel discussion. In the afternoon of the first dayparticipants paid a visit to the headquarters of the National Foundation for EducationalResearch in England and Wales where they had the opportunity to discuss the ongoingprojects in which they were most interested, with the project directors.The second and third days were devoted to discussing the two main themes. The presentchairman of the Educational Research Committee, 1)/fr. L. Legrand, summarised his paperon "Policy of Educational Research Organisations" and Professor K. Harnqvist, GoteborgUniversity, reported on his study on "Training and Career Structures of EducationalResearchers". Both lectures were followed by panel discussions and thereupon themeeting split into small groups to discuss the main themes on the basis of simulationpapers prepared by Professor W. Taylor. Finally, spokesmen of each group reported to theplenary on the results of the discussion.In the concluding session Professor G. de Landsheere, 1,1..ge University, summed up theresults of the discussions and the recommendations made.The two main recommendations were :

to examine subjects and methods for co-operative educational research projects on aEuropean scale and, eventually, the creation of a European Foundation for the Pro-motion of Educational Research and Development to be structured similarly to theEuropean Youth Foundation at present under discussion, and

-- to study the possibilities for reforming and harmonising the training and career struc-tures of educational researchers in member States.

The results of the Coloquium and in particular the recommendations will be discussed atthe next meeting of the Educational Research Committee.Documents : Simulation Papars ; DECS/Rech (71) 19 ; 20.

26

Second Part

COUNCIL OF EUROPE STUDIES

The Council of Europe, and in particular its Committee for Generaland Technical Education, has devoted special attention to theproblems inherent in. traditionni examination methods and theintroduction of new techniques for assessment. The ad hoc Confe-rence of European Ministers of Education, held at Strasbourg in1967, emphasised the importance of these questions and, followingits Resolution No. 4/1967 on the place of examinations in the schoolsystem, the Committee for General and Technical Education com-missioned the following studies : "New techniques for assessment ofpupils' work", by A. D. C. Peterson (Oxford), and "Secondary schoolleaving examinations", by E. Egger (Geneva). Mr. Peterson's study,which is set out in full below, will be supplemented by extractsfrom Mr. Egger's report.

NEW TECHNIQUES FOR ASSESSMENT OF PUPILS WORKby A. D. C. PETERSON

1.0. GENERAL INTRODUCTION

This study is concerned with the formal assessmentof pupils' work resulting in a publicly availablegrade, rank order or orientation. Of course teachersare continuously assessing their pupils' progressas a matter of personal judgement, but the techni-ques by which they do this are too subjectiveand the results too rarely formulated to be thesubject of a study such as this. Moreover it is theformal assessments and their published outcomewhich are of immediate concern to educators inall countries.

When Professor F. Hotyat (1) published in 1962his magisterial survey of examinations he wasconcerned very largely with examinations for theselection of pupils at the point of entry to second-ary education, the age of eleven or twelve. Thefact that selective procedures at this stage domi-nated research work in the first two decades afterthe second world war can be seen from the listof references which Hotyat quotes. It was naturalthat this should be so. In terms of human destinies,and therefore politically, it was selection at this

stage which was of crucial importance. A studywhich concentrates today on methods of assessmentat the point where they most vitally affect boththe pupil and the educational system must moveforward to the assessment of achievement at theend of the secondary stage, to selection for entryto the tertiary stage and to the continuing processof orientation (2).

In doing so we should be wise not to neglect thelesson of the immediate past. Events overtook thosedocimologists (students of examinations) wholaboured with such devotion to improve the reli-ability of selection at 'eleven plus'. As the demo-cratisation of education at the secondary stagebecomes a reality and secondary education through-out Europe is genuinely opened to all, the selectionprocedures whose reliability once seemed andindeed was so crucially important have begunto vanish before their eyes.All the evidence seems to confirm that it will.Yet the procedure may well be slower than in thecase of selection for secondary education. Even onthe general theories of the sociologists, which

(1) Hotyat, F. (1962) : Les Exaraens. Paris. Editions (2) Pieron, H. (1963) : Examens et Docimologie. Paris.Presses Universitaires de France, p. 4.Bourrelier for the Unesco Institute, Hamburg.

27

predict that the achievement of universal educa-tion at any one stage in education leads to openentry at the next within a generation, Europe willstill be concerned for many years to improve itsmethods of assessment and selection at the end ofthe secondary stage : and the economic barriers tothe achievement of universal tertiary educationmay prove more formidable than did those blockingthe way to universal secondary education. More-over, it is clear that as continued education ofsome sort is increasingly enjoyed by all andselection, in the sense of acceptance or rejection.dies out, so orientation to different types of ex-tended education becomes more important. Fororientation to the appropriate type of courseessessment both of potential and achievement con-tinues to be of the greatest importance. The factthat it now contributes to guidance rather thanto allocation, means simply that the factor of thepupil's desire and commitment now enters intothe process to a degree which was not possiblewhen the decision had to be made by an externaljudge, that is by the examiners. It will be one ofthe contentions of this study that this combinationof external and internal factors does not in facthappen so long as the purpose of assessment is thedistribution of a limited number of opportunitiesfor extended education among a larger number ofapplicants, even though it would in fact improvethe validity of the process if it did. The reasonfor this is clear. So long as extended education is,in economic terms, a 'good' in limited supply, madeavailable to some and not to others and providedby the State out of the common resources of society,the method by which those who are to enjoy it7re selected is of paramount importance to socialjustice. This means that it must at least seem tobe impartial as between rich and poor, free fromfavouritism of any kind and conducted with openand scrupulous accuracy. Hence the great concen-tration on improving the reliability of the assess-ments on which the allocation of such importantlife-chances' is made.

Throughout Europe there has developed in theyears since the Carnegie Commission's report of1934 (3) a growing amount of dissatisfaction withtraditional examinations. This dissatisfaction ex-tends from the protest of the libertarian student

International Institute Examination Inquiry (1936) :La Correction des Epreuves Ecrites dans les Exa-mens Enquête Experimentale sur le Baccalaureat.Paris. A la Maison du Livre.andHartog and Rhodes (1935) : An ExaminationExaminations. London. Macmillan.

28

against any form of judgement or classification ofhuman beings, at one end of the scale, to thescepticism of the professional educator about thereliability of current examining practices at theother. Many examinations which were once theobject of vehement criticism, the 'eleven-plus' inEngland, the studentexamen in Sweden or the firstpart of the baccalauréat in F'rance, have eitherdisappeared or are in the process of disappearing.

Nevertheless, so long as society demands certifi-cates of competence from the entrants to suchprofessions as law, medicine, engineering andaccountancy, certification examinations will benecessary and as long as there are more youngpeople seeking to enter a particular stage in theeducational process, whether higher secondary ortertiary, than the institutions in this stage canaccommodate, selection devices, including differenttypes of examination, will persist. The more educa-tion is democratised, in the sense that educationalopportunity, however limited, is equally open tothe rich and the poor, the more crucial becomesthe role of the selection procedure. Before thesecond world war most European countriescontrolled the proportion of the age groupentering tertiary education by eliminating mostof the poor. If ressources do not allow the wholeage group to be admitted and poverty is not tobe the barrier, then someone else must be elimi-nated. In a democratic society this can only bethose who fail selection tests. The purpose forwhich selection examinations are actually used,therefore, is often to provide something whichpeople can fail. It is on the assumption that exami-nations either for certification or selection willcontinue to be necessary for many years thatmany educators have been turning their attentiono improvements in the technique of assessing

pupils' work. This study is concerned with thesearch for these improvements and in it no distinc-tion will be made between the 'examination' andthe 'test'. Even so short a time ago as 1958S. Wiseman, later Director of the National Foun-dation for Educational Research, wrote : "Teachers,on the whole, are responsible for 'examinations',psychologists for tests" (4) This may have beena necessary distinction then, but in the last tenyears the rapprochement which he advocated hastaken place, and 'objective testing' now plays alarge part in many school examinations.

Many factors enter into the judgement of whatis or is not, in any particular case, a good techni-

(4) Wiseman, S. (ed.) (1961) : Examinations and EnglishEducation. Manchester University Press, p. 134.

que of assessment. Of these the most importantare : 'backwash', validity, reliability, cost andspeed. Attempts to improve assessment techniquesusually concentrate on one or other of these andit often turns out that an improvement in onefactor can only be achieved at the cost of a deterio-ration in another. In England, for instance, at-tempts to improve the reliability of GCE examina-tions since the report of the Carnegie Commissionhave had considerable success : but they have beenpaid for in cost, speed and backwash effect. Hencethe importance of clearly evaluating the relativemportance of the factors.

By 'backwash' is meant the effect which any par-ticular examining technique will have on theteaching and learning which goes on in the perioddevoted to preparing for the examination. Forinstance, if an examination in Chemistry rewardswith success the capacity to reproduce from mem-ory a large number of formulae, teachers will tendto spend a great deal of time on memory drill andneglect laboratory work : if on the other hand itrewards practical manipulative skill, but providesthe pupil with any formulae which he will needin his answers, teachers will not bother to maketheir pupils learn them, but will drill them inten-sively in manipulative skills. This whole importantarea was analysed for the Council of Europe inProfessor A. Agazzi's Report on "The EducationalAspects of Examinations" (5).

The concept of validity is simple. An examinationis valid to the extent that it measures what itpurports to measure and not something else. Ageography examination which gives great weightto the beauty and neatness of the maps drawn bythe candidates is to that extent measuringdraughtsmanship, not geographical understanding.It may be that draughtsmanship is an importantskill for the geographer, but in that case it shouldbe made clear to candidates that this skill, as wellas their geographical knowledge and under-standing, is being tested, and examiners must beaccurately briefed as to the weight to be givento it. One of the great difficulties in assessingvalidity is, in fact, that educators have been veryslow to state in clear operational terms what arethe objectives of their courses. Unless we knowwhat the pupil is expected to have gained fromthe course, how can we judge whether the exami-nation is validly measuring that gain ?

Another difficulty often arises from the nearimpossibility of finding another measure of the(5) Agazzi, A. (1907) : Les Aspects Pedagagiques des

Exarnens. Strasbourg. Council of Europe.

stated objectives, when they are stated, againstwhich the examination, as a measuring device,can be calibrated. Let us suppose that one of theobjectives of a course in literature is the develop-ment and refinement of the moral sense. Whatother measure have we got of the extent to whichthis has happened than the examination itself ?And if we have no other at least equally goodmeasure, how can we judge to what extent theexamination is a valid measuring instrument ? Acommon process for estimating the validity of anew type of examination is to correlate it withthe teacher's careful estimate of the pupil's ability,but this suffers from a high degree of subjectivityand if we were really satisfied that the teacher'sestimate was the most valid measurement, theexamination might be unnecessary. Neverthelessattempts to improve the validity of e-aminationsare well worth making. The most reliable exami-nation in the world will do nothing but harm if ithas little or no validity. We could select candidatesfor tertiary education with almost complete reli-ability by weighing them.

There is one special form of validity which hasmuch concerned docimologists. This is 'predictivevalidity'. Here it is assumed that the purpose ofthe course was not to bring about changes of anyintrinsic and specified nature in the behaviour ofthe pupil, but merely to prepare him for thesuccessful completion of the next stage in theeducational process. The examination is thereforenot intended to measure any existing qualities ofthe candidate but to. forecast his future perfor-mance. Neither of the problems outlined abovearise in this case and the predictive validity ofany examir:..aon is often measured simply by theextent to which it predicts success in the nextexamination.

It was on the reliability of commonly used exami-ning techniques that the report of the CarnegieCommission cast serious and alarming doubt. Ifthe competence of a professional man or the futurecareer of a student is to depend on an examinationresult, then it is desirable that this result shouldrepresent a consistent judgement and not a fluc-tuating assessment dependent on chance factors.Yet the report of the commission showed that insuch examinations different examiners wouldmark very differently the same paper and eventhat the same examiner would mark it substan-tially differently at six month intervals. H. Pierondescribing the conclusions of the French commis-sion on the baccalauréat goes so far as to concludethat it showed that "to predict the mark of acandidate it was more important to know the

29

examiner than the candidate " es). It is not sur-prising therefore that much of the work which hassubsequently been done on the improvement ofexamining techniques has been devoted to impro-ving the:r reliability.

It is natural that work on reliability should leadon to cost. To mark questions of the 'essay' typewith any high degree of reliability it seems neces-sary to resort to multiple marking, the averageof a number of even fairly superficial markingsby different examiners proving more reliable thancareful marking by a single examiner. But exam-iners in most examination systems have to bepaid. Here it should be sufficient to insist thatthe true cost of arty systerri of examining shouldbe calculated in terms of opportunities foregoneopportunities for teachers to teach and studentsto learn.

Finally speed is an important factor in any exami-nation. Here I am referring no longer to theopportunity foregone in terms of the time occupledduring the examination period which might havebeen more educationally spent in the class-roomor library, but of the importance of rapid publi-cation of results, particularly where these determineentry to the next stage of education. In England,for instance, the time gap between the taking ofthe first paper in the General Certificate of Educa-tion at Advanced Level and the publication ofresults may be as much as twelve weeks. Exami-nations start early in June and results are pub-lished half way through August. In France theresults of the baccalaureat are published withinthree weeks and in Germany, as in Europe gener-ally, the results of equivalent examinations areavailable equally quickly and before the schoolsdisperse in late June or early July for the summerholidays. The disadvantages of the longer perioddevoted to assessment, both in terms of studentand family anxiety and of inconvenience for the'receiving' institution, which does not know untilalmost the last moment which of its candidateshave reached the required standard, have to beweighed against the increased reliability whichmay be achieved by more prolonged scrutiny.

The value of any new technique of assessment.therefore, will depend on the balance of advantagealong these five parameters.

2.0. EFFORTS TO IMPROVE RELIABILITY

Recent attempts to improve the reliability ofexaminations by introducing new tecln' .es of

(6) Pieron, H. : Op. Cit. p. 24.

30

assessment have mainly concentrated on dimin-ishing the unreliability due to the examiner'ssubjective judgement. This unreliability arises notonly from genuine dif ferences of opinion betweendifferent examiners about the worth of a particularanswer, but because examiners tend to be affected,and affected to different degrees, by factors whichmay or may not be intended to enter into theassessment. Thus a pupil who excels in literarystyle may secure not only a high mark in theliterature examination but also a higher markin history than another pupil whose actualunderstanding of history is greater, but whosepowers of expression are inferior. Professor W D.Furneaux (7) has shown in England that with con-ventional 'essay type' examinations there is acommon element of "general examination-passingability" which enters into the assessment of allsubjects and which may therefore carry undueweight in what purports to be a balanced assess-ment based on number of tests in different areasof the curriculum. Professor K. Ingenkamp reportssimilar evidence of the influence of extraneousfactors from studies in the 1960s on the GermanAbitur (8).

It has long been recognised that in extendedexamination answers at the upper secondary ortertiary level this so-called 'halo' effect willoperate. An examiner, judging on general impres-sion of an extended piece of writing or of anextended interview, will be influenced by unusu-ally good or bad performance at one stage ofthe written or oral answer to form a general, butsubjective, opinion of the candidate's work and toextrapolate from that opinion in judging the wholeof the rest of his performance. We all know howprone we are to assess anything, whether it be anew acquaintance, a painting or an examinationperformance, in accordance with our preconceivedexpectations. It would seem that, unless fullyrecognised and deliberately intended as part ofthe assessment procedure this could be particu-larly dangerous in systems where the final decisiondepends on the views of a 'jury'.

2.1. Objective tests

One way to avoid this halo effect and also to avoidthe subjective differences inevitably arising fromthe judgements of different examiners, some severe

(7) Furneaux, W.D. (1962) : The PsycholoMst and theuniversity. Universities Quarterly. Vol. 17 No. 1.

(8) Ingenkamp, K. (1969) Moglichkeiten und Grensendes Lehrerurteils und des Schultests. Deutscher Bit-dungsrat : Begabung und Lernen. Stuttgart.pp. 409-410.

and others lenient, some tired at the end of theday and others optimistic at the beginning, is tobreak down the qualities and knowledge whichit is intended to examine into measurable unitswhich can be assessed separately and objectively.It is the search for this kind of 'objective' reli-ability which has j.ven rise to the so-called'objective test'.

The objective test does not ask the candidate todevelop in his own terms, either written or oral,the full answer to a general question, but to respondto a very specific question or to select from anumber of possible answers the one which seemsto him the most appropriate.

Since the justification of the technique dependsupon this prior analysis into measurable units ofthe skills or information which it is intended toassess., the starting point in the construction ofobjective tests is always the definition of theoutcome required, the truism that accurate measu-rement only becomes possible when one is clearabout what is being measured. Here the variousclassifications of different kinds of learning whichwill be discussed later under the heading of 'vali-dity' have made it possible to use tests to measurethe ability, not merely to recall facts but to under-stand and apply general principles induced fromfacts. Thus it is claimed that in this fashion it ispossible to assess a whole range of learning. In the

Figure I.

IEA study, for example, eight objectives were listedin the testing of physical sciences (") :

Obtaining scientific informationInterpreting scientific informationTheorisation ; ConstructionTheorisation ; UtilisationComprehensionApplication of scientific knowledgePersonal and social objectivesPhilosophical aspects

Questions were then devised to test each of these.Once the objectives of the course have been classi-fied under this or some similar system it is possibleto consider which of them can best be tested bythis method, what balance struck between themand what weight given to each in the final globalass-ssment. When it has been decided to makeuse of this type of testing to assess performanceeither in a whole course or in some part of it, thenext step in the g Inerally accepted procedure isto construct a 'grid', a simplified illustration ofwhich is given in Fig. 1. drawn from a grid de-signed to test the outcome of courses in civiceducation.

(9) U.S. Department of Health, 17-1ueation and Welfare :Cross-National Study of Educational Attainment.Stage 1. investigation in Six Subject Areas. FinalReport. Feb. 1969. p. D-3.

Contentof syllabus(civics)

Learning outcome

Knowledge offacts and principles

Understanding offacts and principles

Application offacts and principles

Constitutions

Historicaldevelopment

Local/centralcontrol

The legislative .

The executive

The judiciary

31

The text constructor must then consider what sortof questions, usually known as 'items' to includein each box of the grid. The main difference be-tween them is between those which require thecandidate to supply an answer and those whichrequire him merely to select the most appropriatefrom a list of answers presented in the questionpaper. Both have the advantage that they testthe ca !idate's knowledge or ability over a muchwider range of the course which he is supposedto have studied than can a limited number of essayquestions, and so diminish the unreliability dueeither to 'halo' effect or to the 'luck of the draw'in a candidate's finding questions in the examina-tion which relate either to his strongest or to hisweakest points.

The supply type, however, which is often knownas the 'short answer' question cannot be markedcompletely objectively, either by clerks or machinesince answers will differ and someone competentto do so must judge whether or to what extentthey are acceptable. It is possible that in very largescale examinations it may become possible to over .come this obf -tenon by programming a machine toaccept a large number of specified variations andso to mark automatically a wide range of freeresponses, but even this procedure would not beable to reward adequately the brilliant and totallyunexpected response. A typical 'short-answer' item,drawn from the International Baccalaureate exam-ination in Physical Science for 1969 is the follow-ing :

"At 273 K and atmospheric pressure (101.3kPa) 10-" rn" of a certain gas are found tohave a mass of 0.18 g.(a) Calculate the number of moles of gas

presentCalculate the molecular weight of the gasCalculate the r.m.s. velocity of the gasmolecules at 273 K."

(b)

(c)

Beneath each question an adequate amount ofspace is left for the candidate to write an answerof the length required.The 'selection' type of time requires no more ofthe candidate than to choose between alternativeanswers presented to him. He writes nothing, butsimply ticks or underlines the correct response.Items of this type may take several forms. Thecandidate may merely be required to say whethercertain statements presented to him are true orfalse. Yet even this apparently simple type iscapable of considerable sophistication as is shownby the following example, quoted from Objective

32

Testing hy H. G. Mackintosh and R. B. Morri-son ( ') :

"DirectionsEach of the following items may be true with-out qualification, true with qualification, orfalse. If it is true without qualification, circlethe T and mark a 3 in the space provided. If itis true with one of the listed qualifications.circle T and mark the number of the appro-priate qualification in the space If the item isfalse, circle the F.

StatementsT F The total resistance in an electrical

circuit is equal to the sum of theindividual resistances.

T F . . . . The total current in an electricalcircuit is equal to the sum of thecurrents in the individual parts ofthe circuit.

T F The total current in an electricalcircuit is equal to the electromotiveforce in the circuit divided by theresistance of the circuit*The power supplied to a circuit isequal to the product of the totalresistance and the amount of currentin the circuit."

Qualifications1 if the resistances are connected in parallel.2 if the resistances are connected in series.3 no qualification.

He may on the other hand be asked to select froma number of possible responses, usually five, theone which is appropriate. This is the typical 'mul-tiple choice' question. It is capable of testing morethan the mere recall of factual information as willbe seen from the examples given below, but itrequires considerable skill in composition.

(a) "Which of the following had as their majorpurpose the achievement of national inde-pendence ?

I The OctobristsII The Sinn Feiners

III The Young TurksIV The CarbonariV The One Thousand(A) I and III only(B) II and V only

(10) Mackintosh, H.G. and Morrison, FLE: (1969) Objec-tive Testing. London. University of London Press.

(C) I, III and IV only(D) II, IV and V only(E) I, II, III, IV and V"(quoted from College Entrance ExaminationsBoard Achievement Tests 1965) (11).

(b) "500 houseflies were kept in a cage. Onecubic centimetre of 1 `,4 DDT solutionwas discharged into the cage and as a result95 (;4: of the flies died. The survivors weretransferred to a fresh cage and allowedto reproduce. The resulting adults werethen submitted to the same DDT treat-ment. The survivors were transferred toa fresh cage and the procedure wasrepeated. The mortality rate declined witheach generation until it was 34 % for the14th generation.

The most acceptable scientific explanation forthese results is( ) a. repeated DDT treatment causes house-

flies to become resistant) b. DDT treatment causes mutations in the

genetic material( ) c. a few flies get a sub-lethal dose and

then become immune( ) d. interbreeding causes mutation leading to

DDT resistanceDDT resistance occurs naturally and isinheritable."

(quoted from International Baccalaureate High-er Level Biology 1970).

The essentials of a good multiple choice item are :(a) That the preliminary information, known as

the 'stern' should contain all the informationthat a candidate ought to need, but no guidance

(b) That there should be one and only one answerthat is acceptable.That the other answers, known as the `distrac-tors' should be sufficiently plausible not to rulethemselves out of court, but not so nearlycorrect as to allow of genuine difference ofopinion.

(d) That the degree of skill or information requiredto pick the right answer should be sufficient todistinguish between the more and the less wellprepared candidates.

Clearly this latter criterion will vary from item toitem and it is usually considered wise to begin atest with a number of easy items, which almost all

(c)

(11) College Entrance Examination Board (1963) :Achievement Tests. New York. CEEB p. '75.

candidates will get right and proceed gradually tomore difficult items.The extreme difficulty of composing fifty or ahundred items of this kind makes it desirable thatmultiple choice tests should be composed by a teamof experienced examiners working together. Evenso the first version of the test is likely to containsome items which in practice do not call forth theresponses that the team expected, and it is usualto administer the test in advance to a populationsimilar to, but different from, those pupils forwhom it is being designed. In this way the faultyitems, particularly those which fail to discriminate,in the sense that they are answered correctly byall or by none, can be identified and rejected. Acommonly accepted rule is that the limits of accept-able discrimination lie between correct responsesfrom thirty and from seventy per cent of thecandidates taking the test.

The advantages claimed for the multiple-choicetype of examination are many and it will be con-venient to tabulate them and then to discuss themseparately.

(a) t is extremely reliable in that it eliminatesfrom the examination the subjectivity of the exam-iner. 'Elch examination paper will score the same,whoever marks it and at whatever time it ismarked. Indeed it can be and often is marked bymachine. In view of what we know about theunreliability of examinations of the 'essay' typewhen marked by a single examiner this is a verysignificant advantage. It is sometimes argued thatthe multiple choice type of examination merelyremoves the element of subjectivity from the mar-king of the answers to the composition of thequestions. This is, of course, true, but the numberof people involved in the construction of a multiplechoice paper, whether as test constructors, itemwriters or pre-test participants, is so great thatthe final result cannot be regarded as representingthe judgement of one man, as can the assessmentof an 'essay' type paper set and marked by a singleexaminer. Moreover, even if an element of subjec-tivity enters into the construction of the paperand so favours a whole group of candidates withone cast of mind rather than another, at leastthe performance of all candidates on that paperis assessed equally, reliably and without the fluc-tuations due to the subjectivity of differentexaminers.

(b) Provided that it is really true that the qualities,skills and information which it is intended toassess can be analysed and broken down into alarge number of separate and independent parts,

3

then the multiple choice paper can sample thewhole area to be tested much more fairly than canthe traditional 'essay' type examination.

Lei us illustrate this irom the examining of foreignlanguages and let us assume that one purpose ofour examination is to discover whether the candi-eate has a wide knowledge of the syntactical andidiomatic conventions of the language. If we askhim to write an essay, or even to make a trans-lation into the language we shall learn little abouthis competence over the whole of this field. Thesubject set for the essay may or may not demandthe full range of his knowledge. If he uses ,,-hatappears to be a restricted range, it may be becausehe has a genuine purity of style which seeks alwaysthe simplest forms. If, on the other hand, his proseexemplifies a wide variety of structures, he maybe artificially dragging in constructions and idiomswh; 'h, while representing the extreme range of hisown capacity, still fall short of the best we couldcxpect.

On the other hand if we ask him to answer fiftyto a hundred multiple choice questions, coveringthe full range of all that we might expect him toknow, we can discover with a great deal of accu-racy how far his range of syntactical and idiomaticusage extends. We shall not, of course, discoverhow far he is capable of using that range insensitive and meaningful prose, but that is notwhat, in this particular part of the examination,we were setting out to discover.

(c) As soon as the number of candidates passes acertain figure, which will vary with the subjectbeing examined, this method becomes progressivelycheaper than the conventional 'essay' type exami-nation or the oral. For multiple choice papers themain cost lies in the construction and pre-testingof the examination : the actual papers can be verycheaply marked by clerks or a machine. For 'essay'type papers the actual constntetion of the exami-nation is not very expensive and is sometimes doneby a single examiner in a few hours. What mattersis not so much the questions he asks as the answersthe candidates give. But this means that the ans-wers must be marked by highly qualified acade-mics, perhaps as highly qualified as the examinerwho has set the paper : and the time of highlyqualified examiners is valuable and expensive.We must remember here the point made earlierthat the costs of an examination should be mea-sured not in cash expended but in opportunitiesforegone. One of the questions which all industri-alised countries have to ask themselves is whethertoo high a proportion of the time of their limited

34

supply of highly qualified academics is being usedin examining rather than teaching.(d) This method is quick. With machine scoringthe results of tens of thousands of examinationscan be provided within a few days. It was pointedout in the introduction that there is a certainimportance in this criterion, if the results of exami-nations are to be used to determine transition fromone stage in education to another. To achieve thesame speed in the provision of results on 'essay'type examinations for a comparably large groupof candidates, it would be necessary to employhundreds of different examiners. And the greaterthe number of different examiners employed thegreater the unreliability of the results.

It seems therefore that objective tests have consi-derable advantages from the point of view ofreliability, cost and speed. They are criticised ongrounds of validity and backwash effect.

Early criticisms of their validity, made when theyfirst appeared were based on the view that theycould test no more than the factual recall of infor-mation and that they favoured random guessing.This may have been true of the earliest type oftrue /false item (e.g. "Bergen is the capital of Nor-way" : True ''''alse), but it is manifestly not trueof some of the more sophistiea:-ed items now inuse. Moreover a certain amount of retention in thememory of factual information is an essentialpart of any academic study or, indeed, of anyeffective intellectual activity at all. It is, there-fore, one of the factors which any process ofassessment should take into account, providedit is not made the dominating factor. A moresophisticated criticism is that many of the morerecently developed items which purport to testjudgement or interpretation of evidence are reallytesting not these activities themselves, but the re-collection of them, as previously undertaken duringthe course either by the pupil or his teacher. Thismay well be true : but it is a criticism which canbe equally well made of examination questions ofthe conventional essay type. Other criticisms ofvalidity have perhaps arisen from inflated expec-tations. Few tests are perfect in terms of the crite-ria proposed above. Professor B. Hoffman hasshown in The Tyranny of Testing (12) that someitems can be so ill-designed as positively to discri-minate against the subtle or more inventive thinkerand this is undoubtedly true. Even in the mostscrupulously designed tests it is still possible thatindividual bad items will appear, but the influence(12) Hoffman. B. (1964) The Tyranny of Testing. NeW

York. Collier-Macmillan.

f these on scores, when they are one or two outof fifty is slight, and the test, as a whole, may stillbe sampling the whole syllabus in a more compre-hensive and reliable way than a conventionalexamination. 'Objective testing' has acquired amystique and perfectionism of its own ari-1 it issometimes forgotten that, for certain purposes.even a faulty multiple choice Lest may be a bettermeasuring instrument than a traditional exami-nation.

The objection that tests of this kind favour randomguessing is part of a wider criticism of their vali-dity arising from the fact that they do rely on anumber of separate and unrelated responses. It isof course true that a candidate who neither knewnor thought anything could expect to score twenty-five on a hundred item test where there were fourchoicer, or fifty on a true /false test. Although thiscan be corrected for by the use of a simple formulamany test constructors are in fact avoiding thetrue/false type. Where four, or more commonlyfive, alternatives are offered in a selection tast,they argue that candidates do not, in fact, guessat random and that a high percentage of right'guesses' simply indicates a range of competencewhich, although not complete, is very considerablez,nd should be assessed as such. They thereforemake no statistical adjustment for guessing atull ("). It may, however, be significant that theAmerican College Entrance Examination Boardwhich has greater experience of these tests thanany other body, continues to arrive at its finalscore by applying the formula suggested by theEducational Testing Service :

R -

Where R is the number of right answers, W thenumber of wrong answers and n the number ofchoices offered.

The justification for this procedure is clearly des-cribed in the Board's pamphlet on the ScholasticAptitude Tests : "When the SAT is scored, a per-centage of the wrong answers is subtracted fromthe number of right answers as a correction forhaphazard guessing. Mere guessing, therefore, isas likely to lower your scores as to raise them. If,however, you are not sure of the correct answerbut have some knowledge of a question and areable to eliminate one or more of the answer choicesas wrong, your chance of getting the right answer

(13) Ebel,R. (1965) : Measuring Educational Achievement.New York. Prentice-Hall. pp. 98-99.

is improved, and it will to yout advantage toanswer such a ques ion (").The real objection to tr 1,-g objective tests simplyto replace conventional examinations was givenwith great eloquence by M. Desclos, President ofthe French Commission at the time of the Carnegieenquiry :

"Si l'on se bornait a considérer les examensdu seul point de vue de leur utilité en tantqu'instruments de selection d'apres les con-naissances acquises, on pourrait ètre amen&a envisager des épreuves quasi mécaniquescapables d'inventorier rapidement le stock desnotions que possedent les candidats et da four-nir des resultats parfaitement exacts ou objec-tifs. Mais si notre ambition va au-dela, si nousstmons que l'objet des etudes est moins d'ac-

cumuler des connaissances que de les coor-dormer pour en tirer une philosophie et pourles mettre en ouvre, moins de meubler l'espritque de l'assouplir, l'affiner et le vivifier, sinous attachons du prix a la liberté de juge-ment, au sens critique, a l'effort personnel, al'imagination créatrice, au gont et é la mesure,

toutes les richesses spirituelles, subtiles etfugitives qui constituent la culture, il faut nousen tenir a des epreuves d'examen moths exac-tes sans doute, mais capables d'en décelerl'existence, de les évaluer sinon de les mesurer,et qui permettent en memo temps d'utiliserpour leur preparation des exercices qu'uneexperience pédagogique seculaire a mis aupoint et dont l'efficacite est incontestable " (I").

Refinements of the technique of objective testingmay have enabled us to extend its use to assessingmore than 'the stock of ideas', but the truth remainsthat essential qualities involved in the orderingand marshalling of thought, in creativity, in imagi-nation and style, to say nothing of what is nowknown as 'the affective domain' are difficult ifnot impossible to assess by a method which sumsthe responses to a number of isolated stimuli. Thiscan easily be seen from the example quoted earlierof assessment of command of a foreign language.The multiple choice test will tell us better thanany essay how wide and accurate is the candidate'scommand of lexis. It will not tell us whether heis capable of using this command to develop acoherent argument in the language, whether hisimagination is sterile or rich, or whether he has

(14) College Entrance Examination Board (1965)Scholastic Aptitude Test. New York CEEB

(15) International Institute Examination Inquiry : Op.Cit. p. 385.

any 'style' in his use of language. Even the attemptswhich have begun in America to test literaryappreciation by multiple choice questions, althougninteresting in themselves, have so far proved oflimited value.

When we come to the most important question ofbackwash effect objective tests are heavily criti-cised. The main objection is that which Descloshints at in his last lines, that if this is known tobe the sole method of assessment, teachers willbe tempted to neglect the development of thdrpupils' capacity to write continuous prose. Tt hasbeen said of these tests that they require tl- can-didate to read and think, where traditional exami-nations require him to think and write : andwriting is a very important accomplishment. Sostrong was the reaction in the United States thatthe College Board tried to counteract it by requi-ring a 'writing sample' as well as the completionof the tests from each candidate. Unfortunately itproved impossible to achieve the degree of reli-ability in the assessment of writing samples thathad come to be expected and the requirement wasdropped. This negative backwash effect remains,however, a very serious one in any country whereteaching methods are strongly influenced by exam-i ling procedures.

A positively harmful back ash effect has beenseen in the danger that constant practice in com-pleting this type of test will condition pupils tothe belief that intellectual activity consists in aseries of responses to a set of problems, for eachof which there is one and only one completelyright answer in fact that the manifold diversityof our experience is best conceived as a programmefed to us by the heavenly computer. M. Trow, theAmerican sociologist is recorded as having found"that at one institute in America students had somany objective tests during their first year thattheir approach to their studies became dominatedby the need to pass exams. The next two yearswere spent curing them" (").

2.2. Multiple marking

One technique for securing greater reliability inthe grading of 'essay' type questions is to havethem marked by more than one marker and toadopt as the final grade the average grade awar-ded. There is little doubt that this does improvereliability. Its disadvantages lie in cost and speed.

If a very quick superficial reading by three separateexaminers is substituted for a careful assessment by

(16) (Reported by) Cox, R. : New SOciety 24.5.1966.

36

a single examiner taking three times as long overthe work, the cost in terms of examiners time neednot be any greater ; and it is claimed that thistechnique did give improved reliability in theassessment of English essays in the 'eleven-plus'examination in England (17). Even here howeverthere is an extra cost in photo-copying and postageor an extra delay and administrative risk in re-quiring one reader to post scripts on to the nextor in bringing together readers to a central point.Moreover a technique which gave improved resultsin the assessment of a more or less free compositionwritten by pupils aged between ten and elevencannot be assumed to be of equal value in assessingother forms of work.

At the upper secondary level multiple marking isat present employed in Luxemburg. Each writtenpaper is marked by three examiners drawn fromthe 'commissions' for three different schools whoserd their grades, independently and without con-sultation, to the government assessor. If there is asignificant difference between the grades the asses-sor calls the three markers together to re-examinethe script. If this confrontation does not produceagreement the assessor submits the script to thecommission for the school from which the can-didate comes, for a final decision. Presumably thisbrings in at the crucial stage an element of contin-uous assessment since the school's judgement islikely to be affected by what they know of thecandidate's previous work.

It may well be that this system is possible onlyin a small and homogeneous system where suffi-cient examiners with comparable standards andeasy access to each other are available. In Franceexperiments were started with double-markingafter the report of the Carnegie Commission butwere subsequently abandoned on grounds of costand speed.

In England double-marking of essays was intro-duced in the first part of the GCE (Ordinary level)which is normally taken between 15 and 16 by theNorthern Board in 1967 and extended in 1969 toover 40,000 candidates.

Multiple marking is, of course, much easier toachieve in an examination system which is largelyinternal and marking of written scripts by twoexaminers, one from the school itself and onefrom a neighbouring Gymnasium, with an officialassessor giving the final adjudication where the

(17) Wiseman, S. (1949) : The Marking of English Com-positions in Grammar School Selection. BritishJournal of Educational Studies XIX. pp. 200-209.

ades differ, has been practiced in some Landerfor the German Abitur. In terms of real costs thisrather more than doubles the cost per candidate ofactual marking, but this does not appear in moneyterms where those engaged in the process are notpaid separately for their examining work butundertake it in lieu of teaching or administrativeduties. That there is little loss of speed in such asystem is probably due to the comparatively coarsegrading system of the Abitur on a six point scale.If a twenty point scale similar to that of the Frenchbaccalauréat or the new proposals for the EnglishGCE were useci. differences of opinion between thetwo markers and the consequent necessity for theassessor to read the scripts, thus increasing bothreal cost and delay, might be much increased.

2.3. Train-marking

Another result of the Carnegie Commission andthe publication of Hartog and Rhodes Exami-nation of Examinations, which was based on itsfindings, has been much more elaborate arran-gements for the setting and marking of essay typepapers. The most exhaustive attempts to improvereliability by this type of sophistication have pro-bably been made by the English GCE examiningboards. These have tried to avoid the unreliabilitydisclosed by thc Commission's finding of very poorcorrelations between a number of independentmarkers by organising the markers into a 'team'.

The first step in this process is the preparation bythe Chief Examiners of extremely detailed markingschemes for each paper, to which his assIstantexaminers are expected to adhere very closely. Toquote the 1969 report of the Northern UniversitiesJoint Matriculation Board : "When the examinersfirst draft their questions they provide 'notes forthe answers' indicating what in their opinion can-didates might reasonably be expected to includein their answers_. Much more detail is howeverrequired in the marking scheme since this is in-tended to ensure that all members of the panelof examiners responsible for marking a paper (ora section of a paper) use the same method of assess-ment and adopt a similar standard" (18).

This marking scheme breaks down the expectedresponse to a long piece of translation or an 'essay'type question into comparatively small elementsand fixes the allocation of marks to be given foreach one. We can see already therefore an approachto the sort of reliability to be expected from a

(18) Northern Universities Joint Matriculation Board(1969) : The Work of the Joint Matriculation Board.Manchester. J.M.B. p. 12.

number of short answer or even multiple choicequestions 0 r) the other hand with every step inthis direction we are getting further from Desclos'justification of the 'essay' type questions as flexibleand humane instruments encouraging creativity,ima :tion and flexibility of thought.

The pi ovision of a detailed marking scheme is onlya beginning. The first stage after the actual exami-nation is for each examiner on the panel to senda batch of papers which he has provisionallymarked to the Chief Examiner as a sample of thatyear's performance. This is followed by a meetingattended by all the examiners at which themarking scheme is revised and standardised in thelight of the performance of this sample of thecandidates who have taken it. The examiners thenreturn home and begin their definitive markingDuring this period they continue to send samplesto the chief examiner who can thus assess whethera particular examiner is proving 'severe' or 'lenient'

or worst of all inconsistent, in which case allhis papers have to be redistributed for remarkingby other examiners. When all the marking is com-pleted the Chief Examiner and his assistants pro-ceed to adjust the marks by increasing those of'severe' examiners or dimi-lishing those of 'lenient',until they are satisfied that a common standardhas been reached. Only at this point do they decidethe mark which is to be accepted for that yearas a 'pass mark', and then the panel proceeds toreview all scripts whose marks fall around thisborderline. This final review procedure is insome cases carried out by the Chief Examiner butin others involves a final general meeting of exam-iners which is called the 'Award' and whichmay take several days. The document quotedabove, for instance, records that in 1969 the reviewof borderline scripts in Paper B of English Lan-guage at Ordinary Level (the first part of the GCE)occupied ten examiners for ten days (").

The improved reliability of 'essay' type examina-tions marked with this meticulous care is paid forin terms of cost and, above all, speed. The delaysof postage and travelling, added to the time occu-pied in standardising and reviewing, mean thatthe English GCE candidate sometimes does notknow the results of the examination he took atthe beginning of June until late August and doesnot know whether he will be admitted to a univer-sity until two or three weeks before the sessionstarts. The human cost here in prolonged anxiety,which is known to many English families, and the

(19) Northern Universities Joint Matriculation BoardOp. Cit. p. I.C.

37

administrative cost to the universities and otherinstitutes of te,;.iary education, which do not knowthe examination results of their candidates untilthe new academic year is almost beginning, hasalready been referred to.

The financial cost may be estimated from the factthat the fee charged for each single examinationin each subject at the 'Advanced Level' is now twopounds sterling. For the average university can-didate taking six subjects at Ordinary level andthree at Advanced level the cost would be 13.

How far then has this meticulous proceduresucceeded in reducing the unreliability of suchexaminations exposed by the Carnegie Commis-sion ? It is remarkable how little research hasbeen done on this topic since the publication ofHartog and Rhodes report in 1935. The EnglishExamining Boards have some justification for com-plaining that most docimological studies are stillquoting, as their main evidence for the low reli-ability of 'essay' type questions, studies which arethirty-five years out of date. On the other handtheir uwn research has been mainly concentratedon establishing, or seeking to establish, the compa-rability of standards between one Board andanother, rather than the reliability of markingwithin each Board. It would be extremely inter-esting, for instance, to set up controlled experi-ments to establish whether or to what extent, theelaborate procedures and long review period havegiven a higher reliability index to the marking of'essay' type question in the GCE `A' level thanin the much more quickly assessed French bacca-laureat, or in the triple-marked Luxemburg system.

One study, that of E.L. Black of the University ofManchester, though carried out on a small scale,reproduced as nearly as possible the conditionsand safeguards now built into the GCE system (20).Nineteen different examiners, who had been 'brief-ed' in the standard way and were part of ateam, were given the same script in English Lan-guage to assess ten days after the briefing andwhile they were in the middle of marking theirofficial scripts. The experiment was repeated infour separate years. The marks of the differentexaminers varied from 53 % to 31 % in year one,from 61 % to 40 Cic in year two, from 65 % to49 % in year three, from 61 (7( to 43 % in yearfour When it is remembered that a difference ofa few percentage marks can make the differencebetween a 'pass' and a 'fail' it is clear that, even

(20) Black, E.L, (1962) : The Marking of G.C.E. Scripts.British Journal of Educational Studies. Vol. 11.pp. 61-71.

though the correlations found are considerablybetter than the extremes disclosed by the CarnegieCommission, the subjectivity of the examiners'judgement has been by no means eliminated fromthis type of assessment. The Schools Council inEngland has now proposed that the examiningboards should change from their present sevenpoint scale to a twenty point scale and publishsome guidance to those who interpret examinationresults about the degree of accuracy which can beexpected. Their conclusion is that "Enough isknown about the reliability of examinations ofthe types at present commonly set at 'A' level toestabliah that on the twenty point scale a standarderror of les,s than two scale points would be verydifficult to obtain."

One of the factors which may contribute to thisdegree of unreliability is the technique employedfor translating the raw marks of different exam-iners on to a common scale of grades for publicationand use in selection or orientation.

2.4. Procedures for expressing rawgrades

marks as

Various procedures may be used to transform rawmarks into grades and these may affect the reli-ability or validity of the ass,'ssment. There arefour types of unreliability which such proceduresmay be intended to mitigate: that which arises fromexaminers being called upon to assess the workof schools with widely different mean levels ofability ; that which arises from an unequal spreadof marks in different subjects ; that which arisesfrom candidates choosing different questions toanswer from a paper where, for instance, the can-didate is asked to answer three out of twelvequestions ; and that which arises from differentstandards adopted from one year to the next. In allcases there is a possible conflict between reli-ability and validity and in all cases the correctdecision in this conflict will depend upon the pur-pose for which the assessment is being made.

If it is assumed that there really is such a thingas an objective standard of achievement or abilityand that it is the purpose of the examination tomeasure as nearly as possible the pupil's perfor-mance against this standard, then it is clearlyimportant that the standard should not vary fromschool to school, question to question or year toyear. Where global assessments are based on thesumming of results in different subjects, as inSweden, it is important that it should not varyfrom subject to subject.

To assure a reliable comparability of standardsbetween school and school it would be desirablethat each examiner, each 'team' of examiners oreach jury should receive scripts to correct from a'Tresentative sample of the population as a whole

would be dif ficult, for example, for an examinerwho saw scripts only from the Lycée Henri Quatrcor Manchester Grammar School to form a justappreciation of the national standard. J. Fetchclaims that in the General Certifica te of Educationin England this is achieved by allocating schoolsto examiners 'alphabetically by names of towns'and thus achieving something approaching arandom sample.

The danger arises, however, in any system wherethe examining is entrusted to a number of auto-nomous boards, commissions or juries, that thewhole of the sample being assessed by one boardmay differ substantially in academic potential fromthe whole of the sample being assessed by another.H. B. Miles and G. E. Shipworth have shown thatover a five year period the mean IQ of entrantsfor the different General Certificate of EducationBoards in England varied very substantially (e.g.betwfi.en 125 /7 in one board and 109/11 in another)and yet that "it is difficult to extract from thfigures any consistent relationship between IQof entrants and grades obtained. Board 5 for exam-ple with lowest IQ's on en.ry is more generous inits grades on mathematics over the years thanBoard 2 with candidates of much higher IQ onaverage" (21).

The most carefully designed procedure to ensurethe maintenance of a common standard is thatwhich has recently been introduced in Swedenand which is described in section 3 of this study,in relation to the even greater problems of corn-parability involved in the use of continuous assess-ment by teachers

The need for some sort of statistical adjustment toproduce comparable gradings as between differentsubjects arises from the well established fact thatexaminers, except perhaps in mathematics, tend tobunch their marks about the mean and make littleuse of the extreme ends of the scale, reserving.as is said in France, the top mark for 'le bon Dieu'.Thus the Northern Universities Board in Englandfound that in a typical year the marks, out. of anominal 200, ranged from 200 to 0 in mathematicswith few below 48, while those in geography

(21) Miles, H.B. and Shipworth, G.E. : The Times Edu-cational Supplement 2.10.1070.

1

ranged from 144 to 11 with few below 54 (22). Bothsets of marks had, at that time, to be translatedfor publication on to a five point scale roughlycategorised as 'excellent, very good, acceptable,insufficient. failed'.

If the grades are to be published separately andneither global assessment nor compensation be-tween subjects is concerned, it can be argued thatnothing should be done to standardise the spreadof marks between subjects, that pupils at this stagedo differ much more widely in their mathematicalability than in the battery of skills and knowledgerequired to write a geography examination, andthat the spread of marks therefore represents atrue judgement and not just the inability of theexaminers to perceive distinctions. If this is so,then to adjust the marks statistically so as to pro-duce a comparable pattern of spread betweensubjects will be to distort the validity of the assess-ment.

If, on the other hand, results in different subjectsare to be regarded as comparable or summed toproduce a global assessment, then it is importantthat the spread of marks in different subjectsshould also be comparable. Otherwise a subjectlike mathematics which normally has a widespread will dominate the process of arriving at arank order on which, in practice, selection depends.The best way to normalise raw scores for thispurpose seems to be to translate them into 'Tscores', i.e. scores expressed in terms of a mean of 50and a standard deviation of 10, giving effectiveupper and lower limits of 80 and 20. T scoresexpress the raw score not in term3 of a fixed scale,invariable as between subject and subject, but interms of deviation above or below the average ofall candidates for each particular subject. Thusthey ensure that marks in a subject where there isan unusually wide spread of marks or where ahigh proportion of candidates score highly (e.g.Theology in the Abitur) do not carry undue weightin the global assessment. The procedure for cal-culating T scores can be found in standard workson educational and psychological measurement (23).

In many examinations of the conventional essaytype, e.g. History cxaminations in the Englishsystem, a candidate may be given as many astwelve questions from which he is required toanswer no more than three. The purpose of thispattern is clear. It ensures that the whole syllabus

(22) Fetch, J. (1966) : Marks and Marking. Manchester.Northern Universities Joint Matriculation Board.

(23) (See, for instance) Guilford, J.P. (1956) : Fundamen-tal Statistics in Education and Psychology. NewYork. McGraw Hill.

39

is covered in the examination and gives the candi-date an opportunity to show what he can do well.To set three compulsory questions only introducesso great an element of chance in favour of thecandidate who happens to have concentrated onthe areas chosen for the examination that it oftenappears more as a test of what the candidate cannot, do, or of his teachers expertise in 'spotting'the questions likely to appear.

Yet the wide choice of questions means in effectthat a candidate choosing questions 1, 2 and 3 isanswering a totally different examination paperfrom one choosing 10, 11 and 12 : and a simple cal-culation of the permutations makes clear howmany different examinations are in fact beingoffered. How then are we to approach compara-bility between the students taking them ? Rese-arch on the reliability of examinations where sucha choice of questions is permitted has been carriedout at Oxford University and by the NationalFoundation for Educational Research (21).

There are two possible ways of achieving themaintenance of comparable standards from year toyear, dependence on the experience and subjectivejudgement of the examiners and the assumptionthat with sufficiently large numbers of comparablepopulations the distribution and level of the quali-ties which it is intended to assess will not varysignificantly from one year to the next. In practicemost systems rely on a combination of the two, theexaminers having a rough idea of the proportionof candidates to be placed in each grade, butchecking this against their subjective impressionthat standards have risen or fallen.

There are a number of points worth noting here.While it is true that in each succeeding pair ofyears a marked change in the statistical distribu-tion of grades would be most unlikely to be justi-fied, this might well not be the case over a se-quence of a number of years and the change, forbetter or for worse, might well be so gradual asnot to be noticed even by the most sensitive andexperienced examiners. This could happen parti-cularly where, through the democratisation of edu-cation, the population taking the examination wassignificantly changed. It is also worth rememberingthat where the purpose of an assessment is selec-tion rather than evaluation the 'pass mark' willvary from year to year according to the policy ofthe accepting institutions. Thus a considerably high-er mark is now required to secure entry to anEnglish university to read Arts and a considerably

(24) Backhouse, J. (1971) : Report to the Schools Council.

lower mark to read Science than was the case tenyears ago. The proportion of entrants who passedthe French baecalaureat was for a long time moreor less static lind 60 ';; . The drop to 50 ; in1966 and the rise to over 80 % in 1968 were theresult of outside evente rather than of any signifi-cant change in the quality of the pupils beingassessed.

3.0. EFFORTS TO IMPROVE VALIDITY

Three main criticisms of the validity of traditionaltechniques of assessment have been made by doe:-mologists in recent years. Of these the most impor-tant, because the most fruitful, has been that theassessment is falsified because it relies on theperformance of the pupil on a single occasion whichis governed by what have come to be known as

.ition conditions'. This has led to a growinginterest in techniques of 'continuous assessment'.The other main criticisms have been that the skillsand knowledge tested in conventional examinationsare not, in reality, the skills and knowledge whichthose examinations purport to test and that, evenif they were, there is some doubt whether thescare really the skills and knowledge which aremost valuable at a late' ,tage.

One obvious flaw in the single terminal test carriedout under examination conditions, is that the pupilmay do himself more or less than justice becausehe, or even more probably she, is affected on thatparticular day by accidental physiological or psy-chological factors. These may work either to theadvantage or disadvantage of the candidate. Thereare some pupils who are stimulated by the dramaticatmosphere of the examination room and thesingle examination, which they treat as a challenge,and who therefore consistently perform better inthe examination than they have done throughoutthe course. It is they, presumably, who cling toall the formalities, including special forms of dress,which have been associated with the occasion.There are others, notably perhaps slow, hesitantand scrupulous thinkers, who find it feverish orclaustrophobic and who perform worse. And thereare those who have, on the day of the examination,a migraine, or a menstrual period.

Moreover examinations of this kind by their verynature, favour the pupil with the ready pen, forwhom the rate of nervous response between brainand fingers is rapid and unimpeded. They favour,in some cases almost demand, the type of memorywhich can retain not so much facts we havegot beyond that but prestructured interpreta-tions of facts, long enough to reproduce them in

examination room in legible and coherentform. The pupil who is asked, in the French bac-calauréat to write an essay on such a subject as"lin critique a Mini Alfred de Vigny 'un cornelienmélancolioue' ; qu'en pensez-vous ?" or in thcEnglish GCE at Advanced Level to discuss, intwenty-five minutes : "Milton's Satan moves usbecause he alone is able to convey dramaticallywhat goodness is" has no recourse but to his me-mory of what either he or his teacher thoughtabout the subject on a previous occasion and tothe facility of his pen. He has, in the English systemat least, no time to think, and if he had he cannotrefer to the text, as would anyone_ seriously con-sidering such a question. It is not surprising thatin such circumstances the examiners regularly com-plain that candidates answer, not the questionwhich has been set, but a more or less closeapproximation to it, which they have preparedbefore entering the examination room. The skilllies in disguising this memorised reasoning as aresponse to the question on the examination paper.One device intended to mitigate the reliance onmemory which has been tried recently in a num-ber of systems is to allow pupils to bring referencebooks into the examination, but whatever valuethis may have it does not eliminate the effects ofrequiring pupils to work to a strict time limit on asingle occasion.

The harmful bac wash effects of such examina-tions in promoting the tacit acceptance of receivedideas and opinions, rather than the exercise ofthought, has been as much criticised as theirfailure to assess the qualities which they purportto assess. Before we condemn them altogether,however, in favour of some more continuous formof assessment it is worth considering why theyhave enjoyed the long ascendancy which, in somecountries, is only now being challenged.

There is some justification for the view that theskills and qualities which enable the first type ofpupil described, the 'good examinee', to performwell in such examinations are important andshoulei play their part in any assessment. Manypur.ils would themselves pay tribute to the valueof such examinations as incentives. The case forthis view was well stated as long ago as 1911 inthe Report of the Consultative Committee on Exa-minations in England and Wales, -which includesamong the good effects of terminal -examinationson the pupil "that they train the poWer of gettingup a subject for a definite purpose, even thoughit may not appear necessary to remember it after-wards a training which is useful for parts of

the professional duty of the lawyer, the adminis-trator, the journalist and the man of business" (a).

There seems to be no justification, however, forthe view that the encouragement and measurementof these abilities should play the dominant par tin assessment, and it is partly the realisation of theundue importance which they have assumed, andin many systems still assume, that has led to thedemand for more continuous assessment. Af ter all,educators are not engaged simply in training andselecting lawye-s, administrators, journalists andmen of business and even in the academic worldthere is a growing recognition that there is novery high correlation between success in terminalexaminations of this type and capacity for con-tinued research.

The real reason why these examinations continueto play so large a part in our process of assessmentat the most decisive point in the educational pro-cess is surely their reliability and their demonstrableimpartiality. Impartiallty, which is demonstrableto parents and students, is of crucial importanceat any point where life chances depend on accept-ance or rejection, and the reliability of this typeof examination, though low as we have seen, isstill higher than that of most techniques of con-tinuous assessment. If we are to replace themwith some method of continuous assessment whichdoes not suffer from their inherent weaknesses,we must try to preserve within the new form ofassessment as much as possible of the virtues ofthe old.

3.1. Continuous assessment

Clearly one way to avoid the weaknesses of thesingle terminal examination as a method of assess-ment is to spread the process over a considerableperiod of time. The term 'continuous assessment'has been used somewhat loosely, however, and itwill probably be convenient to distinguish threemain usages.

The first, which we may call periodic assessment,substitutes for the single terminal examination anumber of tests taken on different occasionsthroughout the course. This is not unlike the'course credit' system in American universities,with its regular tests and grades at the end ofeach semester contributing to the final award ofa degree. In Europe the German Abitur and th..,!Swiss Cantonal Maturité have long given weight tothis type of continuous assessment.

(25) Report of the Consultative Committee on Exami-nations in Secondary Schools (1911). London. HMSO.

4 1

In Germany the procedure may differ slightly fromone Land to another, but in a typical system theassessment of Klassenarbeit' will be based on aseries of tests spread over the last two years of thesecondary course (e.g. one in the middle of thepenultimate year, one at its end and one in themiddle of the last year). These tests are given bythe teacher in each subject and their administrationis closely controlled. In some Lander, for instance,the number of tests to be given is specificallyrelated to the number of hours per week for whichthe subject is studied, or the tests may not begiven on a Monday, nor two tests on the same day.The cumulative grades for these tests establish the'Vornote', which may carry as much as half theweight in establishing the final grade in the Abitur.In the Swiss Cantons the standa-d procedure is togive half the weight in establishing the grade inthe Maturité to this periodic assessment.

Periodic assessment of this type almost certainlyimproves validity by eliminating or mitigatingsome of the physiological or psychological hazardsof a single terminal examination ; but it is stillopen to the criticisms which c' n be justly levelledagainst the traditional written or oral examinationas a measuring instrument. The fact that thisinstrument may be used on as many as f- -0 occa-sions eliminates some of the reasons which lead usto question its validity when used on a single occa-sion, it may even improve slightly its reliability ;the problem remains however that what is mea-sured may be recall rather than either understan-ding or creativity. And because it is administeredaria marked by the teachers in each school orcollege questions immediately begin to arise as toits impartiality and comparability between oneschool and the next. It is perhaps significant thatwhereas in Switzerland the Cantonal Maturitérelies to this extent on continuous assessmentbecause the teachers making it will be employedby the Cantonal authorities and known to theparents, the Federal Maturité, which is requiredfrom pupils in independent schools, does not.

The second type of continuous assessment, whichseeks to break away from the restrictions of singleoccasion tests altogether, relies upon a generalassessment of the studen,.i work in each subjectover the whole of the course. This we will callcumulative assessment. Cumulative assessmentmakes it possible to avoid all the pitfalls of thesingle occasion test, whether used terminally orperiodically, but it increases still further the sub-jective element in the assessment, since in mostsystems the teacher is the only judge who has beenin contact with the pupil throughout the course

42

and must therefore accept responsibility for theassessment.

An interesting variant of nimulative assessmentwhich spreads the responsibility more widely andalso introduces a certain element of self-assessmenthas been developed in connection with the assess-ment of projects in which a number of pupils havecollaborated over an extended period. Here theproblem is not merely to assess the success of theproject as a whole, but to assess how much eachindividual in the group has contributed to it. Thetechnique is to ask each pupil and each teacherwho has taken part in the project to allot a gradeto each participant, including, in the case of thepupils, himself. In order to avoid over or under-marking based upon prejudice, a device used inInternational Games, when panels of nationaljudges assess diving or skating, is adopted : fromeach series of grades the top and the bottom gradeare eliminated and the final grade is the mean ofall the others. This method has been tried experi-mentally in the assessment of architecture stu-dents, both in England and Germany, and reportsupon it are encouraging (").

Periodic and cumulative assessment are commonlyused together as in the German Abitur, wherethe teacher's assessment based on the Klassen-arbeit not only contributes to the final Abiturgrade but is also decisive in the important questionof whether a pupil is ready for promotion to thenext class or must repeat the year's work. Whereasin the English and French systems approximately35 % of all candidates fail the single terminalexamination, in Germany failure is minimal be-cause a process of continuous assessment hasensured that candidates do not take the examina-tion until they are ready to pass it. Thus manyof the tensions and injustices of the baccalauréatand the GCE are avoided.

It has been noted above, however, that improve-ments in one feature of assessment are often paidfor by concomitant disadvantages in another. Theprice of the long period of continuous assessmentin the years leading up to the Abitur has been asteady rise in the average age at which it is taken,until it has now reached 20.5, with a very highrate of drop-out on the way. H. Peisert and R.Dahrendorf, working with a total cohort of 5383children entering the first year of the Gymnasiumcourse, found that this had been reduced to 1579

(26) Oxford Polytechnic (1970 ) Department of TownPlanning. Occasional Papers No. 2.

on entry to the penultimate year, of whom 1236finally passed the Abitur (27).

Combinations of periodic and cumulat've assess-ment naturally vary according to circumstancesbut the following regulations for assessment inthe Diploma in Education of Bristol Universitygive some idea of the process where examinationis wholly internal :

"Assessment of course work is us d insteadof tradfflonal final examinations. Normallythis means that for each course studied, awritten assignment will be required. Assign-ments take various forms : longer essays, aseries of short papers, seminar papers,case,7iudy reports, etc., at the discretioe ofindividual tutors. It should be noted that coLIsework assessment does not preclude the use oftests of either the traditional essay or theshort answer type. Such tests, if used, aregiven during the session.

Assignments are so phased as to spread thework load throughout the session. They mustbe completed by the set date, and when re-turned by tutors, must be kept together ina folder, for submission to the examiners atthe conclusion of the course.

Systems of assessment are of necessity com-parative, and it is not always easy for aninstitution to cater with any real refinementfor individual differences. In addition, com-plete objectivity is not possible ; in markingwork, tutors attempt to take account of thefollowing factors :

The qw lity of the work submitted in rela-tion to le rest of the group ;The quality of the work in relation to thetutors' assessment of general standards inthe subject.

Inherent in the evaluation problem are factorsof an affective nature which are complex andvariable; and an attempt is made to takeaccount of these factors in the assessment ofstudents' work.

Obviously, different tutors have differentmarking styles and produce different distribu-tions of scores. This is covered very largelyb-J a final assessment meeting at which notonly grades but also ranking orders are consi-

(27) Peisert, H. and DahrendOrf, R. (1967) : Der Vor-zeitige Abgang vom Gymnasium. KultusmintsteriumBaden-Württernberg. Reihe A Nr 6.

dered in reaching a final overall grade foreach candidate.

Two further points should be made :It is unfair to both tutors and students ifwork schedules are not adhered to. Onlyin special circumstances may work com-pletion dates be postponed.Attendance below the 70 `,;. minimum forany of the couy.-%es is not acceptable andwill be considered as constituting failure ofthe total course" (28).

The examples quoted so far have illustrated thecombination of periodic or cumulative assessmentswith each other or with terminal examinations,when the same examiners are essentially respon-sible for both. A typical example of this processwhere the terminal examination is in the handsof external examiners is the use of the 'livretscolaire' in the French baccalauréat. This cumula-tive record book, in which are recorded the teach-ers assessments of course work over the year, ismade available to the examiners, who take it intoaccount, both in confirming the immediate accept-ance of those who have scored a mark of 12 orabove on the 20 point scale in the first group oftwo written and two oral examinations, and todecide borderline cases both for admission to L-esecond group of examinations and for finalacceptance or rejection after their completion.

In the International Baccalaureate teachers' assess-ments are made available to the examiners forconsultation after the preliminary marking, as anadditional check on the grading, a practice alsoused by some of the English GCE Boards.

In the examinations for the 1--1-.0 National Diplo-ma (a professional qualific f, engineeringand kindred professions) in F.., 30 % of theweight in the final grading is gi..unt to periodicassessments, carried out by the teachers sometimeson as many as twelve occasions, and the remainderto terminal examinations on syllabuses submittedby the teachets, but controlled by assessors, as inthe German Abitur. It is noteworthy that in this,as in the Bristol example quoted above, or inAmerican universities, a minimum attendance rate,in this case 67 % of all possible- attendances, isrequired. Such a proviso is of course unnecessaryin secondary schools, but it is difficult to see howcumulative assessment, at least, could be justifiedwithout such a requirement, which might be hardto enforce, in many areas of tertiary education.

(28) Bristol University (1969) : Facutty of EducationProspectus. p. 17.

43

In periodic assessment it is, of course, possible toavoid specific tests and to assess the quality ofpupils' work on laboratory note-books, extendedessays or portfolios representing work done duringthe course, without controlling attendance atclasses, but this clearly involves both a risk ofcheating in the form of work which has not beendone by the pupil at all a a very fine decisionas to the extent that a pupil may legitimately be'helped' in its preparation, for instance by an eldersister at the university. The best control here,which should certainly avoid deliberate cheating,is probably an oral examination on the worksubmitted, but it is in the nature of continuousassessment that in practice such oral examinationscould only be carried out by the pupils' ownteacher.

The reference to "factors of an affective nature"in the Bristol regulations leads us on to the thirdusage of the term 'continuous assessment', toindicate a complete assessment of the whole per-sonality and record of the pupil. This, which wouldclearly provide the best criterion either for orienta-tion or selection, we shall call "global assessment."It is, however, manifestly much more subjectiveand therefore unreliable than even cumulativeassessment. Consequently it is at present usedonly by ialependent colleges whose assessmentcannot be %aestioned on grounds of partiality, or,in official systems, for orientation rather thanselection.

3.2. Orientation

In an ideal educational system, unaffected by eithereconomic limitations, academic snobberies or theshortage and inexperience of teachers, all assess-ment would be for orientation rather than selection.For such a purpose a global assessment of thewhole personality of the pupil would no doubtprovide him with the best guidance in plotting hiscourse through life. This ideal has long beenrecognised, particularly in France, where the term'classe d'orientation' dates back to 1937.

In reality, however, educational systems are onlyable to approach towards this ideal where pupilsare moving from a less to a more differentiatedstage within the period of universal education.Even then the pure concept of orientation is com-promised and selection enters in as soon as one ofthe different channels in 'he second stage is attrac-ting, for whatever reason, more applicants fromthe first stage than A can absorb.In the current European situation, therefore, wefind the nearest approaches to a genuine system

44

of orientation at the point of tranaier from lowerto upper secondary education rather than fromupper secondary to higher.

The problems which such orientation poses havebeen very clearly set out in a recent report ofthe Institut Pedagogique National in Paris (2").Whether the actual advice on orientation is to begiven by a school counsellor or by a commission, ithas to be based on a dossier which records not onlythe social and family background of each pupil, butan assessment of his qualities by all those teacherswho have been mainly responsible for teachinghim. This means, as R. Gal has said, that theteacher must become a psychologist, and manyteachers who arc excellent teachers of Latin orMathematics, and who can judge very well apupil's performance in Latin or Mathematics, areill-equipped for this wider function.

In the design of a 'dossier scolaire' there is aconstant conflict between the points of view of theadministrator who wants it to be a complete andtidy record, the teacher who wants it to be easyto fill in and to deal with, the counsellor or com-mission who, perhaps dealing with several hundredunder pressure of time, want it to be short andcontain only essential information, and the poten-tial research worker who wants it to contain allconceivably relevant facts. The report quoted abovegives two specimen dossiers one of twelve andone of twenty-two pages and appends some of thecriticisms of those who used them. The first corn-meni of a teacher, on even the shorter dossier,was that a packet of 35 or 40 dossiers for a singleclass made a heavy and cumbersome load to carryhome. The criteria for a good dossier suggested bya 'user' were also significant :

It should be short ;ft should give a synthetic vie but withoutneglecting important details ;It should stress the 'particularities' of thepupil.

It is important to remember that both teachers and'assessors' are human beings with human limita-tions and that there is as much danger in deman-ding from them, or feeding to them, too muchinformation as too little. It is presumably possibleto envisage so structuring the information whichis required that it could be coded and processedby a computer, but this would still require a highly

(291 Institut Pedagogique National (1969) : L'OrientationScolaire et la Recherche des Aptitudes. Paris. Ser-vice &Edition et de Vente des Productions de l'Edu-cation Nationale. Brochure N. 34.

skilled assessment by the teachers in the firstplace.

In so far as the teacher can be trained or trainhimself as a psychologist this type of global assess-ment seems possible, but it will undoubtedly betime-consuming, even if the professional psycho-logists come to his aid with batteries of tests.Moreover, it increases the individual teacher'sresponsibility, and as long as the advice based onhis assessment has any mandatory force on thepupil there will be a need for some recourse to asecond opinion. Otherwise the emotional pressureon teachers from pupils and parents is likely tobecome intolerable.

This recourse is provided in France by the factthat orientation to the Lycée, the most favouredchannel, is the procedure only within the statesystem. Pupils from independent schools have topass an entrance examination, and it is left opento pupils from the state system who have not been'oriented to the Lycee, and who disagree with theirorientation, to take this examination also and, ifsuccessful, to enter. This device of providing asystem of continuous assessment, with an exami-nation open to those who disagree with the assess-ment, is one which might prove useful elsewhere.

The procedure followed in Sweden is very similar,except that standardised achievement tests play alarge part in decidin7 the advice on orientationwhich the teachers give to the parents. These testsare, however, characterised by Henrysson asbeing 'monitorial' only in character and "it isconsidered to be of value for teachers in theirmarking to include qualifications in their-studentsthat cannot be evaluated by tests" (30).

The advice given by the teachers at the point ofentry to the pre-gymnasium stream in the com-prehensive school is not mandatory on the parentsand whatever the views of the school authorithany parent may insist on his child entering thisstream. Up to the age of 16 therefore we have hereorientation in its pure form. At this point howeverselection enters in. The proportion of the agegroup in the pre-gymnasium stream over the lastthree years has been 45 % in 1967/8, 44 % in1968/9 and 42 % in 1969/70 (31) ; yet by the termsof a Parliamentary decision the maximum propor-tion which can oe admitted to the gymnasium is30 %. Selection is therefore based on the appli-

(30) Marklund, 1-lenrysson and Paulin (1968) : Korn-petensutredningen III. Stockholm. National Educa-tion Board. (English summary).

(31) Statistika Centralbryan Reports 1968 U2, 1969 and1970 U5. Stockholm. National Education Board.

41.47

canfs school marks as 'monitored' by the stan-dardised national tests (5). It seems possible thatthe gradually falling proportion in the pre-gymne,slum stream represents a growing tendency on thepart of parents to accept the teachers' advice atthe earlier stage.

England and Wales are the only Fui apean countrieswhich make use of a nationally organised exami-nation of the conventional type, the General Certi-ficate of Education at Ordinary Level or the Certi-ficate of Secondary Education, for orientation atthis point and some small elements of continuousassessment are now beginning to appear in theseexaminations also. If the recommendations of theSchools Council that these two examinationsshould be merged and should no longer play anypart in University selection are implemented. itseems likely that the element of continuous assess-ment will increase.

The Fr nch experiments seem the most ambitiousattempt yet made at orientation through globalassessment in a national system, bnt, as Legrandpoints out in the introduction to the report quotedabo .-e, there are still very serious problems to beovercome, both in the training of teachers asassessors and in the conceptual analysis of thequalities to be assessed.

3.3. The reliability of continuous assessment

Th2 advantages of continuous assessment, whetherperiodic, cumulative or global, in terms of validity,backwash and speed are apparent. It is not sur-prising therefore that much effort is now beinggiven to improving its reliability. The most notableexample of this is undoubtedly the new systemwhich has been adopted in Sweden, both for orien-tation within secondary education and admissionto higher education.

In Sweden the studentexamen, which previouslyclosely resembled the German Abitur, has beenreplaced by continuous assessment of a cumulativetype based on the school marks in a range ofsubjects. The procedure introduced to improve thereliability of this assessment and particularly thecomparability between one school and the next isof great interest.

The first step was the establishment of stan-dardised national achievement tests at the uppersecondary level. This was done by asking gymna-

(32) Orring, J. (1969) : School in Stheden. St- _kholm.National Education Board.

45

sium teachers to submit items for their respectivesubjects. Those items we.-e then reviewed and themost promising pre-tested in the most rigorousfashion, including, for instance, the pre-testing oflanguage items in the countries where the languageconcerned was the mother tongue. The tests werethen standardised over a large and representativesample of the Swedish school population. It issignificant here that Henrysson reports that ruralschools gave as good results as those in the town.From these tests a national mean performance ateach stage in the upper secondar3 course is calcu-lated and it is assumed that results throughoutthe country, if related to the national mean, willcorrespond to a pattern of normal distributionThese tests are then sent to each school wherethey are administered and scored by teachers.Tests at the gymnasium level have now beendevised by the central research unit for Swedish.foreign languages, mathematics, physics, chemistryand economics, i.e. the subjects normally presentedin Europe for written matriculation examinations.Where the tests are wholly of the objective' typescoring is comparatively simple, but in languages,as Li the USA it has been found desirable torequire some piece of extended writing. In orderto assist teachers in assessing this, corrected scriptsfrom the standardising sample are sent to teacherswho, if they are still in doubt after consultingthem, may can in a second examiner.The purpose of these standardised tests is, as wehave seen above 'monitorial'. They do not deter-mine each pupil's final assessment, but from themthe teacher can judge how the general standardof his class and the intervals between pupils withinit conform to the national norm. He is expectedto make other assessments of his pupils' work overthe year, but when at the end the final assessmentscre translated on to a iive point scale the teachertakes into account the pattern established withinhis class by the national tests in determining thedistribution of his grades.

Let us suppose, for instance, a class of 30 pupilswhose raw scores in a certain subject range from83 to 15 with a mean of 48 on a national test forwhich the national average is 53. Normal distribu-tion of the grades would be as follows :

Grade 5 4 3 2 1

umber ofPupils 2 7 12 2

Looking at his raw scores the teacher sees that hehas two pupils well ahead of the rest with scores

46

of 83 and 82. Although his experience tells himthat on the whole the class is rather below averageand this is confirmed by their score on the nationaltest, these two clearly deserve a mark of 5. Simi-larly he finds a group of seven between 77 and 62who are clearly ahead of the next candidate at 59.He retains the normal distribution therefore forthe mark of 4. Below that he has less thanwould normally be expected bunched about thenational mean and so his grade di.stribution forthis class comes out as below (normal distributionin brackets) :

GradeI II 3 2_I 1

Number ofPupils 2 (2) 7 9 12 9 (7) I

Examples of possible distributions quoted by theNational Board of Education for different typesof class of 30 are given below (")

Mark

Natureof class 5 4 2I I

Average 2 7 12 I7 I

1

Good 4 9 9 I 7 1 1

Poor 1 4 12 I 10 I 3

Even 1 7 14 11

1

Uneven 4 7 9 I 4

Considerable liberty is therefore left to the teacherto distribute the grades in accordance with his ownassessment of the whole of his pupils' work but indoing so he has their performance in the nationaltests as a 'monitor'. Similarly the whole resultsfrom a particular school may be above or belowthe national norm, but is the business of theinspector to ensure that if this occurs there arcreasons which justify it.

Tliis system, which is still in an experimentalstage, has several important advantages. It intro-duces an objective control into continuous assess-ment and it ensures that the spread of grades iscomparable as between different subjects. This isparticularly important in Sweden where a pupil'sfinal assessment is expressed as a sum of his g adesand the difference between an average grade of

(33) National Education Board (1968) : The New gym-nasium in Sweden. stocknolm. p. 52.

4.3 and one of 4.5 may imply acceptance or rejec-tion by the most exclusive faculty of the university.

Neither this nor the use of standardised tests in thecomprehensive school, however, is proving whollypopular and there are still problems involved in thecombination of objectivity with continuous ,ssess-ment which have not been solved. It is easy to seefor instance the resentment which might be causedunder this system when identical individual per-formances on a standardised objective test are trans-lated into different grades on the five point scalebecause of differing distribution patterns in theclasses.

Moreover, the National Board of Edur.ation havefound that, in the competitive situation whichpersists with regard to entry to the most favouredchannels or faculties, continuous assessment has notsubstantially reduced the competitive pressure forvvhich the studentexamen was blamed. Issue num-ber 371970 of the Council of Europe Newsletterreports as follows :

"The National Board of Education and theMinistry have declared their intention to re-form the present grading system within thecomprehensive school. The grading system hasbeen the subject of considerable criticism, insome eases resulting even in the boycotting bypupils of st-ndardised achievement tests.It is generally admitted that the present systemof grading has several defects. There are manywho have come to the conclusion that theseare so serious that an entirely new approachwill have to be adopted to find, in particular,more adequate selection methods for rkstrictedintake lines Above all, it is felt that the pres-ent grading system counteracts cooperationand collaboration between pupils, contrary tothe goals of the comprehensive school.A decision is soon expected to be taken togreatly reduce grading within the compre-hensive school. Experimental and investiga-tional work is being done to obtain a ldasis forsuch reforms. Suggestions and ideas from bothpupils and teachers being discussed at presentin periodicals and newspapers, will also betaken into account" (34).

Experiments are also being carried out in Englandto determine and to improve the reliability of con-tinuous assessment. In 1964 the Department ofEducation and Science gave a grant to Leicester

(34) Documentation Centre for Education in Europe :News-Letter 3/70. Strasbourg. Council of Europe.p. 16.

49

University -to study ways of examining other thanby conventional written papers." In introducingthe record of their experiments which were carriedout in the context .)f the cnrtificate of secondaryeducation Professor Eggleston writes : "providedthe assessments arise directly from an adequatespecification of the educational objectives, the con-tinuous evaluation of attainment has importantconsequences. These include morr- emphasis onimmediate and intermediate gair rather than onterminal or more remote outcomes ; higher levelsof student motivation ; and encouragement to usea wider variety of teaching methods" (5).

The emphasis herr laid on the specification ofobjectives was reflected in the experimental me-thod. Taking the physics test as an example, teach-ers were first asked to specify the objectives oftheir course, as in the IEA experiment quotedearlier. These were then compared with the exam-iner's specification and an agreed list preparedconsisting in this case of 'Inference', 'Organisationof Data', 'Observation', and 'Application of Factsand Principles to problem solving'. The procedurefor assessment was then as follows :

The teacher decides that pupils will exercisea particular ability drawn I -pm the listduring a specific lesson.The teacher then organises the lesson so asto draw a response requiring that ability,preferably in writing and 'under test con-ditions', during the lesson.The teacher grades these responses on a fivepoint scale.

This was done on an average three times over aneight week period. Teachers agreed that althoughthis kind of assessment was difficult it could beimproved by practice and did not interfere withgood teaching. The assessments were then checkedagainst a 'moderating test' of the 'objective' mul-tiple-choice type based on the same objectives. Theteachers were warned of the 'halo' effect and askedas far as possible to isolate the specific ability beingtested.

Three indications from this test may be of generalinterest :

Although the average correlation with theobjective test was reasonable (0,45), someteachers achieved markedly worse corre-lations than others and there was someindication that even a small amount oftraining would improve results,

(35) Eggleston, J. and Kerr, T. (1969) : Studies in Assess-ment. London. English Universities Press. p. 3.

47

The in tercorrelations between the gradeson the tests of separate abilities were muchhigher on the teachers' assessment (between0.44 and 0.54) than on the objective test(between 0.14 and 0.34), indicating thatteachet:s canducting continuous aessmentwere considerably less capable of 'isola-ting' specific abilities than test constructors.The test constructors complained, as usual,that the teachers did not use the full rangeof the scale to which the teachers repliedthat 'pupils in a streamed class might wellrepresent a band of the spectrum of abilityso narrow that differentiation even on afive point scale might be difiicult'.

All these seem to point in the direction that conti-nuous assessment by teachers requires specialtraining and is more appropriate for certificationor orientation than for selection.

In the test of practical work in Biology they foundthe reliability of cumulative assessment greatefthan that of periodic and of the same order as inteam-marking of essay type questions. The intro-duction of a 'control' in the form of a number ofquestions in the written papers which referred topractical skills, a device now used by those GCEBoards which rely on teachers assessments for themeasurement of practical work, was also investi-gated and regarded as providing reliability at anacceptable level for a test carrying not more thanone third of the total weight of the L-it7essrnent.

The general conclusion of the investigating teamfor Physics was ! "When the extreme cases ofdisparity between grades were examined acrossall three assessment procedures (e.g. continuousassessment by teachers, objective test and conven-tional examinations) there was no evidence to makea case for one being a more accurate assessmentof attainment in Physics than another. The decisionto use one method of assessment rather than an-other win presumably depend on a priori assump-tions about the known or presumed backwasheffects of the procedures" ("). For the purposes ofassessments made at a point of transition withina full secondary education, and concerned onlywith certification and orientation, this may wellbe true. It may be argued on the other hand thatthe cost of these elaborate teacher-controlled pro-cedures in training, attendance at meetings andassessment is greater than the outcome justifies,and that the increased involvement and enthusiasmwhich they generate as an experiment would eva-porate when they became a routine.(36) Eggleston, J. and Kerr, J. : Op. Cit. p. 76.

46

3.4. Analysis of abjectites

The experiments reported above laid great stresson the clear specification of objectives as an essen-tial preliminary to assessment. Unless we are to beconcerned solely with predictive validity, it seemsclear that in order to make a valid assessment ofwhat gains in knowledge. skills or attitudes pupilshave made from following a course, we do needto know what knowledge, skills or attitudes thatcourse is seeking to develop. Since the publicationof B.S. Bloom's important work in 1956 this hasoften been known as a taxonomy of educationalobjectives (17). But the analysis of objectives canbe a fruitful method of assessing the value of acourse as well as the progress made by individualpupils. If a course at upper secondary level leadingto a public assessment, whether continuous or bysingle occasion examinations, is intended to preparepupils for the next stage in their lives, then weneed to know not only whether the method ofassessment is testing reliably the objectives of thecourse, but also whether those objectives are reallyrelevant to the needs of the next stage.

That this is a useful exercise for those responsiblefor course and assessment planning seems beyonddoubt, but it must be realised that it raises boththe profoundest issues of educational philosophyand the most practical problems of administrationand teacher training. Some of the teachers whotook part in J. Eggleston's experiments werestrongly of the opinion that a consideration of themethods of assessment was of great value inhelping to clarify objectives.

On the other hand the historians were so convincedthat the use of original sources was important asa teaching method that they finally abandonedany attempt to relate items in their tests to specificobjectives and preferred to make the use of originalsources the criterion of a good assessment proce-dure. The outcome, in their opinion, was that 'thechildren are better historians as a result'. But thequestion whether the objective of teaching historyto fourteen and fifteen year olds of average or lessthan average ability is to make them 'better histo-rians' raises profound issues of educational philo-sophy, to say nothing of semantics, and ought not,perhaps, to be left to the history teache-,s. Mostteachers seemed to agree that objectives in whatBloom terms the 'affective domain' were of greatimportance, but also that it was difficult, if notimpossible, to assess how far they were beingachieved,

(37) Bloom B.S. et at. (1966) : Taxonomy of EducationalObjectives. New York. David McKay.

This very understandable view, combined with theview that consideration of assessment procedureshelps to clarify objectives, raises the doubt whetherwe are not faced with the risk which Deselos sawin objective testing that in our attempt to breakdown the objectives of education into specific,operational and testable items of behaviour, weshall produce assessment procedures which are nomore valid, because we have eliminated from ourcourses those elements, often the most valuable,which are difficult or impossible to assess. As oneof the most distinguished empirical researchershas said, the relative emphasis on the knowledgefunction in comparison with the non-knowledgecomponents in school courses is 'a question of edu-cational philosophy and not very well suited forempirical research' ('). If this is so, it may well beimportant that the analysis of objectives shouldnot be 1-ft too much in the hands of the specialistteachers or empiricists with a new found enthu-siasm for taxonomies, and should quite clearlyprecede the design of the assessment procedureThe argument that assessment should always relatedirectly to the course which the teachers areteaching, and not to the course which the exam-iners think they ought to be teaching, should notarise in a system in which both teachers and exam-iners are working within a framework of agreedobjectives.

In drawing up such a framework or taxonomy, fewwould seriously quarrel with the kind of hierarchyof learning at the basis of Bloom's work, at leastin the cognitive area, but its application to courseand assessment uesign is by no means simple.From a teaching point of view a Piagetian approachmight lead us to advocate a much deeper analysisof the stages, including the tension between estab-lished schemata and incoming experience, throughwhich a particular level of cognitive ability is bestdeveloped. This might lead to the specification ofobjectives in terms more of a 'spiral', to useJ. Bruner's term, than of a series of levels, and sopoint in the direction more of continuous assess-ment than of sampling on single occasions. On theother hand, we know from experience, confirmedby Eggleston's experiments, that teachers find icextremely hard to differentiate their assessmentinto any but the simplest categories. 'Observation'may be, as Legrand has pointed out, a far fromunitary activity, yet the teachers in Eggleston'sexperiment clearly found it quite difficult to assessit separately from even three other widely different'objectives'.

(38) Dahl lbf, U. (1963) : The Contents of Education withregard to demands for different jobs and for furtherstudies. Stockholm. National Education Board.

951

Although much more fundamental work needs tobe done in clarifying the objectives of educationalcourses and relating them to assessment, there areclearly certain easily identifiable skills, in language,for instance, or mathematics, where a more detaileddefinition of the level expected and the weight tobe given to each area of performance might con-tribute to improving the validity of assessment.In such a detailed taxonomy attention shouldsurely be given to the second problem, the rele-vance of skills taught at one stage to the needsof the next. An exhaustive study of this problemwas carried out by the Swedish National Board ofEducation in connection with the reform of thegymnasium and is summarised by Professor U.Dahlkif ("). The procedure was to break downthe curriculum of the gymnasium into sixty-fiveseparate areas of content and fifteen separate"general study-skills". University professors andindustrialists were then asked to rate on a fivepoint scale both the importance of each of thesefor further study in his own field or tor employ-ment, and the degree to which the present prepa-ration of gymnasium students was adequate. Theresults from the universities emphasised the impor-tance, in the area of content, of foreign languages(especially English and German), elementary math-ematics and statistics, among which German and

atistics were found to be inadequately treated.i_mong study-skills the greatest emphasis wasplaced on "rapid reading in order to identify themain points in the text, the making of notes, thecollection of information from a library and its usein an essay or memorandum." These skills mayseem rather obvious, but in how many uppersecondary courses is their development a consciousobjective or significantly tested in the assessment ?

4.0. SCHOLASTIC APTITUDE TESTS

Some of those who have been primarily concernedwith the predictive validity of current assessmentprocedures rather than their validity as certificatesof secondary education, have turned their attentionto the possibility of using tests of aptitude forhigher studies rather than of achievement at anearlier stage.

It might reasonably be assumed -that, providedthere is a good 'fit' between the content of uppersecondary and university courses, success in thefirst would be a good predictor of success in thesecond and therefore a good criterion for selection.The whole process of selection at this point in the

(39) Dahlldf, U. (1963) : Op. Cit.

49

educational process has in fact been based on thisassumption throughout Europe. Recent research ina number of countries has thrown considerabledoubt upon it. Dr. Bagg, in England, working withstudents of chemical engineering and comparingGCE Advanced Level results .ith university firstyear and final examinations found a poor anddecreasing correlation between them and concludedthat "only 13.5 % of the v.riance of the Part ITFinals marks can be accounted for by the three 'A'Level results taken together" ("r). Other studiesin England have shown a fair correl.,Lion between'A' Level and first year examinations but littlebetween 'A' Level and final university examina-tions.

Similar research in Germany appears to point inthe same direction. A study reported by E. Wein-gardt of correlations between average Abitur gra-des and the 'Erster Lehrerprtifung' in four Pada-gogisehe Hochschulen showed correlations of justover 0.40 in three cases (although only 0.29 in thefourth), but correlations with the 'Zwischenpril-fung' in Science in five Universiti, s ranged be-tween 0.18 and 0.37 and the correlation betweenthe Abitur grade in Chemistry and the final uni-versity examination in Chemistry in one studywas 0.06 which is even lower than Bagg's findingfor Chemistry in England (41). Orlik found onlyslightly better correlations between Abitur gradesin individual subjects and university success inthat subject with actually a negative correlationfor medical students (42).

These results are confirmed by a much more exten-sive survey of 174 different studies of correlationbetween schoor or examination marks and 'success'in higher education in the Scandinavian coun-tries carried out by Marklund. Henrysson andPaulin. In these studies involving approximately30,000 pupils a mean correlation of 0.27 was foundbetween matriculation n'arks and results in dif-ferent kinds of further education using a widcvariety of criteria. This accords well with otherEuropean findings. Findings in the USA mayapproximate to 0.50 but, as the Swedish reportpoints out there is a far greater degree of simi-

(40) Bagg, D. (1968) : The Correlation of GCE ,A, LevelGrades with University Examinations in ChemicalEngineering. British Journal of Educational Psy-chology. Vol. 38. Part 2. June 1968.

(41) Weingardt, E. (1968) : Der Voraussagewert desReifezeugnisses fur wissenschaftliche Prilfungen.Roth, E. (ed.) Begabung und Lernen. Stuttgart.Klett. pp. 433-447.

(42) (Reported by) Flitner, A. (1966) : Das Schulzeugnisha Lichte neuerer Untersuchungen. Zeitschrift filrPddagogik. Jg 12. Heft 6. December 1966.

50

laritv, both in structure and methods of assessment,between senior high school and university firstdegree work in the USA than in Europe. Thisreport, which includes a valuable survey of theproblems involved in defining 'success' in tertiaryeducation, also records a mean correlation of 0.28between the results of test batteries and results infurther education, using the same criteria as formatriculation marks (4'). In view of these almostexactly equal correlations and of the adversebackwash effects of matriculation examinationsalready referred to, it is not surprising that anumber of European countries are now investi-gating urgently the employment e -11 tests foruniversity selection.

In compiling such scholast aptitude tests theyare able to draw on the considerable experienceof educators in the United States. The assumptionthat success in upper secondary courses, in so faras this was validly and reliably assessed by matri-culation examinations, was a good predictor ofsuccess at the University depended on a largedegree of University control of both courses andexaminations in the upper secondary school. Thi3.of course, has prevailed in Europe but notthroughout North America. Consequently the needfor tests, both of scholastic aptitude and achieve-ment which were not dependent on a prescribedsyllabus was felt all the sooner in the NorthAmerican context.

The College Entrance Examination Board has beenconducting Scholastic Aptitude Tests since 1926.The present tests are three hour objective testsdesigned to measure the development of mathema-tical and verbal skills. They are used by AmericanColleges as one element in a battery of assessments,on which they base their selection procedureBecause they have been in use for so long andhave the financial backing of a continent-widesystem they are based on cumulative experienceand very substantial research. They are designed,however, for a population rather younger and alsoless homogeneous, both in social and academieterms, than that which is now seeking entry tohigher education in Europe.

Typical questions from the American CullegeBoard Scholastic Aptitude Tests are given below :

I. "As warfare has come to engross an in-creasing proportion of the belligerant popu-lations, so military ... has grown far beyondthe problems of varying terrain.

(43) Marklund, Henrysson am, Paulin (1968) Op. Cit.

(A)(B)(C)(D)(E)

customlifehistorystrengthgeography."

This is a relatively difficult question. An analogyis drawn between the general expansion of war-fare and whatever it is that is indicated by themissing word. A careful examination of the sen-tence should indicate that the missing word issomehow related to the problems of "varyingterrain" and its effects on some aspect of militaryoperations. Of the five choices, (A) and (C) areobviously incorrect ; military custom and militaryhistory have changed and grown and were neverlimited in their concern to problems of varyingterrain. (B) and (D) seem plausible ; military lifeand strength do involve problems relating toterrain, but this is not their major concern. Of thefive alternatives given, only (E) geography relate.:specifically to terrain, so this is the correct answer.

2. "If x > 1, which of the following in-crease(s) as x increases ?

IL(x2 - x)

1

1

III. 4x - 2x=(A; I only, (B) II only, (C) III on y, (D) I andIII only, (E) I, II and III."

This question is slightly above average in.difficultyand requires numerical judgement in a relativelynew situation. Two principles must be understoodand applied in this problem : (1) If the denominatorof a fraction increases while the numeratorremains constant, the entire fraction decreases(2) If x is greater than 1 and increases, then x"increases more rapidly than x"-1 ; i.e., x2 increasesmore rapidly than x and 4x3 increases more rapidlythan 2x2. Thus one can show that expressions Iand III increase as x increases, whereas II doesnot. Therefore the correct answer is (D) (44).

Another type of item used to test verbal skills isthe provision of a relatively sophisticated piece ofprose which then forms the basis of items designedto test comprehension. Scholastic aptitude testsare now being investigated in England for theCommittee of Vice-Chancellors and Principals, in

Germany by the Planungsgruppe Ph ago,is, heDiagnostik sponsored by the Volkswagen Stiftungand in Sweden for the National Board of Educa-tion. The intention is clearly to use these as supple-mentary, rather than substitute, methods of assess-ment in order to reduce the dominance of tradi-tional examinations or of periodic assessment. Thepromoters of the English project say that "it ishoped that if the test results prove their worththey will eventually relieve the pressure on theschools to concentrate on high achievement in Alevels to the exclusion of other aspects of thecurriculum" and toe Swedish report referred toabove : "Short-sighted striving for high marks mayeasily eclipse the more long term work of develop-ment of personality in the school. This experienceis by no means new, but its significance has beengreatly underestimated. If, in addition to schoolmarks, other instruments are used in selection, thederogatory effects will be reduced."

Typical questions m the English ExperimentalTest of Academic Aptitude (15) designed for use atthe European university entrance level are quotedbelow :

1. "Directions. In each of the following ques-tions, a related pair of words, printed in capitalletters, is followed by five pairs of worthlettered A to E. Select the lettered pair whichbest expresses a relationship similar to thatexpr,ssed in the pair printed in capitals :

REFORMA conscien eB changeC correction

exerciseE legislation

PROGRESSvirtueresultsimprovementhealthwelfare."

2. "Di:factious. Solve the following problems.You may use any blank spaces for roughwork. Answer each question by marking theappropriate lettered space on your answersheet :

The miles per gallon of petrol obtainedby a certain car falls uniformly from 40at 40 m.p.h. to only 20 at 80 m.p.h. Howmany miles per gallon will be obtainedat 52 m p.h. ?

B 34C 35D 36E 37."

(44) College Entrance Examination Board (1965) : Op. Cit. (45) Association of Commonwealth Universities (1970) :pp. 13 and 26. Experimental Test of Academic Aptitude.

51

Na3

Another type of question used in these tests is toprovide a question and two statements, the can-didat being asked, not to answer the question,but to state whether the data provided in eitherur both of the statements is sufficient to answer it.

The report of the German commission is alreadycompleted and pupils who took the first batch ofEnglish tests before entry to University will becompleting their first degrees this year, so thata considerabk amount of evidence about theirpredictive validity should be available very shortly.If it should prove that this is as high or higherthan that of assessments, whether single or perio-dic, of achievement, the case for using them inselection will be strong since they have greatadvantages in cost and speed ; but it may well bethat they test different qualities from those testedin achievement tests. Since achievement tests arelikely to be retained, both for certification and fortheir incentive value, the decision to considerthem as supplementary forms of assessment willprobably be maintained. This is also the recom-mendation of the Swedish 'survey which suggeststhat there are more opportunities of improvingthe validity of school marks (based on both cumu-lative and )eriodic assessment) as predictors thanthat of tests, and tiat the general test of scholasticaptitude usually proves a better predictor thaneither the specialised test designed to measureindividual skills or the attempts to test qualitiesin the 'affective domain'.

5.0. ORAL EXAMINATIONS

.iral examinations fall into two clearly definedtypes : tests of oral skill, that is the ability tounderstand and to speak a language, and tests ofall the other knowledge, skills and attitudes dis-cussed in previous sections by oral rather thanwritten methods. It will be best to distinguish thetwo, although purely oral skill almost certainlyaffects the assessment in all other subjects, as wesaw pure skill with the pen contributing to successin all written examinations.

Oral tests must form a part o:f any valid assessmentof competence in foreign languages. If, however,they simply take the form of an expose by thepupil or a conversation between examiner andexaminee, they suffer from grave defects both invalidity and re.iiability. Many of these, the relianceon the subjective judgement of a great number ofexaminers, the variations in psychological rapportbetween different candidates and examiners, theproblem of the over-helpful or over-forbiddingexaminer, the difficulty of carrying on a conversa-

52

lion and an assessment simultaneously, arc com-mon to both types of oral examination. There areothers, such as sampling adequately the range ofphonemes to be distinguished, which are peculiarto language examining. Fortunately the rapidspread of electronic recordings, combined with themore detailed analysis of objectives, is producingnew methods of oral language examining whichshould be much superior to the old.

Even the simplest analysis of objectives disclosesthat the.te are two separate skills to be testedhere : aural comprehension and oral expression.Aural comprehension can be objectively tested byplaying tapes, which include a wide distributionof the most commonly confused phonemes, togroups of candi es who are asked to record onanswer sheets thc,r responses to a series of ques-tions which test what they have heard. This notonly introduces objectivity into what has been theleast objective of examinations, but greatly reducesthe cost and increases the speed of the operation.

Tests of oral expression require a sligl tly moreelaborate procedure : the candidate's response toa series of questions, posed to him by the examiner.can be recorded on tape, as can a dialogue earriedon with his teacher on a subject chosen by theexaminer, and the recordings assessed at leisureeither by an examiner or by a jury. It has beenobjected that candidates will be rendered nervousby having to .,peak into a microphone, but this is,perhaps, to ignore the extent to which taperecorders- are now used in language teaching andalso the extent to which other candidates arerendered nervous by the attitude of a severeexaminer. Experiments with this type of oralexamination have been carried out by the Interna-tional Baccalaureate Office as well as in nationalsystems and though there are undoubtedly somecandidates who feel happier with the traditional'face-to-face' oral there are others who prefer thetape recorder.

Oral examinations of the second type, designedto assess pupils' work in subjects other thanlanguages, present much greater problems. Theyhave a far longer tradition than written exami-nations and in many national systems play a veryimportant part. The various forms which theymay take and the problems of reliability whichthey pose have been admirably analysed by Profes-sor G. Panchaud ('") of 'he University of Lausanne,who draws attention to the fact that little or no

(46) Panchaud, G. et a/. (1969) : La Valour Objective desExamens. Etudes P&Lagogiques. Lausanne. EditionsPayot. pp. 55-72.

research has been done on the various extraneousfactors which may affect oral assessment, whethercarried out by the candidate's own teacher, in thepresence of an assessor, or by a stranger. Nor domost systems seem to have examined very thor-oughly the desired role of the examiner, with aview to laying down general instructions and soensuring a greater harmony of approach betweenexaminers. Experiments carried out in relation tothe French baccalauréat. for instance, show thaexaminers vary greatly in the proportion of timeduring which they, rather than the candidate, aretalking (17). Some seem to be giving a lesson, somelc be 'drawing the candidate out' and others tobe conducting an inquisition. One of the studieswhich have been carried out seemed to indicatethat there is a considerably greater correlationbetween the grades awarded by different oralexaminers when they are assessing a number ofspecific abilities and then averaging their gradesthan when they attempt to mark on `global im-pression' ("). It may be, therefore, that the reli-ability of oral examinations could be improved bygreater specificity about objectives as well as aboutprocedures.

What then are the objectives of the oral, as opposedto the written examination, in such subjects asLiterature, History or Physics ? Their use as acontrol of independent work submitted by the pupilhas already been referred to and can be carriedout, as in the International Baccalaureate, byquestions posed on tape, but this is only a specialcase of a general objectiva. Panchaud points outthat a special feature of the oral examination isthat it enables the examiner to make sure thatthe candidate does not misunderstand the question,if necessary by reformulating it, and even moreimportant, that the candidate re.:lly understandshis own answer, by posing further questions. Thiscan make a most important contribution to thevalidity of examining, particularly, but not solely,where the oral is conducted by an examiner whohas already read the written scripts. How oftendoes an examiner find himself saying of a writterscript "This is excellent for a pupil of this agebut does he really understand what he has wre- an,or is he merely repeating as jargon what he hashalf understood from his teacher ?" The difficultiesin introducing such a concrot are practical ones,

(VI Pieron. Reuchlin and Bacher (1962) : Line rechercheexperimentale de docinologie sur les exarnens dephysique au niveau du baccalaureat de mathema-tiques. Biotypologie. March/June 1962.

(48) Timble. 0. (1954) : The oral examination : its va-lidity and reliability. School and Society. New York.Vol. 39. pp. 550-55'z.

5

of which the greatest is the conflict between speed,validity and reliability. Unless examining is dis-tributed among a very large number of highlylocalised juries, the delay involved in completingthe ,arking of the written scripts first and thensem. the examiners who have marked themround the schools as oral examiners is likely tobe very great.

Yet the distribution of examining among a greatnumber of local juries, each working independentlyof the others, is a well-known source of unreli-ability. It is possible that a solution to this dilemmamay be found in the 'moderatior of oral exami-nations by th submission of taped recordings offace-to-face examinations to a central commission.Some very interesting work on different methodsof recording oral examinations has been done inthe context of the French baccalaureat ("), butclearly we shall need much more experimentbefore we can hope to arrive at a satisfactorymethod of giving adequate reliability to oralexaminations. In the course of these experimentsthe factor of cost is likely to play an importantpart. Educational researchers, like so many otherresearchers, are inclined to seek perfection withoutconsidering whether the outcome of their rese-a rches could be generalised and applied on a widescale within the bounds of the resources available.

Oral examinations have other valuable functionsbeyond this testing of the pupil's real understand-ing. The quickness of mind which enables a pupilto grasp a new idea, to see the irnplications of anew piece of information, or to appreciate imme-diately a flaw in his own reasoning when it is putto him, is a quality invaluable both in higherstudies and in future life. Its development shouldbe an objective of upper secondary courses andcan only be assessed orally. I have myself found,when examining for the International Baccalau-reate, that one of the most important objectivesof literature courses, the development of a genuinedelight in good literature, likely to last beyondthe period of formal education, can be betterassessed in an oral than in any written examination.It is to these qualities which Panchaud refers, inrecommending oral exar-inations as a way ofassessing "la vivacite d'esprit du candidat, sonhabileté a se tirer d'embarras,' sa fawn de s'expri-mer, la solidité de sea connaissances, son émotivité,etc." ("). Yet, as the same time, the 'global' natureof such assessments increases still further the riskof unreliability.

(49) Panchaud, G. (1969) : Op. Cit.(50) Panchaud, G. (1969) : Op. Cit. p. 60.

53

It seems possible that the rapid developmentthe video-tape recorder may help towards theachievement of greater reliability, by providingus with better means both of training oral exam-iners in this type of assessment and of moderatingthe assessments made. There seems some justifica-tion for the expenditure of some resources in thisdirection, since unless reliability can be improvedthere is a danger that this type of examination willbe abandoned and a process of assessment whichis, in the opinion of many contemporary students,too impersonal already depersonalised still further.

One further type of assessment to which video-tape or film might contribute is the assessmentof practical laboratory work in the sciences. Wehave already seen that in some examining systemscontinuous assessment by teachers is being adoptedin place of the single practical test and that thisprobably has superior validity ; but some kind ofcontrol of reliability by means of a more objectivetest would be desirable. If the skills which it isdesired to test in science practicals are analysed,it becomes apparent that while so.ne such asmanual dexterity, can be best tested through con-tinuous assessment, others, such as observation.the formulation of hypotheses and the design offurther experiments, could be tested by the useof a film or video-tape which represented a typicallaboratory situation. Candidates could be shownthe film, perhaps more than once, and perhapsstopping at certain points, and asked to recordtheir observations, interpretations or suggestedfurther experiments, an, to identify certain piecesof equipment, on prepared answer sheets, as forthe aural test in languages.

The final advantage of oral examinations whichmust be recorded is their backwash effect. Theyencourage a 'dialogue' form of teaching moreappropriate to modern youth than the magisterial ;they develop powers of oral communication, in-creasingly important in the age of the telephoneand dictaphone ; and in so far as they employaudio-visual aids in assessment they encouragetheir use in teaching.

6.0. CONCLUSION

Assessment procedures can be used for certifica-tion, selection or orientation. The borderline be-tween these three purposes is sometimes ratherblurred. Many secondary school terminal examina-thns, for instance, were originally certification

7edures. As such they worked reasonably well,tong as full secondary education was confined

' an elite and there was genuinely no numerus

54

clausus at entry to universities. Today, however.with government grants dependent on the gradesachieved, with an official or unofficial numerusclausus in many faculties, and with extreme com-petition for entry to the most favoured faculties ofthe most favoured universities, they have becomein effect selection procedures. This is true evenfor countries like the USA, Japan and Swedenwhich have approached most nearly to open entryto tertiarT education. On the other hand procedureswhich were designed ostensibly for orientationsuch as the 'eleven plus' in England or the newprocedure in the Swedish comprehensive school,described in section 3.3 above, also become selectionprocedures when the proportion of candidatesanxious to follow one of the `channels' availableexceeds the capacity of that channel to absorbthem.

This conflict of purposes is responsible for muchof the confusion about techniques. For certificationa combination of periodic and cumulative assess-ment is probably the most valid measure availableto us and it may be significant that this is widelyacceptable even for certification of professionalcompetence provided no selection or numerusclausus is involved. Its reliability however demandsa high degree of training in techniques of assess-ment and a high degree of professional integrityfrom the teachers, who are the only assessors ina position to carry it out. The first of these, as wehave seen in section 3.2, can be very expensive inteachers' time and in the opportunities i Dr teaching,rather than assessment, which must be foregone.The Northern Universities Board in England whichhas been experimenting with this type of examin-ation in English for the lnst four years found, whenil sought to extend the experiment in 1969 thatof 256 schools approached, only 41 finally joinedin the experiment. Their report states : "Thereasons put forward for not taking part are ofsome interest. Not a few teachers feel that it istheir duty to teach and the Board's to examine" (51).The second demand raises more delicate issueaand the extent to which it can be met will dependon the differing social conventions and socialpressures in different countries. Here it is perhapsenough to say that few examiners or inspectorswith international experience would simply sub-scribe to the somewhat idealistic optimism of A.Agazzi's view that "there must be con Iidence inthe teacher's honesty, sense of responsibility andsense of vocation" (52).

(51) Northern Universitier: Joint Matriculation Board(1970) Sixty-Sixth Annual Report. Manchester.JMB p. 9.

(52) Agazzi, A. (1969) : Op. Cit. p. 56.

For orientation there seems every reason to supposethat global assessment is the ideal procedure. Butglobal assessment is even more expensive and timeconsuming than periodic or cumulative assessmentand involves so many factors that its reliabilityis almost impossible to measure and quite impossi-lile to demonstrate. How, indeed, could we demon-strate what would have happened to a student, whohas been oriented into one channel, if he had in factbeen oriented into another, any more than one candemonstrate that a woman would have beenhappier had she chosen one suitor rather thananother. Confidence in a global process of orienta-tion depends on our confidence in a whole batteryof assessments, many of them subjective. Thisconfidence may well be justified, but it will not beachieved if many students are oriented, againsttheir will and that of their parents, into channelswhich carry less social prestige and inferior lifechances.

Thus orientation procedures also are affected assoon as the element of selection enters into them.Reviewing the state of research and innovation inEurope, it would seem that the most promisingline of development is the 'examen bilan'. What isneeded is a process of assessment which is as validas possible, in the sense that it really assesses thewhole endowment and personality of the pupil inrelation to the next stage of his life, but whichis at the same time sufficiently reliable to assurepupils, parents, teachers and receiving institutionsthat justice is being done. Yet such a process mustnot by its backwash effect distort good teaching,nor be too slow nor absorb too much of our scarceeducational resources. Would not the best way towork towards such a process be to analyse in sig-nificant, not over-sophisticated terms, the qualitieswe want to measure and to adopt for each theappropriate measuring technique, objective test',for some part of the course, cumulative projectassessment including self-assessment for another ?And in arriving at this balance perhaps we shall

decide that the reliability of all our methods isso questionable that the method with tile best'backwash effect' is the one to be preferred. Thedilemma is well posed by J. Valentine of theEducational testing service in Princeton : 'good'examination from the educational impact point ofview, that provides an effective model of desiredstudent behaviour, suffers as a measuring devicebecause ef the limited sample of- behaviour itproduces and the unreliability of marking. A 'good'examination from a strictly measurement point ofview, on the other hand, that generates an ade-quately large and representative sample of be-haviour, is likely to resort to efficient but essen-tially indirect and artificial measurement devices,such as multiple-choice questions, which havelimited value as classroom exereices" ('").

The xperience of four years in the InternationalBaccalaureate Office has shown how much Europehas to contribute to this programme of improvingassessmene but often also how ignorant we areof each other's systems and how much we shouldgain by coordinating our research. Is there, forinstance, any cross-national comparative study ofthe costs of assessment, in time spent by pupilsand teachers both on taking examinations andpractising to take them ? Is there any comparativestudy of the improvements made in the reliabilityof written examinations in different countries sincethe report of ale Carnegie Commission ? If so Ihave been unable to find them, and ProfessorPanchaud's study of oral examinations quotedabove indicates our equally thinly covered fieldfor them. Considering the vital part which thebaccalauréat or its equivalent plays in the lives ofso many young people today, there is surely acase here for a concerted European programme ofresearch studies.

(53) Valentine, J. (1969) : The Unbearable Burden ofExternal Examinations in England and the UnitedStates. Comparative Education. Vol. 5 No. 2.

55

SECONDARY SCHO -LEAVING EXAMINATIONSby E. EGGER

The report on "Secondary school-leaving examinations- IDoc.CCC/EGT (71) 6], drawn up by Professor E. Egger, Uninersity ofGeneva, comprises two parts. In the first, Professor Egger discussesthe results of an enquiry conducted by the Committee for Generaland Technical Education Council of Europe member States andoutlines the present position of various examination systems aswell as the observed trends. The second part consists of individualcontributions by specialists, who make suggestions relating to thevarious problems inherent in examinations. Three of these papersare reprinted below.

Examination research: Results thus

Interest in the problems raised by examinationshas never been so keen ; it is shown in a largenumber of publications, many of which containexpressions of opinion and reflection or describeplanned or attempted reforms. Accounts of eventsobserved or systematic experiments using materialgathered in such a way that facts can be confirmedor hypotheses objectively tested are less frequent ;and it is these observations and experiments whichconstitute the field of examination research do-ci mology ( ' ).

The earliest work in this field was almost cer-tainly that done in 1922 by H. Pieron, M. Pieronand H. Laugier, on the French primary school-leaving certificate, comparing exam results withthose produced by the same pupils in a series ofpsychological tests. The object then was to findout whether school examinations could be used asa criterion of the value of psychological tests ;results were not conclusive. They are mentionedhere, however, because they indicate a trendwhich must be noted : the first research proposedto establish experimentclly which of two testingprocesses was most satisfactory. One fact becameso instantly and disturbingly clear, however, thatit may well have driven the new field of research

Cf. H. Pieron, Examens et doeirnologie, Paris, PressesUniversitaires de France, 1963.

56

ar and outlook for the futureby M. REUCHLIN, Paris

off its original path : the fact that traditionalexamination methods were monumentally inade-quate. Almost immediately criticism of thosemethods became the chief activity in the field,seeking principally and almost exclusively to accu-mulate objective, verifiable data confirming thefaults of traditional methods of evaluating schooland university achievement. This attempt washighly successful, although its success gives nocause for rejoicing. Examples of such work werethe following : Etudes docimologiques, publishedby H. Laugier, H. Pieron, M. Pieron, E. Toulouseand D. Weinberg (in the series Travail humain) ;the international enquiry financed by the CarnegieCorporation on Les conceptions, les méthodes, latechnique et la portée péclagogique et sociale desexarnens et concours ; a volume published by theEnglish Carnegie Commission in 1936 (An exami-nation of examinations, twice reprinted since,including a 1941 edition by Macmillan, entitledThe marking of English); and a publication bythe French committee, the same year, called Lacorrection des epreuves écrites dans les examens.In 1956, the French advisory council for scientificresearch and technological progress, at the sugges-tion of H. Laugier, included the problem of exa-minations on its list of subjects requiring urgentnational investigation and financed a new courseof research to be carried out by the national

Inttu t d'etude du travail et d'orientation profes-slonnella, the results of which have been publishedia several periodicals (Bulletin (le 'i'estitut natio-

d'otude du travail et d'orientation profession-nelle, Travail humain, Biotypologie). This work,directed by H. Pieron, M. Reuchlin and F. Baeher,remains largely critical, but also attempts to make

positive contribution to the investigation ofexaminations, thus returning to the original and,

may be, too quickly abandoned, intention of thefirst studies in the field.

Current investigations are continuing in this di-rection. The teams carrying them out have beenpredominantly trained in psychology, and employall the techniques of measurement and controldeveloped by that science since the beginning ofthe century. The adoption of these methods ofobjective evaluation necessitates an explanation ofthe value-scales employed, and consequently ofthe aims of education as well.

A situation report on docimology

These very cursory historical notes explain why anexperiment-based critique of traditional examina-tion and marking methods forms such a largeproportion of the work done in the field. Theresults being without exception convergent, andhaving been abundantly confirmed, there is nolonger any research of this type being carried out.In so far as they give teachers (especially thosewho agree to participate in them) informationconcerning the unreliability of their usual methodsof assessment, however, such experiments are stillvaluable.

With regard to the French baccalauréat in parti-cular, statistical analysis of series of marksactually given in the examination has beenrelevant.

For example, a comparison was made between theaverage marks given in the same examinationsubject by 17 boards in the philosophy exam and13 in the mathematics exam (July 1955 examina-tions), the candidates in each set being allocated tothe boards at random (in alphabetical order).Variations in averages were found to be consider-able from 5.81 to 9.06 for the written paper inmaths, from 8.2 to 9.5 for the written paper inphilosophy, from 8.3 to 13 for the physics oral(mathematics section), from 9 to 14.4 for thenatural sciences oral (philosophy section), etc.These fluctuations are greater than one wouldexpect for repeated estimates of an average basedon a series of random samples from a single popu-lation group.

These differences in averages in the markingscales used by different examiners lead naturallyto similar variations in percentages of candidatespassed by the different boards. In the examplegiven above, pass percentages ranged from 48 %to 61 '; in philosophy, and from 31 ' ; to 53 ; inmathematics, depending on the board.

The consistency in marks given to the same can-didates in different subjects has also been studied,and has been found to be very low indeed, even asbetween pairs of subjects such as physics andmathematics, or the written and oral parts of anexamination in the same subject.

To establish theae conclusions with greater ac-curacy, experiments have been specially conductedusing not the marks actually awarded to examina-tion candidates, but figures specially obtainedexperimentally.

In the Carnegie project. sets of 100 scripts wereselected from the examination (baccalauréat) officefiles, five copies of each set were made, andissued, for marking to five different teachers, allexperienced examiners. Averages varied as widelyin this instance (where the material was identicalin all 5 cases) as in the analysis of actual examina-tions : from 6.32 to 10 in French composition ;from 7.01 to 9.16 in maths ; from 7.65 to 11.23 inphilosophy ; from 7A I to 9.48 in physics, etc.Consistency among examiners (which is affectedonly by the rank assigned to scripts, not by themarking scale) varied considerably from subjectto subject and also between pairs of examiners.

Also as part of the Carnegie project, 3 Frenchcompositions were marked by 76 different readers,with results ranging, respectively, from I to 13, 3to 16 and 4 to 14.

More recently, expe-iments have been carried outinvolving the multiple evaluation of identical datain physics orals (2nd part of the baccalauréat inmathematics). Twenty oral examinations wererecorded on tape and listened to by 16 lycee tea-chers with considerable experience of examiningorally for that particular examinatien, each teachermarking each unit separately. Averages rangedfrom 13.4 to 8.03.

These difficulties in assessment are not, of course,restricted to the baccalauréat. Other experimentsat other levels (from the primary school-leavingcertificate to university first-degree examinations)reach similar conclusions. Nor are they peculiar toFrench examinations, which have been used asexamples here. The general tenor of the EnglishCarnegie Commission report is identical.

57

Such critical material, which exists in abundance.was the firsi contribution of examination research.The uses it has been put to are mistaken and in asense deceptive in my opinion : it has been takeoas evidence that examinations create a false pro-blem, or adduced in favour of solutions to theproblem which, although offering the appeal ofsimplicity, are probably false solutions.

If critical research into examination evaluationshows that the traditional type is a poor solutionto the problem of examining, the problem never-theless remains ; it remains to be solved, and theimperfections of the present systems are no justi-fication for the total abolition of any form ofexamination (9.

The advances in science and technology whichhave revolutionised our world have given greaterimportance than ever before to the qualificationshierarchy. Qualifications have lost most of thespecificity they may have possessed in a craft-based production system, and now serve mainlyto distinguish between individuals in terms oftheir general level of education. Workers can moverelatively easily from one job to another at thesame level of qualification ; but it is extremelydifficult today for them to make any vertical pro-gress up the qualifications ladder in the exerciseof their trade or occupation. The level of generaleducation reached before leaving school (or at theprice of great effort while working) is the decidingfactor.

The school and university machinery which pro-vides this general education can no longer concernitself exclusively with the transmission of dis-interested culture, as it could at a time whenprotracted study was possible only for a privilegedminority not subject to economic pressures. Thevast majority of today's pupils and students comeseeking access to the most highly qualified occu-pation possible. The mere length of time spent atschool is not in itself any guarantee of the educa-tion actually received by anyone. If the universitydid not trouble to make individual assessments ofsuch education, the "consumers" themselves wouldassuredly do so ; and in accordance with scales ofvalues which, however distasteful they may be tothe universities, would nonetheless tellingly affectthe direction of instruction.

It is hard to imagine that the problem so imper-fectly dealt with by examinations would not arisenaturally in the course of an education. Long years

(2) Cf. on this point the controversy between M. Lobrotand M. Reach lin in Pourquai des examens P Paris,Societe des editions rationalistes, 1968.

58

of study must inevitably carry some people to thchighest levels of qualification ; and our observationtells us 1 hat all individuals are not equallycapable of reaching those heights, or equallyprepared to submit to the long hours of effort andway of life required of those who do want toreach them and remain there.

Long study also means diversified stuctir. Somesubjects are spontaneously chosen by a number ofstudents out of all proportion to the employmentavailable at the end of their study. In technolo-gically advanced societies, there are other reasonswhy some form of check upon individual capacitiesbecomes inevitable. Instruments of considerablepotency may be put into the hands of individualsin a wide range of activities ; because of theircost to the public and their potential danger,society lays down very strict conditions for estab-lishing the abilities of those who use them. Howmany citizens would consent to the measuresproposed by M. Lobrot, writing on the subject ofexaminations : "Examinations must be replaced bysomething else. But what ? The answer is : bynothing. It should be enough for a man to walkinto an industry and announce, 'I have learnedchemistry' (perhaps signing a document to thateffect), his employers would judge him by hiswork, and would soon see whether he has told thetruth. It should be enough for a man to hang upa sign and say, Tm a doctor', and his patientswould judge him by his cures.

If we will admit the reality of the problem whichtraditional examinations are struggling to solvewithout much success, we must also admit thereality of certain related problems which willpresumably be with us as long as the first one is :whatever process is used to guide pupils in theirstudies, and assign them to a particular occupa-tional level, that process will affect the impartingand receiving of education and the emotions andstability of students.

Critical research on examinations, then, cannotmake the problems disappear by proving thattraditional examinations are a poor solution tothem. Nor can it give unreserved support to morerecent alternatives whose only value may be intheir apparent simplicity.

Suggestions for measuring scales and types of tes

Some of these would-be solutions are technical :certain reformers hold that it is desperatelyimportant to abandon the 1 to 20 scale in favourof the 1 to 5 scale. A scale with fewer gradeswould lessen the likelihood of two cerrectors

60

placing the same essay in different categories. itis true ; but by using a scale with a single grade,the risk would be clone away with altogether ! It isequally clear that each single error would bemagnified in proportion as the number of gradesdiminished. The net result of this reform, thus,would be to convert a total sum of error, whichwould itself be in no way reduced, into largerunits. It is possible that a five-grade scale wouldbe better suited than a 20-grade scale to thesensitivity and accuracy of that measuring instru-ment which is the corrector : using a five-gradescale would at least avoid the absurdity of usingtenths of a millimeter to talk about somethingwhich has been measured with a dressmaker'stape. But this is only a hypothesis, which couldand should be tested before being adopted. Untildoubt on this point has been dispelled, the dangersof imposing a broad-band scale (increasing theweight of each single error) are greater than thoseof putting up with a narrow-band scale for thetime being.

Proposals to replace numerical evaluations by ver-bal ones form a second set of non-solutions.According to them the five grades mentionedabove would not be designated by the numbers 1to 5, but by words, such as Poor, Fair, Average,Good, Very Good. Some teachers, the literature-minded in particular, hope great things from thistype of reform. Partisans of this view apparentlysee a significance in words which could providean absolute scale of reference, by offering ameans of expression in language understood by all.Experience (') unfortunately shows that languageis understood differently by diffeient assessors :evaluated ye bally, the same set of homeworkpapers produces even greater disagreement amongcorrectors than when those same correctors usenumbers.

But some say that words can be used otherwisethan as a direct translation of numerical marks.They can be arranged into sentences expressingthe overall impression made on a teacher by eachof his pupils. This proposal is often associatedwith the idea that a teacher, having had regularcontact with a pupil throughout a year (chiefly insecondary education), is well-placed to make asolidly-Lased general assessment of him, withoutrecourse to any formal means of testing what hehas learned.

A discussion of this proposition, which involves

(3) M. Demangeon, S. Larcebeau, tine experience de cor-rections multiples, B.1.N.O.P., 1958, special issue,pp. 131-156.

o461

more fundamental considerations than the pre-vious ones, would demand more time than can begiven to it here (I). I would only offer two orthree reminders. In the first place, the materialconditions, in which teaching takes place in manycountries (the teacher-pupil ratio in particular) aresuch that a teacher cannot always know everypupil personally ; as things are now, he may evenbe completely ignorant of the home, family andinterests of a child who has attended his classesfor a year. Where closer relations have developedbetween teaches and pupil, each may have reachedconclusions regarding the other which are felt tobe self-evident and beyond question, but this self-evidence and certainty are, of course, totally sub-jective, based wholly on one system of personalrelations which would be different with anotherteacher, and in some cases permanently influencedin nature and manner by one small detail.

The comparison of impressions by several teachersmeeting in "council" is a palliative. Its value islimited : the pupil may have developed a tem-porary and superficial attitude towards the entireteaching staff ; also, the group comparison itselfcannot escape the laws of group dynamics, whichconfer very different degrees of persuasivenessupon the testimony of different persons, accordingto the structure of relations between those persons.I would add that individual judgments of this typeinevitably reflect the average level of the wholeclass, and levels vary far more than one mightsuppose. They also reflect the values of eachteacher, the relative importance he attaches toeach of the elements considered in the overallevaluation.

Here again, objective research has shown thatdifferent teachers have very different scales ofvalues. It would be extremely optimistic to sup-pose that all these sources of potential error wouldultimately cancel each other out in a group assess-ment, provided only that it covered everything.There is every reason to fear, on the contrary,that errors are not independent, and in particular,that the emotional tone of relations between oneteacher and his pupil will pull several other sour-ces of potential neuralisation into the same orbit.As a result, an overall assessment, even coveringa relatively long period of time, will riot solve theproblem of examinations. It has been established ()

(4) Cf. M. Reuchlin, P. Bache'', L'appréciation des &evespar leurs projesseurs, Revue frangaise de pédagogie,1968, No. 2, pp. 19-25.

(5) F. Bacher, M. Reuchlin, Le cycle d'observation. En-quête sur l'ensemble des &eves d'un département,B.1.N.O.P., 1965, 21, No. 3, pp. 149-236.

59

that at the end of five years of primary school, ateacher's general estimate of his pupils' probableperformance in lower secondary education wasless accurate than a standard one-hour achieve-ment test, when the two predictions, made at thesame time, were compared with actual results atthe end of the seventh year of school. This doesnot mean that the teacher's observations do notcontain a mine of potentially precious informa-tion ; the problem is how to render that informa-tion useful. It will not be solved by recourse tointuition, generalisations, and verbalism.

At an even higher level of generalisation, we findanother pseudo-solution. This consists in affirmingthat examinations create problems only if viewedas an "eliminating selection" but not if they areseen as an "advancement selection" ; or thatreplacing selection by orientation would solvethem. Clearly, this is more rhetoric, which maybe justified by particular social circumstances butdoes nothing to alter the terms of the problem.It is essential that those who do not w ish or areunable to embark upon studies requiring verticalor steep ascension should be able to find a hori-zontal or more gently graded alternative at everylevel of education, It is essential that their choicecease to be between success or nothing, and be-come a choice between different successes. Theywill be qualitatively different, of course, but itcannot be seriously argued that they are notprimarily hierarchical. The development of thishierarchy is not a political issue ; it is inextricablylinked to the decisive role being played by scienceand technology, in every system. This is the pro-blem with which the present examination systemis failing to contend, this is the problem whichthe non-solutions I have mentioned either displaceor deny.

Examination research cannot pretend to offer anysatisfactory, full solution. To the extent that itadopts a positive, constructive attitude towards itstraditional critical function an attitude toooften lacking in the past it can, however, offersome partial suggestions at the technical level, andalso attempt to state the wider problem moreclearly.

As far as techniques are concerned, simple statis-tical operations may be of some value. For exam-ple, a group of examiners making a randomselection from a large set of papers should findtheir marks distributed around equal averages.Where there is a large gap between the averagesof two correctors, both may justifiably be askedto alter all the marks they have given (by adding

60

to or subtracting from each the same number ofpoints), in order to lessen the disparity. Thisprocess has been recommended to baccalaureatboakcis of the Academie of Paris.

in many fields standardised questionnaires mayhe used to check factual knowledge. They containa large number of questions and cover an entirosyllabus. Answers may be open or multiple-choice.A good deal of the opposition to this form ofcheck is based solely on ignorance or prejudice.

An interesting variant involves building up apublic "question bank" during the course andwith the aid of teachers, from which test questionsare subsequently drawn. Another worth-whileexperiment would be to give standard tests toextremely large groups of students. By consultinga published account of the results (percentage ofcorrect replies to each question), teachers wouldbe able to place thAr own classes in relation tothe large group and make the necessary deductionsabout their own teaching and assessment scales.

Another and very interesting possibility is the"open book" test, in which the candidate is givenall the material necessary for making a synthesis,following a line of reasoning, etc. The preparationof such tests (choice of subjects and material) andtheir assessment, however, are very difficult. Mul-tiple, independent correcting is probably the only;and expensive) means of achieving an acceptablelevel of -objectivity here.

Introduction to docirnology

In addition to these suggested measuring scalesand types of test, it is to be hoped, in broaderterms, that all teachers might be given an intro-duction to docimology marking in th e. course ofthrir training. This should, without fail, cover fiveaspects :

-- The purposes served by pupil assessment ;Elementary statistics ;Study of published research ;Experience of multiple correcting ;Study of new types of checks.

Such an introduction would not provide teacherswith ready-made solutions, for these are still to befound. But it would make them conscious of theproblem, and enable them to take a more activepart in the search for solutions, both in practice(the work of the English examinations boardsmight serve as an example) and in research.

At a still higher level of generalisation, work onexamination evaluation leads in turn to an effortat clarification ("). A better solution to the pro-blem which traditional examinations -So-lye sopoorly cannot be found withmt first defining theexact function academic assessment is supposed toperform, e.g. does it relate mainly to teaching, orshould it simply describe attainment, or is itspurpose piincipally to estimate suitability for fur-ther study of some particular type ?

(6) M. Reuel-11in, Lu docirnologie, effort d'explication.Les Annis de Sevres, 1968, No. 2, pp. 3 -40.

This effort to clarify the functions of asse smentand examination leads in turn, however, to aneffort to clarify the aims of education sinQethe object of any such operation, after all, is tofind out whether or not they have been achieved.It is certainly not for teachers alone, and still lessfor the examination research experts, to definethose aims, nor is that the intention of recentresearch, in particular, that of B. S. Bloom ; but itis part of their task to state the question in termsthat will lead towards a useful answer, and thisis at least one of the directions which modern workon examinations should pursue.

Objective testing and educational assessment

The doubts raised on all sides about the educa-tional value of conventional examinations haveincreased in recent years. In particular theirvalidity as a means of accurate grading and differ-entiation has been called into question. Too much,i1 is alleged, is left to chance. Most "essay-type"examinations preclude a systematic sampling ofthe knowledge, skills and behavioural attitudes oflearning. The setting of only a few questions toanswer means that large areas of what has beenlearnt remain untested. If these questions containalternatives the unfairness to the candidate be-comes even more manifest : only rarely. dan suchalternative questions be of the same order ofdifficulty and complexity. Above all, it is alleged,subjectivity in making is great, and may lead toerratic results (I). On the purely material plane,moreover, there are great difficulties surroundingconventional examinations : the security of thequestion papers is only one aspect of this ; thephenomenal growth in numbers of candidates,particularly in the vital examinations of secondaryeducation, is another. Thus everywhere nationaleducation systems are seeking out new methods ofevaluating their pupils. In Sweden, experimentsare being made in continuous assessment, usingobjective tests as one element in this process.Elsewhere, more attention is being paid to theschool record and to the interview, but there are

(1) Hartog and Rhodes : An Examination of Examina-tions, 1935 : also the Report of the Commission Fran-gaise pour l'Enquete Carnegie sur les examens et con-cours en France.

by W. D, HALLS, Oxford

insuperable difficulties in standardising these twoprocedures. It is for these reasons, therefore, thatobjective testing is put forward as yet anotheralternative. The purpose of this paper is to discussits validity, as a method of evaluation.Objective testing may be defined as a systematicmethod for evaluating by sampling procedures anindividual's psychological behaviour, mainly inrelation to ability, aptitude or achievement. Thefirst example of such a test was the "intelligencetest". Although its historical origins can be tracedback to the work of Galton, Pearson and Cattellin the late nineteenth century, the credit fordevising general tests of scholastic ability must goto A. Binet, who, with Simon, first devised inParis a scale of intelligence from which wasderived the concept of the Intelligence Quotient(IQ). From this European work in individualtesting the Americans elaborated group testing oflarge numbers. The first achievement tests weredevised by E. L. Thorndike, working at Teachers'College, Columbia, in New York in the early 1900s.Testing as a method of evaluation first supple-mented and then largely supplanted conventionalexaminations, so much so that today in NorthAmerica most children by the age- of 12 haveundergone at least half a dozen intelligence orachievement tests of one kind or another. Theextent of the "testing industry" may be gaugedfrom the fact that in 1964, 148 million test bookletsand accompanying answer sheets were sold. Des-pite their European ancestry, however, tests as ameans of assessment, have not been widely adopted

61

this side of the Atlantic. (United Kingdom, wherethere were few linguistic difficulties, is a possibleexception ; certainly in England so-called "intelli-gence tests"' were generally in use as part ofthe notorious 11 examination for selection tosecondary education.) One American writer ascri-bes this reluctance on the part of Europeans toadopt testing as a routine procedure as due to"basic ideological and cultural differences of opi-nion about the nature of human abilities" and to"technical and social problems which make thelarge-scale use of objective tests either difficult orimpractical (2). That this attitude in Europe ischanging there can be no doubt. More and more,at the upper levels of secondary education, andparticularly in relation to access to higher educa-tion, testing is being eonsiderd. It is this "classifi-cation and promotion" aspect of testing, ratherthan its more frequent use as a "diagnostic forcounselling and treatment", which is undoubtedlyat present of the greatest interest.

O'ojective test

What, precisely, is an "objective test" Accordingto Ebel ("), a test contains "a small but statisticallysignificant number of short-answer questions'items' designed to test the most important areasof knowledge, skills and behavioural attitudes".The adjective "objective" is mainly applied tocontrast with the subjectivity inherent in markingthe convention examination. Certainly the scoringof a test is usually so simple that no bias can enterinto the marking and no technical expertise isrequired. Where, however, subjectivity remains adanger is of course in the devising of the test,which is a lengthy, highly-skilled and extremelycostly process, and in the pre-determining of whatis the most appropriate answer from a numberof possibilities. Tests, in fact, contain a number of"items" which may be of different kinds. Thesimplest and crudest sort is the one which posesa short question and supplies two possible answers,from which a choice has to be made between the"true" and the "false" one. A refinement of thisand the most widely-used type of item is the"multiple-choice" question where several possibleanswers are supplied perhaps as many as five

all of which are responses of differing degrees

(2) D.A. Goslin : The search for ability : standardisedtesting in social perspective, New York, 1966. This isthe best work on the social impact of testing.R.. L. Ebel : Measuring educational achievement,Englewood Cliffs, New Jersey, 1965. This is one ofthe best standard works on the techniques of educa-tional measurement, and is easily understandable bythe non-SPecialist.

02

of plausibility, but only one of which is absoiutelycorrect. To answer this kind of question requiresboth knowledge and judgment, Another kind ofitem may be described as "classificatory" : thecandidate has to assign each object in a list IA itsappropriate class ; the "objects" may consist, forexample, of names, descriptions, pictures andstatements. A further kind of item is termed thematching" one : two lists of statements or sym-

bols have t, be exactly matched, detail by detail,against each other, using the principle of "closestassociation". To avoid the effect of guessing, amathematical correction can be applied to thefinal score, if the above types of items are used.A final possibility, although this does not fallstrictly within the purview of "objective testing",is o present a number of short-answer questions

usually described as being "open-ended" towhich the pupil may give a "free" answer, asdistinct from the other items, where his choice islimited in advance. This latter kind of test ob-viously requires the corrector to exercise judgmentas well.

Some of the advantages of objective testing havealready been mentioned. The ease with which largenumbers of candidates can be dealt with and thetests scored are not inconsiderable merits. It isalso clear that, with a larger number of items, orquestions, a much more systematic "sampling" ofthe content of a syllabus can take place. Withthe help of a taxonomy, "weighting" can be ap-plied to various parts of the test ; a selector forcandidates for university arts faculty may, forexample, be more interested in how well potentialentrants answer questions demanding verbal ra-ther than mathematical ability, and can "weight"his marking accordingly. Moreover, it is alsoclaimed that a test has great reliability ; if well-devised, it will be consistent in its measurementof what it is intended to measure. Thus the assess-ment made of pupils in one year can be comparedwith assessment of different pupils made in pre-vious years, by standardising scores. An evenmore tangible advantage, for example, in selectionfor higher education, is the claim that tests havehigh predictive validity (although in fairness itmust be added that most authorities agree that acandidate's school record is as good a predictor.The drawback in using records, however, is thatalthough as a whole they are usable, the indivi-dual cases may be assessed on different yeard-sticks so many variables, from teacher quality tostandard of instruction, lie totally outside thecontrol of the selector). All in all, therefore, theadvantages of using tests for assessment are veryconsiderable.

erpretatiou of the scores

Are they, however, overwhelming ? Vital to theunderstanding of objective testing is how thescs,res should be interpreted. Any test, howevercarefully devised, does not yield a score that isanything more than an approximation : it meansthat the estimate of the candidate's ability willfall within a certain range, of which the score heobtains is the mean. The extent of this rangerepresents the "standard error of measurement".The large claim is made that this standard erroris less than in conventional examinations. But, asfar as is known, no European examining body,using conventional methods, has ever publishedwhat the standard error of its examinations is.Until this is made public, the only verdict possiblehere on the testers' claim must be one of "notproven".

A more substantial objection concerns the develop-ment of new theories on the nature of intelligenceThese have called into question the permissibilityof objective testing. We now know that intelli-gence is to a large extent environmentally con-ditioned. The tests are no better than conventionalexaminations in eliminating an element possiblythe preponderant one which gives a socio--ailtural advantage to children from educogenicfamilies. An even more serious criticism is thatobjective tests arid here they may even bemore biased than conventional examinationsfail to measure creative capacity. Too often"intelligence" has been conceived in terms ofwhat has been called "convergent" thinkingthe ability to see relationships that, once perceived,appear obvious. But intelligence, in its new defini-tion, must also embrace "divergent" thinkingthe ability to perceive unusual relationships, whichentails the use of creative thought. Tests have asyet not been devised which could measure thiscapacity.

By their modus operandi existing tests require theanswer that a majority would give ; moreover, inshort "responses" it is impossible to justify anyanswer that is unusual, however ultimately well-founded that answer may be. Yet it is this abilityto discover the unusual, rath-eT than the usualrelationships, to see connections that have notbeen discerned before, which is a quality thatmust be highly prized in modern society, whereinnovative capacity is rare. In a conventionalexamination, where creativity and imaginationmay be given free rein, although its spark cannotbe accurately measured, a qualitative assessmentof it is possible. The more rigorous selection is,

as for example, for entrance to higher education,the more this creative aspect of intelligence is ofvalue. In tests, in any question that calls forrelating one phenomenon to another, the tester,because he cannot interrogate the candidate tofind out whether an unusual answer is well-founded, must automatically penalise those whodo not arrive at the expected answer. Such anobjection to present forms of testing seems grave,and not easily overcome.

Linked with this, is the allegation that tests cannotmeasure powers of synthesis and analysis, or thecapacity to follow through a chain of thought toits logical conclusion, whether in discursive proseor in the symbolic language of mathematics.Testing is regarded as a passive process, in whicheven the ability to express oneself well in themother tongue a prerequisite for success in anyfield of human activity is not required norfostered.

Further objections concern the undoubted factthat tests are, despite thorough checking of items,far from perfect ('). Items lend themselves toambiguities and obscurities. But perhaps this mayoe described as a mere fault of design, which canbe remedied with greater care in drawing up thetest, and in pre-testing it before it is actually used.

A last objection concerns the abuses to whichtesting may give rise. Chief among these is the"backlash" effect upon teaching. There is little orno evidence that well-designed tests such as thoseof the US College Entrance Examinations Board(CEEB) yield vastly different scores when coachinghas taken place. A gain of a few points may beregistered, but since we have been warned thattest scores should be interpreted as demonstratingthat a candidate's ability lies within a certainrange, and cannot be determined as an exact ma-thematical quantity, this is no insuperable draw-back. Yet there have grown up in the l_arger UScities coaching "schools" to prepare pupils fortests. The College Entrance Examinations Boardhas expressed its concern at this detrimentaldevelopment, "because we see the educationalprocess unwillingly corrupted in some schools to

(4) Mention must be made here of an attack on testingby B. Hoffman : The tyranny of 'testing, New York,1962. In 1968, when the present writer attended theannual meeting in Chicago of the College EntranceExaminations Board, which uses the services of theEducational Testing Service of Princeton, and is tnelargest testing organisation in the world, an « Anti-Test Protest was under way. To the present writerit seems that not the principle of testing is calledinto question by the movement, but the abuses towhich indiscriminate testing may lead.

63

gain ends which we believe to be not only un-worthy, but ironically, unattainable". Nevertheless,such commercial institutions continue to exist, andare even emulated by public schools wbo see thesalvation of their pupils encompassed by submit-ting them to endless testing.

Scholastic aptitude tests

Objective tests have been used to test achieve-ment, scholastic aptitude, and personality andcharacter -traits. Unfortunately, the tests of per-sonality and character that have been devised upth now have been the least accurate, yet it isprecisely these which at the upper secondary levelwould be of the greatest utility. Of particularinterest, however, in view of the difficulties ofprediction of success in higher education, arescholastic aptitude tests, a refinement developedfrom the old-style "intelligence" tests. In the USthe College Entrance Examinations Board hasdeveloped a Preliminary Scholastic AptitudeTest, taken in the eleventh grade, and a ScholasticAptitude Test proper taken a year later. Theirpurpose is to serve as a guide to universities andcolleges of the future academic success of theiiwould-be entrants.

In England these tests have aroused so muchinterest that the Committee of Vice-Chancellors,in its search for new methods of university selec-tion has conducted pilot experiments with Englishadaptations of the tests on a wide scale to deter-mine their feasibility as a selection instrument. Nodefinite conclusions have yet been reached. Theaptitude tests have no passing mark as such, butsimply yield a score (which is to be interpreted asaccurate within a given range) which is passedon to the appropriate college or university that acandidate seeks to enter. The score is standardisedon a national scale, and it is up to the receivinginstitution to decide whether it is sufficient tojustify admitting the candidate to the particularcourses it offers. It must be emphasised that thetests are aimed at supplementing the evidenceprovided by the schools, not at replacing it Usedby the Ivy League universities as Harvard, thereis no doubt that their value is much appreciated.

The tests have as their object the measurement ofgeneral ability. They are not yardsticks of attain-ment, but seek to measure the basic learning skillsrequired for university success. They are of twokinds, mathematical (SAT-M) and verbal (SAT-V).Whereas the SAT-M is alleged to measure ability"to reason with numbers", the SAT-V avowedly

64

measures the ability "to read with skill and tounderstand and use words correctly". As well asthe experiment mentioned above, a trial of thetests, without any alternation in their formulation,was c:,o-riocl out on pupils in England. Tho generalconclu.tzion reached was that "United States testscan work well in other countries but that certainitems in any test might have to be changed to takeinto account cultural differences". If this is so asbetween two countries where linguisti- and cul-tural differences are comparatively small, it iseven more so where such differences are large.

An even more challenging experiment is beingmounted in Canada. A Canadian version of theSAT is being worked out in English and Frenchas a criterion for university entrance. Such a test,in a bicultural situation, must be of equal fairnessin both languages, particularly in provinces suchas Quebec and Ontario, each of which has res-pectively a strong English-speaking and French-speaking minority. (Incidentally, the Quebec pro-vincial Department of Education has pioneered themodern use of objective tests in French, and thereare indications that French-speaking nationseverywhere are interested in its work.)

In order to compile a SAT-V test a very compli-cated procedure is followed by the CEEB. Eachyear some 2,000 items are drafted by a team whichincludes a psychologist, English specialists and anumber of "laymen". After pre-testing, only 90items are retained in the final test. The first partof the test can be broken down into items that askfor sentence composition, antonyms, and analogies,and relate to categories such as the esthetic andphilosophical, the world of practical affairs,science, human relationships and general matters.The second division of the test is of items that askquestions on passages of reading comprehensionrelating to the sciences, the humanities and socialsciences. Such passages may consist of straightnarrative or discursive prose. The candidate isasked questions on the ideas, the inferences to bedrawn, the logic of the argument, and its style.Similar care goes into the compilation of theSAT-M test, where the main branches of mathe-matics are used but no previous specialistknowledge is assumed and types of thinkingconnected with computation and numerical judg-ment, and relational thinking, are evaluated. Eachyear a certain number of "dummy" items (whichdo not count towards the candidate's final score)are slipped into the SAT tests ; this constitutes anexperimental section for pre-testing items. Afterthe SAT tests have been worked a most detailedanalysis of each item is made to see whether it

discriminated well between the brighter and theweaker ean,'_Oates, and was otherwise viable.

The difficulties of "transfer" of such tests from onelanguage to another of transliteration or eventi anslation there can be no question seemalmost insuperable. Nevertheless, the prospect ofsuch a test being used as providing additionalevidence uniculturally for selection to higher edu-cation is an attractive one. So also is the prospectof devising satisfactory multicultural one on aEuropean scale as a new instrument to be usedin solving the problem of equivalences, or at leastthe "acceptability" of students from foreign coun-tries. A first step might be the production ofsuitable tests to serve French and German-speaking areas of Europe. It is also likely that itwould be feasible to produce a test which mightserve both the Italian and the Spanish-speakingpeoples. Those in English already exist, and couldserve as exemplars, if not models.

Achievement tests

If one now turns to consider the use of achieve-ment tests in the various disciplines it can be seenthat their utility will vary considerably accordingto the subject. In the mother tongue, where "freecorm:osition" and literary appreciation are requir-ed, they would seem to be of only marginal value.In modern languages, where the aim is to testgrammar and comprehensive and the correct useof language, they would seem to serve a moreuseful purpose. In aural-oral tests, easy standardi-sation is possible. But as with the mother tongue,neither creative writing or literay appreciationwould seem capable of adequate testing in thisway. (We omit translation altogether.) In historyand geography much would depend on the natureof the test. If a series of questions, for example,were asked regarding an historical document, oreven a picture, much could be learned about acandidate ; in geography the best American testsnow consist, for example, in presenting a map,from which deductions have to be made. In thesciences, where much factual knowledge is stillrequired, the test offers a rapid way of systemati-cally checking on the candidate's memorisation offacts, and is also useful for verifying his ability tosolve small problems. The same holds true formathematics.

What has to be avoided at all costs is that testsare used exclusively, and solely as a means ofensuring that a candidate possesses the requisiteknowledge. This was the trap that the Americans

fell into initially. As ,on, however, as it wasemphasised that a curriculum consisted of morethan its content, and postulated general cognitiveaims of learning such as flexibility. judgment.'ntuitive ability and other intellectual qualities, aswell as aims intrinsic to a subject, the characterof achievement tests changed rapidly and radically.But to test such qualities systematically by theuse of items that ae difficult to devise andcannot be used too frequently (although they canobviously be used more than once, for no candi-date can keep the "question paper" when he leavesthe examination room ) is obviously a difficultprocess.

Whole books have been written on the socialimpact of testing. Its pedagogical repercussionshave already been touched upon. There is nodoubt that testing can build up neuroses amongpupils and parents not to mention teachers.What is needed is a clear idea of the limitationsof the testing process. The widespread use of testsas a means of making decisions of vital impor-tance to individuals, whether in school, in businessor industry, has caused much heartsearching inthe USA. In school, apart from their use inselection for higher education, tests have beenused as part of the counselling and guidance pro-cess, for differentiating pupils according to theirability, and for the weeding-out of gifted orretarded children to be placed in special classesand schools. It must be emphasised, however, thatin the context of schooling, only rarely is testingused as the sole criterion : before :decisions aremade other evidence, such as teachers' evaluationsor school records are almost invariably used.Testing is therefore no panacea for all the educa-tional problems connected with evaluation thatface us in Europe. There would seem to be astrong case for a series of controlled multinationaland multidisciplinary experiments in evaluation inwhich the advantages of testing are weightedagainst those other modes of evaluation in cur-rent use, from conventional examinations to classwork and the use of school records. Correlationsof prachctive validity, for example, might beestablished through using each of the evaluationprocesses mentioned on all candidates for entranceto higher education and then comparing theirefficiency. In any case, if the return across theAtlantic, to where it originated, of objective testinghas served any purpose, it has been to re-awakeninterest in producing a more systematic dareone use the word "scieatific" ? process of eva-luation than the subjective methods, too oftenbased upon prejudice and dogmatism, which havehitherto held sway in Europe.

65

From point-in-time" examination to general assessment

In October 1966, the Council of Europe held acourse on examinations and access to higher edu-cation, in Brussels ; in a report read on thatoccasion, I proposed certain definitions of exami-nation types which I have again adopted in thefollowing -pages.

The examination was defined as an operationdesigned to assess a candidate in relation to adefinite goal.

With regard to the structure of the examinationitself, a distinction was made between the "imme-diate" type which I have since called a "point-in-time" exam (examen ponctuel), using a term whichunderlines its distinctive feature, and the "long-term" or general assessment based on a largenumber of sources of information.

The hazards of the pointin-ti examine _ion

The particular feature of this examination is thatit is performed in a period of time which isextremely short in relation to the period of pre-paration for it and to the period during which thepupil may be observed and assessed by his tea-chers. In essence, it selects, more or less at random,one point on the graph of the pupil's performancein the subject throughout the whole of an aca-demic year or course of study.

It may be of several types :

the candidate may be given a series of psycho-technical tests ;he may be "interviewed" on more or lessnarrowly defined subjects by a small group of"judges" ;he may be asked to write papers referring to apre-determined syllabus in a series of academicsubjects, and judged according to the quality ofhis performance (written or oral).

In schools, the third type is by far the most com-mon and what I have to say hereafter referschiefly to it.

The point-in-time exam is unreliable because itjudges a candidate solely on his reply at one par-ticular moment not of his own choosing to aproposition selected from a number of possiblepropositions, to which he is required to address

66

by J. CAPELLE, Bergerac, Dordogne

himself. The quality of his reaction cannot be thesame at every moment and for every proposition.

Any numerical assessment of his performanceaccordingly involves a margin of uncertainty,bearing in mind his possible response to the ques-tions which might have been put to him insteadof the one that was. Given that success dependsupon his obtaining a certain minimum mark, theelement of chance will mean something verydifferent for the candidate well above the mini-mum and for the one who comes close to it.

In addition to this uncertainty, which we may callintrinsic, there is another, extrinsic element ofchance, resulting from the diversity of assessmentswhich different correctors may assign to the sameperformance, or which the same corrector mayassign to the same performance at different times.

If the evaluation of a candidate's capabilities in aparticular subject is subject to both these elementsof uncertainty, their evaluation on the basis ofwork produced at one moment of time representsan extrapolation which adds yet a third element

and a sizeable one it is, too to the fog ofuncertainty surrounding the traditional examina-tion system.

Must point- -time examinations be abolished ?

The love of gambling and fighting, that is, thelove of risk, is inbuilt in us ; it is one of our mosteffectives motors. Therefore, it would be absurdto lose the stimulating effect of a point-in-timeexamination. What needs to be carefully watched,and possibly changed is the aim of such exams :the stake must not be so high that the candidatewill be thrown off his stride by undue nervoustension.

Stimulus-examinations can be used with profit,when given moderately often and as a competitivegame, like a contest between athletes. But thepupil who does not do well must not be perma-nently harmed by his failure on one occasion ; likethe athletes, he must be able to tell himself that hewill do better "next time", for there has to be a"next time"_

When the point-in-time examination is also aturning-point examination, there may be no "nex "

time, and this what causes the stress and dis-ruption which are prejudicial, not favourable, tocontinued study.

This being so, is it necessary to abolish this formof competition w 2n its results have a decisive,ffect on a young person's career ?

I do not think it is, provided that throwing opena prized career to competition of this sort does notruin all a student's chances for fulfilling his am-bition and desires in his chosen field of study.

Competitive exa -iation as a means of access tosome careers is not an evil if it is not the onlyacccss to then-i, and if there are a large enoughnumber of similar occupations which can beobtained through other and, in so far as possible,less hazardous routes. The situation of a partici-pant in a competitive examination must be a littlelike that of the worker who plays the pools : ifone of his attempts is unsuccessful he may bedisappointed, but he will not be cast into despair.

A second point must be made : competitive point-in-time examinations become virtually indispen-sable, or in any event more acceptable, whentheir object is to detect exceptional ability ; thosepossessing it seldom do poorly in this type ofexamination, and a challenge merely brings outtheir virtuosity. The "concours general" in Frenchlycées has no adverse effect on those who fail, butthose who succeed are never undistinguished, andthe prospect of being allowed to sit for it providesan excellent stimulus for pupils in those schools.

The international mathematics Olympics in Mos-cow may be seen in the same light. Offering acompetition of this type in certain subjects to thebest pupils in European secondary schools cer-tainly could not lessen the effectiveness of second-ary education they would be striving for a successwhich, like the pine branch for which Olympicathletes competed, would be symbolic and no lessephemeral, but it would have a definite stimulat-ing effect and would provide a valuable oppor-tunity for singling out the most gifted.

Continuous assessment

Evaluating the capabilities of the young, prin-cipally at the time when they must make a choicewhich will decide the success of their futurestudies and career, is a goal as difficult of achieve-ment as it is desirable.

The potential of an individual is to some degreecontingent, no doubt ; it will evolve with hispersonality, his physical development and the

maturity he acquires with experience. It can riseand fall in response to a change of circumstance(discovery of a motivation, dettm-mina Con to sur-mount an obstacle, assumption of unforeseen res-ponsibilities).

There are so many imponderables among theelements which propel an individual towards acareer and which, more often than not, deposithim in a vocation for which he never suspectedhe was prepared.

But setting all these difficulties aside, along withthe knowledge that at any given moment thecomponents of an individual personality are notall intrinsic, we must still be able to define thosecomponents far more objectively than is possiblewith the patently fortuitous deductions that can beobtained from a student's performance on a singleexamination.

It is in this context that the long-term assessmentbecomes relevant.

Unfortunately, the more we seek to define thisform of assessment, the more we realise how littlewe know about the art of evaluating the indivi-dual. For too long, university circles may wellhave regarded educational research as being oflittle importance, even incapable of reachingbeyond conventional verbalism ; and new horizonsmay well have been slow to appear for thatteason ; but research on examinations which, soto speak, perform the same function for educationas inspection in the factory does for productionis undeniably even more inconsistent in doctrine,and has even less to show for itself in actualresults. In order to define as concretely as possiblethe concepts that go to make up the idea of long-term assessment, we shall look at the specificproblem of assessing a pupil at the end of .theupper secondary course (usually lasting threeyears), with a view to establishing a summary ofhis attainments and a portrait of his potentialqualities.

These two aims call for two types of evidence,one relating to what the pupil has already achievedand the other to what he may be able to achievehereafter. The former is analytic, the latter syn-thetic.

Analytic evidence

The object here is to produce a condensed and ifpossible standardised account of the pupil's attain-ments in each subject throughout the course (e.g.in the French system, during the three years of

67

the Ina write). This account should not, of course,be confined to a list of marks or place obtained onthe traditional composition. The profession of re-sults obtained over a period of time on otherforms of exercises or tests should also be noted.

One possibility would be to give tests throughoutthe course to all pupils in schools of the same levelin one region, or all over the country. The use ofthese tests, which would act as stimulants and alsomake comparison more easy, should never giverise to the tension and agitation caused by theinstitution in February 1960 of a part-session ofthe baccalaureat exam.

After the Easter holidays, in each of the threeyears of the course, pupils would write an essayon a subject drawn by lot (and for which thenecessary documentary material would be sup-plied, so as to dispel all needless anxiety). Con-tinuing with this hypothesis, the subject maths,French civilisation, physical sciences, etc. wouldnot be disclosed until the exam, In this way, pupilscould both work and sleep undisturbed during thepreceding days. The essay would be correctedcarefully, following instructions given by a re-gional or national examinations board.

The boards themselves should work in close asso-ciation with the educational research instituteswhose creation within universities was urged bythe Caen Conference in November 1956.

Such tests, being "point-in-time", should not, ofcourse, be decisive ; the results would be recordedin the analytic summary alongside other results,as an additional source of information. Theirimportance could be considerable, however, andnot only because they would facilitate comparisonand increase comparability between school niark-ing systems.

Whatever the fate of this proposa, the object foreach teacher remains to mark and classify perfor-mances on a relatively large variety of exercicesand over a relatively long period of time, in orderto build up a picture of the pupil's attainmentswhich shows the work he has done and hisefficiency.

Synthetic evidence

Going beyond this account of attainments, theobject now is to compose a portrait of the pupilhimself, which should be viewed as a syntheticassessment of his potential performance in differ-ent branches of study and response to the res-ponsibilities he may have to assume as a student.

68

From a look at subject-teachers' end-of-t rm orend-of-course reports in, say, the school record,we can plainly see the hazards of such an under-taking. The comments are vague and cannot becompared they run along the lines of "poor"."good worker", "achievement poor", "does hisbest", "intelligent" "can do better", "able buterratic", etc.

The truth is that we have not defined what quali-ties we want to assess, we have not determinedtheir places on the spectrum of a pupil's possibleresponses in each subject, and we have not agreedupon terms for expressing that place. The portraitI am thinking of could be compared to the diagno-sis which a group of medical experts wouldproduce, each for his own specialisation, to des-cribe a person's state of health. Each part of thediagnosis would be expressed in standardisedterminology, and the degree of quality or defi-ciency expressed by this terminology could beconverted numerically. If the same person isexamined by a different group of doctors repre-senting the same specialisations, their diagnosiswill be very much like the first, in both the termsemployed and their numerical translation.

In this respect, the field of examination researchis far behind the medical profession !

Nevertheless, let us try to suggest how the portraitwe want might one day be composed, howeverindistinct its outline now. We will not attempt tohave it show every aspect of a personality, whichis always complex.

It could eXpress two types of features :

polarised features, or those revealed in the pu-pil's behaviour in his relations with each subjecttaken separately ;general features, or those which are constant inthe pupil's personality regardless of the objectof his activity.

These features are composed of qualities whichneed to be defined precisely enough to distinguisheach from the others, and evaluated by means ofa convention which must be as simple as possibleand hence numerical.

Even without any decimals, the 20-point scale,however accepted it may be in France, implies aprecision of evaluation well beyond that withwhich human judgment can express itself objec-tively, in an area so full of contingencies and sodifficult to defend against subjective interpre-tation.

It would seem that a quality could be sufficientlysubtly graduated on a five-degree scale, expressedby the numbers 1 to 5 (for insufficient, adequate,fair, good, excellent).

Polarised feptures

For each subject, the qualities would be assessedby the teacher in charge. Greater comparability ofmodes of assessment and greater reliability wouldbe achieved by having teachers of the same orrelated subjects work together. A teacher shouldbe able to follow his pupils' progress throughouta course by consulting the rest of his department

in contrast to the present professorial compart-mentalisation where every teacher scrupulously"respects" the isolation of every other, even whenthey are all teaching the same subject.

The next question is, what qualities do we meanto assess at the level of individual subjects ?

It is convenient, and indeed desirable, to consider

the saint, qualities for all subjects, specifyingwhere necessary how one of them imagination,for example is to be interpreted in a particularinstance.

1.-1 the following scheme, which may be over-simplified and excessively arbitrary, I have sug-gested six qualities.

Three are constant :

aptitude for analysisaptitude for _synthesis and composition,imagination and creativity.

Three are dynamic :

rapidity of assimilation,curiosity,initiative.

On this basis the table of polarised features couldbe presented in highly condensed form, as follows :

Analysis Synthesis Imagination Subject Rapidity Curiosity Initiative

4

3

5

3

3

2

French

History

Mathematics

2

3

3

4

2

3

General features

The assessment of these qualities requires us to goeven more deeply into the individual's personality.

Here we are no longer concerned with qualities asthey may relate to one or another subject of study .we want a diagnosis of the candidate's mentalattitude, intellectual worth, morality, and acces-sibility.

To some extent a portrait thus composed would bea synthesis of the polarised features listed above,but it would go beyond that to reveal the basiccharacter of the individual.

Two problems immediately arise, one of objectivityand the other uf ethics.

As regards the former, it would be necessary todefine these features which can be deemed cha-racteristic, and which would indicate the type and

amount of iesponsibility an individual mightassume ; and a means of evaluating them wouldhave to be found. The evaluation would be per-formed by the staff as a body, including in parti-cular those teachers who have had occasion toobserve the pupils outside the classroom (physicaleducation instructors, study supervisors). Testsmight also be used, and their results comparedwith those of more academic assessment processes,etc.

The ethical problem arises when W2 cease to judgethe work produced by a pupil and begin to assessthe value and deficiencies of his personality.Proceeding in the order outlined, with all thecaution and reservations required in the interestsof objectivity, we should arrive at a true portraitof the individual's basic personality structure, andthus obtain the best possible basis on which tocompare him with others and place him in relation

69

to the requirements of a particular type of studyor career. We should also, however, be trespassingupon his consciHice and privacy.

It is possible to state on a school certificate thata candidate received a particular mark in mathe-matics without inflicting pain ; but it is a far moredelicate matter to record some personality defi-ciency there.

Teachers in a position to compose such a portrait,disclosing every feature of an individual's per-sonality (assuming the thing to be possible), shouldbe pledged to secrecy on the same terms and forthe same reasons as medical practitioners.

The continuous assessment, which achieves thefullest possible knowledge of an individual, alsoencroaches upon the realm of the confidential. Itsfull contents, therefore, can never be made public ;but the person concerned should himself beinformed of them, and should be able, if he sodesires, to communicate them to whatever author-ity takes the decision on applications he maysubmit.

70

To conclude I do not ask for the abolition of"point-in-time" examinations, whose value asstimulants and detectors of exceptional abilitycannot be questioned. But they must not be usedfor other purposes : that is, in conditions whichleave the pupil gasping and his work in tatters.

The long-term assessment would appear to be thebest bas,s for those decisions which have to betaken at the end of a course of study concerningfuture study or the choice of a career. It alone canattest to the continuity of work done, and leavethe candidate with the comforting thought of anevaltuqion which is free from the vagaries ofchance and human moods.

But our schools and univers ties have not beenadequately prepared for the responsibilities itentails. Sustained effort is required in two direc-tions : research on examinations, and acceptanceby teachers and parents alike of a new principle,i.e. that the best guarantee of objectivity is thateacher who knows the pupil, not the one whodoes not.

Publications

The two series of educational works "Education in Europe" and the "CompanionVolumes", publishod in English and French by the Council of Europe, recordthe results of the studies of experts and intergovernmental surveys carried outwithin the framework of the programme of the CCC. We 7.2.re present thelatest publications in both of the series, obtainable from the Council of EuropeSales Agents, as well as some other books published with the support of theCouncil for Cultural Co-operation of the Council of Europe.

Series "Education in Europe"

THE TEACHING OF MATHEMATICS ATUNIVERSITY LEVEL

by F. FIALA

Published by George G. Harrap & Co. Ltd.,London, 1970, 163 pages, 1.50.

This book is a comparative study of mathematicsat the university level in various western Europeanuniversities.

A questionnaire was sent to 150 higher educationinstitutions in 16 countries, and some 50 usablereplies were received. It is mainly on this in!or-mation that the book has been based. The firstdraft, prepared by the late Professor F. Fiala, haobeen modified to take into consideration the com-ments and additional information provided by agroup of mathematicians at a meeting held inStrasbourg in February 1969.

The book concentrates on mathematics at theundergraduate level, and on those students forwhom mathematics is a principal subject. Infor-mation is given concerning university admissionrequirements, length of study, the various degreesor diplomas offered, course content, general orga-nisation of examinations, teaching conditions.Within these various sections brief details aregiven concerning the situation in the variouscountries. Teaching methods applicable to mathe-matics in higher education institutions are touchedon indirectly.

The aim of the book is to contribute "to a search

for criteria enabl ng some sort of equivalence tobe established between the materials studied, inthe hope of ultimately being able to attain a legalrecognition of equivalences between the academicqualifications". In so doing, attention is drawn tothe great divergence in university studies not onlyat the international level but also, in certain cases,at the national level.

Co panion Volumes

HOW TO VISIT A MUSEUM

by Pierre REBETEZ

Strasbourg, 1970, 186 pages. Distribution free ofcharge.

The main aim of the study is to encourage schoolsand museums to unite their efforts to further theuse of the latter for teaching purposes and topromote the full development of creative facilities.In emphasising its educational function, the bookseeks to show the different capacities of the mu-seum : its vitality, its possibilities of contact withthe public, its organisation.

The relationship museum-school is examined inthree chapters : the museum, . its aim.; and itsmeans ; the school curriculum and the TrIllE eurn'sactivities ; the museum as a school.

The author is more specifically concerned with the13-18 age group. He also stresses the importanceof better collaboration between authorities, tea-chers and curators, so that museums may be usedmore effectively by schools.

71

Other publica ions

PAEDAGOGICA EUROPAEA :THE CHANGING SCHOOL CURRICULUM INEUROPE. VOL. VI.

L. C.G. Malmberg, N. V. Uitgever and Georg Wes-mann Verlag, 1971, 268 p.

The theme of the latest issue of the EuropeanYearbook of Education Rese- :ch is "The ChangingSchool Curriculum in Europe', a theme whichdominates the European educational scene. Emi-nent authors from France, the Federal Republic ofGermany, Sweden, Switzerland and the UnitedKingdom have contributed seventeen stuthes whichcover the major aspects in the field of curriculumresearch and development and which show theconvergence of developments and trends. Theeditor, Professor S. J. Eggleston summarises thisconvergence as follows : "The moves to 'curriculumautonomy' have introduced a number of furthervariables previously unknown and certainly un-anticipated by the educators who initiated 'curri-culum development'. Most notably these are the

72

decision m; king powers that have been claimedand won by the clients of the system theparents, students and pupils."

The reader will find this volume to be a mostuseful, and indeed necessary, handbook on thechanging nature and determination of the schoolcurriculum in western Europe. He may regret,however, that the British element dominates thisissue and that the French and Scandinavian lan-guage areas are underrepresented. There is noItalian. eontribut.on. It is hoped that the next twovolumes which are to deal with the diversificationof post-secondary education and guidance andassessment respectively, will remedy this imba-lance.

Paedagogica Europaea has now found its place androle in the international book market as a Euro-pean forum for the discussion of educational issues.The possibility of it being published biannually,and in a cheaper edition, would most certainly bewelcomed. This would enable Paedagogica Euro-paea to follow more closely developments in therapidly expanding field of research and innovationin education in Europe, and also to reach a widerpublic.

Editor : The Director of Education and of Cultural and Scientific AffairsStrasbourg

7S

V

1