This article was downloaded by: [University of Tennessee At Martin]On: 04 October 2014, At: 23:23Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH,UK

Measurement: InterdisciplinaryResearch and PerspectivesPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/hmes20

Assessing what students know,how they know it, or both?Derek C. Briggs aa University of Colorado ,Published online: 05 Dec 2007.

To cite this article: Derek C. Briggs (2007) Assessing what students know, how theyknow it, or both?, Measurement: Interdisciplinary Research and Perspectives, 5:1,62-65, DOI: 10.1080/15366360701293618

To link to this article: http://dx.doi.org/10.1080/15366360701293618

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all theinformation (the “Content”) contained in the publications on our platform.However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness,or suitability for any purpose of the Content. Any opinions and viewsexpressed in this publication are the opinions and views of the authors, andare not the views of or endorsed by Taylor & Francis. The accuracy of theContent should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for anylosses, actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly orindirectly in connection with, in relation to or arising out of the use of theContent.

This article may be used for research, teaching, and private study purposes.Any substantial or systematic reproduction, redistribution, reselling, loan,sub-licensing, systematic supply, or distribution in any form to anyone is

expressly forbidden. Terms & Conditions of access and use can be found athttp://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Uni

vers

ity o

f T

enne

ssee

At M

artin

] at

23:

23 0

4 O

ctob

er 2

014

62 COMMENTARIES

Mitzel, H. C. (2005). Consistency for State Achievement Standards Under NCLB: Council ofChief State School Officers. (Accessed March 19, 2007) http://www.CCSSO.org/content/pdfs/CAS%20series%202.final.pdf

Office of Elementary and Secondary Education, US D. o. E. (2004). Standards and assessments peerreview guidance: Information and examples for meeting requirements of the No Child Left BehindAct of 2001 [Electronic Version], 1–63 Retrieved August 31, 2006 from http://www.ed.gov/policy/elsec/guid/saaprguidance.pdf#search=%22Standard%20Assessments%20Peer%20Review%20Guidance%20%22.

Wilson, M. (Ed.). (2004). Towards coherence between classroom assessment and accountability.Chicago: University of Chicago Press.

Assessing What Students Know,How They Know It, or Both?

Derek C. BriggsUniversity of Colorado

In “International approaches to Science Assessment” Paul Black and DylanWiliam describe eight major elements of assessment systems and contrast someof the differences among these elements across seven countries. Of particularinterest to me is an issue that cuts across many of these elements, namely, whatgets assessed as part of these systems. That is, how do countries manage thetradeoff between assessing the “facts” that student know versus the depth oftheir understanding for how these facts are interrelated? In science assessmentthis issue is especially important: can scientific inquiry be content free? Canor should process skills be assessed independently of content knowledge, orvice-versa? In what follows, I illustrate this issue within the context of a UnitedStates history assessment that has recently been in the news.

How well do college students in the United States understand Americanhistory? Not very well at all, according to a recent report published by theIntercollegiate Studies Institute (ISI, 2006). The ISI study was conducted duringthe fall of 2005 and involved a random sample of 14,000 college freshman andseniors from 50 colleges and universities in the United States. The students inthis sample were given 60 multiple-choice items intended to measure knowledgein the subject areas of American history, government, America and the world,and the market economy. The findings were that college seniors answered, onaverage, about 53% of the items correct, about the same as college freshman,who answered about 52% correct. Eugene Hickock, a member of ISI’s National

Dow

nloa

ded

by [

Uni

vers

ity o

f T

enne

ssee

At M

artin

] at

23:

23 0

4 O

ctob

er 2

014

COMMENTARIES 63

Civic Literacy Board that commissioned the study, was quoted as saying “Werisk a generation of young people not knowing what America stands for � � � Thisisn’t saying students are not doing a good job. It’s saying institutions are notdoing a good job” (Marklein, 2006).

With this background, here are five sample items from the ISI assessment:

1. This battle brought the American Revolution to an end.

(a) Saratoga(b) Gettysburg(c) The Alamo(d) Yorktown(e) New Orleans

2. The federal government’s largest payout is for:

(a) Military(b) Social security(c) Education(d) Foreign aid(e) Welfare

3. The phrase “We hold these truths to be self-evident, that all men arecreated equal� � �” is from:

(a) The Federalist(b) The Preamble to the Constitution(c) The Communist Manifesto(d) The Declaration of Independence(e) An inscription on the Statue of Liberty

4. What kind of government is a junta?

(a) Military(b) Religious(c) Populist(d) Social democratic(e) Parliamentarian

5. Among which of these groups would Saddam Hussein have found hismost reliable supporters?

(a) Islamic brotherhood(b) Baath Party

Dow

nloa

ded

by [

Uni

vers

ity o

f T

enne

ssee

At M

artin

] at

23:

23 0

4 O

ctob

er 2

014

64 COMMENTARIES

(c) Communist Party(d) Hamas(e) Israelis

The answers to these items were (d), (b), (d), (a), and (b). Here is the concludingrecommendation provided in the executive summary of the ISI report:

ISI offers this report with the hope that it will stimulate corrective action andaccountability � � � It is still possible to improve the teaching at our colleges anduniversities of America’s history and institutions, and thereby to forestall thecoming crisis in citizenship. (p. 7)

Black and Wiliam have for some time now pointed out the tension betweensummative and formative uses of assessment, and between items (typicallyconstrained) that assess low-level skills such as factual recall, and items (typicallyopen-ended) that assess higher-level thinking skills. Obviously, the ISI studyinvolved a summative assessment with items primarily testing factual recall ofcontent knowledge. This assessment does not appear to have been designedwith any process skills in mind—those who developed the test would probablyflunk any set of multiple choice items drawn from the content of Knowing whatStudents Know (2001). Indeed, I suspect many would agree that this is the sortof assessment (and report) that would be likely to send most people with abackground in educational measurement screaming into the night.

Yet juxtaposed against Black and Wiliam’s paper, perhaps the findings ofthe ISI report do constitute an indictment of the current system of teaching,learning and assessment in American high schools and colleges—though not thekind of indictment the authors of the report have intended. The principal ironyhere is that American students have for some time now been held much moreaccountable for what they know than how they know it. Outside of some excep-tional advanced placement courses, high school students are typically taught the“facts” of American history, civics and economics—and they get tested on theirability the recall these facts with a wide range of standardized tests. It seemsto me that the solution to the “coming crisis” is not to teach, learn and assessmore factual knowledge, but to figure out a way to make factual knowledgestick by providing the right context for deeper understanding. Along these linesit becomes critical to assess not what students know, but how they know it.I am curious about how this issue is being addressed and/or resolved in othercountries.

REFERENCES

Intercollegiate Studies Institute, Inc. (2006). The coming crisis in citizenship: higher education’sfailure to teach America’s history and institutions. Wilmington, DE. Available online athttp://www.americancivicliteracy.org/resources/downloads.aspx.

Dow

nloa

ded

by [

Uni

vers

ity o

f T

enne

ssee

At M

artin

] at

23:

23 0

4 O

ctob

er 2

014

COMMENTARIES 65

National Research Council. (2001). Knowing what students know: The science and design of educa-tional assessment. Committee on the Foundations of Assessment, J. Pellegrino, N. Chudowskyet al. Washington, DC, National Academy Press.

Marklein, M. (2006). College students fall short on U.S. History. USA Today. September 26.

Balance in Assessment

Richard WhiteMonash University

The review by Black and Wiliam of national systems makes clear the complexityof assessment, and identifies important issues. One of these is balance: balancebetween local and central responsibilities, balance between the weights given tovarious purposes of schooling, balance between weights for various functionsof assessment, and balance between validity of assessment and expediency.Differences in where these balances are placed lie behind differences betweensystems.

Balance Between Local and Central Responsibilities

Black and Wiliam describe how systems vary in the degree to which teachersshare in the assessment of their pupils. The main concern about the involvementof teachers is fairness: will teachers’ assessments be comparable? Though theyavoid judgment, Black and Wiliam note that in Queensland, where the balancelies the most towards teacher-managed assessment, the system works well andis accepted as publicly accountable.

Balance Between Weights Given to Various Purposesof Schooling

Over time, a nation will change its purpose for the learning of science. A devel-oping nation that wants to build up as fast as it can the specialists that willhelp promote industries and health and new agriculture might prefer schools toprepare elite students for tertiary study of science rather than to aim at “sciencefor all”. Production of a mass of scientifically literate citizens can come later,when advances in the economy permit. A shift in purpose would require a shiftin mode of assessment. All of the countries that Black and Wiliam compareare, however, advanced technologically and economically. Why then do their

Dow

nloa

ded

by [

Uni

vers

ity o

f T

enne

ssee

At M

artin

] at

23:

23 0

4 O

ctob

er 2

014