assessing student learning of newton’s laws: the force and motion...

Assessing student learning of Newton’s laws: The Force and MotionConceptual Evaluation and the Evaluation of Active Learning Laboratoryand Lecture Curricula

Ronald K. ThorntonCenter for Science and Mathematics Teaching, Departments of Physics and Education, Tufts University,Medford, Massachusetts 02155

David R. SokoloffDepartment of Physics, University of Oregon, Eugene, Oregon 97403

~Received 6 November 1995; accepted 14 July 1997!

In this paper, we describe the Force and Motion Conceptual Evaluation, a research-based,multiple-choice assessment of student conceptual understanding of Newton’s Laws of Motion. Wediscuss a subset of the questions in detail, and give evidence for their validity. As examples of theapplication of this test, we first present data which examine student learning of dynamics conceptsin traditional introductory physics courses. Then we present results in courses where research-basedactive learning strategies are supported by the use of microcomputer-based~MBL ! tools. Theseinclude ~1! Tools for Scientific Thinking Motion and Forceand RealTime Physics Mechanicslaboratory curricula, and~2! microcomputer-basedInteractive Lecture Demonstrations. In bothcases, there is strong evidence, based on the test, of significantly improved conceptual learning.© 1998 American Association of Physics Teachers.

tunge

icskadin

,leidedn

sti

osin

albn

entwroisoth

t en-ntsoupwilluped

tohis, in

lus-tionc-

eryve

anys of

ofsre

ws

uchuldc-

sesnts

I. INTRODUCTION

In this paper, we discuss the Force and Motion ConcepEvaluation1 ~FMCE!, and its use to evaluate student learniin introductory physics courses. This research-basmultiple-choice assessment instrument was designedprobe conceptual understanding of Newtonian mechanResults obtained on a subset of questions that were abefore and after instruction demonstrate that studentslittle affected by the traditional approach. In an effort to adress this problem, we have developed two active learnmicrocomputer-based laboratory~MBL ! curricula,Tools forScientific Thinking Motion and ForceandRealTime PhysicsMechanics. We have also developedTools for ScientificThinking Interactive Lecture Demonstrations, a general strat-egy to encourage active learning in large lecture classesin any situation in which only one computer is availabAfter describing the test and giving arguments for its vality, we describe its application in the assessment of thefectiveness of these three curricula in helping studentsvelop a functional understanding of the first and secolaws.2

II. CONTEXT FOR THE INVESTIGATION

While we and others have evaluated large numbers ofdents at many colleges, universities, and high schools wthe FMCE, we have had the opportunity to do our mextensive controlled testing at the University of Oregon,the noncalculus~algebra-based! and calculus-based generphysics lecture courses and in the introductory physics laratory, and at Tufts University in both the noncalculus acalculus-based courses with laboratories.

At Oregon, the noncalculus lecture course generallyrolls between 400 and 500 students divided among eitheror three lecture sections. The calculus-based course en60–90 students in one lecture section. The laboratoryseparate course, with weekly experiments, enrolling ab250 students from both of these lecture courses. The fact

338 Am. J. Phys.66 ~4!, April 1998

al

d,tos.edre-g,

or.-f-e-d

u-tht

o-d

-ollsa

utat

as many as half of the students in these courses are norolled in the laboratory has allowed us to divide the studein each course into two research study groups. One grconsists of students who are only enrolled in lecture, andbe referred to below as the NOLAB group. The other grois enrolled in both lecture and laboratory and will be referrto as the LAB group.

At Tufts, the noncalculus physics course enrolls 160200 students in a single lecture section. As part of tcourse, almost all of these students take the laboratorywhich they do a new lab every two weeks.~Enrollment in thelab sections is a mix of students from both the noncalcuand calculus-based courses.! In addition to a large calculusbased course in the Fall, Tufts offers an off-semester secin the Spring. Almost all of the 50–70 students in this setion also are enrolled in a laboratory which meets evweek. These students are statistically more likely to hatrouble with physics than those in the Fall course, and mhave postponed taking physics to increase their chancesuccess.

III. DESCRIPTION OF THE FMCE

In previous studies, we have reported on the evaluationstudent understanding ofkinematicsconcepts using a serieof multiple-choice questions.3 Some of these questions aincluded in the FMCE.4 In this paper, we will focus on foursets of questions from the FMCE that probe students’ vieof force and motion~dynamics! concepts, the ‘‘Force Sled’’questions~questions 1–7!, the ‘‘Cart on Ramp’’ questions~questions 8–10!, the ‘‘Coin Toss’’ questions~questions 11–13!, and the ‘‘Force Graph’’ questions~questions 14–21!.Appendix A contains the complete FMCE.1 Most physicsprofessors thought initially that these questions were mtoo simple for their students and expected that most woanswer in a Newtonian way after traditional physics instrution at a selective university. After seeing typical responto these questions in which fewer than 10% of the stude

338© 1998 American Association of Physics Teachers

inqu

thtyual-hurecsuifcoore-kneth

onin

tegh

rolte

in-re-

ichateicistered

eonn-ard

ents.u-re-nal

tu-thiser-wayve

inoteues-byarens

to aas

theionss torcens

rc

lcudaf

l in-cal-stions9–

Sledraphan-

seem to change their views of dynamics after traditionalstruction, some professors suggested that perhaps thetions are not significant~or valid and reliable! measures ofstudents’ knowledge. Our research does not supportpoint of view, and we will discuss evidence for the validiof the questions after looking at some actual research res

Figure 1~a! and ~b! shows the percentage of 240 noncculus NOLAB students at Oregon during 1989 and 1990 wanswered the Force Sled questions and Force Graph qtions in a Newtonian way. The questions were administeboth before and after traditional instruction on dynamiwhich included standard lectures, homework problems, qzes, and exams. These results show that less than 20% ostudents answered dynamics questions in ways that aresistent with a Newtonian view of the world either beforeafter traditional instruction.~See the discussion of ForcGraph question 15 in Sec. IV, below.! These results are typical and not unique to Oregon. Identical questions were asbefore and after instruction. The pre-test was not returnor discussed with the students. It is clear that askingsame questions twice could not have had a large instructieffect, since the total change before and after traditionalstruction averaged about 7%. The results from this pre-~and those that follow! show that very few students enterina university general physics course, including those whave previously studied physics, understand dynamics fa Newtonian point of view. Unhappily, the post-test resushow that only a small percentage of students adopt a N

Fig. 1. Effect of traditional instruction on students’ answers to the FoSled ~Natural Language! and Force Graph~Graphical! questions from theFMCE. Percent of a matched group of 240 University of Oregon noncalus general physics students who answered~a! each Force Sled question an~b! each Force Graph question in a Newtonian manner before andtraditional instruction in 1989–1990.

339 Am. J. Phys., Vol. 66, No. 4, April 1998

-es-

is

lts.

oes-d,z-then-

eddeal-st

omsw-

tonian framework, even after well executed traditionalstruction. These observations are consistent with othersearch into student understanding of dynamics.4–15

The results of the Cart on Ramp and the Coin Toss, whare shown in Fig. 2 for this same group of students, indicthat very few students answer these questions as a physwould. We consider that these questions have been answfrom a Newtonian point of viewonly when the choiceson allthree questionsindicate a constant force downward for thCoin Toss and downward along the ramp for the CartRamp.~Essentially all student models result in the Newtoian answer for the question in each set referring to downwmotion.!

These questions have been asked of thousands of studThe fact that traditional instruction has little effect on stdents’ beliefs about force and motion, as shown by thesults in Figs. 1 and 2, is confirmed by considerable additioresearch.

IV. DISCUSSION OF THE DYNAMICS QUESTIONSON THE FMCE

The FMCE is most useful when correlations among sdent answers on different questions are examined. Inpaper, however, we are concerned primarily with the pcentage of students answering questions in a Newtonianin order to evaluate traditional instruction and our actilearning curricula.

It will be useful to look more closely at the questionsorder to determine their ability to probe students’ ideas. Nthat although both the Force Sled and the Force Graph qtions explore the relationship between force and motionasking about similar motions, the two sets of questionsvery different in a number of ways. The Force Sled questiomake no reference to graphs and no overt referencecoordinate system. They use ‘‘natural’’ language as muchpossible, and they explicitly describe the force acting onmoving object. On the other hand, the Force Graph questuse a graphical representation, make explicit referencecoordinate systems, and do not explicitly describe the fothat is acting. It is easier to ask more complex questio

e

-

ter

Fig. 2. Student understanding of dynamics before and after traditionastruction. Percent of a matched group of 240 University of Oregon nonculus general physics students who answered groups of dynamics quein a Newtonian manner before and after traditional instruction in 1981990. The Natural Language Evaluation is a composite of the Forcequestions, and the Graphical Evaluation is a composite of the Force Gquestions. All three Coin Toss and Cart on Ramp questions had to beswered correctly for the answers to be considered Newtonian.

339R. K. Thornton and D. R. Sokoloff

e’ r

bfet

it

ure

ripsfot

edrc

arn-liegino-

thd

bththg’’hestt

nere6

inues-ulrs.o

wfivew

phinlnrena

lle

asenwi

d 16thehe

-es-the

ver-

n’sandlts.sicsd

allyal

d

ra-nedndm-

on

using the more precise graphical representation. In spitthese differences in the two types of questions, studentssponses are very similar where there is an exact analogtween the Force Sled and Force Graph questions. In thecases where students do very much better answeringForce Graph questions than the Force Sled questions,possible that their English language skills are weak.

Some questions serve specific purposes. Force Sled qtion 5 is intended to identify statistically students who ajust beginning to consider Newton’s First Law as a desction of the world. It is very similar to question 2 but idesigned to elicit the Newtonian answer of zero net forcemotion at constant velocity. Students who are beginningaccept a Newtonian view statistically choose ‘‘no appliforce needed’’ for this question, even while answering FoSled question 2~and Force Graph question 14! by choosing aconstant force in the direction of motion. Students whofar from consistently adopting a Newtonian view, still aswer Force Sled question 5 by picking a nonzero appforce. Thus question 5 helps identify students who are bening to have doubts about their previous views but areentirely convinced. It is not a good indicator of firm Newtonian thinking. Note that many more students answeredquestion in a Newtonian way both before and after trational instruction than other similar questions.

Force Sled question 6 was originally intended to prodirectly whether students believed that the net force is indirection of the acceleration, but further research showedmany students who believed this still chose the ‘‘wronanswer.~Note that the answer is directly contained in tquestion.! When asked to reconsider their answer, thesedents would change it to B, the Newtonian answer. Up40% of physics faculty also choose an answer other tha~almost always F!. After discussion, they agree the answshould be B, but see no way to make the question cleaThis result indicates that a ‘‘wrong’’ answer to questiondoes not necessarily indicate non-Newtonian reasonSince some people very consistently answer all other qtions from a Newtonian viewpoint while still missing quetion 6, we must interpret the results cautiously. Such resconfirm the value of asking a variety of questions of diveaudiences to probe understanding of particular concepts

The combination of Force Sled questions 1–4 and 7,the other hand, indicate reliably~in a statistical sense! theprevalence of non-Newtonian and Newtonian student vieWe will use a composite average of the results on thesequestions in the comparisons that follow and label this avage as the ‘‘natural language evaluation.’’ Figure 2 shothis average for the data already presented in Fig. 1~a! beforeand after traditional instruction.

Some questions are asked to make sure that studentsread English, can understand the format, and/or can intergraphs. Question 15 in the Force Graph sequence, wasks for the force on an object which is at rest, is the osingle question in which the most common student view aNewtonian view are the same. Consequently, it is answethe same way by a physicist and by 85% to 95% of studeeven before instruction. If a large percentage of studentsswer this question incorrectly~common for middle schoostudents!, it is likely that many of these students are unabto read graphs.

Force Graph question 20 is, statistically, one of the lquestions in this set answered correctly, even by studwho answer the other questions in a Newtonian way. We


ofe-e-wheis

es-

-

ro

e

e

dn-t

isi-

eeat

u-oB

rr.

g.s-

tse

n

s.e

r-s

canretchydd

tsn-

ttsll

use a composite average of the seven questions, 14 anthrough 21, to make comparisons and label them‘‘graphical evaluation.’’ Figure 2 shows this average for tdata already presented in Fig. 1~b! before and after tradi-tional instruction.~It is not possible to compare directly student ability to answer natural language or graphical qutions in terms of the averages we have just defined, sinceForce Sled and Force Graph questions included in the aages are not direct analogs.!

We have tracked conceptual understanding of NewtoFirst and Second Laws of incoming students at OregonTufts over a number of years and find very stable resuSome results for students entering noncalculus phycourses are shown in Fig. 3~a!. Results for calculus-baseintroductory courses are shown in Fig. 3~b!. Notice that thevariations between courses or institutions are generlarger than the small changes which result from traditioninstruction~shown in Figs. 1 and 2!.

V. CONCEPTUAL UNDERSTANDING OFDYNAMICS BEFORE AND AFTER ACTIVELEARNING LABORATORIES

The Tools for Scientific Thinking (TST) Motion anForce16 and RealTime Physics (RTP) Mechanics17 MBLlaboratory curricula, developed for the introductory labotory at universities, colleges, and high schools, are desigto allow students to take an active role in their learning aencourage them to construct physical knowledge for the

Fig. 3. Comparison of understanding of dynamics before instruction in~a!noncalculus and~b! calculus-based introductory physics courses at Oregand Tufts.


e-

raeeatio

arthpree

oc

,gin

l oithT

ft?a

eur

Bte

e

e

eso-

metersig-

venhadpts.of

n anal

ory

tenory

achpre-om-on

994

avi-

s-alsol.

ame

en

ns.

selves from actual observations. These curricula have bdescribed in more detail elsewhere.4,7,18They make substantial use of the results of physics education research.4–15 Bothwere designed to introduce active learning into the labotory portion of a traditionally structured introductory coursBoth use the MBL force probe and motion detector to msure and display the force applied to an object and its mo~position, velocity, and/or acceleration! simultaneously inreal time.19 They use a guided discovery approach andintended for student groups of two to four. They supportpeer learning that is possible when data are immediatelysented in an understandable form. They engage studthrough a learning cycle which includes predictions, obsvations, and comparisons. In addition, they pay attentionstudent alternative understandings that have been dmented in the research literature.

The TST Motion and Forcecurriculum, released in 1992was designed to help improve conceptual understandinmechanics by substituting a small number of active learnlaboratories for traditional ones. TheRTP Mechanicscur-riculum, released in 1994, has the more ambitious goacompletely replacing the entire introductory laboratory wa sequenced, coherent set of active learning laboratories.origins of RTP Mechanicslie in TST Motion and ForceandWorkshop Physics.20

How well do students understand dynamics concepts acompleting theTST and RTP active learning laboratoriesDuring 1989–1991, the LAB students in the Oregon noncculus general physics course completed all of theTST Mo-tion and Forcelabs. In 1991, they were evaluated with thFMCE a number of times during the term, largely to measthe effectiveness ofInteractive Lecture Demonstrations. ~SeeSec. VI.! However, we were also able to evaluate the LAgroup’s conceptual understanding after they had completheTST Motion and Forcelabs. As can be seen in Fig. 3~a!,the pre-test results for the group of students in 1991 are vsimilar to results in other years. The same questions~with theorder rearranged! were asked of the LAB group after thstudents completed the twoTST kinematics and twoTSTdynamics laboratories, about five weeks after the pre-tDuring this period, the students also experienced aboutlecture of special kinematicsInteractive Lecture Demonstrations.

Fig. 4. Effect ofTST Motion and Forcelabs andInteractive Lecture Dem-onstrationson understanding of dynamics in 1991 Oregon noncalculus geral physics.


en

-.-n

eee-ntsr-tou-

ing

f

he

er

l-

e

d

ry

t.ne

The first two bars of Fig. 4 show the results for the sagroup of 72 LAB students before instruction and then aflectures and laboratories. As can be seen, there are verynificant improvements even though the post-test was giimmediately after the last laboratory, before the studentsmuch time to assimilate their knowledge of these conce~As shown in the last two bars of Fig. 4, more than 90%these LAB students were answering most questions iNewtonian manner by the end of the course. This additioimprovement was achieved through dynamicsInteractiveLecture Demonstrations, as described in Sec. VI.!

From 1992 to 1994, students in the Oregon introductlaboratory completed theRTP Mechanicslaboratories. Be-cause research, as shown in Fig. 3~a!, shows that the pre-tesresults vary little from year to year, no pre-test was givduring these years, but the FMCE was part of the laboratfinal examination. In Fig. 5~a!, the final results for the LABgroup from the noncalculus general physics course in eyear, 1992 through 1994, are compared to the averagetest results for 1989–91. These results should also be cpared to the 7% gain achieved by traditional instructionthese questions, shown in Fig. 2.~The additional improve-ment in the Coin Toss and Cart on Ramp questions in 1is attributable to a curricular change inRTP Mechanics,which placed more emphasis on motion caused by the grtational force.!

A much smaller LAB group of students in the calculubased general physics course at Oregon in 1992–94completed theRTP Mechanicslaboratories and the lab finaThese results are shown in Fig. 5~b!. While no pre-test wasgiven to these classes, the 1995 pre-test results for the s

-

Fig. 5. Effect of RTP Mechanicslabs on understanding of dynamics i1992–1994 Oregon~a! noncalculus and~b! calculus-based general physic


abn

uotna

nan

timtu

h

eso

ableof

1 a

npas-ure

ut-

ndis-ur-s.

crip-

ninghotforral

assn-

r ofind.pts

thforereofIn

eri-

n

andin-
course are included for comparison. This seems reason
since the pre-test data for calculus-based courses showFig. 3~b! are all very consistent with one another.

The students in the Spring~off-semester! introductorycalculus-based course at Tufts also did theRTP Mechanicslaboratories during the Springs of 1994 and 1995. The reson the FMCE for a combined group of 88 students from byears is shown in Fig. 6 before all instruction and on the fiexamination.

Figure 7 compares results on the FMCE after traditioinstruction in introductory noncalculus physics courses aafter experiencing theRTP Mechanicslabs in noncalculusand calculus-based courses. The significant improvemenconceptual understanding as a result of the labs is very slar in calculus and noncalculus courses and similar for sdents at Oregon and at Tufts.

VI. CONCEPTUAL UNDERSTANDING OFDYNAMICS BEFORE AND AFTER INTERACTIVELECTURE DEMONSTRATIONS

Despite considerable evidence that traditional approacare ineffective in teaching physics concepts,3–15most physicsstudents in this country continue to be taught in lecturoften in large lectures with more than 100 students. Al

Fig. 6. Effect of RTP Mechanicslabs on understanding of dynamics iSpring, 1994–1995 Tufts calculus-based introductory physics.

Fig. 7. Understanding of dynamics in noncalculus~NC! and calculus-based~C! courses at Oregon and Tufts which includedRTP Mechanicslabs com-pared to that with traditional instruction.


le,in

ltshl

ld

ini--

es

s,,

many high school and college physics programs are unto support hands-on laboratory work for large numbersstudents because they have only a few computers.

After several years of research, we formalized in 199procedure for theTools for Scientific Thinking InteractiveLecture Demonstrations~ILDs!, which engages students ithe learning process and, therefore, converts the usuallysive lecture environment to a more active one. The procedinvolves students recording individual predictions of the ocomes of simple experiments on a Prediction Sheet~which iscollected!, discussing their predictions with neighbors athen comparing their predictions to the actual results dplayed for the class with MBL tools. We have published fosequences of mechanicsILDs to enhance the learning of kinematics and dynamics, including Newton’s Three LawMore details on the procedure and design, as well as destions of these four ILD sequences, will be foundelsewhere.4–7,21,22

Here we report on assessments of conceptual leargains using the FMCE for introductory physics students wexperienced series ofILDs on kinematics and Newton’s Firsand Second Laws. In the Fall of 1991, as a substitutetraditional instruction, students in the noncalculus genephysics class at Oregon experiencedabout two full lecturesof ILDs on kinematics and dynamics. A similar set ofILDswas carried out in the noncalculus introductory physics clat Tufts, during Fall, 1994. At Tufts, all students were erolled in the laboratory, where they completedonly the twoTST Motion and Forcekinematicslaboratories.16 At bothOregon and Tufts, students were awarded a small numbepoints toward their final grades for attending and handingtheir Prediction Sheets, but their answers were not grade

Figure 8 compares student learning of dynamics concein traditional instruction to learning in identical courses wiILDs. As we mentioned in Sec. IV, the pre-test resultsOregon students in 1991 and Tufts students in 1994 wvery similar to those of the combined 1989–1990 groupOregon students, which we show in the first bar of Fig. 8.comparison, the last two bars show that the effect of expencingabout two full lecturesof ILDs is very substantial.

Fig. 8. Understanding of dynamics in non-calculus courses at OregonTufts with ILD-enhanced instruction compared to that with traditionalstruction.


sowia

Orup

rtnft,n

fina

er

ican

tioouinerosta

onav

fhao

thcotur

o

h of

.nts

fterut

an

in-e-

partar-re

4he-an-

hehelsohly

ring

erethe

sery

gen-10.al

dtive-

onere

outio

wingin-

VII. INSTRUCTIONAL AND ASSESSMENTHISTORIES

In order to illustrate the proper use of the FMCE in asseing the effectiveness of learning strategies and to give mdetail on the experimental protocol used in this research,present here a more detailed account of the timelines ofstruction and testing at Oregon and Tufts. Figure 9 showsassessment history for the noncalculus NOLAB group ofegon students, while Fig. 4 shows results for the LAB groThe bars labeled ‘‘Before Instruction’’~meaning before alldynamics instruction! show the results on the FMCE aftetwo traditional kinematics lectures.~In all of these studies aboth Oregon and Tufts, neither the evaluation questionsthe answers were returned or posted at any time until athe final examination.! After two more weeks of lecturesincluding several lectures on Newton’s Laws, the studeexperienced a total of about one lecture of kinematicsILDs.About a week later, after all lectures on dynamics wereished, dynamics questions from the FMCE were givenpart of the midterm examination. The bars labeled ‘‘AftTraditional1’’ in Fig. 9 show the effect ondynamicscon-ceptual understanding of enhancingkinematicsinstructionwith the kinematicsILDs. The NOLAB students improvedon the natural language questions by 14%, on the graphquestions by 24%, on the Coin Toss questions by 47%,on the Cart on Ramp questions by 35%.

These improvements after enhanced kinematics instrucand traditional dynamics instruction can be explained byprevious research on the learning hierarchy formed by kmatics and dynamics concepts. We have shown that imping student understanding of kinematics also improvesdent learning of dynamics,even if dynamics is taught intraditional manner.4,5

We have reported previously on substantial gains in cceptual understanding of kinematics by students who hexperienced our active learning laboratories.3 Our analysiswas based on averages of students’ responses on thevelocity and five acceleration questions on the FMCE. Wabout the effect ofILDs? Whereas at Oregon an averageonly 35% of noncalculus NOLAB students can answeracceleration questions and 70% the velocity questionsrectly after traditional instruction, 80% of these same sdents answer the acceleration questions and 90% answe

Fig. 9. Assessment history for the 1991 Oregon noncalculus NOLAB grshowing understanding of dynamics before, during, and after instrucwhich included kinematics and dynamicsILDs.


s-ree

n-n-.

orer

ts

-s

ald

nr-

v-u-

-e

ourt

fer--the

velocity questions correctly after a combination of the twkinematics sequences ofILDs. For noncalculus LAB stu-dents at Oregon and for Tufts noncalculus students, botwhom also completed the twoTST kinematics labs, thesepercentages rise to 90%–95% on both sets of questions

At the time of the midterm assessment, the LAB studehad also completed the fourTST Motion and Forcelabs. Theeven larger increases seen in the bar in Fig. 4 labeled ‘‘ATST Labs,’’ have already been mentioned in Sec. V. Abo70% of the LAB students are answering in a Newtonimanner.

About a week after the midterm examination, the 40 mutes of ILDs on Newton’s First and Second laws were prsented to the students. In the lecture following theILDs, thesame dynamics questions from the FMCE were asked asof a quiz—with the order of questions and choices reranged. The results on this quiz for the NOLAB group ashown by the bars labeled ‘‘After ILDs’’ in Fig. 9. Figureshows the results for the LAB students. The effect of tdynamicsILDs on the NOLAB group seems truly remarkable in that nearly 70% are now answering both natural lguage and graphical questions in a Newtonian wayafter onlyan additional 40 minutes of interactive presentation of tconcepts in lecture. Students are doing even better on tCoin Toss and Cart on Ramp questions. The LAB group ashows some additional improvement, so that now roug80% are answering the questions in a Newtonian way.~Sincesuch a large percentage of the LAB group was answethese questions correctly before the dynamicsILDs, we ex-pect that those who were still missing these questions wamong the weakest students in the class. The fact thatILDs still were effective in changing the view of dynamicfor approximately half of these weaker students seems vencouraging.!

The assessment history of students in the noncalculuseral physics course at Tufts in Fall, 1994 is shown in Fig.One difference from Oregon was that at Tufts all traditioninstruction in kinematics and dynamics was completedbe-fore any ILDs were presented. ~The timelines at Oregon anTufts were necessitated by our desire to assess the effecness of theILDs independently from traditional instruction.!The ‘‘Before Instruction’’ evaluation represents resultsthe FMCE given on the first day of class. The students w

pnFig. 10. Assessment history for the 1994 Tufts noncalculus course shounderstanding of dynamics before, during, and after instruction whichcluded twoTSTkinematics labs, and kinematics and dynamicsILDs.


n

e

re

utr

er

ebn

b

eeethth

gta

oCamm

c-lts

ltobats—

thndu

ee

th

len

cs

ues-rceentpts.

en-

tu-gonicheseinWed atew-tere ofec-

thisofby

stu-larlyntsysi-

sti-mp

werianeyAsein a

evi-

regonTheegonof theonsame

given traditional lectures and problems on kinematics adynamics. They also completed the first twoTST Motion andForce labs on kinematics. After all traditional instruction onmechanics~plus the kinematics labs!, the students wereevaluated again. Since the kinematics instruction washanced by the two labs, the data are labeled ‘‘Traditional1’’in Fig. 10.

During the next week, two 40-min sequences ofILDs onkinematics and Newton’s First and Second Laws were psented. The day after the second set ofILDs, the dynamicsquestions with rearranged choices were included as 45 o100 points on the second hour exam in the course. Thesults are very gratifying. In Fig. 10 the bars labeled ‘‘AftILDs’’ show a gain of almost 50% from 80 min ofILDinstruction, with almost 90% of the students answering qutions in a Newtonian way after instruction enhancedILDs. The total gain of over 75% from before instructioshould be compared to the 7%–10% gain we have seensulting from traditional instruction.

VIII. RETENTION OF CONCEPTUAL KNOWLEDGEGAINED FROM ACTIVE LEARNINGLABORATORIES AND ILDs

Retention of the Newtonian conceptual view seems tovery good for students who have completed theTSTor RTPlabs. Whenever questions were asked again up to six wafter instruction in dynamics had ended, the percentagstudents answering in a Newtonian way increased ratherdecreased. We attribute this increase to assimilation ofconcepts.

It might not be too surprising if the improved learninfrom the ILDs were more ephemeral. The research dahowever, seem to show that theILD-enhanced learning alsis persistent. As a test of retention, the Force Graph andon Ramp questions were included on the Oregon final exanation. The final was given about six weeks after the dynaics ILDs, during which time no additional dynamics instrution took place. The bars labeled ‘‘Final’’ show the resufor the NOLAB group in Fig. 9 and for the LAB group inFig. 4. As can be seen, assimilation apparently has resuin a 6% increase for the graphical questions and a mmodest increase for the Cart on Ramp questions. Thelabeled ‘‘Final’’ in Fig. 10 show the results on the Tuffinal, which was seven weeks after dynamics instructionincluding ILDs—had ended. There is a 7% increase ongraphical questions, a 23% increase on the Coin Toss, a36% increase on the Cart on Ramp questions, even thothere was noadditional relevant instruction. ~It should benoted that the post-test labeled ‘‘After ILDs’’ was given thnext day after the dynamicsILD, so that there was no timfor assimilation.!

IX. SUMMARY OF EVIDENCE FOR THE VALIDITYOF THE FMCE

Our observations that 70%–90% of students answerFMCE dynamics questions from a Newtonian perspectivethe end of the term after completing theTSTor RTP labora-tory curriculum and/or participating in theILDs, while lessthan 20% do so after traditional lecture instruction, hassome to question the validity of the test. Are the questiosignificant indicators of student understanding of dynami


d

n-

-

ofe-

s-y

re-

e

ksofene

,

rti--

edrers

ea

gh

eat

ds?

To some extent, we have already addressed important qtions that might be raised. Student answers to the FoGraph questions correlate with answers to the very differformat Force Sled questions which probe the same conceThe correlation holds both before and after traditional orhanced instruction.

To explore further the significance of the Newtonian sdent responses, we also included on the final exam at Oreand Tufts a new set of simple conceptual questions whhad not been asked previously. Figure 11 shows ten of thnew questions which are of a different format and setrather different contexts than the Force Graph questions.only consider students at Oregon and Tufts who answereleast seven of the eight Force Graph questions from a Ntonian point of view. The first number in parentheses afeach Newtonian answer in Fig. 11 indicates the percentagOregon students giving the Newtonian answer, while the sond number is the percentage of Tufts students givinganswer. The results were impressive, with nine out of tenthese questions answered from a Newtonian point of view80% or more Oregon students and 86% or more Tuftsdents. The results on these new questions were particugratifying, since previous work had shown us that studeoften do not generalize in ways that seem obvious to phcists.

After traditional or enhanced instruction, students statically answer the Coin Toss questions and the Cart on Raquestions in a non-Newtonian way even after they ansmost of the other questions on the FMCE in a Newtonmanner.4 Many of their answers seem to indicate that thassociate force with velocity rather than acceleration.shown in Sec. III, after traditional instruction only 5% of thstudents at Oregon answer the Coin Toss questionsNewtonian manner, while after the laboratories andILDs atOregon and after theILDs at Tufts, over 90% do.

The Coin Toss questions and analogs provide more

Fig. 11. Alternative assessment questions asked on the final exam at Oand Tufts which test understanding of dynamics in different contexts.first set of numbers in parentheses show the percent of the 1991 Ornoncalculus general physics students who had answered at least seveneight Force Graph questions in a Newtonian manner who also did sothese questions. The second set of numbers in parentheses shows thepercentages for the 1994 Tufts noncalculus students.


tiota

sigwt ohinger

nh

tortoinae

tio

t

swce

otoere

eseCE

ghsirein

iceer-dedsuf-m-ul-

’’use

stu-dentheyse

sweronscor-e wehedomom-or

ands to

ing.

heircanls.whoe al-ith-

ho

an-ly

ve-ionrcenlyop-cese

t

orceblecon-

m

dence that students who answer the Force Graph quesfrom a Newtonian point of view have made a fundamenbelief change. If we look again at the sample of studentOregon and Tufts who answered at least seven of the eForce Graph questions from a Newtonian point of view,find that 93% of these students also did so on the CarRamp questions. Figure 12 shows a coin toss analog wthey had not seen previously, a block sliding into a spriNinety-two percent of these same students again answfrom a Newtonian point of view.

As with all the questions on the FMCE, students who aswered correctly were also able to describe in words wthey picked the answers they did.5 Statistically one of the lasquestions to be answered in a Newtonian manner is the fon a cart rolling up a ramp as it reverses direction at the~Cart on Ramp question 9!. Students were asked to explahow they determined this force. The following are typicwritten explanations from students who answered this qution from a Newtonian point of view:

‘‘After the car is released the only net force acting on itis the x-component of its weight which has a net forcedown the ramp in the positive direction.’’‘‘When the car is at the top of the ramp, its velocity is0 for just an instant, but in the next instant it is movingdown the ramp, v22v15a pos number so it is accel.down. Also, gravity is always pulling down on the carno matter which way it is moving.’’‘‘The only two forces involved were gravity and fric-tion. At the top of the ramp the net force was down-ward because gravity is higher in magnitude than fric-tion ~unless the tires & the ramp were sticky!.’’

Typical student answers for those who answered as if moimplies force were:

‘‘At the highest point, the toy car’s force is switchingfrom one direction to another and there are no netforces acting upon it, so it is zero.’’‘‘Because at the one instant the car is at its highespoint it is no longer moving so the force is zero for thatone instant it is at rest5net force50.’’

The agreement between the multiple-choice and open anresponses is almost 100%. Such results give us confidenthe significance of student choices.

In summary, most students answer the Force GraphForce Sled questions as if they held a Newtonian pointview, and also are able to answer Coin Toss and coinanalog questions and other questions that they have nseen before. In addition, students’ written explanations ag

Fig. 12. Alternative coin toss analog questions asked on the final exaOregon in 1991 and Tufts in 1994.


nsl

atht

en

ch.ed

-y

cep

ls-

n

erin

orf

ssvere

with their choices on these multiple-choice questions. Thresults support the usefulness of the questions on the FMfor evaluating student understanding.

X. WHY IS THE FMCE INSTRUCTIONALLYUSEFUL?

The difficulties in convincing physics professors and hischool teachers to give up course time for testing, our deto make evaluation less subjective, and the effort involvedanalyzing large samples moved us to use multiple-choquestions on the FMCE. Although a more complete undstanding of student learning can be gained by an open-enquestioning process, the FMCE has allowed us to gatherficient data at many different institutions to counter the comon response that ‘‘my students do not have these difficties you describe.’’ Almost all answers, ‘‘right or wronghelp us to evaluate student views about dynamics. Becathe available choices in the questions were derived fromdents’ answers to free response questions and from stuinterviews, students almost always find an answer that tare satisfied with. Guessing correctly is very difficult becaumany of the questions require students to choose an anfrom up to nine choices. The correlations among questihave been examined and individual questions have beenrelated with more open-ended student answers. Becausare able to identify statistically most student views from tpattern of answers and because there are very few rananswers, we are also able to identify students with less cmon beliefs about motion and follow up with interviewsopen-ended questions. The use of an easily administeredrobust multiple-choice test has also allowed us and othertrack changes in student views of dynamics4,5 and to separatethe effects of various curricular changes on student learn

Student answers correlate well~above 90%! with writtenshort answers in which students explain the reason for tchoices, and almost all students pick choices that weassociate with a relatively small number of student mode5

Testing with smaller student samples shows that thosecan pick the correct graph under these circumstances armost equally successful at drawing the graph correctly wout being presented with choices.

The great majority of students at Oregon and Tufts wcompleted the MBL laboratory curricula and/orILDs an-swered the FMCE dynamics questions in a Newtonian mner. It would be a mistake to imagine that students uniformapply a consistent model~at least from the point of view ofmost physicists! to all ‘‘manners’’ of motion. Many studentsconsider speeding up, slowing down, moving at constantlocity, and standing still to be independent states of motthat do not require a consistent relationship between foand acceleration or velocity. Numbers of students commorequire an object slowing down to have a constant forceposing the motion while requiring an ever increasing forfor an object which is speeding up. The dynamics of thechanging student views is described elsewhere.5 The fact thatmost students are using ‘‘models’’~even if they are incorrecor are applied in very limited circumstances! is a good be-ginning for instruction.

XI. CONCLUSIONS

We have developed and refined the research-based Fand Motion Conceptual Evaluation so that it is now a reliameans of assessing student understanding of mechanics

at


tutenllt

blml

hitaindestenibssr

n

isverd-lee,ei-

alsoandrenislly,tionca-e

cepts, and it is easily administered to large groups of sdents. Our studies of conceptual understanding using thisshow that introductory physics students do not commounderstand kinematics and dynamics concepts as a resuthorough traditional instruction. Since the choices availato students on the FMCE allow us to distinguish among comon student views about dynamics,5 this test has been usefufor guiding the development of instructional strategies. Tresearch and that of others, along with the developmenuser friendly microcomputer-based laboratory tools, havelowed us to extend our computer supported, active learnlaboratory curricula to dynamics and have allowed us tovelop a strategy for more active learning of these conceptlectures usingILDs. Assessments using the FMCE indicathat student understanding of dynamics concepts is sigcantly improved when these learning strategies are sututed for traditional ones. In a future paper we will discuassessment of learning of Newton’s Third Law in laboratoand lecture using the FMCE.


-st

yof

e-

sofl-g-in

fi-ti-sy

ACKNOWLEDGMENTS

We are especially grateful to Priscilla Laws of DickinsoCollege, co-author ofRealTime Physics, for her continuingcollaboration which has contributed significantly to thwork. The laboratory and lecture curricula which we hadeveloped would not have been possible without the haware and software development work of Stephen BeardsLars Travers, Ronald Budworth, and David Vernier. Wthank the faculty at the University of Oregon and Tufts Unversity for assisting with our student assessments. Wethank the students for participating in these assessmentsin the testing of our laboratory and lecture curricula. We agrateful to Lillian McDermott and the Physics EducatioGroup at the University of Washington for reviewing thpaper and offering many constructive suggestions. Finathis work was supported by the National Science Foundaand by the Fund for Improvement of Post-secondary Edution ~FIPSE! of the U.S. Department of Education. SeRef. 2.

APPENDIX A: COMPLETE FORCE AND MOTION CONCEPTUAL EVALUATION


347 347Am. J. Phys., Vol. 66, No. 4, April 1998 R. K. Thornton and D. R. Sokoloff

n,h

hin

ndE

en-for

ific

ng

deeaanci-

rce

ch-iron-s

rystra-

s,’’

nts

is

ge

en-

1Revisions and updates of the Force and Motion Conceptual Evaluatiowell as conceptual evaluations under development in other areas of pics, are available from the Center for Science and Mathematics TeacTufts University, 4 Colby Street, Medford, MA 02155, or from theWork-shop Physics On-line Instructor Resource Guideon the World Wide Webat http://physics.dickinson.edu.

2This work was supported in part by the National Science Foundation uGrant No. USE-9150589, ‘‘Student Oriented Science,’’ Grant No. US9153725, ‘‘The Workshop Physics Laboratory Featuring Tools for Scitific Thinking,’’ and Grant No. TPE-8751481, ‘‘Tools for Scientific Thinking: MBL for Teaching Science Teachers,’’ and by the FundImprovement of Post-secondary Education~FIPSE! of the U.S. Depart-ment of Education under Grant No. G008642149, ‘‘Tools for ScientThinking,’’ and number P116B90692, ‘‘Interactive Physics.’’

3R. K. Thornton and D. R. Sokoloff, ‘‘Learning motion concepts usireal-time, microcomputer-based laboratory tools,’’ Am. J. Phys.58, 858–867 ~1990!.

4R. K. Thornton, ‘‘Using large-scale classroom research to study stuconceptual learning in mechanics and to develop new approaches to ling,’’ in Microcomputer-Based Labs: Educational Research and Stdards, edited by Robert F. Tinker,Series F, Computer and Systems Sences156, 89–114~Springer-Verlag, Berlin, 1996!.

5R. K. Thornton, ‘‘Conceptual dynamics: Changing student views of fo


asys-g,

er--

ntrn--

and motion,’’ inThinking Physics for Teaching, edited by C. Tarsitani, C.Bernardini, and M. Vincentini~Plenum, London, 1995!.

6R. K. Thornton, ‘‘Changing the physics teaching laboratory: Using tenology and new approaches to learning to create an experiential envment for learning physics concepts,’’Proceedings of the EurophysicStudy Conference, The Role of Experiment in Physics Education, edited bySeta Oblak and Nada Razpet~Ljubljana, Slovenia, 1993!.

7Ronald K. Thornton, ‘‘Learning physics concepts in the introductocourse: Microcomputer-based Labs and Interactive Lecture Demontions,’’ in Proc. Conf. on the Intro. Physics Course, edited by Jack Wilson~Wiley, New York, 1997!, pp. 69–85.

8A. B. Arons, A Guide to Introductory Physics Teaching~Wiley, NewYork, 1990!.

9L. C. McDermott, ‘‘Research on conceptual understanding in mechanicPhys. Today37, 24–32~1984!.

10L. C. McDermott, ‘‘Guest comment: How we teach and how studelearn—a mismatch?,’’ Am. J. Phys.61, 295–298~1993!.

11L. C. McDermott, ‘‘Millikan lecture 1990: What we teach and whatlearned—closing the gap,’’ Am. J. Phys.59, 301–315~1991!.

12J. A. Halloun and D. Hestenes, ‘‘The initial knowledge state of collephysics students,’’ Am. J. Phys.53, 1043–1056~1985!.

13D. Hestenes, M. Wells, and G. Schwackhammer, ‘‘Force concept invtory,’’ Phys. Teach.30~3!, 141–158~1992!.


foAt

fo

rs’

ein

Unn

arey,

ay

reach.

sultsale

14L. C. McDermott, P. Shaffer, and M. D. Somers, ‘‘Research as a guideteaching introductory mechanics: An illustration in the context of thewood’s machine,’’ Am. J. Phys.62, 46–55~1994!.

15M. Wells, D. Hestenes, and G. Schwackhammer, ‘‘A modeling methodhigh school physics instruction,’’ Am. J. Phys.63, 606–619~1995!.

16Ronald K. Thornton and David R. Sokoloff,Tools for ScientificThinking—Motion and Force Laboratory Curriculum and TeacheGuide ~Vernier Software, Portland, OR, 1992!, 2nd ed.

17David R. Sokoloff, Priscilla W. Laws, and Ronald K. Thornton,RealTimePhysics: Mechanics V. 1.40~Vernier Software, Portland, OR, 1994!.

18Ronald K. Thornton and David R. Sokoloff, ‘‘RealTime Physics: ActivLearning Laboratory,’’ inThe Changing Role of Physics DepartmentsModern Universities: Proceedings of the International Conference ondergraduate Physics Education, edited by E. F. Redish and J. S. Rigde


r-

r

-

~American Institute of Physics, Woodbury, NY, 1997!, pp. 1101–1118.19The MBL probes, interface, software, and curricula discussed here

available from Vernier Software, 8565 Beaverton-Hillsdale HighwaPortland, OR 97225-2429.

20P. W. Laws, ‘‘Calculus-based physics without lectures,’’ Phys. Tod44~12!, 24–31~1991!.

21David R. Sokoloff and Ronald K. Thornton, ‘‘Using interactive lectudemonstrations to create an active learning environment,’’ Phys. Te35~6!, 340-47 1997~in press!.

22Copies of the complete mechanicsInteractive Lecture Demonstrationpackage, including teacher instructions and student Prediction and Resheets are available from Vernier Software, 8565 Beaverton–HillsdHighway, Portland, OR 97225-2429.


assessing student learning of newton’s laws: the force and motion...

Documents