effect of explicit problem solving instruction on high school students' problem-solving...

Effect of Explicit Problem Solving Instruction on High School Students’Problem-Solving Performance and Conceptual Understanding of Physics

Douglas Huffman

Department of Curriculum and Instruction, 150 Peik Hall, University of Minnesota,159 Pillsbury Drive SE, Minneapolis, Minnesota 55455

Received 6 September 1995; revised 4 March 1997; accepted 17 March 1997

Abstract: In this study a two-sample, pre/posttest, quasi-experimental design was used to investigatethe effect of explicit problem-solving instruction on high school students’ conceptual understanding ofphysics. Eight physics classes, with a total of 145 students, were randomly assigned to either a treatmentor comparison group. The four treatment classes were taught how to use an explicit problem-solving strat-egy, while the four comparison classes were taught how to use a textbook problem-solving strategy. Stu-dents’ problem-solving performance and conceptual understanding were assessed both before and after in-struction. The results indicated that the explicit strategy improved the quality and completeness of students’physics representations more than the textbook strategy, but there was no difference between the two strate-gies on match of equations with representations, organization, or mathematical execution. In terms of con-ceptual understanding, there was no overall difference between the two groups; however, there was a signif-icant interaction between the sex of the students and group. The explicit strategy appeared to benefit femalestudents, while the textbook strategy appeared to benefit male students. The implications of these results forphysics instruction are discussed. © 1997 John Wiley & Sons, Inc. J Res Sci Teach 34: 551–570, 1997.

Rationale for the Study

In a review of research on physics problem solving, Maloney (1994) stated that althoughthe research to date in physics problem solving is informative, one of the key issues in need ofinvestigation is the role of problem solving in learning physics concepts. According to Maloney,this lack of research is surprising given that many physics instructors unquestionably assumethat problem solving is a vehicle for learning physics. This assumption clearly has not been wellresearched; one of the purposes of this study was to address the lack of research on the role ofproblem solving in learning physics concepts.

We know from previous research that preparing students to become effective problem solversand helping students to understand concepts are both difficult goals to achieve. Numerous studiesof problem solving indicate that even after instruction, many physics students still have difficultysolving problems and continue to use novice problem-solving techniques rather than more ad-vanced problem-solving techniques (Maloney, 1994). Likewise, numerous studies of conceptualunderstanding indicate that even after instruction, many physics students continue to have thesame alternative conceptions they had before instruction (Wandersee, Mintzes, & Novak, 1994).

JOURNAL OF RESEARCH IN SCIENCE TEACHING VOL. 34, NO. 6, PP. 551–570 (1997)

© 1997 John Wiley & Sons, Inc. CCC 0022-4308/97/060551-20

One instructional method that has been used to address both problem-solving performanceand conceptual understanding is explicit problem-solving instruction. Explicit problem solvingis instruction that directly teaches students how to use more advanced techniques for solvingproblems. Textbook problem solving, on the other hand, often provides only a general outlineof steps to follow for solving problems and does not usually provide explicit instruction on howto apply those steps. Textbook problem solving also tends to emphasize primarily the quantita-tive aspects of problem solving, while explicit problem solving tends to emphasize both thequalitative and quantitative aspects of problem solving. This emphasis on the qualitative aspectsof problem solving may help students not only to improve their problem-solving performance,but also to better understand the concepts and principles of physics. Therefore, to test the effectof explicit problem-solving instruction, this study was designed to determine if high school stu-dents who were taught how to use an explicit problem-solving strategy exhibited more im-provement in problem-solving performance and more improvement in conceptual understand-ing of physics than students who were taught how to use a textbook problem-solving strategy.

Review of the Literature

This study is based on a large body of research on physics problem solving. Numerous stud-ies have documented the problem-solving techniques of novices and have compared them to thetechniques that experts use when they solve problems (Chi, Feltovich, & Glasner, 1980; Larkin,McDermott, Simon, & Simon, 1980; Larkin & Reif, 1979). This research identified importantdifferences between the ways experts and novices solve problems, which in turn have been usedto help teach students how to solve problems. For example, experts tend to represent a problemqualitatively in terms of fundamental physics concepts before they translate the problem intomathematical equations, while novices often tend to plunge into mathematical manipulations ofequations with little if any qualitative description of the problem (Larkin et al., 1980). Expertsalso tend to engage in more planning than novice problem solvers. Experts tend to carefully con-sider alternatives and to develop plans of attack before manipulating equations, while novicestend to immediately work from first impressions without developing plans (Larkin, 1981; Larkinet al., 1980; Reif & Heller, 1982). Experts also tend to carry out solutions to problems differ-ently from novices. Among other things, experts tend to solve problems in a more logical, or-ganized manner than novices (Woods, 1989). When experts know the solution to a problem,they tend to work forward using the givens in the problem to select the appropriate equationsand calculate the desired unknown variable (Larkin et al., 1980). When experts are faced witha problem for which they do not immediately know the solution, they tend to solve equationsalgebraically by working backward, starting with the unknown variable to be found, substitut-ing the givens into this equation, and then solving for the unknown variable (Larkin, 1983b).

Research on the differences between experts and novices has led in turn to the developmentof explicit problem-solving strategies designed to teach students how to use more advancedtechniques. Numerous studies have reported that explicit problem-solving instruction can helpimprove students’ problem-solving performance more than traditional or textbook problem-solv-ing instruction (Mestre, Dufresne, Gerace, Hardiman, & Touger, 1993; Heller, Keith, & Ander-son, 1992; Van Heuvelen, 1990; Wright & Williams, 1986; Heller & Reif, 1984; Larkin & Reif,1979, Reif, Larkin, & Brackett, 1976). Each of these studies measured slightly different aspectsof problem-solving performance, but in general, students who learned the explicit problem-solv-ing strategies exhibited more advanced problem-solving performance, including better qualita-tive descriptions of problems, more extensive planning, and more complete solutions.

Teaching students how to solve physics problems is only one of the many goals of physics

552 HUFFMAN

instruction. Another important goal of physics instruction is to help students develop an under-standing of the concepts and principles of physics. It is clear that beginning physics studentshave great difficulty learning physics concepts. Numerous studies have documented the alter-native conceptions held by beginning physics students; in fact, there have been over 300 stud-ies devoted to the conceptual understanding of mechanics alone (Wandersee et al., 1994). Muchof the work on students’ conceptual understanding has focused on identifying and documentingthe extent to which students’ understanding of mechanics deviates from accepted scientific un-derstanding. Research indicates that alternative conceptions of mechanics are very prevalentamong high school and college students, and that they are highly resistant to change (diSessa,1982; Clement, 1982; Minstrell, 1982; McClosky, 1983; McDermott, 1984; Gunstone, 1987).

One technique that has been used to address both problem solving and conceptual under-standing is explicit problem solving. Explicit problem solving tends to emphasize both the qual-itative and quantitative aspects of problem solving, and it is this dual emphasis which may helpstudents not only to improve their problem-solving performance, but also to better understandconcepts. Textbook problem solving, on the other hand, tends to emphasize primarily the quan-titative aspects of problem solving, and some physics educators claim that an emphasis on thequantitative aspects of problem solving may actually obscure students’ understanding of thephysics (Hewitt, 1992).

Classroom-based experimental studies that focus specifically on the effect of problem-solv-ing instruction on students’ conceptual understanding of physics are relatively rare in the prob-lem-solving literature. The majority of research related to problem solving and conceptual un-derstanding focuses on the relationship of conceptual understanding on problem-solvingperformance, rather than vice versa. In one of the few studies on the effect of problem solvingon students’ physics knowledge, Heller et al. (1992) reported that during the group problem-solving process of justifying statements, clarifying ideas, and elaborating on explanations, stu-dents appeared to be deepening their understanding of physics concepts and principles. How-ever, this claim has not been formally tested.

There is also a lack of research on the differences between male and female problem-solv-ing performance and conceptual understanding. In a study with introductory university physicsstudents, Heller and Hollabaugh (1992) investigated the impact of homogeneous and heteroge-neous cooperative group structures on physics problem-solving performance. They reported thatthe best problem-solving performance was achieved with cooperative groups of three studentsthat were composed of either all male students or 1 male and 2 female students. It was observedthat male students tend to dominate the group problem-solving process, and this domination didnot necessarily lead to better group problem-solving performance. When male students wereplaced in groups with a female majority, the groups tended to demonstrated better problem-solv-ing performance. Heller and Lin (1992) conducted one of the only analyses of problem solvingand conceptual understanding by sex in physics. They found that male and female introductoryphysics students performed equally well on physics problems after students were taught how touse an explicit problem-solving strategy in a cooperative group environment. In terms of con-ceptual understanding, they reported that among those students who had taken high schoolphysics, the male students scored significantly higher than female students at the beginning ofthe course, but by the end of the course there were no significant differences between male andfemale scores on a multiple-choice test of conceptual understanding. It was reported that a cog-nitive apprenticeship instructional approach using cooperative group instruction, and an explic-it problem-solving strategy appeared to help close the gap between male and female physics stu-dents. In both of these studies using the explicit problem-solving strategy (Heller & Hollabaugh,1992; Heller & Lin, 1992), it was reported that there appeared to be differences in male and fe-

EFFECT OF EXPLICIT PROBLEM SOLVING 553

male students’ willingness to use the explicit problem-solving strategy, with a larger proportionof women than men adopting the explicit strategy.

All in all, the research on physics problem solving and conceptual understanding has im-portant implications regarding the design of the present study. The research on expert and noviceproblem solving indicates there are distinct differences in the procedures novices and expertsuse to solve problems. This research in turn has led to the development of instructional meth-ods to help students develop more advanced problem-solving techniques. The research on prob-lem solving indicates that instruction can help students to use more expertlike techniques; how-ever, there is a noticeable lack of classroom-based research on the role of problem-solvinginstruction in students’ conceptual understanding of physics.

Given the need for more instructional studies of problem solving, the lack of research onthe effect of physics problem solving on students’ understanding of concepts, and the hypothe-sis that explicit problem solving may actually enhance students’ conceptual understanding ofphysics, this study was designed to test the effect of explicit problem solving on high schoolstudents’ conceptual understanding of physics. Specifically, this study was designed to deter-mine if students who were taught how to use an explicit problem-solving strategy exhibitedmore improvement in problem-solving performance and conceptual understanding of physicsthan students who were taught how to use a textbook problem-solving strategy.

Method

Research Design

This study was conducted during an 18-week-long semester at a large suburban high schoolin the midwest. In this study, a two-sample, pretest/posttest, quasi-experimental design was usedto determine whether students who were taught how to use an explicit problem-solving strate-gy exhibited more improvement in problem-solving performance and conceptual understandingof physics than students who were taught how to use a textbook problem-solving strategy.

This study included two phases: problem-solving instructional and conceptual instruction.During the problem-solving instructional phase, eight physics classes, with a total of 145 stu-dents, were randomly assigned to either the treatment or comparison group. For each of the twoteachers who participated in this study, two of their classes were randomly assigned to the treat-ment group and two of their classes were randomly assigned to the comparison group. The fourtreatment classes were taught how to use an explicit problem-solving strategy, while the fourcomparison classes were taught how to use a textbook problem-solving strategy. Throughout theproblem-solving phase, students practiced solving problems using their respective strategies inweekly group problem-solving sessions using cooperative learning as defined by Johnson, John-son, and Holubec (1986).

The background characteristics of the students were analyzed to establish the quasi-equiva-lence of the two groups. The results of this analysis indicated that there were no significant dif-ferences between the background characteristics of the two groups. The students were similarin terms of their grade point averages, science and mathematics course backgrounds, and in-tention to enroll in college.

After the students learned how to use their respective problem-solving strategies, Phase twoof the study began. During this phase, students in both groups were taught an identical unit onNewton’s laws. Both groups continued to solve problems in weekly cooperative group problem-solving sessions using their respective strategies. At the end of Phase two, the conceptual un-derstanding and problem-solving performance of students who used the explicit strategy wascompared to that of students who used the textbook strategy.

554 HUFFMAN

Description of Problem-Solving Strategies

The textbook problem-solving method used in this study included two components: a five-step procedure for solving problems and the use of traditional one- or two-step physics prob-lems found in most physics textbooks. The textbook problem-solving methods were designedto correspond as closely as possible to the type of problem-solving methods used in most highschool physics textbooks. In general, textbook problem solving is characterized by a multistepprocedure used to solve one- or two-step physics problems. The strategy used in this study in-cluded the following five steps: (a) Draw a sketch; (b) define known and unknown quantities;(c) select equations; (d) solve equations; and (e) check the answer.

According to the textbook strategy, the first step in solving a physics problem is to draw asketch of the problem situation. This sketch usually includes a simple drawing of all relevantobjects and interactions. The second step is to define the variables by identifying both the knownand unknown quantities in the problem. The third step is to select the equations. Usually thisinvolves selecting a mathematical relationship that contains the unknown quantity to be foundand any other relationships that contain necessary unknown quantities. The fourth step is tosolve equations by substituting the values that are given in the problem into the mathematicalrelationships and solving for the unknown. The final step is check the answer; this is usuallydone by substituting the answer into a different equation to verify the accuracy of the answer.

The students in the explicit classes were taught how to solve physics problems using theexplicit problem-solving strategy described by Heller et al. (1992). The method includes a de-tailed five-step procedure to solve real-world, context-rich physics problems rather than simpletextbook physics problems. The steps of the strategy were influenced by the work of Reif andHeller (1982) and Larkin (1983b) in physics problem solving, and Schoenfeld (1985) in math-ematical problem solving. The five steps of the strategy are: (a) focus the problem; (b) describethe physics; (c) plan the solution; (d) execute the plan; and (e) evaluate the solution. An exam-ple of how to solve a problem using the explicit strategy is included in Figure 1.

The first step of the explicit strategy is to focus the problem. This is done by translating thewritten words into a visual description of the problem situation. The description includes asketch of the problem situation, the given information, a simple question about what one wantsto find out, and a general approach that can be used to solve the problem. The second step is todescribe the physics. In this step, the sketch from the first step is translated into a simplifiedphysics description. This step includes three parts: a physics diagram, a definition of variablesincluding a target variable, and selection of quantitative relationships. In the example solution,the physics diagram is a simplified version of the sketch, drawn on a coordinate axis with allimportant variables defined below the diagram at each time interval. The target variable is thedistance one needs to find to answer the question. Finally, the quantitative relationships are thephysics principles or mathematical relationships that can be used to solve the problem. The thirdstep is to plan the solution. In this step, the physics description is translated into specific math-ematical equations which are used to solve the problem. This step includes three parts: con-structing specific equations, checking for sufficiency, and outlining the math solution. The con-struction of specific equations always begins with an equation containing the target variable, andinvolves the successive combination of equations in which unknown variables are related toknown variables. In the example solution, the plan begins with an equation containing the dis-tance to be found. This equation contains one additional unknown; therefore, a second equationis constructed that contains the second unknown variable. The check for sufficiency involves acomparison of the number of equations to the number of unknowns. A mathematical solution ispossible when one has as many equations as unknowns. Finally, the math solution is outlinedto provide a bridge to the actually mathematical execution. In the example solution, the outline


starts with the second equation and progresses backward to the target variable. The fourth stepis to execute the plan. In this step, the equations are combined algebraically according to theplan, to produce an equation with a single unknown target variable. The units of each term inthis equation are checked to make sure they are correct, and are converted if necessary. Final-ly, the known quantities are inserted into the equation to calculate the value of the target vari-

556 HUFFMAN

Figure 1. Example solution using explicit problem-solving strategy.

able. The final step is to evaluate the solution. In this step, the solution is checked to ensure thatit is properly stated, reasonable, and complete. In the example solution, the units are checked toconfirm that they are in units of distance, the magnitude of the answer is checked to ensure thatit is reasonable, and the answer is checked to confirm that the original question has indeed beenanswered.

Although on the surface the explicit strategy appears to be quite similar to the textbookstrategy, the explicit strategy provides a much more detailed series of instructions for studentsto follow, including bridging steps that help students move from one step to the next. The ex-plicit strategy can be viewed as a systematic series of translations, in which each step requiresthe student to translate the problem into increasingly more abstract and mathematical represen-tations of the problem. In contrast, the textbook strategy merely provides a general frameworkfor solving a problem and provides fewer specific instructions on how to perform each step andhow to make the translation from one step to the next.

Another component of the explicit problem-solving strategy is the use of context-rich, real-world problems. Table 1 includes an example of a textbook problem and a context-rich prob-lem. Context-rich problems are real-world problem situations that tend to have the followingcharacteristics: (a) real-world contexts, (b) problem statements that do not explicitly identifywhat one needs to find, (c) extra information that may not be necessary for solving the prob-lem, and (d) a lack of explicitness that requires the student to make reasonable assumptions tosolve the problem (Heller et al., 1992). They tend to be much more complex than traditionaltextbook problems, and as a result, are much more difficult to solve.

Context-rich problems were used because the complexity of the problem played an im-portant role in promoting the use of the strategy. When students were asked to solve tradition-al one- or two-step textbook problems, they would often resort to their novice problem-solvingstrategies, because many of the textbook problems could be successfully solved with novicetechniques. However, when students were faced with context-rich problems, their novice prob-lem-solving techniques were not sufficient to solve the problem, and hence the students had aneed to learn more expertlike techniques (Heller & Hollabaugh, 1992). For these reasons, thestudents in the explicit group solved context-rich problems rather than the one- or two-step prob-lems found in most physics textbooks.


Table 1Problem-solving pretest and posttest

Runner problem: Ann and Bill are practicing for their next competition race. Ann runs at a constantspeed of 3 m/s down a long, straight road. After 5 min, Bill starts from the same place and runs at afaster constant speed of 4 m/s. How far from the starting point will the runners be when Bill catches upto Ann?

Asteroid problem: It has been proposed that dinosaurs and many other organisms became extinct 65 mil-lion years ago because Earth was struck by a large asteroid. The idea is that dust from the impact waslofted into the upper atmosphere all around the globe, where it lingered for at least several months andblocked the sunlight from reaching Earth’s surface. On the dark and cold Earth that temporarily resulted,many forms of life then became extinct. Available evidence suggests that about 20% of the asteroid’smass ended up as dust spread uniformly over the earth after settling out of the upper atmosphere. About0.020 g/cm2 of dust, which is chemically different from the earth’s rock, covered the earth’s surface.Typical asteroids have a density of about 2.0 g/cm3. It has been suggested that such an asteroid collisionis likely to happen again, perhaps causing the extinction of the current dominant life-form on earth (us).As we scan the space for this danger, how large an ateroid should we be watching for?

Instruments

Problem-solving performance was assessed before and after instruction through the use ofboth a traditional textbook problem and a context-rich problem. The problems used in this studywere developed by Heller and Hollabaugh (1992) as part of their study on cooperative groupproblem solving. The treatment group had more practice with context-rich problems during thecourse of the study, while the comparison group had more practice with textbooks; thus, bothtypes of problems were included on the problem-solving test in an attempt to equalize the ad-vantage each group may have had on the test. The two problems are presented in Table 1. Theywere administered in two consecutive days during the first and last weeks of the study. Becauseof the time-consuming nature of solving context-rich physics problems, students were given afull 50 min to solve each problem.

Scoring Written Problem Solutions. The quality of the solutions was judged using the char-acteristics of expertlike problem solving identified in the problem-solving literature (Larkin etal., 1980; Woods, 1989). These characteristics included detailed qualitative descriptions of prob-lems; mathematical solutions that match the qualitative descriptions; logical, well-organized so-lutions; and correct mathematical executions. The scoring rubric used in this study was designedto measure these general characteristics of expertlike problem solving and is presented in Table2. Tests were originally scored using the 15-point rubric in Table 2. To make comparisons moreeasily between the five different characteristics that were scored, each of the five subscores wereequally weighted by translating them to a 5-point scale for a total maximum score of 25 points.The interrater reliability of the problem scoring was .95.

The Force Concept Inventory. To measure changes in students’ conceptual understandingof Newton’s laws, two different written tests were administered both before and after instruc-tion: (a) a multiple-choice conceptual test, and (b) an open-ended test on Newton’s laws. Themultiple-choice test used in this study was the Force Concept Inventory (Hestenes, Wells, &Swackhamer, 1992). This test is a nationally recognized, 29-question, multiple-choice test thatis designed to measure introductory physics students’ conceptual understanding of selected as-pects of mechanics. The test is designed in such a way that each question includes one New-tonian answer and several non-Newtonian commonsense alternatives that tend to be believed bymany introductory physics students. Each of the questions on this test was worth one point. Inthis study, the reliability of the results as measured by Cronbach’s alpha was .73 on the posttest.

Newton’s Laws Test. To provide a more in-depth measure of students’ understanding thancould be provided with a multiple-choice test, an open-ended test was also used in this study.Three open-ended questions that focused respectively on the three Newton’s laws were admin-istered both before and after instruction. The Newton’s Laws Test included open-ended ques-tions developed by Heller and Hollabaugh (1992) as part of a study of cooperative group prob-lem solving. The Newton’s Laws Test was designed to reveal students’ understanding ofNewton’s laws by having students respond in writing to open-ended situations. Responses to theopen-ended questions were analyzed in two steps. The responses were first classified into logi-cal categories, and then each category was assigned a ranked score. The interrater reliability ofthe scoring was .95. Follow-up interviews with a sample of students were also conducted to con-firm students’ written responses.

558 HUFFMAN

The first open-ended question involved a ball on a table and was designed to focus primarilyon students’ understanding of the first law. In this question, a ball sitting on a table is given aquick push so that the ball rolls off the end of a table (Figure 2). Students were asked to de-scribe the forces on the ball at four different times: before the ball was pushed, while the ballwas being pushed, while the ball was rolling across table, and when the ball was in the air.

An understanding of the first law was suggested by a response indicating that the forces onthe ball are balanced when the ball is at rest, and when the ball is rolling across the table at aconstant velocity. Responses were therefore classified into one of two categories: balanced orunbalanced forces on the ball. Points were assigned according to the following criteria: 2 points5 balanced forces on a ball at both rest and at a constant velocity; 1 point 5 balanced forceson a ball at either rest or a constant velocity; and 0 points 5 unbalanced forces on a ball at restand at a constant velocity.

The second open-ended question was the cart and block question, designed to focus pri-marily on the second law. The purpose of this question was to probe students’ understanding ofthe relationship between force, mass, velocity, and acceleration. In this question, a mining cartwas pulled along a track by a large block that hung over the edge of a deep canyon (Figure 3).Students were asked to describe what would happen to the velocity and acceleration of the cartif the block were suddenly made much smaller than the cart.


Table 2Problem-solving scoring rubric

1. Quality of physics representationIs problem translated into appropriately?

0 5 Nothing written1 5 Problem not translated into appropriate physics2 5 Problem is translated into appropriate physics

2. Completeness of physics representationIs the physics representation complete?

0 5 Nothing written1 5 Representation is barely there (i.e., most variables missing)2 5 Representation is incomplete (i.e., picture and some variables defined)3 5 Representation is complete

3. Match of equationsAre the mathematical equations consistent with representation?

0 5 Nothing written1 5 Equations do not match representation2 5 Equations match representation

4. Organized progressionDoes the solution appear to be organized?

0 5 Nothing written1 5 Unorganized progression (i.e., haphazard manipulation of equations)2 5 Incomplete organized progression3 5 Complete means–end organized progression4 5 Complete forward solution (i.e., from general principles to answer)

5. Mathematical executionAside from minor mistakes, is the mathematics correct?

0 5 Nothing written1 5 When obstacle is encountered, inappropriate math is used2 5 When an obstacle is encountered, execution is terminated3 5 Executes correctly and completely, but uses numerical manipulations4 5 Executes correctly and completely (or with only minor mistakes)

An understanding of the second law was suggested by a response that if the mass of theblock were made much smaller, the acceleration of the cart would decrease proportionally, whilethe velocity of the cart would continue to increase. Students frequently failed to discriminatebetween velocity and acceleration, and often indicated that velocity is proportional to forcerather than acceleration. Responses to this question were therefore classified into three differentcategories according to the relationship between acceleration, velocity, and force: (a) accelera-tion is proportional to force, and velocity is not proportional to force; (b) acceleration and ve-locity are both proportional to force; and (c) acceleration is not proportional to force. Pointswere then assigned according to the following criteria: 2 points 5 acceleration is proportionalto force, and velocity is not proportional to force; 1 point 5 acceleration is proportional to force,but velocity is also proportional to force; and 0 points 5 acceleration is not proportional to force.

560 HUFFMAN

Figure 2. Ball on table question.

Figure 3. Cart and block question.

The third open-ended question was the bug and car question, designed to focus primarilyon the third law. In this question, a car traveling at 55 mph collides with a bug (Figure 4). Thestudents were asked to draw all of the forces acting on the bug and the car at the moment of im-pact, and to describe each force in words. An understanding of the third law was suggested bya response indicating that the force of the car on the bug is equal in magnitude to the force ofthe bug on the car. Students frequently wrote that the larger object exerts a larger force. Re-sponses to this question were classified into one of two categories: (a) the force of the car onthe bug is equal to the force of the bug on the car; or (b) the force of the car on the bug is notequal to the force of the bug on the car. A portion of the responses did not include enough in-formation to determine the forces on the bug and the car. These responses were classified as un-able to be coded. Scores were then assigned according to the following criteria: 1 point 5 theforce of the car on the bug is equal to the force of the bug on the car; and 0 points 5 the forceof the car on the bug is not equal to the force of the bug on the car.

Results

To determine whether students who were taught the explicit problem-solving strategy ex-hibit more improvement in problem-solving performance than students who were taught how touse a textbook problem-solving strategy, gain scores on both the asteroid and runner problemswere compared. Because the problem-solving scoring rubric was an ordinal scale, the nonpara-metric Wilcoxon rank sum test was calculated on the gain scores to determine if there was a sig-nificant difference between the two groups. The problem-solving gain scores were also analyzedby sex within groups.

Runner Problem Results

Scores on the runner problem had a possible range from 0 to 25 and are presented in Table3. The explicit group had a mean gain score of approximately 21%, while the textbook grouphad a mean gain score of approximately 3%. A Wilcoxon rank sum on the gain scores indicat-ed that there was a statistically significant difference between the two groups (z 5 4.07; p ,.0001). In this study, all statistical tests used two-tailed tests with an alpha level of .05.


Figure 4. Car and bug question.

Subscores on the runner problem were also analyzed to compare specific characteristics ofthe problem-solving solutions. The total score on the runner problem was composed of five dif-ferent subscores: quality of the physics representation, completeness of physics representation,match of equations with physics representation, organization of solution, and mathematical exe-cution. The subscore gains on the runner problem by group are presented in Table 4.

The gain scores for each of these characteristics had a range from 0 to 5 points. On a per-cent basis, the explicit group had a 46% gain in representation quality, and the textbook groupa 6% gain. On representation completeness, the explicit group had a 40% gain, and the textbookgroup a 4% gain. On the match of equation, the explicit group had a 14% gain, and the text-book group a 0% gain. On organization, the explicit group had a 0% gain, and the textbookgroup a 4% gain. Finally, on mathematical execution, the explicit group had a 2% gain, and text-book group a 0% gain.

562 HUFFMAN

Table 3Runner and asteroid problem pretest, posttest, and gain scores, by group

Pretest Posttest Gain

Group M SD M SD M SD

Runner problemExplicit (n 5 86) 8.8 4.3 14.1 5.4 5.2 6.7Textbook (n 5 59) 10.0 4.3 10.7 4.4 0.8 5.4

Asteroid problemExplicit (n 5 86) 4.9 2.4 5.5 4.7 0.6 4.6Textbook (n 5 59) 5.2 2.6 5.1 3.4 20.1 3.7

Table 4Runner and asteroid problem subscore gains, by group

Gain Scores

Problem-solving Explicit (n 5 86) Textbook (n 5 59)

Characteristic M SD M SD z p

Runner problemRepresentation quality 2.3 2.3 0.3 2.1 5.23 ,.0001*

Representation complete 2.0 2.3 0.2 1.8 5.04 ,.0001*

Match of equations 0.7 1.7 0.0 1.3 2.98 .003*

Organization 0.0 1.2 0.2 1.1 0.44 .66Mathematical execution 0.1 1.8 0.0 1.8 0.29 .77

Total score 5.2 6.7 0.8 5.4 4.07 ,.0001*

Asteroid problemRepresentation quality 0.8 1.6 0.1 1.5 2.24 .03*

Representation complete 0.6 1.4 0.1 0.9 2.30 .02*

Match of equations 20.4 1.3 0.1 1.1 1.27 .21Organization 20.2 1.0 0.0 0.8 2.09 .04*

Mathematical execution 20.1 1.1 20.2 1.0 .21 .84Total score 0.6 4.6 20.1 3.7 .79 .43

*p # .05

A Wilcoxon rank sum test on the subscores indicated that there were statistically significantdifferences on 3 of the 5 problem-solving characteristics. The explicit group had significantlyhigher gains on the quality of the physics representation (z 5 5.23; p , .0001), on the useful-ness of the physics representation (z 5 5.04; p , .0001), and on the match of equations withphysics representation (z 5 2.98; p 5 .003). There were no significant differences on the orga-nization of the solutions or the mathematical executions.

The runner problem scores were also analyzed by sex within groups and are presented inTable 5. A Wilcoxon rank sum test by sex indicated that there was no statistically significantdifference between the two groups on the gain scores. For the explicit group, z 5 1.29, p 5 .20,and for the textbook group, z 5 1.30, p 5 .31. It was not possible to test for interaction usinga Wilcoxon rank sum test; however, a graphic examination of gain scores was conducted andindicated that there was no apparent interaction.

Asteroid Problem Results

Scores on the context-rich asteroid problem were analyzed in the same way as the runnerproblem. Mean pretest, posttest, and gain scores for the asteroid problem are presented in Table3. The explicit group had a mean gain of approximately 2%, while the textbook group had amean decline of approximately 1%. A Wilcoxon rank sum test on the asteroid gain scores re-vealed that there was no significant difference in the gain scores between the two groups (z 5.79; p 5 .43).

The subscore mean gains on the asteroid problem are presented by group in Table 4. On apercent basis, both groups had approximately no gain on match of equations, organization, ormathematical execution. On representation quality and representation completeness, the explic-it group had 16% and 12% gains, respectively, while the textbook group had 2% gains. AWilcoxon rank sum test on the subscores indicated that there were statistically significant dif-ferences on 3 of the 5 subscores. The explicit group had higher gains on the quality of thephysics representation (z 5 2.24; p 5 .03) and on the completeness of the physics representa-tion (z 5 2.30; p 5 .02), while the textbook group had higher gains on the organization of thesolution (z 5 2.09; p 5 .04). There were no differences on the match of equations with repre-sentations or the mathematical executions.

The gain scores on the asteroid problem were also analyzed by sex within groups; the re-sults are presented in Table 5. In the explicit group, the female students had a 4% gain and themale students a 2% gain, while in the textbook group female students had a 2% decline andmale students a 1% gain. A Wilcoxon rank sum test by sex indicated that neither of the gain


Table 5Runner and asteroid problem gain scores, by sex, within group

Female Male

Group M SD n M SD n

Runner problemExplicit (n 5 86) 6.8 6.0 29 4.5 7.0 57Textbook (n 5 59) 1.3 6.0 30 0.3 4.8 29

Asteroid problemExplicit (n 5 86) 1.0 .5 29 0.5 4.8 57Textbook (n 5 59) 20.5 4.0 30 0.3 3.3 29

score differences were statistically significant. For the explicit group, z 5 .44, p 5 .66; and forthe textbook group, z 5 .76, p 5 .45.

Force Concept Inventory Results

To provide a general measure of students’ understanding of Newton’s laws, scores on theForce Concept Inventory were analyzed both before and after instruction. The results of pretest,posttest, and gain scores on the Force Concept Inventory are presented in Table 6. There was amaximum score of 29 on the Force Concept Inventory. On a percent basis, the explicit grouphad a gain of 18%, while the textbook group had a gain of 22%. The gain scores on the ForceConcept Inventory were also analyzed by sex. These data are also presented in Table 6. On apercent basis, the female students in the explicit group had a gain of 20%, and the male studentsa gain of 17%. For the textbook group, the female students had a gain of 18%, and the malestudents a gain of 25%.

Gain scores on the Force Concept Inventory were analyzed by group and sex using a two-way analysis of variance (ANOVA). The results of the two-way ANOVA on the Force ConceptInventory gain scores by group and sex are presented in Table 7. The ANOVA indicated thatthere were no significant differences between the two groups, F(1, 144) 5 2.98, p 5 .09, or be-tween the sexes, F(1, 144) 5 0.16, p 5 .69; however, there was a significant two-way interac-tion between group and sex, F(1, 144) 5 5.18, p 5 .02. An examination of the mean gain scoresshowed that in the explicit group, female students had higher gain scores than male students,while in the textbook group, male students had higher gain scores than female students. Theseresults suggest that the problem-solving strategies had differential effects on male and femalestudents’ conceptual understanding.

Newton’s Laws Test Results

To provide a more in-depth examination of students’ conceptual understanding of Newton’slaws than could be obtained with the Force Concept Inventory, students in this study also com-pleted an open-ended test before and after instruction. The Newton’s Laws Test included threeopen-ended questions that focused on the three Newton’s laws, respectively. The pretest,posttest, and gain scores on the three Newton’s laws questions are presented in Table 8.

To compare changes in students’ responses to the Newton’s Laws Test, gain scores werecalculated by group. Gain scores on the first law and second law question ranged from 22.0 to12.0, and on the third law question ranged from 21.0 to 11.0. A Wilcoxon rank sum test on

564 HUFFMAN

Table 6Force Concept Inventory pretest, posttest, and gain scores, by sex

Pretest Posttest Gain

Group M SD M SD M SD

Explicit (n 5 86) 9.6 3.8 14.7 4.3 5.1 3.8Female (n 5 29) 7.8 2.7 13.7 3.9 5.9 3.6Male (n 5 57) 10.5 4.0 15.2 4.4 4.8 3.9

Textbook (n 5 59) 9.3 3.4 15.6 5.0 6.3 4.3Female (n 5 30) 8.2 2.8 13.4 4.5 5.3 4.7Male (n 5 29) 10.5 3.5 17.8 4.4 7.3 3.6

the gain scores indicated that for the first law question, the textbook group had a significantlyhigher mean gain score than the explicit group (z 5 2.25; p 5 .02); for the second law ques-tion, there was no significant difference in the gain scores by group (z 5 .97; p 5 .33); and forthe third law question, there was no significant difference in gain scores by group (z 5 1.39; p5 .17).

The Newton’s Laws Test was also analyzed by sex within group to determine if there wasa significant difference between the gain scores of male and female students. Gain scores on theNewton’s Laws Test by sex within group are presented in Table 9. A Wilcoxon rank sum on thegain scores indicated that there were no significant differences in gain scores by sex on any ofthe three Newton’s laws questions. On the first law question, for the explicit group, z 5 1.62, p5 .10; and for the textbook group, z 5 .36, p 5 .72. On the second law question, for the ex-plicit group, z 5 1.28, p 5 .20; and for the textbook group, z 5 .06, p 5 .95. On the third lawquestion, for the explicit group, z 5 .92, p 5 .36; and for the textbook group, z 5 .40, p 5 .69.A graphic analysis did not reveal any indication of an interaction between sex and group.


Table 7Two-way ANOVA of Force Concept Inventory gain scores,by group and sex

Effects SS df MS F

Group 47.13 1 47.13 2.98Sex 2.48 1 2.48 .16Interaction 81.95 1 81.954 5.18*Residual 2229.56 141 15.81Total 2358.80 144 16.38

*p # .05

Table 8Pretests, posttests, and gain scores on Newton’s Laws Test, by group

Explicit Group Textbook Group (n 5 86) (n 5 59)

M SD M SD

First lawPre .96 .94 .66 .86Post 1.41 .90 1.47 .86Gain .46 1.02 .81 1.22

Second lawPre .58 .64 .64 .58Post .71 .72 .66 .78Gain .13 .88 .02 .78

Third lawPre .07 .27 .10 .31Post .35 .48 .47 .50Gain .26 .51 .40 .58

Discussion

Previous research indicates that explicit problem-solving instruction can help improve stu-dents’ problem-solving performance more than traditional instruction (Mestre et al., 1993;Heller et al., 1992; Van Heuvelen, 1990; Wright & Williams, 1986; Heller & Reif, 1984; Larkin& Reif, 1979; Reif et al., 1976). The results of this study indicated that explicit problem-solv-ing instruction can help improve some, but not all aspects of students’ problem-solving perfor-mance. One of the primary differences between the two groups was in the qualitative descrip-tions of the problems. The students in the explicit group demonstrated more improvement in thequality and completeness of their physics representations; this difference appears to have beendue to the different problem-solving instruction that the students received. Both groups weretaught to draw a sketch and define variables, but in addition, the explicit group was taught torewrite the question, decide on an approach, draw a physics diagram, select a target variable,and select quantitative relationships. These additional instructions appear to have helped im-prove the explicit groups’ physics representations more than the textbook group. The problem-solving literature suggests that improvements in problem representation can have practical im-plications for students. Larkin (1983a) claimed that representations are one of the most crucialaspects of problem solving, because they can determine the direction of the entire solution. Dia-grams can be useful because of the way they help organize information into chunks that can beused in the problem solution (Larkin & Simon, 1987).

The results on the organization of problem solutions were contrary to what was expected.It was hypothesized that teaching students how to plan a solution to a problem would improvetheir organization of the solution more than students who were not taught how to plan. How-ever, there were no significant differences between the two groups on organization of the solu-tions, although the explicit group was directly taught to write a plan to the solution and the text-book group was not. The explicit group was given detailed instructions on how to constructequations, check for sufficiency of the equations, and outline the math solution of the problem,while the textbook group was given no instructions on how to plan a solution to the problem.The additional planning instruction that the explicit group received apparently did not result ingreater improvement in students’ logical organization of the solution. Although these results aresurprising, they are reasonable when one considers the previous research which indicates thatplanning is one of the most difficult problem-solving skills for novices to learn (Dalby, Tourni-aire, & Linn, 1986). One of the reasons why planning may not have improved students’ orga-nization of the solution as much as expected is that novices lack problem-solving experience,problem-solving procedures, and an understanding of the domain, all of which are helping inplanning solutions to physics problems (Eylon & Linn, 1988). The results of the present study

566 HUFFMAN

Table 9Newton’s Laws Test gain scores, by group and sex

Explicit Group Textbook Group

Female (n 5 29) Male (n 5 57) Female (n 5 30) Male (n 5 29)

M SD M SD M SD M SD

First law .69 1.11 .32 .96 .83 1.32 .79 1.15Second law .00 .85 .19 .90 .03 .81 .00 .76Third law .35 .49 .21 .51 .43 .60 .36 .58

appear to support the notion that planning is one of the most difficult problem-solving skills forstudents to use, and one that may not improve novices’ organization of the solution.

There were also no significant differences between the two groups in terms of students’mathematical execution. In fact, neither group gained even more than 5% on their mathemati-cal execution score. The lack of difference in the mathematical execution gain scores is also sur-prising, because the two groups were given different instructions on how to execute the math-ematics. The explicit group was given instructions on how to execute the mathematics inaccordance with the plan, and how to solve algebraically for the target variable before insertingnumbers into the equations, while the students in the textbook group were merely instructed tosolve the mathematical equations any way they wanted to obtain an answer. Despite the differ-ent instructions, there were no significant differences in the improvement of the two groups interms of mathematical execution.

Another purpose of this study was to determine if students who were taught how to use anexplicit problem-solving strategy exhibited more improvement in conceptual understanding ofNewton’s laws than students who were taught how to use a traditional textbook strategy. Ac-cording to Maloney (1994), there is a lack of research on the role of problem solving in learn-ing physics concepts, and it is important to investigate the issue given that many physics in-structors unquestionably assume that problem solving is a vehicle for learning physics. Theresults of the Force Concept Inventory and the Newton’s Laws Test both indicated that the ex-plicit strategy did not improve students’ conceptual understanding of Newton’s laws more thanthe textbook strategy. The explicit group did not gain more than the textbook group on the ForceConcept Inventory or any of the three questions on Newton’s laws. These results are surprisinggiven the conceptual nature of the explicit problem-solving strategy compared to the computa-tional nature of textbook problem solving. Despite the direct steps taken to help students focuson the conceptual aspects of the problems, it did not appear to significantly affect students’ un-derstanding of concepts.

Although there was no significant difference in the conceptual understanding of the twogroups, there was evidence that both groups did have reasonable gains on these measures. Nu-merous studies on high school and college students’ understanding of mechanics indicate thateven after instruction, anywhere from 25% to 50% of the students still fail to answer concep-tual questions correctly (McDermott, 1984). In the present study, the percentage of incorrect re-sponses on the Force Concept Inventory and the Newton’s Laws Test after instruction is simi-lar to what has been observed in pervious research.

Despite the lack of difference between the conceptual understanding of the two groups,there was evidence that the explicit problem-solving strategy differentially affected male and fe-male students. An analysis of the Force Concept Inventory results by sex revealed that therewere no significant differences in the gain scores by sex; however, there was a significant in-teraction between sex and group. The explicit strategy appeared to benefit the conceptual un-derstanding of female students, while the textbook strategy appeared to benefit the conceptualunderstanding of male students. It was hypothesized that all students in the explicit group wouldgain more than students in the textbook group because of the conceptual nature of the explicitstrategy; however, this did not appear to be the case.

Previous research indicates that in high school physics, male students generally make larg-er gains in conceptual understanding than female students. Based upon the results of the Me-chanics Diagnostic Test (Halloun & Hestenes, 1985), which is similar to the Force Concept In-ventory, Heller and Lin (1992) reported that in high school, male students made significant gainsin conceptual understanding, but female students did not appear to have learned anything in theirhigh school physics classes. In the present study, as expected, the male students in the textbook


group had a larger gain than the female students, while the male students in the explicit grouphad surprisingly less gain than the female students. These results suggest that the explicit strat-egy had a differential effect on the conceptual understanding of male and female students.

One possible reason for the interaction between group and sex may be the motivation ofmale students. Based on discussions with the classroom teachers and observations of classes,more male than female students appeared to dislike the new strategy and resisted its use. Theresistance to the strategy could have affected the male students’ scores on the Force Concept In-ventory and the Newton’s Laws Test. Previous research in physics problem solving has shownthat some male students are resistant to using new techniques. For example, in a study using theexplicit problem-solving strategy with introductory university physics students, Heller and Lin(1992) reported that a larger proportion of women than men adopted the strategy, and that asmall proportion of men were resistant to the explicit strategy. More research is still needed onthe different effects of explicit problem-solving instruction on male and female students’ levelsof motivation to better understand the interaction that was observed in this study.

Overall, the results of this study have several implications for physics instruction. The re-sults indicated that explicit problem-solving instruction helped improve the quality and com-pleteness of students’ problem representations more than students who used textbook tech-niques. However, there was no evidence that explicit instruction helped improve students’organization or mathematical execution more than textbook instruction. In terms of conceptualunderstanding, the results indicated that despite the emphasis of explicit instruction on the con-ceptual aspects of problem solving, explicit problem-solving instruction should not necessarilybe viewed as a means of helping students to improve their understanding of the concepts morethan textbook problem solving. Hewitt (1992) claimed that in high school, problem-solving in-struction actually obscures students’ understanding of the concepts; he recommended teachingthe concepts and principles of physics instead of problem solving. If Hewitt is correct, the timeand effort that teachers devote to teaching problem solving may be better spent on the conceptsand principles of physics. More research is needed to determine if a conceptual course actuallyresults in more conceptual understanding than a course that combines problem solving and con-cepts.

The results of this study also indicated that explicit problem-solving instruction may dif-ferentially affect male and female students’ conceptual understanding. There were indicationsthat male students who were taught the explicit strategy did not improve their conceptual un-derstanding as much as expected. In physics, male students typically have greater gains in con-ceptual understanding than female students; however in this study, the male students in the ex-plicit group had the lowest gain scores on 3 of the 4 measures of conceptual understanding.Based on these results, physics instructors need to be aware of the potential for new problem-solving strategies to differentially affect male and female students. If the interaction betweensex and group was due to the negative reaction of the male students toward the explicit strate-gy, instructors that use the explicit strategy need to address students’ dislike for the strategy andfind ways of encouraging the use of more expertlike techniques without alienating groups ofstudents. All in all, more research is clearly needed on the differential effect of problem-solv-ing instruction on male and female students’ conceptual understanding, to completely under-stand the interaction that was observed in this study.

References

Chi, M.T.H., Feltovich, P.J., & Glasner, R. (1980). Categorization and representation ofphysics problems by experts and novices. Cognitive Science, 5, 121–152.

568 HUFFMAN

Clement, J. (1982). Students’ preconceptions in introductory mechanics. American Journalof Physics, 50, 66–71.

Dalby, J., Tourniaire, F., & Linn, M.C. (1986). Making programming instruction cognitivelydemanding: An intervention study. Journal of Research in Science Teaching, 23, 427–436.

diSessa, A. (1982). Unlearning Aristotelian physics: A study of knowledge-based learning.Cognitive Science, 6, 37–75.

Eylon, B., & Linn, M.C. (1988). Learning and instruction: An examination of four researchperspectives in science education. Review of Educational Research, 58, 251–301.

Gunstone, R.F. (1987). Student understanding in mechanics: A large population survey.American Journal of Physics, 55, 691–696.

Halloun, I.A., & Hestenes, D. (1985). The initial knowledge state of college physics stu-dents. American Journal of Physics, 53, 1043–1055.

Heller, J., & Reif, F. (1984). Prescribing effective human problem-solving processes: Prob-lem descriptions in physics. Cognition and Instruction, 1, 177–216.

Heller, P., & Hollabaugh, M. (1992). Teaching problem solving through cooperative group-ing. Part 2: Designing problems and structuring groups. American Journal of Physics, 60,637–644.

Heller, P., Keith, R., & Anderson, S. (1992). Teaching problem solving through cooperativegrouping. Part 1: Group versus individual problem solving. American Journal of Physics, 60,627–636.

Heller, P., & Lin, H. (1992, April). Teaching physics problem solving through cooperativegrouping: Do men perform better than women? Paper presented at the annual meeting of theNational Association for Research in Science Teaching, Boston, MA.

Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The PhysicsTeacher, 30, 141–153.

Hewitt, P. (1992). Conceptual physics: The high school physics program. Menlo Park, CA:Addison-Wesley.

Johnson, D.W., Johnson, R.T., & Holubec, E.J. (1986). Circles of learning: Cooperation inthe classroom. Edina, MN: Interaction Book.

Larkin, J.H. (1981). Cognition of learning physics. American Journal of Physics, 49,534–541.

Larkin, J.H. (1983a). The role of problem representation in physics. In D. Gentner & A. Stevens (Eds.), Mental models. Hillsdale, NJ: Erlbaum.

Larkin, J.H. (1983b). A general structure for learning or teaching science. In A.C. Wilkin-son (Ed.), Computers and cognition. New York: Academic Press.

Larkin, J.H., McDermott, J., Simon, D.P., & Simon, H.A. (1980). Expert and novice per-formance in solving physics problems. Science, 208, 1335–1342.

Larkin, J.H., & Reif, F. (1979). Understanding and teaching problem solving in physics.European Journal of Science Education, 1, 191–203.

Larkin, J.H., & Simon, H.A. (1987). Why a diagram is (sometimes) worth ten thousandwords. Cognitive Science, 11, 65–99.

Maloney, D.P. (1994). Research on problem solving: Physics. In D. Gabel (Ed.), Handbookof research on science teaching and learning. Washington, DC: National Science Teachers As-sociation.

McClosky, M. (1983). Naive theories of motion. In D. Gentner & A. Stevens (Eds.), Men-tal models. Hillsdale, NJ: Erlbaum.

McDermott, L.C. (1984). Research on conceptual understanding in mechanics. Physics To-day, July, 24–32.


Mestre, J.P., Dufresne, R.J., Gerace, W.J., Hardiman, P.T., & Touger, J.S. (1993). Promot-ing skilled problem-solving behavior among beginning physics students. Journal of Research inScience Teaching, 30, 303–317.

Minstrell, J. (1982). Explaining the “at rest” condition of an object. The Physics Teacher,January, 10–14.

Reif, F., & Heller, J.I. (1982). Knowledge structure and problem solving in physics. Edu-cational Psychologist, 17, 102–127.

Reif, F., Larkin, J., & Brackett, G.C. (1976). Teaching general learning and problem solv-ing skills. American Journal of Physics, 44, 212–217.

Schoenfeld, A.H. (1985). Mathematical problem solving. San Diego, CA: Academic Press.Van Heuvelen, A. (1990). Overview—Case-study approach to introductory physics instruc-

tion. Paper presented at the meeting of the American Association of Physics Teachers, Min-neapolis, MN.

Wandersee, J., Mintzes, J., & Novak, J. (1994). Research on alternative conceptions in sci-ence. In D. Gabel (Ed.), Handbook of research on science teaching and learning. Washington,DC: National Science Teachers Association.

Woods, D.R. (1989). Problem solving in practice. In D. Gabel (Ed.), What research says tothe science teacher. Vol. 5: Problem solving. Washington, DC: National Science Teachers As-sociation.

Wright, D.S., & Williams, C.D. (1986). A WISE strategy for introductory physics. ThePhysics Teacher, 24, 211–216.

570 HUFFMAN

effect of explicit problem solving instruction on high school students' problem-solving...

Documents