Teaching the Process of Science

Submission to: U. S. Department of EducationFund for the Improvement of Postsecondary Education (FIPSE)

The Comprehensive Program, CFDA 84.116BPIs: Anne Egger, Stanford University, and Anthony Carpi, John Jay College, CUNY

ABSTRACT

Many social issues have become “socioscientific” issues, requiring individuals to make

decisions regarding diet, the environment, or other areas of their life based on an understanding

of the nature of scientific research. Yet the majority of people are ill-equipped to make these

decisions because they lack an understanding of the scientific process (Sadler et al., 2004).

The proposed project is a collaboration between the City University of New York

(CUNY), Stanford University (SU), and the University of California Museum of Paleontology

(UCMP) that is designed to expose a greater number of students, science majors and non-science

majors alike, to the scientific process and scientific research by: 1) assembling an

interdisciplinary panel of experts in science and science education to develop a one-semester,

undergraduate curriculum on the Process of Science; 2) creating a set of freely available online

curricular modules and associated teaching materials to be used in Process of Science courses; 3)

evaluating these materials in a Process of Science course to be mounted at CUNY, in a

disciplinary science course at SU, and at four additional institutions; and 4) disseminating the

Process of Science teaching materials into a national program for science education, and creating

a community of professionals to promote their use and continued assessment.

This proposal draws on an existing national-scale project that has developed an

innovative and successful model for the delivery of high-quality teaching content for

interdisciplinary science education. Based on this success and the near-consensus among

educational researchers that study about the nature of science improves scientific understanding,

we now seek to expand this model to catalyze a shift in the way undergraduate introductory

science courses are taught.

Teaching the Process of Science

TABLE OF CONTENTS

GOALS AND

OBJECTIVES................................................................................................................................1

THE STATE OF SCIENCE EDUCATION................................................................................1

TEACHING THE PROCESS OF SCIENCE.............................................................................4

PROPOSED WORK.....................................................................................................................6

Objective 1: Assemble an interdisciplinary panel of experts....................................................7

Objective 2: Create a set of curricular modules.........................................................................9

Table 1: Key Scientific Concepts and Possible Module Topics..........................................9

Objective 3: Test and evaluate these materials........................................................................12

Table 2: Examples of assessment instrument questions....................................................13

Objective 4: Disseminate and develop these materials............................................................16

WHY VISIONLEARNING?.......................................................................................................17

EXPECTED OUTCOMES.........................................................................................................19

REFERENCES CITED........................................................................................References pg. 1

TOC

Teaching the Process of Science

GOALS AND OBJECTIVES

This project aims to catalyze a shift in the way introductory science courses are taught by

moving them away from portraying science as an exercise in memorizing concepts and toward

teaching the process of science, as championed by the educational literature (Reif, 1995;

Schmidt, 1996; Dagher et al, 2004; Flick & Lederman, 2004). This work is a collaboration of

major public and private institutions, and existing national-scale educational projects, thus

assuring its broad dissemination. Toward this goal, this work has four specific objectives:

Objective 1: Assemble an interdisciplinary panel of experts in science and science education to

develop a one-semester, undergraduate curriculum on the Process of Science.

Objective 2: Create a set of curricular modules and associated teaching materials to be used in

Process of Science courses and launch the materials on the Visionlearning website.

Objective 3: Evaluate these materials in a Process of Science course to be mounted at the City

University of New York, in a disciplinary science course at Stanford University, and at four

other institutions.

Objective 4: Disseminate the Process of Science teaching materials into a national program for

science education, and create a community of professionals to promote their use and

continued assessment.

This project addresses FIPSE Invitational Priority A by providing all undergraduate students

access to scientific research through freely available, high quality materials that elucidate the

process of science and highlight modern research questions.

THE STATE OF SCIENCE EDUCATION

The poor state of science education in the United States is all too well known and has

serious consequences on American culture and society (Dye, 2000; Augustine, 1998). The most

1

The Process of Science Carpi

recent Science and Engineering Indicators study shows scientific knowledge among adults in the

United States is declining, and over 61% of respondents lack a clear understanding of the nature

of scientific inquiry (NSB, 2006). On a local level, the lack of public understanding of the

scientific process has enabled school boards in Kansas, Dover, PA, and other regions to argue, at

least temporarily, that the teaching of science should include a discussion of belief systems such

as Intelligent Design (NSB, 2006; NCSE, 2005; Allen, 2005). On a global level, the U.S. is the

only developed country to reject the limits described by international treaties on climate change

because a vocal minority has convinced the public that scientific debate means that the scientific

basis for this change is uncertain, unreliable, or unproven (Oreskes, 2004).

While responses by the public to scientific issues often perplex scientists and educators,

they are a natural product of the manner in which science is taught in the U.S. Traditional college

lecture courses present science as a simple series of facts and conclusions made by long-dead

scientists, overlooking the natural course of debate and discussion that delimited those

conclusions and denying students the opportunity to understand the process by which science

derives its authority (Allchin, 2003). As a result, the discourse that accompanies current

scientific theories is seen as surprising and out-of-character to students, and elicits a natural

response to question the very validity of that which is being debated, rather than understand that

the debate is itself crucial to scientific understanding (Rudolph & Stewart, 1998).

Almost every discipline presents itself through both process and application courses:

English pairs creative writing and effective speaking courses with literature and poetry classes;

mathematics, possibly to a fault, focuses on process as opposed to its application in accounting,

statistics, or other areas; foreign languages teach the process of speaking, writing, and

understanding the language in addition to foreign literature and culture. And yet the sciences are

almost completely devoid of courses on the process of science at the introductory level.

Applications of the subject in various fields – chemistry, physics, biology, etc. – are taught at

2

The Process of Science Carpi

nearly every undergraduate institution in the country; however, courses on the process and nature

of science, which convey the common principles of scientific research, are a rarity.

Some faculty believe that students pick up an understanding of the process of science by

performing laboratory experiments. Yet the common laboratory course, in which precise

measures and exact endpoints are already known, does little to convey the process of science

(Hofstein & Lunetta, 2004). Others have shown that students do begin to understand the nature

of science by participating in faculty-mentored research (Ryder et al., 1999). As a result, the U.S.

Department of Education and the National Science Foundation are staunch supporters of

undergraduate research experiences. Unfortunately, traditional research experiences are reserved

for undergraduate students who major in the sciences at a limited number of institutions:

commonly four-year institutions that grant graduate degrees or have access to research facilities.

Non-science majors or low-performing students rarely have access to undergraduate research

internships. Further, minority- and Hispanic-serving colleges and universities are among the least

prepared to offer these experiences to students (NSF, 1998). Additionally, those students who do

have access to research opportunities often do not participate in these experiences until their final

year (Ryder et al., 1999).

Teaching the process of science is critical at the introductory undergraduate level,

however, because many students enter college with minimal understanding of the scientific

endeavor beyond methodical procedure (Moss et al., 2001; Bell et al., 2003). Unfortunately, the

vast majority of students are never taught the scientific process; instead, science is presented to

them as a system of discipline-specific facts to be memorized – comparable to memorizing a

poem in a foreign language without understanding the vocabulary. As a result, non-science

majors express high anxiety and low motivation when confronted with decisions that involve

scientific information (Gogolin & Swartz, 1992). Even science majors express limited

understanding of the creativity involved in the process of science (Ryder et al., 1999). This may

3

The Process of Science Carpi

contribute to the high attrition rates documented in the first two years of the science major

(Seymour & Hewitt, 1997).

TEACHING THE PROCESS OF SCIENCE

Though few studies have been conducted on the effectiveness of teaching the nature and

process of science, the results of those studies are encouraging. Misconceptions regarding the

scientific process often rest on language or terminology that can be easily clarified: Dagher and

Boujaoude (1997) have concluded that one cause of the common student opinion that evolution

is a very tentative theory is a lack of explicit teaching about the nature of the word “theory.”

When instruction in the nature of theories is explicitly included in the study of scientific theories

themselves, student learning is improved (Dagher et al, 2004). When students are explicitly

taught that science is a process and a product of debate, they begin to understand the process in a

more intelligent manner, giving them the perspective to better appreciate current socioscientific

issues such as global climate change (Schmidt, 1996; Monk & Osborne, 1998).

These results suggest that it is important to introduce students to the nature and process of

science early in their undergraduate careers. All students are generally required to take at least

one introductory-level science course in college. These courses have tremendous potential to

provide them with an understanding of the nature and process of science through inquiry-based

experiences and exposure to ongoing research in the literature. Instead, in the vast majority of

introductory courses, the traditional lecture format dominates and science is presented as a

collection of facts determined by content in a textbook (Astin & Astin, 1992; Moulton, 1994).

Textbooks, however, have proven to be poor purveyors of scientific understanding

(Blosser & Helgeson, 1988). Few contain explicit information about the process of science

beyond an introductory chapter about the scientific method; though there are notable exceptions

(see Smith and Pun, 2005). Among other cited shortcomings, many science textbooks have been

4

The Process of Science Carpi

identified as "low-cohesion," presenting science as a string of disconnected facts that make it

difficult for students to see the relationships between concepts (Best et al., 2005). Textbooks

commonly present watered-down, popular versions of science that ignore controversy or

uncertainty, and fail to provide any content to convey the process of scientific discovery (Brush,

1974). They focus on solving isolated problems, ignoring the research that suggests that

discussing the nature of scientific experimentation provides students with a deeper appreciation

of the subject matter and builds better concept recall (Nurrenbern & Pickering, 1987).

The few textbooks dedicated to the process of science are targeted to primary and

secondary school teachers (Rezba et al., 2003), or to primary and secondary school children

themselves (Pearson, 2005). Unfortunately, these books often fare poorly when externally

reviewed (Bennetta, 1993). Available online resources on the nature of science tend to target the

K-12 audience (NAP, 1998) or are limited to a brief discussion of the scientific process (Farabee,

2002). Thus, the obligation falls upon the instructor to include explicit instruction about the

nature of science in his or her class with little textbook support (Hurd, 1998).

Some guidance for teaching the process of science does exist. Flick and Lederman (2004)

edited a collection of essays on teaching the nature of science. Frederick Reif’s speech and paper

following his receipt of the Millikan Medal for “creative contributions to the teaching of

physics,” provides a discussion of the processes critical for students to properly interpret

scientific concepts and principles (Reif, 1995). And Tobin (1990) has discussed the principles

that should be conveyed in laboratory class to foster inquiry and an understanding of the

scientific process. Yet there is still no widespread integration of these concepts into the typical

introductory science classroom where the vast majority of students could benefit the most.

5

The Process of Science Carpi

PROPOSED WORK

The overall goal of this project is to develop, test, and refine a one-semester,

undergraduate curriculum for teaching the nature and process of science based on collaborative

input from a variety of experts in scientific disciplines, education, and cognitive science. Such a

curriculum has the potential to revolutionize the way science is taught to science majors and non-

majors alike, providing access to high quality science education for students at a variety of

institutions. By integrating the proposed materials into an established and nationally known

website for teaching interdisciplinary science, these materials can be assembled as a stand alone

course, or they can be integrated into introductory science courses in chemistry, earth science,

and other disciplines. As such, this project will not only catalyze a new agenda in science

education reform, but it will present this agenda in a way such that it can be integrated gradually

into existing curricula, avoiding the barriers to adoption that radical curricular change may

encounter. Further, this will allow these reforms to be reinforced in several courses across a

variety of disciplines, as promoted in the literature (Dagher et al, 2004).

The work proposed is a collaboration between John Jay College of CUNY, Stanford

University, and the University of

California Museum of Paleontology.

John Jay College is a leading minority-

and Hispanic-serving institution (Fig.

1), thus this project connects a diverse

set of influential institutions to assure

its wide dissemination and broad impact. Further, this project brings together the Visionlearning

project at CUNY and the Understanding Science project at UCMP. Visionlearning was funded in

2000 and again in 2002 by the National Science Foundation to produce undergraduate curricular

materials for teaching interdisciplinary science. Understanding Science was funded in 2006 by

6

Figure 1

Ethnic Composition of the John Jay Student Body

African American

29%

Asian4%

Caucasian27%

Hispanic39%

Other1%

The Process of Science Carpi

the National Science Foundation to identify critical principles for teaching the process of science,

and produce web-based resources that convey these principles. As such, this project is uniquely

positioned to achieve the stated goal of developing a curriculum for teaching the process of

science to all undergraduates and mounting this curriculum so that it can be taught as a stand-

alone course or integrated into existing disciplinary science courses. Toward achieving this goal,

this project has four specific objectives.

Objective 1: Assemble an interdisciplinary panel of experts in science and science

education to develop a one-semester, undergraduate curriculum on the Process of Science.

Experts in a wide variety of fields have no problem identifying what science is (and what

it is not), despite the fact that the actual procedures may differ significantly between fields.

However, existing studies that evaluate student gains from research projects define the nature of

science only very vaguely (see Seymour et al., 2004, for summary). Our objective is to define the

nature of science such that a core undergraduate course can be developed.

As part of their funded grant, Understanding Science will convene an Advisory Board in

September 2006 that will consist of experts in several scientific disciplines. The Understanding

Science Advisory Board will be charged with defining the concept of natural science, and

establishing a set of characteristics critical to understand the nature of the scientific process

science and scientific research. This work will serve as a foundation upon which we will build,

and in preparation for the proposed work, the Project Coordinator for this proposal (Ms. Egger)

will attend this meeting. Thus, even before a grant is made under the FIPSE program, this

collaboration will have created an extensive outline of key scientific concepts important to

understanding the nature of science.

As soon as a funding notification is made, we will assemble our own panel of

interdisciplinary experts who will be charged with using the Understanding Science outline and

7

The Process of Science Carpi

the existing literature (Tobin, 1990; Reif, 1995; Flick & Lederman, 2004) to develop an

undergraduate curriculum on the process of science and outline a series of teaching modules

(described further in objective 2) to be used as course readings to support faculty teaching this

course. Our expert panel will meet virtually to discuss their objectives, and will then convene for

a 2.5 day face-to-face conference to outline these materials. We have already contacted and

confirmed the participation of several key educators for this panel. Panel participants include

(full Biographical Sketches and Letters of Support are included in the appendices):

Dr. Anthony Carpi (Project Director) is the founder of Visionlearning, recipient of two NSF grants to develop innovative educational materials, and has published broadly on the effect of novel online resources on improving undergraduate science education (Carpi, 2001; Carpi, 2003; Carpi & Mikhailova, 2003).

Ms. Anne Egger (Project Coordinator) is the Undergraduate Program Coordinator in the Department of Geological and Environmental Sciences at Stanford University, where she has redesigned the undergraduate curriculum and directs the undergraduate research program.

Dr. Norman Lederman is internationally known for his research and scholarship on the development of students' and teachers' conceptions of the nature of science and has published widely on methods used to assess student and teacher understanding of the nature of science (Lederman, 1992; Lederman, 1999, Lederman, 2002).

Dr. Cathy Manduca is Director of the Science Education Resource Center (SERC) at Carleton College and has earned a national reputation for her work with projects that support improvements in geoscience and science education.

Ms. Judy Scotchmoor is Assistant Director of the University of California Museum of Paleontology, and the Project Coordinator of the NSF-funded Understanding Science project.

Dr. Natalie Kuldell is the designer and coordinator of the Biological Engineering program at MIT, and has written on the use of research in the teaching of undergraduate science

Ms. Sandra Swenson (Evaluation Coordinator) has extensive experience teaching and evaluating the impact of science teaching resources; she is a doctoral student in science education at Columbia University.

To assure that the objectives of the Department of Education are fully considered in this work,

we will consult with our FIPSE program director to determine three additional panel members

with experience in science research and education. The panel will have the following charges:

1. Compare and contrast how the scientific method is practiced in diverse disciplines. The

scientific method is perhaps one of the most commonly misinterpreted concepts in

undergraduate science education (Bauer, 1992). Current texts contain major errors:

8

The Process of Science Carpi

portraying the scientific method as absolute, suggesting that a single, linear scientific method

exists, and suggesting that the acceptance of scientific knowledge is straightforward

(McComas, 1998). Our goal is to portray the scientific method accurately as it is practiced in

diverse fields of science as a non-linear, self-reinforcing process.

2. Describe key aspects of the process of science. The panel will be asked to develop the key

characteristics of science identified by the Understanding Science Advisory Board into a full

set of curricular materials for undergraduate education.

3. Identify major scientific revolutions, important scientists, major scientific controversies,

and key technological breakthroughs within and across disciplines. Scientific advances are

not always gradual, and understanding the contribution of individuals and the nature of

scientific revolution is key to understanding the process of science.

Objective 2: Create a set of curricular modules and associated teaching materials to be

used in Process of Science courses and launch the materials on the Visionlearning website.

A major strength of this proposal is that it builds on an established national-scale project

for creating and disseminating undergraduate science teaching content. The Visionlearning

project has developed a teaching module design that has proven effective and is widely used by

instructors (described in detail later in this proposal). These modules consist of 3-4 page

explorations of a single concept that support classroom instruction. While specific module topics

will be outlined by our panel, a draft outline of possible module topics is included in Table 1

below to provide a sense of the nature of the materials we will create.

Table 1: Key Concepts and Possible Module Topics to be Developed in this ProjectKey Concept Specific Module TopicsScientific methods o Characteristics of non-linear scientific

methodso Variations in the scientific method between

disciplinesThe evolution of science through time o The development of scientific thinking

o The role of key scientists

9

The Process of Science Carpi

o Instrumentation and analytical advanceso What makes a scientific revolutiono What makes a scientific controversy

The practice of science o Experimentation o Analyzing data o Error, uncertainty, and statisticso Modification of theories

Communicating science: o Scientific writing and peer reviewo Science and policyo Science and lawo Science and belief systemso The language of science: the meaning of

terms like theory, data, etc.o Public perception of scienceo Scientific ethics: fraud and objectivity

Science today o Scientific research profileso Critical reading of the literature

Five key concepts are currently envisioned as making up this work, which will be later

refined by our panel of experts. 1) Scientific methods: will consist of two to three modules that

convey the scientific method as it is practiced - in a non-linear fashion that differs between

different disciplines. 2) The evolution of science through time: will consist of five modules that

describe the origin of scientific thinking, what can cause a scientific revolution or controversy,

and the role of key scientific figures and instrumentation discoveries in the advancement of

scientific thought. 3) The practice of science: will consist of four to five modules that describe

key aspects of scientific research as it is broadly defined across multiple disciplines, with

particularly detailed explanations of language where appropriate (such as the words “theory” and

“hypothesis”). 4) Communicating science: will consist of seven modules that describe the major

modes in which scientific theories and developments are communicated to other scientists and to

the public, and how they can affect policy and law. Within this context we plan to explicitly

address the difference between science and belief systems. Finally, 5) Science today: will consist

of a series of supplemental resources that will be linked to from the core modules. These

10

The Process of Science Carpi

resources will provide profiles of scientists and scientific research, and links to published

research papers to provide a more detailed view of the process of science.

As a proof-of-concept of this proposal, we have already developed and published three

modules on Visionlearning that focus on scientific process instead of content. Visualizing

Scientific Data: An Essential Component of Research, explores methods for the graphical

presentation of data and the importance of these presentation modes in data analysis.1 The

module presents examples of data from the fields of chemistry, earth science, and sociology and

explores the relative effectiveness of various data presentation methods. Additionally, the

module provides a data set on atmospheric CO2 concentrations from the Mauna Loa observatory

that the user can download and graph to experiment with presenting data in a useful manner.

A second module, Scientific Writing: Peer Review and Scientific Journals, describes the

process of peer-review and discusses the differences between popular magazines and scientific

journals.2 The module presents a generalized format of scientific articles and discusses tips for

reading and understanding scientific literature. To provide readers direct experience with the

scientific literature, we have negotiated a relationship with Science magazine, and the research

section of this module provides direct access to the research paper “Ivory-billed Woodpecker

(Campephilus principalis) Persists in Continental North America,” regardless of whether or not

the user has a subscription to the journal (Fitzpatrick et al., 2005). When accessing this article

through our site, teachers and students first pass through a page of targeted questions meant to

focus their reading and stimulate a follow-up discussion of the material in the classroom.

For the “critical reading” component of the “Science Today” concept listed in Table 1,

we plan to expand this relationship with Science and seek similar relationships with Nature,

Geology, and other journals in to provide students broad and seamless access to scientific

research directly in the classroom. This represents a significant benefit to students at smaller 1 http://www.visionlearning.com/library/module_viewer.php?mid=109&l=&c3=2 http://www.visionlearning.com/library/module_viewer.php?mid=123&l=&c3=

11

The Process of Science Carpi

schools and community colleges. Subscriptions to these journals average $1660 per year, and are

among the first expenses to be eliminated when budgets are cut (Willinsky, 2006). By connecting

individual journal articles to content, we will provide all students with guided exploration of the

current research.

A third module displays the type of supporting content that will be developed for the

“scientific research profiles” component of the “Science Today” concept listed in Table 1. Our

biology module Adaptation: The Case of Penguins discusses the scientific concept of organism

adaptation to the environment. The research section of this module links to a series of research

diary entries written by Ms. Iris Saxer, an NSF-funded penguin biologist.3 These diary entries

provide a candid and insightful view of the practice of scientific research. In the proposed grant

period, we will produce a series of profiles about practicing scientists and their work to give

students a more personal perspective on scientific research. These materials will be linked to

from research section of the proposed Process of Science teaching modules.

Objective 3: Evaluate these materials in a Process of Science course to be mounted at

CUNY, in a disciplinary science course at Stanford, and at four other institutions.

This project includes a three-phase evaluation program (outcomes and attainment

measures are further detailed in the evaluation table in the appendices). To assure high quality,

evaluations will be administered by a semi-independent Evaluation Coordinator, Ms. Sandra

Swenson, who will also serve on our expert panel. An independent External Evaluator will

conduct yearly formative evaluations to coincide with our annual reports to the Department of

Education, and will review and oversee all evaluations to assure that they are reliable, valid, and

unbiased. Evaluations will draw on the National Research Council’s report Evaluating and

Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics and

the assessment instruments referenced therein (NRC, 2003).

3 http://web.visionlearning.com/custom/diary/index.htm

12

The Process of Science Carpi

The first phase of our project evaluation will focus on assessing the impact of the

resources developed on student understanding of the nature of science at the collaborating

institutions. Evaluation instruments will be produced in consultation with our expert panel and

will be adapted from the published Views of Nature of Science (VNOS) Questionnaire, the

College Student Report from Carnegie Mellon’s Eberly Center for Teaching Excellence, and

other validated instruments (Ryder et al., 1999; NRC, 2002; Lederman et al., 2002). This project

will benefit significantly from Dr. Lederman’s direct participation on our expert panel. Table 2

contains four sample questions that might be used in our assessment tools to provide an example

of the instruments we will develop.

Table 2: Examples of possible pre- and post-assessment instrument questions

After scientists have developed a theory, does the theory ever change? If you believe theories change, why do we bother to teach scientific theories?

[For GES 1] Explain the concept of plate tectonics. How certain are scientists about plate tectonics? What specific evidence supports plate tectonic theory?

[For NSC107] Explain a scientific concept you recently became aware of outside of class.* Is there evidence to support this concept? If so, what specific evidence exists?

To what extent did† the work in this course emphasize the following activities? Please rank your answer from 1 to 5, 1 being very little emphasis, 5 being very much emphasis.a) Memorizing b) Analyzing c) Synthesizing information d) Making judgments e) Applying concepts in new situations

* The ability to cite a valid scientific concept may itself indicate an understanding of scientific process.† “Did” will be replaced by “do you expect that” in the pre-test.

John Jay College has committed to pilot the implementation of a set of the Process of

Science materials in the College’s interdisciplinary, non-majors laboratory science course

NSC107: An Introduction to Science in Society (see enclosed Letter of Support from the College

President). Once the core teaching materials are complete, two to three lecture sections (n ~ 30

per section) will be taught using these materials as their core readings. The adapted VNOS

instrument described above will be administered as a pre- and post-test in these sections as well

as in an equivalent number of control sections of the NSC107 course taught with traditional

13

The Process of Science Carpi

science content resources. Results will be analyzed by our Evaluation Coordinator, submitted to

our External Evaluator for validation, and presented to our expert panel for review.

Research has suggested that integrating the process of science into traditional disciplinary

courses is a powerful way of building student understanding of the nature of science (Dagher et

al, 2004). Non-project instructors are more likely to incorporate new process of science materials

gradually into their curriculum before committing to a major teaching revision. As such, we will

also evaluate the effectiveness of a subset of the materials we are developing when integrated

into the disciplinary course GES 1: Fundamentals of Geology, at Stanford University. Because

only one section of this course is taught in any given quarter, which precludes using the case-

control design, baseline data will be collected for two quarters prior to the materials

implementation as soon as a funding notification is made. The adapted VNOS will be

administered as a pre- and post test to assess student understanding of the process of science and

comprehension of core theories taught in the course.

In addition, all students enrolled in the sections of NSC107 and GES 1 in which process

of science modules are used will complete a short assessment paired with each new module, a

functionality already in place in the Visionlearning website. We will develop individual quizzes

that contain several questions designed to gauge student understanding of the content presented

in each module, and additional questions to assess opinions regarding the clarity and utility of

each module. This will provide us a measure of the relative strength and utility of each module

developed for the curriculum.

In the second phase of evaluation, we will expand to additional institutions by providing

a small stipend to the first four instructors on our site who teach a course using at least six of

these modules and who agree to participate in the evaluation. The adapted VNOS instruments

will be placed online to facilitate their use. Participating faculty will be asked to administer the

validated instrument as a pre- and post-test. Our Evaluation Coordinator will direct these

14

The Process of Science Carpi

assessments to test the progression in thinking about the nature of science that occurred as a

result of using the materials created in this project.

In the third phase of our evaluation, we will initiate an assessment of teachers to

determine the impact of these materials on teaching strategies. An online questionnaire will be

given to teachers, this questionnaire will be designed to:

1. Evaluate the impact of the developed materials on teachers’ understanding of the process and nature of science;

2. Study how teachers use these materials; and

3. Examine how teachers perceive that the resources affected their students’ understanding of the process of science.

This aspect of the evaluation will be used to guide development of support materials for

teachers that will describe how the Process of Science materials can be used in the classroom.

Visionlearning has a successful model for creating faculty “help” modules designed like our

teaching modules (www.visionlearning.com/teach_tech.php). We will draw on our collaboration

with SERC to develop similar modules into a series of gateway materials to integrate into their

“Starting Point” collection. For many college science faculty, the concept of science as a process

is so inherent to their thinking that they neglect to address it explicitly in the classroom. When

awakened to the learning gains that can be made, many of these same faculty adopt new

practices to engage their students. For this reason, a key component of this project is to give

faculty the tools they need to integrate these materials into their classrooms.

All evaluation results will be submitted to the peer-reviewed literature for publication and

dissemination. In addition, evaluation data will be reviewed by our expert panel to allow them to

prepare a series of recommended revisions to our curriculum that will maximize its effectiveness.

This iterative process of evaluation will assure that the products are successful and will be

widely adopted once they are fully disseminated.

15

The Process of Science Carpi

Objective 4: Disseminate the Process of Science teaching materials into a national program

for science education, and create a community of professionals to promote their use and

continued assessment.

This proposal represents a significant innovation in the teaching of introductory science.

As with any radical change in curriculum, this may result in barriers to its adoption as a

consequence of its modification in teaching content. To overcome these barriers, we will publish

these resources on the existing Visionlearning website in context of existing science content

modules. This will permit their gradual adoption into courses by allowing them to be paired with

science content modules in many different disciplinary course settings.

The Visionlearning project (http://www.visionlearning.com) has a significant existing

user base: with almost 4,000 instructors and more than 19,000 students currently registered with

the site, it receives over 6 million hits per month (Visionlearning, 2006). As of spring 2006, over

300 online classrooms were active on the site, representing 80 different institutions. Thus the

materials will be available to a wide audience as soon as they are published. To facilitate their

adoption and use, we plan to notify all instructors registered with Visionlearning via email when

modules are published (as we currently do for all new content modules). In the past, this has

resulted in near-immediate adoption of these materials.

This project has already negotiated collaborations with digital libraries and science

education centers around the country to help aid in its dissemination. In particular, our

collaboration with the Science Education Resource Center at Carleton College will significantly

aid in the project’s dissemination through their “Starting Point” collection

(http://serc.carleton.edu/introgeo/). Our resources are catalogued in the National STEM Digital

Library (NSDL), the Digital Library for Earth Systems Education (DLESE), and others.

Visionlearning has also negotiated a relationship with Kendall-Hunt Publishers to distribute an

inexpensive textbook based on the learning modules available on the site for teaching

16

The Process of Science Carpi

interdisciplinary science courses. This relationship will be extended in the proposed period to

distribute an affordable text based on the Process of Science materials.

The broadest dissemination of these materials is most likely to occur through the

development of a Community of Practice of science educators focused on teaching the process of

science (Schlager et al., 2000). To facilitate the development of such a community, we plan to

host workshops at major national science meetings (such as the American Geophysical Union,

American Chemical Society, etc.). During these workshops, the benefits of teaching the process

of science will be discussed, and the materials developed in this proposal will be presented as a

means of incorporating the process of science into the undergraduate classroom. Again, we will

draw on the experience of our collaborator Cathy Manduca of SERC in developing and running

similar workshops. In order to allow faculty to continue to stay in contact after these workshops

are complete, we plan to develop a series of teacher-to-teacher communication tools during this

project to facilitate the sharing of knowledge and practices among our users. These tools will

include direct email, teacher blogs, bulletin boards, and chat areas.

WHY VISIONLEARNING?

The Visionlearning project was funded by the National Science Foundation to create an

innovative system for providing interdisciplinary science content to college non-major courses.

This project has achieved or exceeded all of the objectives originally proposed (Boyd, 2006).

The current proposal seeks to expand this successful undergraduate teaching tool into an

innovative new area – the explicit teaching of the nature and process of science. Visionlearning

is particularly well-suited to the development and distribution of these materials.

The Visionlearning website hosts a library of more than 60 original teaching modules -

concise descriptions of scientific concepts that integrate the history of discovery and promote

interdisciplinary learning (http://www.visionlearning.com/library). These modules draw on the

17

The Process of Science Carpi

pedagogical research to take a hybrid approach to the 30-page chapter seen in textbooks and the

<1 page outline/summary approach often seen on the web. They provide a 3-4 page concise

description of a concept that promotes reading and is modeled after the scientific research paper.

Each module integrates a concept description with topical news stories, scientist biographies, and

ongoing research through a series of categorized hyperlinks. Additionally, modules include an

online assessment to provide students a measure of their understanding of the concept and to

provide feedback to the instructor. Modules are written by collaborating scientists and educators,

and undergo extensive review to assure accuracy and clarity before their publication.

The site incorporates a customizable content management system called MyClassroom

(http://www.visionlearning.com/myclassroom). This system contains standard features available

with course management software like WebCT™, including an online syllabus, announcement

system, and email system. Unlike other systems, MyClassroom focuses on content: allowing

instructors to select a custom set of modules for use in their classroom, annotate and customize

these modules to their needs, and create targeted online assessments to gauge student

comprehension. This system will be especially valuable in the proposed project as it will not only

allow faculty to select the specific order in which they choose to present the process of science

modules, but it will also allow them to directly annotate these modules and, of particular

significance, combine process modules with science content modules. Several researchers have

concluded that content about the nature of science should be integrated into disciplinary content,

allowing instructors to discuss the nature of scientific theories while students are learning about

the theory of plate tectonics, for example (Ryder et al., 1999; Brickhouse et al., 2000; Dagher et

al., 2004; Dagher and Boujaoude, 2004). Others have suggested that socioscientific issues (such

as global warming) are the ideal context in which to study the nature of science (Sadler et al.,

2004). Thus, by integrating the proposed modules with the existing site content, this project will

create an exceptionally useful resource for teaching the process of science.

18

The Process of Science Carpi

This project has a demonstrated history of rigorous and high-quality evaluation (Carpi,

2001; Carpi & Mikhailova, 2003). Early stages of the project evaluated prototype teaching

modules against traditional resources. The use of these tutorials reduced attrition in the target

course from over 11% to less than 5%, increased student-instructor communication, and

increased satisfactory performance (a final grade of C or better) from 68% to >80% (Carpi,

2001). In later tests of the Visionlearning modules, students in three sections of an undergraduate

non-major science course had an average score of 77 on a standard science comprehension

assessment, significantly higher (p<0.001) than the average of 62 for students in five other

sections of the course who used a traditional textbook (Carpi & Mikhailova, 2003). When asked

how they felt the Visionlearning materials presented science, almost 70% of students and 80% of

instructors responded that the materials conveyed science as a process of inquiry, as opposed to a

list of facts, and 63% of students stated that Visionlearning helped them to understand the

practice of scientific research (Boyd, 2006). As a result, 88% of instructors stated that they have

recommended our site to colleagues.

EXPECTED OUTCOMES

This is an innovative project that will undertake a major effort to catalyze reform in

undergraduate science instruction. Teaching the process of science is widely cited by experts as

leading to a significant increase in student understanding of the nature of the scientific process

(Reif, 1995; Schmidt, 1996; Dagher & Boujaoude, 1997; Dagher et al, 2004; Flick & Lederman,

2004). And yet, it is not widely taught in undergraduate introductory science courses as a result

of the lack of teaching support materials. This problem is exacerbated at community colleges and

small schools where the cost of supplementary teaching materials, innovative resources, and

subscriptions to research journals may be beyond their budget (Willinsky, 2006). Thus students

19

The Process of Science Carpi

are denied not only access to authentic research experiences, but access to an understanding of

the research process as well. We anticipate the following outcomes from this project:

By drawing on a collaboration of leading institutions and assembling a panel of experts in

the field, we will produce a curriculum and content support materials on the Process of

Science to enable the teaching of such courses at two- and four-year institutions.

By incorporating these resources into a widely used science teaching website, we will

facilitate instruction in the process of science as a stand-alone course and as a supplement

to existing disciplinary science courses.

By publishing our evaluation data in the peer-reviewed literature, we will add

significantly to the research base on the nature of science and student understanding.

By teaching the process of science, we will create a more informed public who can base

their decisions on understanding the complex socioscientific issues we face in the world.

By the end of the proposed three-year project period, we are confident that this work will be a

widely used and broadly cited success, further contributing to the reputation of the FIPSE

program in catalyzing educational innovation.

20

The Process of Science Carpi

REFERENCES CITEDAllen (2005, August 11) “Kansas Moves Closer to Intelligent Design Curriculum,” National

Public Radio, available at: http://www.npr.org/templates/story/story.php?storyId=4795205Allchin, D. (2003) “Scientific Myth-conceptions,” Sci. Ed. 87(3):329-351.Astin, A.W., Astin, H.S. (1992) Undergraduate Science Education: The Impact of Different

College Environments on the Educational Pipeline in the Sciences, Higher Education Research Institute, UCLA, Los Angeles.

Augustine, N. (1998) Essays on Science and Society: What We Don't Know Does Hurt Us. How Scientific Illiteracy Hobbles Society, Science 279 (5357): 1640 – 1641.

Bauer, H.H. (1992) Scientific Literacy and the Myth of the Scientific Method, University of Illinois Press, Urbana.

Bell, R.L., Blair, L.M., Crawford., B.A., and Lederman, N.G. (2003) “Just do it? Impact of a science apprenticeship program on high school students’ understandings of the nature of science and scientific inquiry,” J. Res. Sci. Teach. 40 (5): 487-509

Bennetta, W.J. (1993) “Reviewing a middle-school book in the Prentice Hall Science Series: The Nature of Science,” The Textbook Letter, http://www.textbookleague.org/35natur.htm.

Best, R.M., Rowe, M., Ozuru, Y., McNamara, D. (2005) “Deep-Level Comprehension of Science Texts: The Role of the Reader and the Text,” Top. Lang. Dis. 25 (1):65.

Blosser. P.E, Helgeson, S.L (1988) “Textbooks Contribute to Student Misunderstandings,” Invest. Sci. Ed. 14 (3).

Boyd, H.H (2006) “External Evaluation of Visionlearning Web Resources,” Final Report for Grant 0127246, prepared for the National Science Foundation.

Brickhouse, N., Dagher, Z., Letts, W., and Shipman, H. (2002) “Evidence and warrants for belief in a college astronomy class,” Science and Education 11: 573-588

Brickhouse, N., Dagher, Z., Letts, W., and Shipman, H. (2000) “Diversity of Students’ Views about Evidence, Theory, and the Interface between Science and Religion in an Astronomy course,” J. Res. Sci. Teach.37 (4): 340-362

Brush, S.G. (1974) “Should the History of Science be Rated X?” Science, 183:1164-1172.Carpi, A. (2001) “Improvements in Undergraduate Science Education Using Web-Based

Instructional Modules: The Natural Science Pages,” J. Chemical Education 78(12):1709.Carpi, A. (2003) “Designing Effective Instructional Web Pages,” in Simon, E. J. (ed.) Teachers

Using Technology: Practical Ideas from the Classroom, Kluwer Academic, New York.Carpi, A., Mikhailova, Y. (2003) “The Visionlearning Project: Evaluating the Design and

Effectiveness of Interdisciplinary Science Web Content,” J. College Science Teaching 23(1):12-15.

Dagher, Z.R. and Boujaoude, S. (1997) “Scientific views and religious beliefs of college students: The case of biological evolution,” J. Res. Sci. Teach.34: 429-445

Dagher, Z.R. and Boujaoude, S. (2004) “Students’ Perceptions of the Nature of Evolutionary Theory,” Wiley InterScience

Dagher, Z.R., Brickhouse, N., Shipman, H., and Letts., W. (2004) “How some college students represent their understanding of tscientific theories,” Int. Jnl of Sci. Ed., 26 (6): 735-755

Dye, L. (2000) Science Miseducation: Focusing on Facts Misses the Wonder of Exploration, ABC News, March 22, at: http://www.math.buffalo.edu/~pitman/science_education.html.

References pg. 1

The Process of Science Carpi

Farabee, M.J. (2002) Online Biology Book, Estrella Mountain Community College, http://www.emc.maricopa.edu/faculty/farabee/BIOBK/BioBookTOC.html.

Fairchild, M. (2004) “Ripoff 101: How the current practices of the textbook industry drive up the costs of college textbooks,” CALPIRG

Fitzpatrick et al. (2005) “Ivory-billed Woodpecker (Campephilus principalis) Persists in Continental North America ,” Science 308:1460-1462.

Flick, L.B., Ledereman, N.G. (2004) Scientific Inquiry and the Nature of Science: Implications for Teaching, Learning, and Teacher Education, Kluwer Academic, Dordrecht.

Gogolin, L., Swartz, F. (1992) “A Quantitative and Qualitative Inquiry into the Attitudes toward Science of Nonscience College Students,” J. Res. Sci. Teach. 29(5):487-504.

Hofstein, A., Lunetta, V.N. (2004) “The laboratory in science education: Foundations for the twenty-first century.” Sci. Ed. 88:28-54

Hurd, P. D. (1998) “Science Literacy: New Minds for a Changing World,” Sci. Ed. 82(3):407-416.

Lederman, N.G. (1992) “Students' and Teachers' Conceptions of the Nature of Science: A Review of the Research,” J. Res. Sci. Teach. 29(4):331-359.

Lederman, N.G. (1999) “Teachers' understanding of the nature of science and classroom practice: Factors that facilitate or impede the relationship,” J. Res. Sci. Teach. 36(8): 916 – 929.

Lederman, N.G., Abd-El-Khalick, F., Bell, R.L., and Schwartz, R.S. (2002) Views of Nature of Science Questionnaire: Toward Valid and Meaningful Assessment of Learners’ Conceptions of Nature of Science, Journal of Research in Science Teaching 39(6):497-521

McComas, W.F. (1998) “The Principal Elements of the Nature of Science: Dispelling the Myths of Science,” In W.F. McComas (ed.) The Nature of Science in Science Education: Rationales and Strategies. Kluwer (Springer) Academic Publishers (pg. 53-70).

Moulton, J. (1994) “How Do Teachers Use Textbooks and Other Printed Materials? A review of the literature,” Report to the Improving Education Quality Project, USAID, Arlington, VA, http://www.pitt.edu/~ginie/ieq/pdf/textbook.pdf.

Monk, M., Osborne, J. (1998) “Placing the History and Philosophy of Science on the Curriculum: A Model for the Development of Pedagogy,” Sci. Ed., 81(4): 405-424.

Moss, D. M., Abrams, E.D., and Robb, J., (2001) “Examining student conceptions of the nature of science,” Int. J. Sci. Educ. 23(8): 771-790

NAP (1998) Teaching About Evolution and the Nature of Science, National Academy Press, Washington.

NCSE (2005) “Kitzmiller v. Dover Area School District: Legal Documents, Trial Materials and Updates,” National Center for Science Education, available at: http://www2.ncseweb.org/wp/

NSB (2006) Science and Engineering Indicators 2006, National Science Board, National Science Foundation, Washington, DC, available at: http://www.nsf.gov/statistics/seind06/

NRC (2003) Evaluating and Improving Undergraduate Teaching in Science, Technology, Engineering, and Mathematics, National Academy Press, Washington, DC.

NSB (2004) An Emerging and Critical Problem of the Science and Engineering Labor Force. Companion to the Science and Engineering Indicators 2004, National Science Board, National Science Foundation, available at: http://www.nsf.gov/statistics/nsb0407/

NSF (1998) Scientific and Engineering Research Facilities at Colleges and Universities, National Science Foundation #01-301, http://www.nsf.gov/statistics/nsf01301/c7.htm.

References pg. 2

The Process of Science Carpi

NSF (2002) “Visionlearning: An Interdisciplinary Science Education Web Portal,” NSF, CCLI-EMD grant #0127246.

NSF (2006) Research Experiences for Undergraduates, National Science Foundation, http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5517&from=fund.

Nurrenbern, S.C., Pickering, M. (1987) “Concept Learning versus Problem Solving: Is There a Difference?” J. Chem. Ed. 64(6):508-10.

Oreskes, N. (2004) “Science and Public Policy: What’s Proof Got to Do with It?” Environ. Sci. & Policy 7:369-383.

Pearson (2005) The Nature of Science and Technology, Science Explorer Series, Pearson Education, Upper Saddle River.

Reif, F. (1995) “Understanding and teaching important scientific thought processes,” J. Sci. Ed. Tech. 4(4):261-282.

Rezba, R.J., Sprague, C.S., Fiel, R.L. (2003) Learning and Assessing Science Process Skills, Kendal-Hunt, Dubuque.

Rudolph, J.L., Stewart, J. (1998) “Evolution and the nature of science: On the historical discord and its implications for education,” J. Res. Sci. Teach. 35(10):1069-1089.

Ryder, J., Leach, J., and Driver, R. (1999) “Undergraduate Science Students’ Images of Science,” J. Res. Sci. Teach. 36(2):201-219.

Sadler, T.D., and Zeidler, D.L. (2004) “Sudent conceptualizations of the nature of science in response to a socioscientific issue,” Int. J. Sci. Educ. 26 (4): 387-409

Schlager, M.S., Fusco, J., Schank, P. (2000) “Evolution of an On-Line Education Community of Practice,” in K.A. Renniger & W. Shumar (Eds.) Building Virtual Communities: Learning and Change in Cyberspace. Cambridge University Press, NY.

Schmidt, K. (1996) “Creationists Evolve New Strategy,” Science, 273(5274):420-422.Seymour, E., Hewitt, N.M. (1997) Talking about Leaving : Why Undergraduates leave the

Sciences, Westview Press, Boulder, CO.Seymour, E., Hunter, A-B., Laursen, S.L., and Deantoni, T. (2004) “Establishing the Benefits of

Research Experiences for Undergradutes in the Sciences: First Findings from a Three-Year Study,” Wiley InterScience

Smith, G.A. and Pun, A. (2006) How Does Earth Work? Physical Geology and the Process of Science, Pearson Prentice Hall, Upper Saddle Rive, NJ.

Tobin, K. G. (1990). “Research on science laboratory activities. In pursuit of better questions and answers to improve learning.” School Science and Mathematics, 90, 403–418.

Visionlearning (2006) Site use statistics as compiled on July 01, 2006.Willinsky, J. (2006) The Access Principle: The Case for Open Access to Research and

Scholarship, The MIT Press, Cambridge, MA

References pg. 3