an inquiry into inquiry science teaching in …wp841zm8200/mfigueroa... · an inquiry into inquiry...
TRANSCRIPT
AN INQUIRY INTO INQUIRY SCIENCE TEACHING IN COLOMBIA
A DISSERTATION
SUBMITTED TO THE SCHOOL OF EDUCATION
AND THE COMMITTEE ON GRADUATE STUDIES
OF STANFORD UNIVERSITY
IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE DEGREE OF
DOCTOR OF PHILOSOPHY
Maria Figueroa
May 2011
http://creativecommons.org/licenses/by-nc/3.0/us/
This dissertation is online at: http://purl.stanford.edu/wp841zm8200
© 2011 by Maria Jose Figueroa Cahn Speyer. All Rights Reserved.
Re-distributed by Stanford University under license with the author.
This work is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.
ii
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Richard Shavelson, Primary Adviser
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Edward Haertel
I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.
Maria Araceli Ruiz-Primo
Approved for the Stanford University Committee on Graduate Studies.
Patricia J. Gumport, Vice Provost Graduate Education
This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.
iii
iv
ABSTRACT
Science education in different parts of the world has focused on teaching facts and
concepts transmitted by a teacher in a lecture-style approach. In contrast, some initiatives,
such as inquiry-based science teaching, use scientific inquiry—what scientists do to
generate new knowledge—as a basis for teaching science to students. That is, inquiry-
science teaching focuses on getting students to do what scientists do and how they learn
about natural phenomena. This is not to say inquiry-science teaching ignores facts and
concepts; it goes beyond transmission.
Inquiry-Based Science Education (IBSE) programs have been implemented
throughout the world, with the objective of improving science education. Even though
IBSE programs have received wide attention and substantial funding, the impact of this
approach on students’ learning is unclear. As a small step in clarifying the impact of
IBSE on students’ achievement, a quasi-experiment was conducted and reported in this
dissertation. More specifically, the study examines achievement differences between
inquiry science education and typical science education in five schools in Bogotá,
Colombia for overall achievement, achievement by types of knowledge (declarative,
procedural, mental model) and proximity of the assessment measure to the curriculum
(proximal and distal), and achievement as measured by performance assessments.
Inquiry-based science teaching takes many forms. Moreover, even though studies
compare inquiry teaching with other approaches, descriptions of this type of inquiry
teaching are vague and vary widely as to classroom implementation. Through a review of
the literature focused on empirical studies that compare inquiry teaching with other
v
approaches, I developed a framework used to define inquiry teaching and assess it using a
variety of measurement methods. The framework focuses on three basic elements: 1)
teachers, 2) students, and 3) curriculum materials, and how they tap into inquiry facets or
domains (conceptual, epistemic, and social). This framework guided my comparative
study of an IBSE program in Bogotá, Colombia with a more traditional approach
teaching the same unit, Human Body Systems.
Three types of assessments measured fifth grade students’ science achievement:
paper and pencil tests with (1) multiple-choice and (2) constructed-response questions
and (3) performance assessments. The multiple-choice questions were constructed to test
the different types of knowledge; test items were written proximal and distal to the
curriculum taught. Of the two performance assessments, one was content rich with a
direct link to the curriculum, while the other was content lean with an emphasis on
science process skills.
A total of 365 students from both IBSE and the Control group took the paper and
pencil tests and a sub-group of 140 students from both groups took the performance
assessments. Data were collected from 5 different schools in Bogotá, three that teach
science through an IBSE program and two that use a traditional approach. Data were
analyzed using a nested design (classrooms within schools within treatment condition)
and allowed for a comparison of the IBSE and the Control group science achievement.
The findings were mixed as to the impact of IBSE teaching on achievement.
While there was no statistically significant treatment effect as measured by the paper and
pencil test including the multiple-choice or constructed response questions, there was a
significant treatment effect in the content rich performance assessment as well as in the
vi
content lean. Moreover, even though there was no significant treatment effect on the
paper and pencil tests, IBSE students consistently outperformed the Control students on
all the different measures of science achievement. This result can be explained by the
nature of the nested design, large variation among schools (that served as a significant
part of the experimental error term) and consequently low statistical power.
The results, then, suggest that students who learn science through inquiry are able
to go beyond concepts and apply them in conducting science investigations. Additional
studies with more schools in order to better generalize than I could in this study as well as
to increase statistical power should be done in Colombia and other countries that are
reforming their curriculum through inquiry-based science teaching.
vii
ACKNOWLEDGEMENTS
I am greatly appreciative of all the teachers and students who were part in this
project, who were always willing to help and provide the time, logistics, and feedback
that made this study possible. I am also thankful to the school administrators from
Alianza Educativa, Colegio Las Mercedes, and Gimnasio Sabio Caldas for granting
permission to conduct all the assessments at different times.
My heartfelt gratitude also goes out to my advisor Richard Shavelson who is
responsible for the successful completion of my dissertation. His untiring and constant
effort, commitment, encouragement, guidance and unconditional support helped me
greatly in the understanding and writing of the dissertation.
A number of experts in different fields have offered useful advice and
encouragement. I want to particularly thank Edward Haertel and Maria Araceli Ruiz-
Primo for their valuable suggestions and comments at different stages of this process.
It is a pleasure to thank those who made this thesis possible. ICFES and Pequeños
Científicos provided support in the development of the assessments. The team from
Centro de Evaluación of Universidad de los Andes worked non-stop with me in data
collection. And last, my colleagues from Universidad de Los Andes provided valuable
feedback and guidance during this dissertation.
I offer special thanks to my office mates and friends at Stanford, Alice Fu and Jon
Shemwell. The whole PhD experience was a great ride with you along. Thanks for the
support, patience, and academic growth during these years. To Alice, thank you for the
Wednesday meetings and good vibes during this process.
viii
I am grateful to my family and friends, for their patience and constant words of
encouragement. I thank my mother Vivian for her support and admiration. I owe my
deepest gratitude to my son Emilio, who never complained and always understood the
importance of this work. Finally, it would have been next to impossible to write this
thesis without Camilo´s support, patience and love.
ix
TABLE OF CONTENTS
x
xi
LIST OF TABLES Number Title Page Table 2.1. Characteristics of the Facets of Inquiry that Students Show when
Learning Science through an Inquiry Approach (Adapted from Furtak & Siedel, 2008)
12
Table 3.1. Characterization of the Studies that Compare Inquiry with Other Teaching Approaches
18
Table 3.2. Conceptual Approach - Mapping Study’s Inquiry Conception onto the Inquiry Facets
21
Table 3.3. Research designs
24
Table 3.4. Critique and Drawbacks
25
Table 4.1. Examples Types of Assessments Used to Measure Science Achievement
28
Table 4.2. Outcomes used in Studies that Compare Inquiry with other Approaches
30
Table 4.3. Relationship between Types of Knowledge and Types of Assessments
34
Table 5.1. Distribution of Students in Public and Private Schools in Bogotá in 2009
39
Table 5.2. Managing Institutions of Concession Schools in Bogotá
41
Table 5.3. Schools Participating in the Studies
43
Table 5.4. Schools’ Level According to the Results from the 2010 ICFES Exit Exam.
48
Table 5.5. Schools’ Results in the Science Components of the ICFES Exam (2010).
48
Table 5.6. Schools’ Results in the 2009 SABER Exams.
49
Table 6.1. Teachers’ Classroom Practice as Evidenced from One Lesson
61
Table 6.2. Summary General Information About Each Teacher Based on Interviews
64
xii
Number Title Page Table 6.3. Characterization of What Students do in Class Based on Teacher
Responses to Adapted TIMSS Questionnaire
66
Table 6.4. Mapping Teachers and Students with the Inquiry Facets
67
Table 6.5. Student Participants in the Study by Group, School and Class (sample sizes).
68
Table 6.6. Assessments related with the Paper and Pencil Tests
73
Table 6.7. Composition of Items in the Human Body System Booklets
74
Table 6.8. Composition of Items in the Human Body Systems Mapped into the Facets of Inquiry
74
Table 6.9. Reliabilities of the Paper and Pencil assessments.
77
Table 6.10. Number of students who participated in the Performance Assessments.
79
Table 6.11. Reliabilities of the Performance Assessments
82
Table 7.1. Descriptive Statistics of the Results of the Pre and Post Multiple-Choice Tests
87
Table 7.2. Correlations among the Paper and Pencil Tests
88
Table 7.3. Adjusted Marginal Means of Results for Post-Equal-Pre
89
Table 7.4. Descriptive Statistics of the Results of the Full Post Paper and Pencil Test
89
Table 7.5. Results of the Nested ANCOVA with Constructed Response and Posttest as Dependent Variables
90
Table 7.6. Descriptive Statistics of the Overall Adjusted Means of Science Achievement Depending on the Type of Knowledge by Treatment Group
91
Table 7.7. Correlations of the Results of Science Achievement Depending on the Type of Knowledge
93
Table 7.8. Results of the Nested ANCOVA for Knowledge Types with Pretest as Covariate
93
xiii
Number Title Page Table 7.9. Descriptive Statistics of the Results of Science Achievement
Depending on the Proximity of the Items
94
Table 7.10. Correlations of the Results of Science Achievement Depending on Proximity
95
Table 7.11. Results of the Nested ANCOVA for Proximity with Pretest as Covariate
95
Table 7.12. Descriptive Statistics for the Performance Assessments.
96
Table 7.13. Results of the Nested ANOVA for Performance Assessments
98
Table 7.14. Correlations of the Types of Test
99
Table 7.15. Correlations of the Types of Knowledge and the Performance Assessments
100
Table 7.16. Correlations of the Types of Knowledge and the Performance Assessments
100
Table 7.17. Results of the Nested ANOVA for Types of Test
100
xiv
LIST OF FIGURES Number Title Page Figure 2.1. Continuum representing forms of science instruction (Furtak,
2006)
6
Figure 2.2. Triangle of Inquiry-Based Science Teaching
7
Figure 4.1. Schematic of a multilevel assessment of science achievement. Taken from: Ruiz-Primo et al., 2002, p. 372.
32
Figure 4.2. The relationships between different knowledge types. Source: Shavelson, Ruiz-Primo, Li & Ayala, 2003.
34
Figure 5.1. Distribution of students in public schools according to SES in Bogotá in 2009. Source: Secretaría de Educación del Distrito, 2009.
38
Figure 5.2. Map of schools participating in the studies (adapted from Alianza Educativa, 2008).
43
Figure 6.1. Process of teacher training in Pequeños Científicos. Taken from: Pequeños Científicos. 2004. Institutional Presentation.
54
Figure 6.2. A print-out of pages of the book: “Santillana Casa Ciencias Naturales 5”.
58
Figure 6.3. Crossword puzzle (Digestion in Humans) used in Control group.
59
Figure 6.4. Fill in the blank activity used in Control Groups teaching science.
60
Figure 6.5. Example of a change in a question after the input received in a Think-Aloud.
71
Figure 7.1. Effects size by types of knowledge
92
1
CHAPTER 1. INTRODUCTION
Science education in different parts of the world has focused on teaching facts and
concepts transmitted by a teacher in a lecture-style approach. In contrast, some initiatives,
such as inquiry-based science teaching, use scientific inquiry as a basis for teaching
science to students. Inquiry-Based Science Education (IBSE) programs have been
implemented in at least 30 countries around the world, with the objective of improving
science education (IAP Science Education Programme, 2006). Even though IBSE
programs have received wide attention and substantial funding, the impact of this
approach is unclear. This dissertation studies the impact, if any, inquiry-based science
teaching has on student learning in Colombia.
The Setting Inquiry-based science education programs have been implemented in Colombia
for 10 years by Pequeños Científicos, a program run by University of los Andes in
collaboration with schools and Maloka, the Bogotá Science Museum. Of all the schools
that implement Pequeños Científicos, more than 200 in the country, three schools in
Bogotá were selected for this study. The three Pequeños Científicos schools (hereafter
IBSE schools) and the two comparison schools (hereafter Control schools) come from
similar low socio-economic neighborhoods. The schools were selected using the
following criteria: all were concession schools, all came from similar socio-economic
backgrounds, all had similar results on national standardized tests, all agreed to
participate in this study, and all taught the same curriculum at approximately the same
2
time. Teachers in the three IBSE schools have received training by Pequeños Científicos
in how to teach science using inquiry. The Control-school teachers use a traditional
approach to science teaching, mainly focusing on content coming from a textbook.
As a small step in clarifying the impact of IBSE on students’ achievement, a
quasi-experiment was conducted and reported in this dissertation. More specifically, the
study examines achievement differences between inquiry science education and
traditional science education in five schools in Bogotá, Colombia, for overall
achievement, achievement by types of knowledge (declarative, procedural, mental model)
and proximity of the assessment measure to the curriculum (proximal and distal), and
achievement as measured by performance assessments.
The Problem
Several studies and meta-analyses have examined the impact of IBSE compared
to other science teaching approaches (Berg, Bergendahl, Lundberg, & Tibell, 2003;
Brederman, 1983; Chang & Mao, 1999; Furtak, Seidel & Iverson, 2009; Klahr & Nigam,
2004; Schneider, Krajcik, Marx, & Soloway, 2002; Tamir, Stavy, and Ratner, 1998,
Furtak, Seidel & Iverson, 2009; Minner, Levy & Century, 2010). The findings in the
literature are varied and sometimes contradictory. Some studies present evidence of the
positive impact of scientific inquiry instruction while others present counter evidence.
What appears through the fog as illuminated by meta-analyses is a slight advantage for
IBSE. This current research intends to help lift the fog a little bit.
The varied definitions of inquiry-based science teaching and the multiple methods
used for measuring students’ outcomes are probably two of the causes of the
inconsistencies of the results. Additionally, research done on the impact of scientific
3
inquiry has suffered from design flaws including the lack of a clear definition of the
meaning of inquiry, especially when dealing with achievement. Results from studies that
focus on measuring student science achievement are mixed, with some studies suggesting
positive impact of the methodology while other studies present no impact or negative
impact.
The present dissertation seeks to compare IBSE and Control students in science
achievement. This dissertation begins by presenting a working definition of what inquiry-
based science education is in order to establish the science-teaching framework for this
study. The dissertation continues with a presentation of several empirical studies that
compared inquiry-based teaching with other approaches. The next chapter presents
different components related to measuring student science achievement including ways in
which inquiry has been measured in empirical studies. The following chapter focuses on
a detailed description of the treatment and control groups and the procedures for
conducting the study. Finally, the next chapter reports and discusses findings and a
concluding chapter brings us back to the original question of the effects of IBSE and
possibilities for future research.
The contribution of this dissertation is to provide an in-depth analysis of the
impact of inquiry-based science teaching measured through different assessments that tap
into different types of knowledge. Since inquiry education not only guides students into
learning concepts, but also goes beyond facts, it may present further alternatives to
develop 21st Century skills in students.
4
CHAPTER 2. DEFINITION OF INQUIRY-BASED SCIENCE TEACHING
The term “scientific inquiry,” as applied to science education, has been used in a
variety of ways and contexts (Berg, Bergendahl, Lundberg, & Tibell, 2003; Chang &
Mao, 1999; Klahr & Nigam, 2004; Schneider, Krajcik, Marx, & Soloway, 2002; Tamir,
Stavy, and Ratner, 1998). The problem then, is that its definition is not universal and is
interpreted differently by different people. An example of this includes Chang & Mao’s
(1999) definition, that focuses on inquiry as hands-on activities, gathering and recording
data, interpreting data and their relationships, or Klahr & Nigam (2004), who define it as
experiences where students explore without any teacher guiding questions or direction.
Since there are so many conceptions and definitions of inquiry in science
education, it is important for me to present my perspective of what “inquiry-based
science teaching” means in science education. The development of this conceptual
framework is based on four sources: 1) a definition by the National Research Council that
focuses on students engaging in science activities, 2) a definition by National Science
Foundation that focuses on teachers, 3) a definition of guided scientific inquiry provided
by Furtak (2006), and 4) Duschl’s (2003) facets of scientific inquiry.
The first source informing my definition focuses on students doing activities like
those that scientists engage in and is offered by the National Research Council in the
United States:
Scientific inquiry refers to the diverse ways in which scientists study the natural world and propose explanations based on the evidence derived from their work. Inquiry also refers to the activities of students in which they develop knowledge and understanding of scientific ideas, as well as an understanding of how scientists study the natural world.
- NRC, 1996. p. 23
5
In this case, NRC mentions that inquiry is an approach that scientists use to study
the natural world. It also refers to activities that students engage in, in order to learn and
understand scientific concepts. The combination between what scientists do and what
students do are central to studying inquiry.
The second source, presented by the National Science Foundation, focuses on
what teachers do to engage students in science “inquiry”:
Inquiry teaching leads students to build their understanding of fundamental scientific ideas through direct experience with materials, by consulting books, other resources, and experts, and through argument and debate among themselves. All this takes place under the leadership of the classroom teacher.
-NSF, 1997. p. 7
The National Science Foundation, go a step further with their explanation of what
students do in inquiry instruction. It talks, among other things, about students’ “direct
experience” and “argument and debate” while guided by a classroom teacher. The
leadership of the classroom teacher in setting up the learning process is one of the key
issues of this definition of inquiry.
A third source for defining inquiry-based science education deepens the teacher’s
role in inquiry teaching, and places teacher-student activities on an “inquiry continuum”
ranging from traditional direct instruction to wide-open scientific inquiry (Furtak, 2006).
Most science inquiry instruction commonly falls between these two extremes in a
condition called “Guided Scientific Inquiry” (Furtak, 2009 ) (Figure 2.1), where students
are guided through investigative approaches toward answers that are known by the
teachers. In many cases, curriculum and materials, such as teaching modules and
textbooks, serve to define the location of teaching learning in the inquiry continuum.
6
Traditional,
Direct Instruction
Guided Scientific
Inquiry
Open-Ended Scientific
Inquiry (Discovery) Figure 2.1. Continuum representing forms of science instruction (Furtak, 2006)
Facets in Inquiry-Based Science Teaching The fourth source used for developing my framework of inquiry-based science
teaching comes from Duschl (2003). The author presented a framework of science
inquiry teaching based on previous work in the areas of social and cognitive psychology,
history and philosophy of science, and educational research. He proposed three domains
that are part of inquiry-based science teaching: Epistemic, Conceptual, and Social.
Duschl (2003, p. 42) refers to the epistemic facet as “frameworks used when developing
and evaluating scientific knowledge”, the conceptual facet as “structures and cognitive
processes used when reasoning scientifically”, and the social facet as “the processes and
forums that shape how knowledge is communicated, represented, argued, and debated”.
Ruiz-Primo and Furtak (2004) expanded on these three domains and created four
overlapping facets, dividing Duschl’s epistemic domain into methodological and
epistemic. This division was suggested since it was necessary to differentiate the
methodology required in conducting investigations from the nature of scientific
knowledge.
In Taking Science to School, Duschl et al. (2007) presented four strands of
scientific proficiency where inquiry teaching plays an important role. These four strands
are similar to Furtak’s (2006) and Duschl’s (2003) facets of inquiry: (a) Know, use, and
interpret scientific explanations of the natural world; (b) Generate and evaluate scientific
evidence and explanations; (c) Understand the nature and development of scientific
knowledge; and (d) Participate productively in scientific practices and discourse.
7
Figure 2.2 presents my conceptualization of inquiry-based science teaching based
on the five sources mentioned above. The framework presents three vertices represented
by teachers, students, and curriculum and materials. The interior of the triangle presents
the three facets or domains that inquiry-based science teaching needs to tap into
(conceptual, epistemic, and social) (Duschl, 2003). My framework for inquiry-based
science education is therefore defined by the teacher in his or her actions in the
classroom, including questions asked, activities used, and the set-up of the learning
process, by students in the way they develop knowledge and understanding, and by the
curriculum and materials used. In some cases, teachers follow the textbook and materials
very closely. The curriculum and materials, then, lead teachers to base his or her class on
factual and conceptual material and simple procedures (e.g., calculations).
Figure 2.2. Triangle of Inquiry-Based Science Teaching
In IBSE classes, in addition, teachers can set up an inquiry environment where
students are the ones doing the activities and building their scientific knowledge (Ruiz-
Primo, personal communication, Dec, 2009). Since the three domains that inquiry
Facets Conceptual Epistemic
Social (Duschl, 2003)
Teacher
Student Curriculum/
materials
8
teaching taps into are the center on inquiry teaching, a more detailed description of each
facet is presented below.
Conceptual Facet
Teaching concepts is an important part of inquiry-based science teaching.
Students learn concepts through direct instruction and discovery. Furtak and Seidel
(2008) describe the conceptual facet as the facts, theories, and principles in science.
Concepts are connected both to students’ previous science knowledge and to the
development of complex understandings.
In inquiry science, the development of these concepts is important and needs to
take into account students’ previous knowledge (Duschl, 2003). Inquiry-based science
teaching requires knowledge integration from different areas of science and ways of
reasoning (Duschl, 2003). Inquiry-based science teaching aims to enable students to
extend and refine their understanding of the natural world. Therefore, science education
should focus on identifying important scientific ideas that not only help students
understand the essence of a discipline, but also the manner in which those ideas are
discovered (American Association for the Advancement of Science, 1990; NRC, 1996;
Furtak & Seidel, 2008).
Duschl et al. (2007) present a strand called “Know, use, and interpret scientific
explanations of the natural world.” This strand is directly connected to my definition of
the conceptual facet. This strand is defined as students acquiring facts and conceptual
structures that incorporate scientific knowledge, and using those facts productively to
understand the natural world. In my framework of inquiry-based science teaching,
9
students are led to construct knowledge in a similar way as scientists do, in addition to a
guided process that allows them to discover, think, and individually use those concepts.
Therefore, my definition of the conceptual facet takes into consideration
scientifically based facts, theories, and ideas, that students use to reason about and
understand the natural world. In this process, students’ previous knowledge is taken into
account and is either built upon or changed through the scientific inquiry process.
The conceptual facet is not perceived as the main one in inquiry education.
However, in order to learn science, students need to know specific concepts related with
the scientific topics they are learning and reason about them. In spite of perception, then,
this facet is a major component of science teaching including inquiry teaching.
Li, Ruiz-Primo & Shavelson (2006; Li & Shavelson, 2004) categorize cognitive
outcomes into four types of knowledge: Declarative, knowing that; procedural, knowing
how; schematic, knowing why; and strategic: knowing about knowing in a domain. In the
case of the conceptual facet of inquiry, most cognitive outcomes can be mapped with
declarative knowledge. In addition, there may be several outcomes that go beyond the
facts where students apply their conceptual knowledge, and can be mapped with
schematic knowledge.
Epistemic Facet
The second facet of inquiry-based science teaching is the epistemic one. Duschl
(2003, p. 42) describes this domain as “frameworks used when developing and evaluating
scientific knowledge”. Ruiz-Primo and Furtak (2004) divided Duschl’s epistemic domain
into a methodological and a epistemological facet. The former refers to a series of non-
linear procedures that include experimental design, executing procedures using
10
instruments and data representation. The latter facet refers to the understanding of where
scientific theories and principles come from. This epistemological facet involves
students’ understanding of the nature of science. It is also possible to find the epistemic
facet in Duschl et al. (2007) divided into two strands: “Understand the nature and
development of scientific knowledge” and “Generate and evaluate scientific
explanations”. The former resembles Ruiz-Primo and Furtak’s epistemological facet,
while the later is similar to the methodological facet they propose.
My definition of the epistemic facet focuses on how science can generate
knowledge and its limitations, and is divided into two sub-facets: nature of scientific
knowledge and procedural.
Nature of scientific knowledge. This sub-facet refers to students’ understanding of the
nature of scientific knowledge, its limitations, and scopes. It also refers to students’
understanding of science as a discipline and as a way of knowing (Duschl et al. 2007).
This is a desired outcome in inquiry-based science teaching, and needs to be part of the
teaching process. When students are able to understand how science occurs, they get
closer to thinking like scientists. Inquiry teaching actively leads students in this process.
Procedural. This sub-facet is related to procedures where students design and analyze
investigations constructing and using scientific evidence. It involves planning and
carrying out investigations, reviewing what is already known based on experimental
evidence, using tools to gather, analyze, and interpret data, and proposing answers,
11
explanations, and predictions (NRC, 1996). Through these processes, students are able to
construct knowledge and to understand how scientists warrant knowledge.
This sub-facet was first identified as a separate one by Ruiz-Primo and Furtak
(2004) and described as a methodological one. Their work was based on what Bruner
(1961) called the “heuristics of discovery”, including designing experiments, executing
procedures, recording data, and constructing graphs.
I believe that the procedural and the nature of scientific knowledge sub-facets are
the way through which students start constructing science theory and knowledge. Even
though it is possible to have this epistemic facet in other types of science instruction, it is
an essential component of inquiry. Inquiry-based science teaching may be defined or
thought of as only hands-on experiments, but there is much more to it than this. In order
to go beyond the hands-on component, it is necessary to lead students into a thinking
process that allows them to learn about concepts, procedures, and the nature of scientific
knowledge.
The Epistemic facet can be mapped into the declarative, procedural, schematic ,
and strategic types of knowledge presented by Li, Ruiz-Primo & Shavelson’s (2006). The
declarative, schematic, and strategic types of knowledge include aspects of the Nature of
Scientific sub-facet while the procedural type of knowledge includes aspects of the
procedural sub-facet of inquiry-based science education.
Social Facet
The third facet of inquiry-based science teaching is social. Duschl (2003) and
Furtak (2006) describe this domain as the social processes that shape how knowledge is
communicated, represented, argued, and debated. It involves social interactions among
12
students and is an important element in reasoning and generating scientific knowledge.
Inquiry-based science teaching fosters students communicating with both teachers and
classmates in order to construct their scientific language and thinking process.
My definition of the social facet refers to the communicative processes that are
required to build scientific knowledge. For my definition, the social facet takes into
account that scientific knowledge is constructed in collaborative groups and it is built on
previous work, and how scientists work and interact as part of a community of practice
(Duschl, 2003).
Examples of Actions that Characterize Inquiry-Based Science Teaching
Table 2.1 Characteristics of the Facets of Inquiry that Students Show when Learning Science through an Inquiry Approach (Adapted from Furtak & Siedel, 2008) Facet of Inquiry Numbera Actions students focus on
1.1 Ask Scientifically-Oriented Questions
1.2 Design Experiments
1.3 Execute Scientific Procedures
1.4 Gather and interpret data
1.5 Represent Data
1.6 Do Hands-On Activities
Procedural
1.7 Formulate hypothesis
1.8 Reflect on Nature of Science based on direct experiences
1.9 Draw explanations based on evidence
1.10 Generate and revise theories
Nature of scientific
knowledge
1.11 Apply scientific knowledge
1.12 Draw on/Connect to prior knowledge
1.13 Explain ideas
Conceptual
1.14 Organize concepts and principles
13
1.15 Participate in class discussions
1.16 Argue/debate scientific ideas
1.17 Developing communication skills
Social
1.18 Cooperative learning
a The numbers in this table will be used in Chapter 3 to map the different facets to empirical studies that have compared inquiry with other approaches.
Table 2.1 shows a series of examples of actions that students do that teachers
intentionally enable through the design of inquiry learning environments. These actions
can be linked directly to the Facets of Inquiry described above. This table presents
examples to clarify what types of action can be categorized as inquiry teaching in the
classroom. The examples will hopefully close the gap between my theoretical definition
of inquiry-based science teaching, and what it is expected that students do in an inquiry
classroom.
This table provides examples of actions that can be linked to specific facets. The
elements of the table are applied in the next chapter when evaluating empirical studies
and presenting my own study. By providing these concrete examples, the facets of
inquiry-based science teaching can be made visible, as can the characterization of
teaching practices.
Concluding Remarks about My Definition of Inquiry-Based Science Teaching
My definition of inquiry-based science teaching involves three approaches, 1)
students engaging in science activities, 2) teachers and their instruction, and 3) teachers
and students in their interaction with curriculum materials. Inquiry teaching focuses on
the domains presented by three facets of inquiry: conceptual, epistemic, and social. The
three approaches include the way students engage in scientific activities similar to what a
scientist actually does, how teachers approach their students to foster the actions and
14
thinking of a scientist, and how curricular materials can create a learning environment to
foster interactions between students and teachers to build conceptual, epistemic and
social knowledge and skills. Additionally, the facets allow a further clarification of what
inquiry looks like in the classroom to foster the type of thinking scientists engage in.
The framework presented in this chapter allows me to analyze empirical studies
that have compared inquiry teaching with other approaches, and sets the stage for my
own study, that compares inquiry-based science teaching with traditional approaches. The
use of this framework provides a clear definition of what I refer to by inquiry-based
science teaching.
15
CHAPTER 3. HOW INQUIRY HAS BEEN STUDIED
Even though the framework described above defines inquiry-based science
teaching as comprised of facets or domains, very few studies that compare scientific
inquiry teaching with other teaching methods incorporate this perspective in their
descriptions of inquiry. When I mapped studies to the facets, most of them relate
scientific inquiry with an epistemic facet (Von Secker and Lissitz, 2009; Berg, et.al,
2003; Klahr & Nigam, 2004, Geier, et.al. 2008; Tamir et.al., 1998, Chang & Mao, 1999;
Schneider et.al., 2002; Pine, Aschbacher, Foley, Jones, Kyle, McPhee, Phelps, and Roth,
2006). A few of studies present the epistemic and the conceptual facet in their definition
of science inquiry teaching (Geier, et.al. 2008; Tamir et.al., 1998). However, only a small
group of studies have presented inquiry teaching as the combination of all three facets
(epistemic, conceptual, and social) (Chang & Mao, 1999; Schneider et.al., 2002; Pine,
et.al., 2006). Empirical studies have different definitions of inquiry-based science
teaching, use a variety of research designs, and produce diverse conclusions when
comparing inquiry with other teaching approaches. This chapter provides an analysis of
the empirical studies, identifying drawbacks and lessons learnt that were considered in
my study.
When analyzing the studies, I expected that the greater the number of (up to 3)
facets involved in inquiry science teaching approaches, the greater that teaching impact
would be on students’ science achievement. However, this was not the case (e.g., Pine et
al, Chang & Mao); these studies only showed partial, positive effects of inquiry teaching.
In the studies reviewed, the inclusion of different facets in the treatment, then, did not
16
guarantee measurable differences on students’ learning. But I still think that in order to
have the best test of the science-inquiry teaching hypothesis, it is necessary to have all
three facets in the treatment as well as measured as outcomes. Additionally, none of the
studies reviewed went beyond fairly superficial descriptions of treatments—be they
inquiry or the comparison treatment—and none corroborated these descriptions with
direct evidence from classrooms, where teachers might be applying teaching strategies
and approaches different from the descriptions (cf. Furtak, Shavelson, Shemwell &
Figueroa, in press).
Operational Definitions of Inquiry-Based Science Teaching in Studies
Even though the studies reviewed had a similar purpose, that is, testing the
hypothesis of the superiority of science-inquiry teaching compared to the “standard”
(variously defined) method, they varied in their estimates of the impact of science inquiry
on students’ achievement. Some studies found inquiry to be an effective strategy (e.g.,
Tamir et al., Berg et al., Geier et al.), while others found inquiry to have a partial effect
(Chang & Mao, Schneider et al.) or no impact (Pine et al.). In the most recent meta-
analysis to date, Furtak et. al. (2009) compared nine studies (some included in this
analysis) in a meta-analysis, where she was able to select the studies based on a
framework that included her four facets (conceptual, procedural, epistemic, and social).
Only studies that had a design in which a causal interpretation of the impact of treatments
was warranted were selected (randomized, control experiments or quasi experiments with
a pre-post two-group design, and a cognitive outcome measure). The study reported an
17
average effect size of 0.921 when comparing an inquiry-based science teaching approach
with other teaching approaches. This effect size is considerable taking into account the
different results that are presented previously. Given this analysis, it seems that inquiry-
based science teaching has a large effect when compared to other types of teaching.
However, since the definitions of inquiry-based science teaching differ from study to
study, the results need to be analyzed with more detail.
There are several possible explanations for these inconsistent results. The studies
focused on a wide range of research questions and outcomes (e.g., achievement,
motivation), lacked a unified definition of science inquiry teaching, employed limited
research designs, and a deployed instruments of varied technical quality (reliability and
validity). In addition, the studies varied in science topics taught and grade levels. All this
variation makes it difficult to compare the impact of inquiry teaching reported in the
studies. Additionally, the methodological approaches in the treatments used generated
several threats to validity. In some of the studies including Chang and Mao (1999) and
Tamir et al. (1998), students in the treatment condition were explicitly taught the skills
that were going to be tested; not so in the comparison condition. In other cases,
participants were selected either by the researchers or by school administrators.
Table 3.1 presents a summary of the research questions or purposes presented in
the empirical studies under review here, as well as their content areas, and the grade level
in which the studies took place. The main goal in selecting these studies was to find
research that provided examples of what has been done regarding the impact of inquiry-
based science instruction on students’ outcomes, and that showed designs and
1 The standard deviation for the mean effect size is 1.02. This meta analysis only included six papers and nine studies.
18
methodologies similar to the ones I might use in my own study. Comparative studies
instead of descriptive ones were selected. All of the chosen studies had a group taught
with an inquiry approach and a comparison group that was taught with another approach.
Additionally, all studies used at least one cognitive outcome to measure students’
performance. Also, studies selected for this review were in the science domain,
measuring students’ outcomes in different topics such as Earth Science, Physics,
Chemistry, and Biology. Finally, all the studies were conducted with K-13 students, age
groups that include the grade levels of my study’s participants.
Table 3.1 Characterization of the Studies that Compare Inquiry with Other Teaching Approaches a
Research Study Questions/Purposes
Content Domain Grade Level
Tamir et al. (1998)
E: Evaluate the effect of different instructional strategies, one that emphasized inquiry and another that did not emphasize inquiry, on students’ performance in solving novel problems in the laboratory.
E: Chemistry, physics, or biology
E: 12th graders
Chang & Mao (1999)
E: “Investigate the comparative efficiency between inquiry-group instruction and traditional teaching methods in terms of their effects on student learning of science content and on student attitudes towards the subject matter…”
E: Earth Science E: Junior high
Von Secker & Lissitz (1999)
E: “Provide a baseline evaluation of whether teachers’ decisions to implement the specific instructional emphases recommended in the National Science Education Standards are associated with science achievement and equity.”
E: Chemistry, physics, earth science or biology
E: 10th graders
Schneider et al. (2002)
E: “Whether students in an inquiry-based science curriculum would perform as well as students nationally on achievement test items.”
E: Chemistry, physics, earth science or biology
E: 10th, 11th and 12th graders
Berg et al. (2003)
E: “Will an expository versus open-inquiry version of the same experiment have different outcomes for our students?”
E: Chemistry E: 1st year college students
Klahr & Nigam (2004)
I: Compare the relative effectiveness of discovery learning and direct instruction and test children’s ability to transfer what they have learned once they have achieved mastery.
E: Physics, (balls and ramps)
E: 3rd and 4th graders
19
Pine at al. (2006)
I: Compare the performance of students taught in classrooms with hands-on curricula to students taught in classrooms with textbook curricula.
E: Physical, biological, and earth science
E: 5th graders
Geier et al. (2008)
E: “Whether urban student participation in project-based inquiry science curricula leads to demonstrably higher student achievement on state wide assessments over and above general district-wide efforts at reform.”
I: Chemistry, biology (ecology) and physics.
E: 7th and 8th graders
a E: Take explicitly from the study; I: Taken implicitly from the study.
As mentioned above, the studies generally lacked an explicit definition of both the
science inquiry treatment and the alternative treatment. Even though the studies presented
an overview of their treatment, none provided a formal and explicit definition as to what
each study meant by inquiry teaching. In some cases, this lack of definition is even
reflected by the fact the word “inquiry” itself is used as part of the definition of the
treatment. Studies need to operationalize their treatments in carrying out the study and
reporting results in order to make evaluations of the impact of such type of teaching
possible and comparable (Furtak et al., in press). Additionally, the instructional methods
for the Control groups were not well defined, and in some cases were defined as a “lack
of treatment”. In those cases researchers used samples of students not participating in the
treatment, which makes it difficult to interpret the differences between the inquiry and
“other” treatment(s) due to selection bias.
The framework that defines inquiry-based science teaching in this study is based
on facets. Even though the studies provided limited definitional information, I was able to
map those studies on to the three facets of my framework, to a greater or lesser extent.
While only one study had one of the facets of inquiry-based science teaching, (Klahr &
Nigam’s), others included two facets or sub-facets, (e.g., Von Secker & Lissitz and Berg
et al.), and other studies incorporated three or more facets or sub-facets of inquiry
20
teaching, (e.g., Schneider et al., Pine et al., Geier et al.). Table 3.2 summarizes the nature
of inquiry teaching and the alternative approach following the facets in my framework
(see Chapter 2).
21
Table 3.2 Conceptual Approach - Mapping Study’s Inquiry Conception onto the Inquiry Facets a, b, c
Mapping to Facets and Sub-facets of Inquiry Study Alternative Approach Conception of Inquiry
Procedural NOSK Conceptual Social
Tamir et al. (1998)
I: Curriculum that did not emphasize inquiry: Students who specialized in a physics and/or chemistry curriculum that did not emphasize inquiry.
I: Inquiry oriented curriculum : Group 1. Observation, experimental design, communication and manipulation. Group 2: Same as group 1 with an addition of explicit instruction in the concepts underlying inquiry. * Only applies for group 2.
E: 1.2 Designing
experiments.
* I: 1.8 Reflecting on
nature of science
I: 1.14 Organizing
concepts and principles.
Chang & Mao (1999)
E: Traditional instruction: Lectures given by teachers, use of textbooks and other materials, and clear explanation of important concepts to students. Occasional demonstrations were also included.
E: Inquiry-group instruction: Cooperative learning. I: Hands-on activities, gathering and recording data, interpreting data and their relationships.
E: 1.6 Carrying out
hands-on activities
I: 1.9 Drawing
explanations based on evidence
I: 1.13 Explaining
ideas/ a mental models
E: 1.18 Cooperative
learning
Von Secker & Lissitz (1999)
E: Teacher-centered instruction: Definition not provided.
E: Student-centered instruction: Critical thinking, opportunities for laboratory inquiry, and less teacher-directed instruction
I: 1.6 Carrying out
hands-on activities
I: 1.11 Applying scientific
knowledge
Schneider et al. (2002)
E: National Sample Subgroup: Definition not provided.
E: Project-Based Science (PBS): Collaboration and communication, integrated with computer technology. Gathering information, analyzing data, expressing results, creating scientific models, writing reports.
E: 1.4 Gathering and
interpreting data
I: 1.10 Generating and revising
theories
E: 1.14 Organizing
concepts and principles
I: 1.17 Developing communica-
tion skills
Berg et al. (2003)
E: Expository instruction: Laboratory instructors describe the entire experiment in detail to students.
I: Open-inquiry: Students formulate a hypothesis, propose how to test it, and plan, perform, evaluate and discuss their experiments.
E: 1.7 Formulating hypothesis
I: 1.16 Arguing/debating scientific
ideas. a, E: Explicit; I: Implicit b NOSK: Nature of Scientific Knowledge c If information about the relationship of the treatments to facets is not present in the study then the cell is empty. However, if I am not able to map the information to a facet, then I will indicate that this information is Not Available (NA).
22
Table 3.2 Continues Mapping to Facets of Inquiry
Study Alternative Approach Conception of Inquiry Procedural NOSK Conceptual Social
Klahr & Nigam Option 1 (2004)
E: Direct instruction: where objectives, materials, examples and explanations on how to control variables were provided by the teacher.
E: Discovery-learning: where students explored without any teacher guiding questions or direction.
I: 1.6 Carrying out hands-on
activities
Pine at al. (2006)
E: Text-based curriculum: Textbooks as instruction tools. Multitude of short, fact-based expositions.
E: Hands-on curriculum: Hands-on activities.
E: 1.6 Carrying out
hands-on activities
I: 1.9 Drawing
explanations based on evidence
I: 1.14 Organizing concepts and principles
I: 1.17 Developing communi-cation skills
Geier et al. (2008)
E: Detroit Public Schools: A combined pool of students receiving no intervention or other interventions (different than the inquiry-based science curriculum) in science.
E: Inquiry-based science curriculum (LeTUS curriculum): Inquiry investigations contextualized by driving questions. Supported by technology.
E: 1.4 Gathering and
interpreting data
I: 1.9 Drawing
explanations based on evidence
E: 1.14 Organizing concepts and principles
a, E: Explicit; I: Implicit b NOSK: Nature of Scientific Knowledge c If information about the relationship of the treatments to facets is not present in the study then the cell is empty. However, if I am not able to map the information to a facet, then I will indicate that this information is Not Available (NA).
23
All the studies had in their conception of inquiry information that mapped onto
the procedural sub-facet of inquiry. It seems, then, this sub-facet serves as a defining
characteristic for inquiry teaching in these studies. Unfortunately the studies did not
provide sufficient information for me to characterize their approach, if at all, to the nature
of scientific knowledge sub-facet. So, it is possible to infer that the emphasis of inquiry is
still directed more towards methods and less towards the thinking processes and
reflections of the nature of scientific knowledge.
Research Designs True experimental designs with randomization and appropriate control group that
compare inquiry teaching with other approaches are rare in educational research. This
may be because of the difficulty of conducting randomized studies with teachers and
students in a school setting. Among the several studies reviewed, only one included a
true experimental design (Klahr & Nigam, 2006). Other studies used quasi-experimental
designs, where there is an appropriate control group and extensive pretest data but no
randomization. These studies can also be used to examine causal impact if carefully
carried out (Pine et. al. 2006). There were also some “pre-experimental” designs where
inquiry was compared with another treatment there was neither randomization nor pretest
controls. Table 3.3 presents a summary of the research designs used in the empirical
studies analyzed.
24
Table 3.3 Research designsa, b
Study Experimental Quasi-experimental Ex-post facto
Tamir et al. (1998) I: X1 O1 O2
X2 O1 O2 O1 O2
Chang & Mao (1999)
E: X O1 O2 O1 O2
Von Secker & Lissitz (1999)
I: X1 O X2 O
Schneider et al. (2002)
I: X O O
Berg et al. (2003) I: X O1 O2
O1 O2
Klahr & Nigam (2004)
I: R X O1 O2 R O1 O2
Pine at al. (2006)
I: X O1 O2 O1 O2
Geier et al. (2008) I c: X O
O
a E: Explicit; I: Implicit b Following Campbell and Stanley (1963): X=treatment, Blank=control, O=observation (measurement) c This design was replicated one year later.
Drawbacks and Lessons Learnt of the Studies Table 3.4 shows the main drawbacks of the studies producing a number of lessons
learnt. None of the reviewed studies corroborated treatment labels with observable
classroom practice; teachers could have applied different teaching strategies and
approaches than those labeled. Only one was a true experiment; others were pre-
25
experimental without adequate controls for selection bias. Others used instrumentation
that favored the inquiry treatment.
Finally, there might be different outcomes depending on the facets of knowledge
taught in the treatment. It seems that if students are taught a conceptual facet through
inquiry, they perform better in the conceptual component on a measure of that facet than
does the comparison group (e.g., Chang & Mao, 1999; Schneider et al. 2002, Geier et al.
2008). In contrast, if these same students are taught hands-on activities (procedural facet)
they do not necessarily do better in hands-on (procedural) assessments than students
taught with an alternative approach (Pine et al. 2006).
Table 3.4 Critique and Drawbacksa
Study Critique and Drawbacks Lessons Learnt
Tamir et al. (1998)
(-) Lack of information regarding how both curricula were taught in each class. (-) Lack of detail of the treatment. (-) Difference between students in groups (one group had students specializing in physics and the other one had students specializing in biology). (+) Present the tasks in the appendix.
• Provide the tasks in their entirety for readers.
• As expected, there was greater impact because of the proximity of the outcome as well as the alignment between the treatment and the test.
• Avoid selection bias.
Chang & Mao (1999)
(-) Lack of information regarding how both curricula were taught in each class. (-) Advantage of treatment group because of alignment of treatment with outcomes. (-) The cooperative learning setting used in the treatment might have had an influence on students’ answers to the attitude questionnaire. (+) Random assignment of classes (not students). (+) Presented definitions of the treatment and the alternative approach.
• Use of a close outcome only may limit external validity results.
• It is realistic to consider random assignment by classes.
Von Secker & Lissitz (1999)
(-) Lack of information regarding how both curricula were taught in each class. (-) The definition of the pedagogical practices is generated by a teacher’s self-report questionnaire. (-) No accurate representation of the
• The findings of this study are very limited because of the type of ex-post facto design. The design is critical to what can be concluded.
26
instruction happening in the classroom. (-) Lack of description of the treatment and the outcome.
• Gathering practices only with a self-report provides very limited information.
Schneider et al. (2002)
(-) Lack of information regarding how both curricula were taught in each class. (-) Students participating in the treatment were a selected group from a specific high school and were not randomly assigned. (-) Lack of definition of alternative approach. (-) Test-taking conditions are different for both groups. (-) Authors do not mention how the NAEP item sub-set was selected.
• The outcome is not aligned with the test and it is distal, yet there is a treatment effect. This effect can be explained by the fact of having an alignment between the outcomes and the facet-based inquiry conception.
Berg et al. (2003)
(-) Lack of information regarding how both curricula were taught in each class. (-) The results cannot necessarily be attributed to the approach since open-inquiry students spent more time in the experiments. (-) No measure of achievement was administered. (+) Authors provided examples of the different outcome measures and formats in the text and in appendices. (+) The authors were not involved in teaching students during any part of the courses.
• A measure of achievement is necessary when comparing science performance of students.
Klahr & Nigam (2004)
(-) Lack of information regarding how both curricula were taught in each class. (-) Direct instruction seems more a form of guided scientific inquiry teaching where the teacher interacts with the students. (-) If inquiry is considered only as hands-on activities without instruction, it is not surprising to find that there is no learning. (+) Random assignment. (+) The use of a transfer task (evaluation of posters).
• The use of a transfer task. • The importance of well
defined conditions. If authors had not provided a definition of “direct-instruction”, this category could have been misinterpreted.
• In this study, the alternative approach was much more aligned to the tests than the treatment.
Pine at al. (2006)
(-) Lack of information regarding how both curricula were taught in each class. (-) Unequal assessment tasks (3 physics and 1 biology). (-) Length of one assessment task (three- days long). (+) Use of distal and proximal outcomes. (+) Authors avoided specific content areas and assessed mainly procedural aspects in order to have less aligned outcomes. Focused more on science skills instead of content.
• Importance of monitoring classroom implementation with observation instrument.
• Need to balance the content domains in the assessments.
• Use of distal and proximal outcomes measures.
27
Geier et al. (2008)
(-) Lack of information regarding how both curricula were taught in each class. (-) No sample of the selected MEAP items were provided. (-) There were students in the same school that participated in both conditions. The authors did not find significant difference among those groups. (-) Lack of definition of alternative approach. (-) Teachers that participated in LeTUS were selected by administrators (selection bias). (+) Authors are aware of sample bias.
• Even though no difference was found within students from the same school participating in both conditions, there were significant differences when all groups were analyzed together. Therefore, there is a need to measure different schools.
a (+) Positive aspects; (-) Negative aspects
Concluding Remarks of How Inquiry Has Been Studied
The empirical studies analyzed above provide diverse lessons that were
considered in the design of this dissertation study. First of all, the studies did not clearly
define inquiry teaching. That is why I devoted a whole chapter defining what I refer to
when talking about inquiry-based science teaching. In addition to pointing to the need for
a clear definition of inquiry, the studies made clear to me the importance of developing a
classroom observation tool that could be used to determine whether the conception of
inquiry teaching espoused was enacted in the classrooms. In my study, then, classroom
observation information is compared to my definition of inquiry, to be sure that the
treatment and the control conditions are in fact inquiry or not. The final lesson learned
relates to the nature of the research design. If a randomized experiment cannot be carried
out, a quasi-experimental design, with one or more pretests and a posttest should be
carried out if at all possible and it was possible in this dissertation.
28
CHAPTER 4. MEASURING STUDENTS’ SCIENCE ACHIEVEMENT
There are several different considerations that need to be taken into account when
measuring students’ science achievement. This chapter presents some of these
considerations, based on both empirical studies as well as conceptual articles about
science achievement.
How Students’ Science Achievement is Measured Students’ science achievement can be measured in different ways. The most
common way is to include multiple-choice questions. Assessment alternatives to
multiple-choice questions include evaluation of student science notebooks, Predict-
Observe-Explain (POE) questions, performance assessments and other constructed
response items (Ruiz Primo and Shavelson, 1996; Ruiz Primo, Shavelson, Hamilton and
Klein, 2002; Pine, et al. 2006, Shemwell, Fu, Figueroa, Davis & Shavelson, 2008). Table
4.1 presents a brief description of examples of types of assessments.
Table 4.1 Examples Types of Assessments Used to Measure Science Achievement Type of Assessment Description. Multiple choice Consist of a stem, a correct answer and several distractors.
Constructed response Consists of the question, stimulus material, and a scoring rubric that includes the grading criteria.
Performance assessments Consist of a challenge that is to be solved using materials provided, a response sheet, and a scoring system that includes the procedures and the solution.
POE Students are presented with an initial condition of a situation, with an uncertain outcome and are asked to predict the outcome, observe what happens, and interpret and explain their observations.
29
One of the most common ways in which science achievement is tested is through
multiple-choice exams (Ruiz Primo and Shavelson, 1996). Even though multiple-choice
achievement tests have benefits such as efficiency, cost and high reliabilities since they
are economical to develop, administer, and score, they have received diverse criticisms
that include that they don’t measure some scientific knowledge such as the ability to
formulate a problem or carry out an investigation (Ruiz-Primo & Shavelson, 1996).
However, multiple-choice tests are useful to measure facts and concepts. Nevertheless,
these types of test do not necessarily measure all the outcomes related with knowledge
and skills, since they are not equivalent to what a scientist does (Ruiz Primo &
Shavelson, 1996). Therefore, it is important to consider other types of measures when
assessing students’ science achievement.
However, the assessments per se include diverse other criteria that may determine
how well these can measure students’ science achievement. Therefore, a more detailed
analysis of elements included in item design needs to be considered both from empirical
studies as well as from the way student achievement is measured in practice.
Types of Outcomes in Inquiry Empirical Studies
The empirical studies analyzed in the previous chapter used different measures
student achievement including achievement and local, regional, national or international
standardized tests, students producing questions, and unit problems among others. While
some of the studies provided the full instruments (Pine et al., 2006) as part of the article
and others provided examples of the items (Chang & Mao, 1999), a few only mentioned
or briefly described the tests used. Additionally, of the outcomes described in the studies,
only a few were comparable with each other, mostly state or national assessments. Even
30
though the outcomes varied from one study to the next, a basic comparison of the studies
was possible using the information provided, and the inquiry facets.
Types and characteristics of outcomes
Table 4.2 shows a comparison of the types of outcomes used in empirical studies
that compare inquiry teaching with other approaches. The outcomes can be cognitive,
affective or both. When provided, sample items and the intention of authors were taken
into account in order to classify the assessment into type of outcomes. All studies
presented in Table 4.2 have at least one cognitive outcome. These cognitive outcomes are
further characterized by their proximity and alignment with instruction, and the type of
knowledge they tap into.
Table 4.2 Outcomes used in Studies that Compare Inquiry with other Approaches Study Outcomes measured
Type of outcome a
Tamir et al. (1998) Two variable-based problems (Constructed Response)
C
Earth Science achievement test (Multiple-choice)
C Chang & Mao (1999)
Attitudes towards Earth Science Inventory (Multiple-choice)
A
Von Secker & Lissitz (1999) 10th grade science achievement test (ETS) (Multiple choice)
C
Schneider et al. (2002) NAEP Science Test Items (Multiple-choice) C
Questions asked by students. C Berg et al. (2003) Student’s self-evaluation (Multiple-choice) A Assessment of Control for Variable Strategies (CVS)
C Klahr & Nigam (2004)
Ability to evaluate posters. C Performance Assessment C Pine et al. (2006) TIMSS Achievement (Multiple-choice) C
Geier et al. (2008) Science items from the Michigan Educational Assessment Program. (Multiple-choice)
C
a, C: Cognitive Outcome; A: Affective Outcome
31
Proximity
Proximity refers to the distance between the assessment and what is taught in the
curriculum. Proximity ranges from immediate to a distal and/or remote (Ruiz-Primo,
Shavelson, Hamilton, & Klein, 2002, Ruiz-Primo, Wiley, Rosenquist, Schultz,
Shavelson, Klein, and Hamilton, 1998) (Figure 4.1). Immediate assessments refer to
those that are directly linked with classroom instruction such as notebooks or classroom
tests; close assessments are embedded assessments related with the teaching unit;
proximal assessments measure the same unit but with a different application; distal are
large scale performance assessments from a state or a national curriculum framework;
and remote are national science achievement tests (Ruiz-Primo, Shavelson, Hamilton, &
Klein, 2002). Several studies reviewed had assessments that were close to the content
taught in the treatment condition (Tamir et al, 1998, Chang and Mao, 1999, Berg, et al.,
2003, Klahr & Nigam, 2004), That is, the assessments were parallel to the treatment’s
content and activities. On the other hand, the performance assessment used in Pine et al.
(2006) can be categorized as a proximal assessment since the knowledge and skills
students need are relevant to the curriculum but specific topics can be different. The third
category found in assessments from the studies (Pine et al. 2006, Geier et. Al, 2008, Von
Secker & Lissitz, 1999, Schneider et al., 2002) is a combination of the distal and remote
categories presented by Ruiz-Primo et al. (2002), the former referring to assessment
based on state or national standards, such as large scale assessments including NAEP,
TIMSS, MEAP, and the latter to general measures of achievement. When comparing
inquiry teaching with another approach, there is a need to have an assessment tool that
includes items that are proximal to the instruction as well as items that are distal. Having
32
distal items permits to observe if students’ learning can be extrapolated to other realms of
science. Pine et al. (2006) and Schneider et al. (2003) were able to incorporate these two
types of measures in their studies.
Figure 4.1. Schematic of a multilevel assessment of science achievement. Taken from: Ruiz-Primo et al., 2002, p. 372. Alignment
Alignment refers to the relationship between the test and the “treatment”. Walker
and Schaffarzick (1974) found that there was a greater effect size when comparing
treatment to control conditions when there is close alignment between what the test
measures and what is taught in the treatment condition. In the studies analyzed, alignment
can be seen in two ways. One way is for the test and the treatment to be closely parallel.
Another way is to map the test onto the facets of inquiry. That is, to show how many of
my inquiry facets the test taps into. Some studies with positive effects include
assessments closely aligned with the treatment— most of the time through the conceptual
facet. The positive impact of the treatment, in cases where there was an alignment of the
33
treatment with the outcome, supports Walker and Schaffarzick’s (1974) findings
regarding the positive association between a type of instruction and a test aligned towards
that instruction. For example, Pine et al. (2006) had two outcomes in his study,
performance assessments that he or Shavelson, Baxter and Pine (1991) had constructed,
and multiple-choice and constructed response items from TIMSS. The performance
assessments were aligned with the inquiry-based science curriculum and with the
procedural and the nature of scientific knowledge sub-facets. The TIMSS assessment was
distal to Pine et al.’s treatment and aligned with the procedural and the conceptual facets
of my framework.
Types of knowledge
The cognitive outcomes can also be categorized into four types of knowledge that
are used to conceptualize science achievement (Li & Shavelson, 2004): Declarative,
knowing that; procedural, knowing how; schematic, knowing why; and strategic:
knowing about knowing in a domain. Declarative knowledge refers to scientific
definitions or facts, and can be represented by students either by words or by other means
such as images. Procedural knowledge refers to knowledge of the sequence of steps or of
if-then production rules needed to complete a task. This knowledge can be simple
including measurements of the amount of liquid in a beaker or complex including the
design of an experiment to find out how body temperature changes with exercise.
Schematic knowledge refers to principles or explanatory models and involves relating
complex phenomenon with concrete or common explanations. And strategic knowledge
includes strategies of monitoring or planning and involves knowing where, when, and
34
how to use the other three types of knowledge (Li & Shavelson, 2004). Figure 4.2 shows
how the knowledge types are related with each other.
Figure 4.2. The relationships between different knowledge types. Source: Shavelson, Ruiz-Primo, Li & Ayala, 2003.
Table 4.3, presents a classification of the types of knowledge, the focus of each
type, and the assessments to which the types can be linked.
Table 4.3 Relationship between Types of Knowledge and Types of Assessments Type of knowledge Focus Type of assessment tasks
usually associated with type of knowledge
Declarative • Concepts • Content • Facts • Scientific knowledge
• Multiple choice items • Short answer questions • Concept maps
Procedural • Assessing skills • Design of experiments • Data collection and representation
• Performance assessment • Multiple choice items
Schematic • Explanations of mental models • Analysis • Comprehension
• Open-ended questions • Some multiple choice
questions (e.g., some items from NAEP or TIMSS)
Strategic
• Application of knowledge • Constructed response • Performance assessment • Some multiple choice
questions (e.g., some items from NAEP or TIMSS)
35
Even though there may be a connection between the type of test item and the type
of knowledge tested, this is not always the case. For example, to measure declarative
knowledge, multiple-choice questions are commonly used. However, these types of
questions can also measure schematic knowledge (Li, Ruiz-Primo, & Shavelson, 2006).
Performance assessments can be used to assess the procedural and strategic types of
knowledge in some cases (Paper Towels) and all four types of knowledge in others (e.g.
Electric Mysteries.
Concluding Remarks About Measuring Students’ Science Achievement
Different empirical studies reviewed used a variety of assessments to measure
students’ science achievement. Additionally, the different assessments varied as to
proximity, alignment, and types of knowledge tapped. Each study defined inquiry in
different ways, and none of them used Duschl’s facets or the inquiry framework
presented in this dissertation. However, several of the assessments used in the studies can
be aligned into one or several facets which does relate the proximity of the assessment to
the inquiry curriculum.
The study presented in this dissertation focuses on comparing students who have
learnt science through inquiry to other approaches using diverse assessments. Several
aspects were considered in the design of the assessments. First, the assessments should be
diverse and aligned with different components of inquiry-based science teaching. Second,
the paper and pencil tests should have multiple-choice and constructed response
questions that tap into declarative, procedural, and schematic types of knowledge. Third,
the tests also need to provide tools for students to answer proximal and distal items.
36
Fourth, performance assessments should have both content rich and content lean
components. Finally, I should use as many science facets as possible, so that the
assessment encompasses both traditional methods of science teaching and inquiry-based
science teaching. However, it may be very difficult to incorporate the social facet into the
assessments.
37
CHAPTER 5: SCHOOL CONTEXT
The Educational System in Bogotá, Colombia
Bogotá, the capital city of Colombia, had an estimated population of 7.300.000 in
2009 (Secretaría de Educación del Distrito (SED), 2009). The Secretary of Education in
Bogotá estimated that 22.4% of the population were children between the ages of 5 and
17, generating the demand for preschool, primary, and secondary education (SED, 2009).
In 2009, the city had around 2,400 schools that served school-aged children. Of these
schools, 384 were public, while 1,973 were private, all serving 1.611.808 students. Public
schools have many sites, and they can have two or even three shifts during the day
(morning, afternoon and night). This means that for each of the public schools mentioned
above, there might be from 2 to 18 separate buildings in different campus that house
many students, with a double-shift in each. In 2009, 63% of students in Bogotá attended
public schools while 37% attended private school. The majority of students, therefore,
attend public schools.
In Bogotá, Socio-Economic Status (SES) is set by a scale of “Estratos” that goes
from 0-6, 0 being the lowest and 6 the highest. Figure 5.1 shows the distribution of
students in public schools according to their SES. In terms of schooling, 76% of students
who attend public schools come from the lowest three SES levels. On the other hand, less
than 2,000 students, corresponding to 0.2%, from the higher two SES levels attend public
schools.
38
Figure 5.1. Distribution of students in public schools according to SES in Bogotá in 2009. Source: Secretaría de Educación del Distrito, 2009.
Three models are used in Bogotá’s public educational system: District Official,
Concession Official, and Agreement Official. District Official schools are institutions that
are 100% administered by the city; Concession Official schools are public schools that
are administered by private institutions; and Agreement Official schools are private
institutions that sign an agreement with the Secretary of Education to provide schooling
for non-paying students who otherwise would go to public or concession schools.
Table 5.1 shows the distribution of students in public and private schools in
Bogotá in 2009. Even though the number of District Official schools is low compared to
private schools, each official school operates several sites, administering a total of 715
District Official school sites. The table shows that a majority of students (63%) attend
39
public schools, where the number of institutions is lower than the number of private
schools that educate a smaller percentage of students. Public schools, therefore, have a
larger number of students than private schools.
Table 5.1 Distribution of Students in Public and Private Schools in Bogotá in 2009 Type of School Number of Schools
(Percent of Schools) Number of Students (Percent
of Students)
District Official 359 (15.2)
837,003 (51.9)
Concession Official 25 (1.1)
39,947 (2.5)
Agreement Official 335 (14.2)
143,514 (8.9)
Public Schools (Total) 384 (16.3)
1,020,464 (63.3)
Private Schools 1,973 (83.7)
591,344 (36.7)
Total K-12 2,357 1,611,808
Source: Secretaría de Educación del Distrito, 2009.
The Concession School Model in Bogotá
In the late 1990’s, the city’s administration realized that more than 140,000
school-aged children were not attending school and decided to set out a three-prong
strategy to meet the demand of primary schooling (Uribe, Murnane, Willet and Somers,
2005). The first aspect of the strategy involved an expansion of the public sector, which
included an increase in class sizes, reassignment of teachers from administrative posts to
classrooms, and re-habilitating classrooms in run-down schools. The second, involved
subsidizing private schools to enroll low-income students, and the third, set out to
broaden the coverage as well as the quality of public education through a program called
Concession Schools.
40
Concession schools were built in extremely poor areas of the city where the
demand for primary and secondary education was higher than the number of places
supplied by the official public schools (Barrera-Osorio, 2006). Basically, the Concession
Schools’ program is a partnership between the public and the private education sectors,
where the private schools administer public schools over a 15-year period (Barrera-
Osorio, 2006). In this model, the city provides the infrastructure, selects the students, and
pays a pre-agreed sum per full-time student per year. The concession schools provide
education to a very low-income population assigned to them by the city and must meet
the performance standards set by the Secretary of Education measured through the
Colombian standardized exit exam ICFES (currently called SABER 11).
The Concession Schools were founded on the assumption that they could take
advantage of the experience and high performance of the private schools through their
administration of the public schools (Rodriguez and Hovde, 2002). The program was
designed to overcome several of the most pressing issues faced by public schools
including weak leadership, inability to select their own personnel, lack of labor
flexibility, double-shift of students during each day, and restrictions on enacted
curriculum (Patrinos, 2005). The managing institutions adhered to several standards that
the concession schools must meet including a minimum number of hours of instruction,
establishment of a single-shift of instruction, quality of nutritional provisions, a minimum
qualifications profile for teachers and administrators, facility maintenance standards,
criteria for the availability of instructional materials, a profile of the students to be served,
and the evaluation of achievement by outcomes (Rodriguez and Hovde, 2002). Each
41
managing institution has considerable pedagogic and curricular freedom, since each
institution has its experience in the educational sector.
The private schools were selected through a public procurement process, where
bidders were evaluated on their proposed management plans. Two models were used in
the concessionary management relationship: the one-to-one experience where one private
school administered one public school, and the multiple school experience where an
organization or private group took over the management of several schools (Rodriguez
and Hovde, 2002).
The program was launched in 2000 with 22 schools. Three more schools were
opened soon after, reaching a total of 25 Concession Schools. Currently, almost 40,000
students attend these schools in Bogotá, representing 4% of the city’s public enrollment.
Table 5.2 shows the current managing institutions and the number of schools each has.
Table 5.2 Managing Institutions of Concession Schools in Bogotá Name of Institution Number of Schools
Colsubsidio 5
Alianza Educativa 5
Don Bosco 5
Cafam 4
Fe y Alegría 2
La Salle 1
Nuevo Retiro 1
Gimnasio Moderno 1
Calasanz 1
42
Six years after the program was launched, concession schools showed a lower
dropout rate and higher test scores than similar public schools (Barrera-Osorio, 2006).
The dropout rate in 2008 for concession schools was 1.6% compared to 4.1% in District
Official schools (SED, 2009). Additionally, the concession schools have also increased
the number of students from this population that enter higher education, currently have a
waiting list for admittance, and school violence problems are low in comparison to other
similar public schools (El Tiempo, 2008). Furthermore, the concession schools reach
students beyond academics by providing breakfast and lunch to all students,
psychological counselling and special support for students with learning disabilities, and
by working with the community through workshops directed to parents (El Tiempo,
2008).
Schools Participating in the Studies
All schools selected for the two studies that are part of this dissertation are
Concession Schools. Recall from above, the schools were selected by identifying similar
characteristics including that all were concession schools, that all came from similar
socio-economic backgrounds, and that all had similar results in standardized tests. Three
schools are part of the Alianza Educativa, one school is part of Colsubsidio, and one
school is managed by Gimnasio Moderno. Two schools from Alianza educativa were
excluded from the study since the recently appointed science teacher did not have IBSE
background. Table 5.3 shows the schools that are part of this dissertation research.
43
Table 5.3 Schools Participating in the Studies
Name of School School Administrator
Jaime Garzón Alianza Educativa
La Giralda Alianza Educativa
IBSE
Santiago de Atalayas Alianza Educativa
Las Mercedes Colsubsidio Control
Sabio Caldas Gimnasio Moderno
Figure 5.2 shows a map of Bogotá that represents the distribution of these schools around
the city. Even though some schools come from geographically different areas, all of the
locations are similar in SES and in cultural characteristics.
Figure 5.2. Map of schools participating in the studies (adapted from Alianza Educativa, 2010).
Alianza Educativa: Jaime Garzón, La Giralda, and Santiago de las Atalayas
The Alianza Educativa is a non-profit association whose promoters are three
private schools (Colegio Los Nogales, Colegio San Carlos, Colegio Nueva Granada) and
the Universidad de los Andes. The main purpose of Alianza is “to promote, for the good
of democracy, a high-quality education in Colombia as the best tool and means for
citizens to achieve equal opportunities looking to reach an integral formation that
44
includes the intellectual, social, ethical, and aesthetic education of individuals (Alianza
Educativa, 2008, p. 4.)”
Alianza administers five schools in different low-income communities of Bogotá.
The five Alianza Schools serve 6,200 students from K-11 (K-11 in the Colombian
system) with an average of 40 students per classroom. The regular schedule of classes is
between 7:00 am and 2:30 pm, with extra-curricular activities from 2:30 pm to 4:00 pm.
Alianza currently has 245 teachers and 1817 alumni.
Some of the problems and difficulties present in the communities where the
Alianza works include undernourishment, interfamily violence, sexual abuse, teenage
pregnancy, drug use, theft, academic gaps, and low motivation for studies (Alianza
Educativa, 2008).
Alianza has 5 main goals:
1. To graduate competent high school students.
2. To train a pool of qualified teachers.
3. To develop a model of interinstitutional work.
4. To be a center of influence in the community.
5. To carry out education research.
Pedagogically, Alianza develops its educational model and curriculum based on
constructivist principles (Alianza Educativa, 2008):
1. Constructive processes where students construct their knowledge through a
gradual process.
2. Previous learning where experiences accumulate and contribute to each students’
construction of knowledge.
45
3. Performance and assessments where students produce different actions and
products that show their diverse levels of understanding.
4. Social interaction where learning is augmented through the interaction with
others.
Colsubsidio: Las Mercedes
Colsubsidio is a family compensation fund whose mission is to work for the
integral improvement of life conditions of the population and for the development of a
supportive, harmonic, and equal society (Colsubsidio, 2010). In Colombia, family
compensation funds are private institutions that finance themselves by resources that
come from 2% of the salaries paid by private and public institutions. By law, each
individual and their employer have to be affiliated to one of these funds and both the
individual and the employer share the cost of this affiliation. These funds provide
different services to their affiliates including recreation, health, housing, education, and
training. Colsubsidio is one of the largest family compensation funds in the country (Villa
and Duarte, 2002).
In the educational arena, Colsubsidio offers a model that strives for high quality
schooling of the affiliates’ children. The main objective of Colsubsidio is to raise the
educational level of the Colombian population through a model based on academic
quality focused on the labor world and based on moral and social principles and values.
Colsubsidio has four private schools and currently administers five concession schools
including Las Mercedes (Colsubsidio, 2010).
46
The mission of Colsubsidio’s concession schools is to form citizens with social
and ethical commitment (Instituto para la Investigación Educativa y el Desarrollo
Pedagógico, IDEP, 2010). Colsubsidio’s schools have three main goals (IDEP, 2010):
1. The construction of a community where natural leaders will be identified and
where specific training will be given in communicative processes, conflict
resolution and project management.
2. The implementation of a project where the school accompanies the design of
strategies and instruments, and promotes the process of natural leaders.
3. The community’s evaluation of the impact of the schools’ actions to provide
feedback to these projects.
The pedagogical model used by Colsubsidio is based on constructivism (IDEP, 2010).
Gimnasio Moderno: Sabio Caldas
Gimnasio Sabio Caldas is administered by Gimnasio Moderno, one of the most
traditional schools in Bogotá. The mission of Sabio Caldas is to implement human
growth processes fostering the development of competencies for coexistence and labor
performance that can lead to a productive and happy life.
The school’s goal is to help students find meaning in who they are, and what they
do and learn. The school provides a context where respect, tolerance, responsibility,
nutrition, and social work with families constitute the confidence framework so that the
community feels committed and happy to be an active part of their children’s educational
process (Manual de Convivencia, 2007). Sabio Caldas is a school where the interaction
with the community is a model that aims to transcend to other local and district
47
communities. The school aims to create an “oasis” where each member of the community
will find elements and opportunities that will permit him or her to advance and enrich his
or her life and have a vision of the future (IDEP, 2010).
There are two topics of great importance in Sabio Caldas:
1. Community work, where institutional members work with the community located
close to the school.
2. Formation in values, where the school recognizes the loss of these values and the
role that the school and families have in this process.
The most common pedagogical strategies in Sabio Caldas include integration
centers, pedagogical projects, interest centers, and research. Students in Sabio Caldas are
given an education that allows them to enter the labor force with appropriate skills and
competencies (Manual de Convivencia, 2007).
Schools’ results in standardized tests
The five schools that are part of these studies have participated in several national
standardized tests including the SABER 11exit exam, SABER 5 (fifth grade), and
SABER 9 (ninth grade). The exit exam is high stakes used as the main selection criteria
for entrance into higher education. All graduating students from schools take this
multiple-choice exam that tests knowledge and skills in math, language, biology, physics,
chemistry, social studies and philosophy. SABER 5 and 9 are standardized multiple-
choice tests given to 5th and 9th grade students testing competencies and content
knowledge in language, math and science. These exams are administered every three
48
years, and provide information on school performance rather than individual student
performance.
Based on each school’s results in the ICFES Saber 11 exit exams, each institution
is placed in a scale that ranges from Low to Very Superior. Table 5.4 shows the rating of
each school based on the 2010 ICFES results.
Table 5.4 Schools’ Level According to the Results from the 2010 ICFES Exit Exam.
Name of School ICFES Scale
Jaime Garzón Superior
La Giralda Medium
Santiago de Atalayas High
Las Mercedes High
Sabio Caldas High
Note. The data in column 2 are from http://w6.icfes.gov.co:8095/Clas/
Table 5.5 presents the results of schools in the ICFES exit exams for science
related components (Physics, Biology, and Chemistry). This classification provides a
scale of achievement of each school comparing means with others. The lowest
classification is Low and the highest classification is “Very superior”.
Table 5.5 Schools’ Results in the Science Components of the ICFES Exam (2010). Name of School Biology Chemistry Physics Jaime Garzón 47.12 48.05 46.27 Santiago de Atalayas 47.03 48.48 45.28 La Giralda 44.66 45.19 43.53 Sabio Caldas 45.59 47.44 45.29 Las Mercedes 48.23 47.48 47.23 Note. The data in columns 2. 3 and 4are from http://w6.icfes.gov.co:8095/Clas/
Table 5.6 presents the results of schools in the SABER exams for Math,
Language, and Science. These scores are in general above the national average.
49
Table 5.6 Schools’ Results in the 2009 SABER Exams.
Note. The data in columns 2,3,4,5,6 and 7 are from http://w6.icfes.gov.co:8095/Clas/
According to the information presented in this chapter, the students come from
similar backgrounds, they serve a low SES population in similar contexts, and they are all
concession schools. Statistical analyses of these data are presented in Chapter 6,
Methods.
Name of School SABER 5 SABER 9 Math Language Science Math Language Science Santiago de Atalayas 337 342 340 331 327 324 Jaime Garzón 353 341 335 338 343 337 La Giralda 327 313 308 318 300 314 Sabio Caldas 306 300 299 311 305 300 Las Mercedes 343 350 341 332 328 332
50
CHAPTER 6. METHODS
Overview and Research Questions
This study aims to compare science achievement in students that participate in
inquiry-based science education programs with students that have not participated in such
programs. The comparison between groups looks at total scores in five different
assessments, and also explores differences in the types of knowledge demanded by the
assessments. The research question for this study, in broad terms, is:
How does the science achievement of students participating in the Colombian
IBSE program compare, on average, with the science achievement of students
who have not participated in inquiry-based science education programs?
More specifically, this study addresses the following questions:
1. Is there a difference between IBSE and Control students’ performance in the
Human Body Systems (HBS) paper and pencil tests?
2. Does IBSE and Control students’ science achievement vary depending on the type
of knowledge tested?
3. Does IBSE and non-IBSE students’ science achievement vary depending on the
proximity of the assessment used?
4. Does students’ mean achievement scores differ according to their group (IBSE,
Control), achievement on posttest (high medium, low) or content level (rich or
lean) of the performance assessments?
5. Do students perform similarly on multiple-choice tests and performance
assessments?
51
Participants and Context of this Study
Students from five schools in Bogotá participated in this study. Of these, students
from three IBSE schools are the Treatment group while students from two Non-IBSE
schools are the Control group. Institutions in Colombia (both IBSE and Control) base
their science curriculum on the Science standards presented by the Colombian Ministry
of Education (Ministerio de Educación Nacional (MEN), 2004). The standards present a
broad range of content topics for fourth and fifth grade, as well as the development of
science skills and social skills. The topics related to Human Body Systems (HBS) are part
of the “Living Environment” component and include standards such as (MEN, 2004):
• “I identify the different levels of cellular organization in living things.
• I identify objects in my environment that have similar functions as those of my
organs, and I can justify my comparison.
• I represent the diverse systems and organs of human beings and I can explain their
function.”
Even though all schools in this study base their science curriculum on these
standards, and are comparable in student’s socio-economic status (SES), there are
differences in the way the curriculum is taught. In the following section, both the
treatment and the control groups are described from three perspectives: Curriculum and
materials, teachers, and students. Information about the curriculum and the materials was
obtained directly from each of the modules or books used in classes. The Treatment and
Control groups with their specific curriculum and materials will be presented first,
followed by specific information on teachers and students, which was obtained from
52
classroom observations and from interviews with the teachers. This section of the chapter
will be presented by comparing what teachers and students in each of the groups actually
do in the classroom.
Treatment Group
The Alianza Educativa manages five schools in very low SES neighborhoods in
Bogotá, Colombia. Each school has approximately 1200 students from K – 11, with an
average of 40 students in each classroom. As above mentioned, the communities from
which the student body of Alianza schools are drawn have several problems, including
undernourishment, interfamily violence, sexual abuse, drug use, and low motivation for
studying. However, Alianza schools have increased student achievement and shown
higher scores than other public schools in their neighborhood. Fifth grade students from
La Giralda, Santiago de Atalayas, and Jaime Garzón, three of the Alianza schools
participated in this study. Two of the Alianza schools, Miravalle and Argelia, were
excluded from this study because the teachers are new (started with the Alianza schools
in January 2009) and they have not participated in formal training workshops nor had a
comparable experience to other fifth grade IBSE teachers.
Curriculum and Materials used in the Treatment Group
Since 2002, Alianza schools have implemented an Inquiry Based Science
Curriculum called Pequeños Científicos for grades 0 to 6. The program focuses on the
acquisition of scientific knowledge and skills through direct experimentation that
involves observation of phenomena, elaboration of hypothesis, design and execution of
experiments, analysis of results, and conclusions. Students work in cooperative groups,
53
where each member has a well-defined role including secretary, time-keeper, materials´
administrator and presenter. Each student keeps a written record of the experiences in his
or her notebook. Students are actively encouraged to present their ideas, and to discuss
and argue about results and conclusions. In this process, the teachers are guides who lead
students through questions and observations so that all children construct their own
knowledge.
The program includes visits from scientific researchers, teachers from scientific
disciplines, and university students who enhance the learning process. Students also share
their learning process with their parents and families through homework and assignments.
In class, students are expected to generate predictions, inquire, experiment, and
find evidence bearing on their predictions, keep a written record of their observations and
results, present their conclusions and discuss their presentation with classmates.
Teachers in Pequeños Científicos participate in a rigorous training program on
IBSE that includes workshops, supporting visits and individual work. All teachers
participate in 100 hours of training in inquiry-based teaching. The basic training involves
four workshops over a two-year period; additional training is provided during the
implementation of the module. The initial workshop includes a presentation of the IBSE
strategy, where teachers are “students” in the first module they will teach. This first 20-
hour workshop sets the stage for teachers to begin to develop their knowledge of how
modules work. The workshop incorporates elements of cooperative groups, development
of science skills, and administration of material, among other things. Teachers then go
back to their school to implement their first module in the classroom. During this period,
teachers receive two supporting visits. Six-months later, teachers participate in a second
54
workshop that focuses on the next module that teachers will implement. This second
workshop is 16 hours long and builds on the experience teachers have had during the first
six months of implementation. During the following six months, teachers receive visits
once again and finish the school year with a third 16-hour workshop where the
experience of the first year is reviewed and additional modules are presented. Teachers
are visited during the second year, and then participate in a fourth and final 8-hour
follow-up workshop at the end of this year. Sometimes, this cycle is repeated when there
are new teachers in the schools. Figure 6.1 shows the training process of Pequeños
Científicos.
Figure 6.1. Process of teacher training in Pequeños Científicos. Taken from: Pequeños Científicos. 2004. Institutional Presentation.
55
Science teachers in Alianza schools were selected in 2002 to participate in this
training process as a preparation for teaching the IBSE modules. The schools developed
their 0-6th grade science curriculum based on the modules and the methodology provided
by Pequeños Científicos. Some teachers in Alianza also had a one-hour weekly meeting
when they work on professional development and were able to reflect and continue with
their work in the implementation of IBSE modules. Additionally, the supporting visits,
made by both teachers from the school and staff of Pequeños Científicos provided
feedback on the teachers’ implementation of IBSE modules. The teachers at the three
schools that were part of this study have been working with Pequeños Científicos for
more than eight years.
In the Pequeños Científicos program, several of the modules that students study
during primary school are based on the INSIGHTS modules developed by the Center for
Science Education, a part of the Educational Development Center (EDC). The specific
INSIGHTS module related to this study, Human Body Systems, has been implemented
with fifth graders of Alianza Educativa. The module was translated into Spanish by the
Language Department of Universidad de los Andes. The Human Body Systems module
allows students to explore how body systems work together in order to allow their body
to work. The module starts by introducing students to the needs their bodies have in order
to perform different functions. Then the students work with different body systems,
exploring how they work and how they interact to make their body function properly
(CSE, 2011).
The Human Body Systems module is one of two modules taught in fifth grade.
Students work on the different aspects of this topic for approximately half of the school
56
year (5 months). Students have access to experimentation and to other diverse science
related activities. Additionally, they have science textbooks as reference, including the
same one used in the Control group. In the treatment schools, students answer an
introductory assessment before starting the unit that measures their knowledge about the
topics of the module. When they finish, they answer a final assessment that has similar
questions. Both assessments are composed of constructed-response questions. Students
are evaluated as to their growth from before to after studying HBS. The following
questions exemplify those found in the assessments (Insights, 2003, p. 7-11):
• What is digestion?
• What does the blood do?
• How does the oxygen you breathe in get into your bloodstream?
One science class per teacher in the three IBSE schools was videotaped during
one class period of approximately 50 minutes; the three teachers were also interviewed
toward the end of the study. The information collected in both the videotapes and the
interviews coincide with the information above, where the teachers use an inquiry-based
approach to teach the Human Body System module. Further information on teachers and
students in the Treatment group is presented below.
Control Group
Two other concession schools different from Alianza Educativa were part of this
study. The first school, Las Mercedes, is one of the five concession’s school administered
by Colsubsidio, a family compensation fund. The second school Sabio Caldas, is
administered by Gimnasio Moderno, a traditional private school in Bogotá. Both schools
are located in very low SES neighborhoods in Bogotá, Colombia with a comparable
57
population to that of the Alianza Schools. Each school has approximately 1200 students
from K – 11, with an average of 40 students per classroom. The communities from which
the student body of Las Mercedes and Sabio Caldas are drawn have very similar
problems as those of Alianza schools, including undernourishment, interfamily violence,
sexual abuse, drug use, and low motivation for studies. However, as it is the case with
Alianza schools, both Las Mercedes and Sabio Caldas have increased students’
achievement and shown higher scores than other public schools in their neighborhood.
These two schools are comparable to the Alianza Educativa schools, since they
are also concession schools from a similar SES background and have comparable results
in the ICFES exam (senior-year, school-exit exam). Further information about these
schools is provided in the Context chapter.
Curriculum and Materials Used in the Control Group
Both Fifth grade HBS Science classes in the Control schools are based on the
same textbook: Santillana Casa de las Ciencias Naturales 5. The book provides
information about different body systems, among other topics. This book is the main
source of information for students and teachers. Figure 6.2 shows an image of the type of
information presented in the book. This is an example of the book’s information about
the digestive system.
58
Figure 6.2. A print-out of pages of the on-line version of the book: “Santillana Casa Ciencias Naturales 5” (Editorial Santillana - Casa Ciencias 5. (n.d.)).
Science classes in the two Control schools were videotaped and the fifth grade
teachers in each school were interviewed. The information gathered from the class
observations and from the interview provided general characteristics of the type of
teaching that occurs in each of these teachers’ classrooms. The most prominent
characteristic present in both classes is that teaching is teacher-centered. This means that
teachers in these classrooms do most of the talking in front of students and direct specific
activities carried out by the students. Teachers in these classrooms spend a significant
amount of time providing direct information to students, involving them in rote learning.
For example, one of the teachers frequently asked students to copy a direct text into their
notebook from either the board or the textbook. Most of the work observed in the
classroom focused on teaching declarative knowledge. The activities in these classrooms
59
ranged from a direct narration of information, to revising the previous lesson, or students
filling out a crossword puzzle with HBS vocabulary. In order to develop the necessary
learning teachers mentioned that they used the guides and activities from the book, as
well as posters that describe different human body systems (See interview section).
Figure 6.3 shows one type of activity given by teachers to their students, taken from the
textbook activity bank. The implementation of this activity was actually witnessed
through the videotape.
Figure 6.3. Crossword puzzle (Digestion in Humans) used in Control group taken from on-line resources (Editorial Santillana - Casa Ciencias 5. (n.d.)).
60
Figure 6.4 shows another type of activity, Naming Parts of the Digestive System.
This activity also focused on vocabulary. In this case, students needed to fill out the
names of the different parts of the digestive system.
Figure 6.4. Fill in the blank activity used in Control Groups teaching science taken
from on-line activity bank (Editorial Santillana - Casa Ciencias 5. (n.d.)).
The Human Body Systems unit in four of the five the schools was taught in a
similar time-frame. However, one of the Control schools actually took two months more
than the other four schools working in the unit.
Teaching, Teachers and Students in the Treatment and Control Groups
Implementation check through videos and interviews
Five teachers, in total, participated in this study. They all taught two classes per
school. Observations of each teacher were carried out in order to describe what each
61
implementation of the science curriculum looked like. All teachers were videotaped and
the tapes were observed using the same criteria.
Classroom observation
Fifth grade classes from teachers of both the IBSE and the comparison schools
were videotaped when implementing the HBS unit. One class from each of the five
teachers was videotaped with the objective of gathering additional information about
what occurred in the classroom and obtaining information about the fidelity of
implementation in both, the Treatment and Control groups.
Table 6.1. Teachers’ Classroom Practice as Evidenced from One Lesson
Activities observed in videos
Teacher1
Teacher 2
Teacher 3
Teacher 4
Teacher 5
Type of School IBSE IBSE IBSE Non IBSE
Non IBSE
Teacher transmits information xxx xxxx Teacher grades students notebooks xxxxx Teacher asks declarative questions For example: Which is the main function of the heart?
xx xx x x x
Students answer declarative questions xx xx xx x xxx Students complete words the teacher mentions x xx xxx Students copy information (from documents, book or the board) in their notebooks
x xxxx xx
Teacher asks students about their previous knowledge
xxx xx xx
Teacher walks around the groups asking questions
xxxx xxx xxx x
Teacher gives instructions for experiments xx x xx Students do experiments xxxx xx xxx Students do an activity (not an experiment) Example: writing/completing a crossword
xxx xxx
Teacher gives feedback to students xx xx xx Teacher asks for explanations xxxx xxx x Students discuss in their group xxxx xxxx xx Students answer questions using evidence xxxx xxx xx Students register information with their own words in their notebooks
xxxx x xx
62
Each science class took approximately the same amount of time (60 minutes). These
recordings did not aim to describe teachers’ classroom practices nor characterize their
teaching strategies, but to provide information about the facets of inquiry and traditional
practices visible in each class. Each video was observed and Table 6.1 was created. The
activities in the table show observed moments in the video. In order to describe the
activities, the videos were coded. This coding was carried out in several steps. The first
step was the definition of the categories for which two Master’s students and myself
observed two IBSE and one Control video separately. Each video was observed during
five-minute intervals. After each interval, each observer described what was observed in
that time frame. The main focus during each observation was on the teacher and what he
or she was doing (for example, the teacher asked questions or the teacher rotated from
group to group). There were several subcategories within each category, although each
observer named them differently (e.g. the teacher is teaching versus the teacher is
transmitting information). Common categories were created when sharing the
observations, extrapolating the main observations instead of focusing on specific details
of each teacher. The observations of these three videos provided the general categories
that were used when observing the other two videos. The observations of the last two
videos only added a couple of activities that had not been described previously.
Therefore, for the description of the class session, the presented categories emerged from
the observed classroom videos.
While IBSE teachers spent more classroom time asking declarative questions and
for explanations, and walked around the room, giving feedback to students in their
groups, Control teachers transmitted information and graded notebooks, and placed
63
themselves in the front of the classroom. In general, students in IBSE classrooms were
doing experiments, discussing in their groups, answering questions using evidence, and
registering information with their own words in their notebook. On the other hand,
students in Control classrooms were copying information into their notebook and doing
written activities such as filling in a worksheet or crossword individually (even though
they were seated in pairs in one school and in groups of four in the other school).
Therefore, based on the classroom observations, IBSE teachers characterize what was
defined in Chapter 2 as inquiry teaching, while Control group teachers are traditional in
their approach to science teaching.
Teachers´ interviews
Each teacher was interviewed in March 2011, after all the students’ data was
collected. Information gathered through the interview included teaching experience, years
of teaching, materials used in the science classroom, professional development in science
education, and classroom practices. The interview instrument was translated and adapted
from the TIMSS teacher survey (see Appendix A).
Table 6.2 provides general information about each teacher and characterizes his or
her teaching practices based on their responses to the interview. Four out of the five
teachers were women and all ranged in ages between 30 and 50 years. Four teachers
taught fourth and fifth grades, and one teacher fifth, sixth, and seventh. The number of
students per class ranged from 36 to 43, and 4 out of 5 teachers had between 40 and 43
students in their classes. All teachers had more than 10 years of teaching experience, and
they all mentioned teaching for approximately 10 years in their current school. The
number of hours teaching science varied little with 3.7 weekly hours for the IBSE
64
teachers and 4 weekly hours for the Control teachers. All teachers held a Teachers’
License.
Table 6.2 Summary General Information About Each Teacher Based on Interviews
Teac
her
Gen
der
Ran
ge o
f Age
IBSE
Gra
de ta
ught
#
of st
uden
ts p
er
clas
s
Yea
rs T
each
ing
# of
yea
rs in
the
scho
ol
Wee
kly
Scie
nce
hour
s
Hig
hest
leve
l of
form
al e
duca
tion
Boo
ks u
sed
1 F 40-49 Y 4, 5
41 16 10 3.7 Teaching Licence
Pequeños Científicos Modules, Amigos de la naturaleza
2 F 40-49 Y 4, 5
41 14 10 3.7 Teaching Licence
Pequeños Científicos Modules, Amigos de la naturaleza
3 F 30-39 Y 4, 5
42 10 10 3.7 Teaching Licence
Pequeños Científicos Modules, Amigos de la naturaleza
4 M 40-49 N 4, 5
36 20 10 4 Teaching Licence
Casa de las ciencias naturales. Amigos de la naturaleza
5 F 30-39 N 5, 6, 7
43 15 10 4 Teaching Licence
Casa de las ciencias naturales, Ciencia y vida Amigos de la naturaleza
According to the interviews, teachers have comparable ages, experience,
academic specialization, and number of students per class. One of the main differences
appeared when teachers were asked about their experience and training in IBSE
programs. All three IBSE teachers received, over 8 years ago, training in the modules and
in inquiry teaching from the Pequeños Científicos program. They received additional
training and follow-up classroom visits after their initial workshop. All of them have
been teaching science through inquiry for more than 8 years. Additionally, teachers from
schools one and two have also studied a specialization in science education, focused in
scientific inquiry. None of the Control group teachers had received any training in IBSE,
65
but they have received professional development on topics such as science teaching,
science curriculum, and integration of technology into science classes.
The other main differences that appeared from the interviews are related to what
students do in class, according to the teachers, and are presented in Table 6.3. IBSE
teachers reported that their students plan and design experiments and actually carry out
the experiments in almost all class sessions, while Control teachers noted that this only
happens seldom. Control teachers say that their students memorize facts and principles in
almost all class sessions, while Treatment teachers answered their students never do.
Finally, IBSE teachers have their students relate what they learn in their everyday life to
what they are learning in class, and this is seldom done by teacher 5, and never by teacher
4.
66
Table 6.3
Characterization of What Students do in Class Based on Teacher Responses to Adapted TIMSS Questionnaire
In my class, students Teacher 1 Teacher 2 Teacher 3 Teacher 4 Teacher 5 a) Observe natural phenomena such as the weather or a plant growing and describe what they see
About half the lessons About half the lessons Some lessons Some lessons Some lessons
b) Watch me do a science experiment Never Never Never Some lessons Some lessons
c) Design or plan experiments or projects
Every or almost every lesson
Every or almost every lesson
About half the lessons Some lessons Some lessons
d) Do experiments or projects About half the lessons About half the lessons About half the lessons Some lessons Some lessons
e) Work together in small groups on experiments or projects
Every or almost every lesson
Every or almost every lesson
Every or almost every lesson Some lessons Some lessons
f) Read their textbooks or other resource materials
Every or almost every lesson
Every or almost every lesson
Every or almost every lesson
Every or almost every lesson
Every or almost every lesson
g) Have students memorize facts and principles Never Never Never About half the
lessons Some lessons
h) Give explanations about something they are studying
Every or almost every lesson
Every or almost every lesson Some lessons About half the
lessons About half the
lessons i) Relate what they are learning in science to their daily lives
Every or almost every lesson
Every or almost every lesson
Every or almost every lesson Never Some lessons
j) Work individually at their own pace About half the lessons About half the lessons About half the
lessons About half the
lessons Some lessons
67
Mapping Observed Classes and Interviews onto the Inquiry Facets
The information provided by the description of the modules, classroom
observations, and interviews allows to map, tentatively, each teacher onto the facets of
inquiry presented in Chapter 2 (see Table 6.4). Even though this information is only a
sample of the practices of each teacher, it does provide evidence of the type of facets
present in the classrooms.
Table 6.4 Mapping Teachers and Students with the Inquiry Facets Procedural Nature of Scientific
Knowledge Conceptual Social
Teac
her
Ask
ing
Scie
ntifi
cally
-Orie
nted
Que
stio
ns
Des
igni
ng E
xper
imen
ts
Exec
utin
g Sc
ient
ific
Proc
edur
es
Gat
herin
g an
d in
terp
retin
g da
ta
Rep
rese
ntin
g D
ata
Car
ryin
g H
ands
-On
Act
iviti
es
Form
ulat
ing
hypo
thes
is
Ref
lect
ing
on N
atur
e of
Sci
ence
Dra
win
g ex
plan
atio
ns b
ased
on
evid
ence
Gen
erat
ing
and
revi
sing
theo
ries
App
lyin
g sc
ient
ific
know
ledg
e
Dra
win
g on
/Con
nect
ing
to p
rior k
now
ledg
e
Expl
aini
ng id
eas/
men
tal m
odel
s
Org
aniz
ing
conc
epts
and
prin
cipl
es
Parti
cipa
ting
in c
lass
dis
cuss
ions
Arg
uing
/deb
atin
g sc
ient
ific
idea
s
Dev
elop
ing
com
mun
icat
ion
skill
s
Coo
pera
tive
lear
ning
IBSE Teachers 1 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 2 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! 3 ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! Control Teachers 4 ! ! 5 ! ! ! The Control group teachers focused mainly on the conceptual facet, which reflects
a traditional approach to teaching science while IBSE teachers have a characterization of
their practice that is aligned with my definition of inquiry.
68
Part A - Assessing and Comparing Students´ Knowledge of Human Body Systems with a Paper and Pencil Test
Subjects
Two hundred and forty-seven fifth-grade students from the three Alianza schools
(IBSE schools) and 162 fifth grade students from Sabio Caldas and Las Mercedes
(Control schools), who finished the school year in November 2010, participated in this
study. However, of these 409 students, only 365 students (Table 6.5) have both pretest
and posttest scores, since some of the students were absent on either of the two test dates.
There are no statistically significant mean differences between the Control or IBSE
students that were absent. Additionally, five students in total, 2 coming from the control
group and 3 coming from the IBSE group were excluded because they skipped pages on
the questionnaire.
In Colombia, fifth grade is equivalent to 6th grade in the United States, therefore,
the ages of students who participated in the study ranged from 10 to 12 years. Two
classes were selected in each school, and as above mentioned, each class had the same
teacher as the only one teaching science in fifth grade in each school.
Table 6.5. Student Participants in the Study by Group, School and Class (sample sizes).
Type of School School School and
Class Number of Students with
a Pre and a Post Test 1,1 41
School 1 1,2 39 2,1 38
School 2 2,2 38 3,1 37
IBSE Schools
School 3 3,2 34 4,1 27
School 4 4,2 35 5,1 39
Control Schools
School 5 5,2 37 TOTAL 365
69
Instrument 1-Multiple Choice and Constructed Response Items
The paper and pencil assessment was developed in three steps: (1) item
development, (2) item revision based on trials and think-aloud protocols, and (3) booklet
construction.
Item development.
Test items came from different sources. Some items were developed during a six-
day workshop organized by Pequeños Científicos and led by Professors Maria Araceli
Ruiz-Primo, Guillermo Solano-Flores and myself. Twenty-three IBSE teachers from
Colombia and Panama, as well as disciplinary experts and other science educators were
involved in item development.
Participants were guided in the development of two types of items: some that
were close to the Human Body Systems IBSE module (proximal) and others that matched
Colombia’s national education standards (distal). For the development of the proximal
items, participants used the learning goals and the lesson storyboards of the module.
Participants were also asked to develop items that tapped into three types of
knowledge: (1) declarative knowledge (factual, conceptual knowledge) or “knowing that”
(e.g. name the system that is composed of the heart and blood vessels; (2) procedural
knowledge (step by step procedures) or “knowing how” (e.g. how to interpret a graph);
and (3) schematic knowledge (knowledge used to reason about) or “knowing why” (e.g.
explain why you breathe faster and your heart beats faster when you exercise).
Once the participants developed the items, each item was reviewed for content,
format, knowledge type, and clarity of language. After this review some items were
70
selected and improved, while others were discarded. To insure an adequate number of
items for the final version of the test, two science educators from Universidad de los
Andes and myself developed additional proximal items using both textbooks. .
Distal items were provided by ICFES, the Colombian Institute that carries out all
standardized testing in the country. Specifically, items for this assessment were taken
word-by-word from the released items from the SABER 2009 fifth grade science test. It
was decided to use these distal items instead of those developed in the workshop because
they had been field tested, revised and used on a national scale by ICFES.
Two versions of the paper and pencil assessment were produced. The pre-test
with 28 items contained fewer items than the post-test and focused more on proximal
than on distal items. The post-test contained 37 items and included all of the multiple-
choice (MC) items from the pre-test, several distal items from SABER 2009, and six
constructed response (CR) questions.
Item revision
Different versions of the items were tested on several occasions. On February
2009, a trial was carried out with 68 students in a city close to Bogotá (Tenjo). One result
from this trial indicated that the questions were too difficult for students and lacked
clarity. This led to the incorporation of approximately eight figures, in order to allow
greater clarity and make the questions easier to interpret. Additionally, the amount of
written text that had to be read by students in several questions was reduced.
Other trials were carried out in June and September, 2009 with 38 students in one
of the Treatment classes (La Giralda) and 30 in a Control class (Las Mercedes),
respectively. These students had already finished the HBS module or content but would
71
not participate in this study. This way, the trial provided information of how students
from the same context as those in the study would answer the questions. The results from
this trial led us to see that even though the questions worked better, several of them were
still too difficult. During these trials, students were asked to underline words that they did
not understand in the questions. When two or more students underlined a word, the
wording was modified.
Some students in these trials also participated in think-aloud protocols. Through
this method, students answer the question in the assessment and simultaneously verbalize
what they are thinking in order to gather information regarding the student’s cognitive
processes (Johnstone, Bottsford-Miller, and Thompson, 2006).
Before Think-Aloud Protocols After Think-Aloud Protocols 19. In what part of the following figure does the connection between the circulatory and the digestive systems occur?
19. In what part of the following figure does the connection between the circulatory and the digestive systems occur?
A. 1 A. Esophagus B. 2 B. Stomach C. 3 C. Large intestine D. 4 D. Small intestine Figure 6.5. Example of a change in a question after the input received in a Think-Aloud.
72
Students’ think-aloud protocols showed difficulties interpreting some drawings, as
well as problems understanding the question. An example of the type of change carried
in the items after the think-aloud protocols is presented in Figure 6.5. In this case,
students initially thought that they needed to fill in the blank next to the number, instead
of answering the actual question.
Items from each trial were analyzed using the statistical software Iteman (Item
Analysis) in order to identify items that were not working well and to calculate the
reliability of the scores. Iteman also provided information regarding the difficulty level of
the items as well as their discrimination index.
In general, the results of these trials indicated that the items had a high level of
difficulty and reading load. This led to their revision. In order to assess their difficulty,
the items were presented to elementary school science teachers in a workshop in Chile in
October 2009, where they rated the perceived difficulty level of each item, based on the
percentage of students in their classes that they thought would be able to answer each
question. Teachers classified about 50% of the items as too difficult (where less than 30%
of students would answer the question). Based on the teachers’ rating, items were also
modified, trying to keep their difficulty, but making them easier to understand for the
students. A couple of items were removed from the test.
Excluded items, in general, included those with double negative. Additionally, all
items were revised to unify the distractors so that they were all similar. In some cases,
instead of using a negative or to help students in their comprehension of questions,
keywords were highlighted in bold.
73
The final version of the pretest booklet had 28 questions, while the posttest had 31
multiple-choice questions and 6 constructed response questions. When scoring responses,
multiple-choice questions had a 1-point value while the constructed response questions
had a 2-point value since each constructed-response question had 2 parts. Table 6.6
summarizes the characteristics of the paper and pencil tests. The final versions of both the
pre-test and the complete post-test can be found in the Appendix (Appendix B and C).
Table 6.6. Assessments related with the Paper and Pencil Tests Type of test Description Number of
questions Pre-test The original booklet.
28
Post equal Pre The post-test contains the same questions as the pre-test.
28
Post Total Additional items in the booklet includes more distal items.
31
Post + Constructed Response
Post Total with 6 additional constructed response questions.
37
Booklet construction
The human body systems’ paper and pencil booklets balanced knowledge types
(declarative, procedural and schematic) and proximity of the item to the curriculum
(proximal v. distal). A group of science educators carried out the classification of the
questions according to the types of knowledge type. Table 6.7 shows the distribution of
questions according to the type of knowledge and the proximity to the curriculum. Two
versions of the post-test were created, in order to balance the distribution of the types of
questions (CR or MC) presented the students. Booklet 1 began with the multiple-choice
options, while booklet two began with open-ended questions.
74
Table 6.7. Composition of Items in the Human Body System Booklets Type of Test
Type of Knowledge
Declarative Procedural Schematic
Proximity Proximal Distal Proximal Distal Proximal Distal Pretest Post_equal_Pre
Number of Questions
11 1 4 4 8 0
Proximity Proximal Distal Proximal Distal Proximal Distal Post_Total Number of Questions
12 1 4 4 10 0
Proximity Proximal Distal Proximal Distal Proximal Distal Post+CR Number of
Questions 12 1 5 6 10 3
Additionally, the items can mapped into the facets of scientific inquiry presented
in the framework. Table 6.8 shows the distribution of questions according to the type of
knowledge. In this case, only the conceptual and procedural inquiry facets are
incorporated into the booklets. In general, declarative and schematic items where mapped
into the Conceptual facet, and Procedural items were matched with the Procedural facet.
Table 6.8 Composition of Items in the Human Body Systems Mapped into the Facets of Inquiry
Type of Test Conceptual Procedural Pretest Post_equal_Pre
20 items 8 items
Post_Total 23 items 8 items Post+CR 26 items 11 items
Three examples of translated, revised items selected for the final booklet are
presented below:
This first example was categorized as a proximal item tapping into declarative knowledge. What is the function of the digestive system?
A. Transform food that enters the body.*
B. Carry oxygen through the body. C. Carry nutrients through the body.
D. Regulate the body’s temperature.
75
The second example was categorized as a distal item (SABER 2009) tapping into
schematic knowledge.
When food is digested, where does you body use it?
This third example was categorized as a proximal item tapping into procedural
knowledge.
Doctor Perez records the rhythm of respiration of different people when they are at rest.
He created the following table:
Person Breaths per minute Pedro the baby 38 7 year-old girl 25 7 year-old boy 25 10 year-old boy 20 Mother 16
This table suggest that: A. Boys breathe faster than girls. B. Older people breathe faster than younger people. C. Girls breathe faster than boys. D. Younger people breathe faster than older people.*
This example presents a constructed response proximal question taping into schematic
knowledge.
How are the digestive and the circulatory systems related?
A. Only in the blood of your body. B. Only in the stomach of your body.
C. Only in the lungs of your body. D. In the cells of your body.*
76
Test administration and data collection.
Test givers were trained in the implementation of this test. All test givers
participated in a four-hour training session where they received and read a test
implementation manual and were provided with logistical and technical information
about the data collection (Appendix D). The training and manual helped standardize test
administration and unify instructions and protocols.
All tests were given under standardized testing conditions and classroom setup. In
each classroom, two test givers were present to administer the test. They followed the
instructions found in the training manual and reported, in a provided form, any
irregularities that occurred during the implementation.
Fifth grade students from three IBSE schools and two comparison schools took
the Human Body Systems (HBS) pretest during the first 10 days of February, 2010. The
same students, with the exception of students from Las Mercedes, took the post-test
around the first week of June, 2010. In Las Mercedes, the post-test took place on
November the 8th, since the teacher had not finished the HBS unit before. In all cases, the
post-test was given a few days after the teacher finished the unit. The reliabilities of the
posttest and the posttest scales are in general high. The reliabilities in the pretest are low,
since students had not been exposed to the topics and are probably guessing the answer.
The distal reliability is low given the small number of items, and also due to problems
related with how clear those items were.
Table 6.9 presents the reliabilities for the paper and pencil assessments including the
sub-scales by type of knowledge.
77
Table 6.9. Reliabilities of the Paper and Pencil assessments. Reliabilities based on assessment Complete Data File Reliability Pre Test 0.566 Reliability Post Test 0.791 Reliability Post Test CR 0.600 Reliability Post Test MC + CR 0.831 Reliability Pre Declarative 0.369 Reliability Pre Procedural 0.256 Reliability Pre Schematic 0.258 Reliability Pre Proximal 0.549 Reliability Pre Distal 0.079 Reliability Post Declarative 0.541 Reliability Post Declarative MC - CR 0.636 Reliability Post Procedural 0.576 Reliability Post Schematic 0.569 Reliability Post Schematic MC - CR 0.667 Reliability Post Proximal 0.742 Reliability Post Proximal MC - RC 0.782 Reliability Post Distal 0.503 Reliability Post Distal MC – CR 0.590 Reliability Post equal Pre 0.743 Reliability Post equal Pre Declarative 0.541 Reliability Post equal Pre Procedural 0.411 Reliability Post equal Pre Schematic 0.507 Reliability Post equal Pre Proximal 0.733 Reliability Post equal Pre Distal 0.197 Analysis of data
Descriptive statistics were calculated including the means and standard deviations
per class and per school. Specific analyses were done in order to provide information to
answer each research sub-question.
1. Is there a difference between IBSE and Control students’ performance in the Human
Body Systems (HBS) paper and pencil tests?
A nested analysis of variance (ANOVA) with pretest as dependent variable was
done to see if groups were equivalent. There were no differences in treatment, but there
78
were differences by school. Therefore, a nested analysis of covariance (ANCOVA), with
pretest as covariate and the posttest as dependent variable was performed.
2. Does IBSE and Control students’ science achievement vary depending on the type of
knowledge tested?
A nested ANCOVA for each type of knowledge as dependent variable, and pretest
as covariate was performed. Correlations among sub-scales of types of knowledge2 were
done within each group and compared.
3. Does IBSE and non-IBSE students’ science achievement vary depending on the
proximity of the assessment used?
A nested ANCOVA with the proximal and distal item sub-scale as dependent
variables, and pretest as covariate were performed. Correlations among sub-scales of
proximity where done within each group and compared.
Part B - Assessing and Comparing Students´ Knowledge of Human Body Systems with Performance Assessments: Paper Towels and Pulse
Subjects
Based on students’ performance on the multiple-choice test, the top ten-, the
middle ten-, and the bottom ten-scoring students were selected from each school to
participate in the performance assessments (due time restrictions and student
availability). Even though the top, middle, and bottom students were relative to each
school, if students had been grouped as a whole sample, ignoring school, the selection of
student participants would change less than 10 percent. This led us to answer the
2 Appendix E presents the items included in each scale.
79
following question: Do students perform similarly on multiple-choice tests and
performance assessments?
Table 6.10 shows the number of students who participated in the performance
assessments. It was not possible to carry out the performance assessments for 5 students
in school 4, since they were absent on the programmed day. Two students were removed
from the database since they did not have a pre-test.
Table 6.10 Number of students who participated in the Performance Assessments.
Paper towels Pulse Type of School
School and class All CD* All CD*
School 1,1 20 20 20 20 School 1,2 13 13 13 13 School 2,1 15 15 15 15 School 2,2 17 16 17 16 School 3,1 14 13 14 13
IBSE Schools
School 3,2 17 15 17 15 School 4,1 11 11 11 11 School 4,2 14 13 15 14 School 5,1 15 15 14 14
Control Schools
School 5,2 16 16 16 16 TOTAL 152 147 152 147
*CD = complete data
Instrument 2 - Paper Towels Performance Assessment
Description of instrument. The hands-on Paper Towels performance assessment was taken from the Stanford
Educational Assessment Laboratory website
(http://www.stanford.edu/dept/SUSE/SEAL/Assessments/PaperTowels.htm), and
translated into Spanish (Appendix F). A science performance assessment, in general,
80
includes a challenge as an invitation to perform an investigation, a response format, and a
scoring system. In this case, the challenge requires students to determine which of three
types of paper towels can absorb the most and which the least amount of water? Students
were given materials such as a pitcher of water, a beaker, a scale, different types of paper
towels, an eyedropper, and a stopwatch, among others. There are no specifications of the
steps to be taken in order to solve the challenge; part of the challenge is for students to
create a procedure for carrying out the investigation. Responses were registered in a
notebook and collected after the assessment ended. The time each student spent
responding to the assessment was also recorded in the notebook.
This assessment can be classified as content-lean since no special knowledge is
needed to perform the assessment. More over, students cannot specifically apply the
knowledge and skills learned when studying Human Body Systems. However, this
assessment is process-rich, since students need to apply diverse scientific skills in order
to come up with one of the many solutions to solve the challenge.
Revision of performance assessments The Paper Towels Performance Assessment was tried with sixth grade students.
After this trial, several language adjustments were made and it was decided that the first
page of the notebook would be individually read with each student to ensure that he or
she understood what was called for. Given that there is no large variety of paper towels in
Colombia, many trials were carried out, with as many paper towels as possible, in order
to find extreme differences between the towel that absorbed the most and the towel that
absorbed the least water. The notebook is presented in Appendix F.
81
Instrument 3 - Pulse Performance Assessment
Description of instrument.
This instrument was an adaptation of the Pulse performance assessment, used by
the Third International Math and Science Study (TIMSS). In this 4th Grade performance
task, students determine a baseline pulse rate and collect data on the changes in that rate
with exercise. They then describe the changes in the data and develop an explanation for
their observations. The assessment can be accessed at: http://pals.sri.com/tasks/5-
8/PulseMS/. This assessment was translated into Spanish and is presented in Appendix G.
This assessment can be classified as content-rich since students can specifically
apply the knowledge and skills learned when studying Human Body Systems. This
assessment does not require as much process knowledge as does Paper Towels.
Revision of performance assessments The Pulse Performance Assessment was tried out with fifth grade students from a
private school during March 2010. Students had problems understanding the assessment,
so the instructions were modified. Students had to find for themselves a way in which to
record the data. However, this was very difficult for them. It was decided to keep the data
table that came with the original TIMSS task instructions. Instructions appeared several
times in the notebook and were read at the beginning.
Test administration and data collection for the Paper Towels and Pulse performance assessments. As above mentioned, the ten top, ten middle, and ten bottom performers in that
test were selected to participate in this part of the study. All these students responded to
both the Paper Towels and the Pulse performance assessments during the same session.
82
The students per school were assigned to start with either the Paper Towels or the Pulse
assessment, and then rotated. For each assessment, one trained observer took notes in an
established format (Appendix H), writing all the steps of what the student did.
Scoring
The paper towels performance assessments were graded using the scoring form
taken from the Stanford Education Assessment Laboratory website to assess students’
notebooks.
The first time paper towels was graded, there was very low interrater reliability
(0.495) since there was no common ground about what counted as a correct scientific
method to determine which paper towel absorbs more water, and the difference between
being careful and being sloppy when measuring. After redefining and unifying these
categories the notebooks were graded a second time and the reliability changed to 0.844.
The pulse assessments were graded using a rubric from TIMSS. One research
assistant and myself were trained in the scoring process. Grading of pulse was less
problematic. Additional examples from TIMSS were used for the grading matrix
including the keywords expected in a correct answer. The interrater reliability for this
assessment was 0.945. The grading matrix is presented in Appendix I.
Table 6.11 presents the reliabilities for each of the performance assessments.
Table 6.11 Reliabilities of the Performance Assessments
Complete Data File Reliability Paper Towels 0.531 Reliability Pulse 0.552
Analysis of data
Data from Part B were used to answer the following research question:
83
4. Does students’ mean achievement scores differ according to their group (IBSE,
Control), achievement on posttest (high medium, low) or content level (rich or lean)
of the performance assessments?
To characterize differences in science achievement, a nested ANOVA was done
done by by ability (high, medium, or low) and by each type of assessment as dependent
variable.
Correlations between the two performance assessments and between the
performance assessments and the sub-scales that measure different types of knowledge
were done within each group and compared.
Part C - Comparing Students’ Performance in Paper and Pencil Tests and Performance Assessments
For this part of the study, all the instruments were used to compare the
performance of students of different abilities (high, medium and low) on the different
types of tests (paper and pencil, Pulse, and Paper Towels).
Proposed Analysis of Data
This part of the study aims to answer the following research sub-question: 5. Do students perform similarly on multiple-choice tests and performance assessments?
Descriptive statistics provided information about mean differences and
correlations between paper and pencil and performance assessments provided insight
about the relationship between these two types of measures, especially looking for
differences in correlations between IBSE and Control students. A nested ANOVA by
ability with Pulse or Paper Towels and paper and pencil test as dependent variables were
performed to identified differences among the assessments and interactions.
84
Correlations among the performance assessments and the paper and pencil tests
that measure different types of knowledge were done by treatment in order to identify
relationships among different variables.
85
CHAPTER 7. RESULTS
This chapter addresses the question of the impact of inquiry-based science
education on student achievement. More specifically, it examines achievement
differences between inquiry science education and typical science education in Colombia
for overall achievement, achievement by types of knowledge, and achievement by the
proximity of the achievement measure to the curriculum. The results are organized as
follows: (a) comparison of the achievement of the IBSE and Control groups using the
paper and pencil test, (b) comparison of the groups using performance assessments, and
(c) comparison of results on the paper and pencil tests with the performance assessments.
Part A - Assessing and Comparing Students´ Knowledge of Human Body Systems: Paper and Pencil Test
In this section, data are presented to address the first two research sub-questions.
Each of the questions will be followed by the related data collected and analyzed from
students in the five dissertation schools.
1. Is there a difference between IBSE and Control students’ performance in the Human
Body Systems (HBS) paper and pencil tests?
This part focuses on the results of the paper and pencil tests including the
multiple-choice pretest and posttest, and the constructed response questions. Table 7.1
presents descriptive statistics for students’ scores on the multiple-choice questions by
type of instruction. There are several differences between the means at pretest and
posttest, and for the gain between pre- and posttest.
86
First, consider pretest mean differences. On the one hand, IBSE schools 1 and 2
have the highest means while IBSE school 3 and Control school 4 have the lowest means.
When comparing the treatment and the Control groups, IBSE students obtained an
average score of 11.84 points on the pretest, while Control students obtained an average
of 10.66 points. A nested analysis of variance (ANOVA) with pretest as dependent
variable identified differences among pretest scores3. More specifically, the design was
Class nested within School and School nested within Treatment (IBSE v. Control).4
There was no treatment effect when performing this ANOVA (F(1,3)=.959 ; p>.05).
There are also no differences among classes within school (F(1,3)=.398; p>.05).
However, there is a significant difference in mean performance in the pretest among
schools within each treatment condition (F(3,363)=29.85; p<.05). When comparing
means, there is a significant difference between schools 1 with 3, 4, and 5, and school 2
with 3 and 4. These means even identify differences between IBSE schools, where 1 and
2 are significantly different than 3. These results lead us to conclude that students and
schools do not come from the same population and some adjustments will be needed in
comparing IBSE and Control groups. Perhaps more importantly, a significant school
effect with so few schools reduces the power of all statistical tests as in this nested
design. School enters importantly into the error term for statistical tests of the treatment
effect. Indeed the difference between the IBSE and Control groups (Table 7.3) is 2.75
points and the effect size 0.62, which is not insubstantial yet not significant.
Second, mean differences were observed on the posttest (with the same items as
on the pretest [“Post-equal-Pre”]). Once again, IBSE schools 1 and 2 have the highest
3 At Pretest, there was a gender effect, were boys scored higher than girls. There are no gender effects in the subsequent analyses of this study. 4 The ANOVA assumptions were met including normality and homogeneity of variances (F=.806; p>.05).
87
means, showing a gain greater than 3.6 points in the former and greater than 5.5 in the
latter. The rest of schools, with the exception of Class 10 in school 5, have gains that
range between 1.15 and 2 points. Class 10 has the lowest gain even though this class had
the same teacher as Class 9.
Table 7.1. Descriptive Statistics of the Results of the Pre and Post Multiple-Choice Tests
School Class Pretest mean
Pretest SD
Post-equal-Pre mean
Post-equal-Pre SD
Gain from Pre to Post
1 1 12.88 4.17 16.66 4.42 3.78 1 2 13.33 3.58 16.97 3.54 3.64 2 3 12.29 3.51 17.89 3.45 5.60 2 4 11.66 3.04 17.24 4.00 5.58 3 5 10.41 3.68 11.68 3.63 1.27
IBSE
3 6 10.15 3.16 12.15 3.33 2.00 4 7 10.15 2.32 11.30 2.71 1.15 4 8 9.49 3.06 11.06 3.27 1.57 5 9 11.54 2.96 13.41 3.00 1.87 C
ontro
l
5 10 11.22 3.66 11.68 3.96 0.46
Since there are differences in the pretest, this variable will be used as a covariate
in subsequent analyses.5 For covariate adjustment to be practical, the correlation between
the Pretest (covariate) and the Post-equal-Pre variables should be and is in my data
around .60 for the multiple-choice measure (Table 7.2). Consequently an analysis of
covariance (ANCOVA) was used not only to address selectivity bias (with differences in
pretest scores in this quasi-experimental design) but also to increase statistical power.
The ANCOVA assumptions were checked including normality and homogeneity of
variances (F=.806; p>.20), linearity between posttest and pretest, and homogeneity of
regression slopes.
5 Selectivity bias provides challenges to interpreting difference among groups, my focus, as well as schools and classes. With limited information, the best that could be done is to use the pretest as a covariate to adjust the comparisons at posttest.
88
Table 7.2. Correlations among the Paper and Pencil Tests
I IBSE
(N=227)
CONTROL (N=138)
Pretest Total
Post-Equal- Pre
Posttest Total
Construc-ted
Response Total
Post MC and
Constructed
Pretest Total
1 .684* .709* .513* .711*
Post-Equal-Pre
.545* 1 .980* .645* .952*
Posttest Total
.569 .958* 1 .666* .976*
Constructed Response Total
.304* .285* .331* 1 .790*
Post MC and Constructed
.562* .903* .950* .534* 1
* Correlation is significant at the 0.05 level.
The treatment effect—IBSE vs. Control—was not statistically significant
(F(1,3)=2.635; p>.05) using the Pre-Equal-Post achievement measure. There was also no
class within school effect (F5,354)=1.348; p>.05). On the other hand, there was a school
effect (F3,354)=23.33; p<.05). Table 7.3 presents the adjusted marginal means when
using a pretest as a covariate. There is a covariate adjusted mean difference in favor of
the treatment group, but it is no statistical difference.
89
Table 7.3 Adjusted Marginal Means of Results for Post-Equal-Pre
School Class Post-equal-Pre adjusted mean
Post-equal-Pre standard Error
Overall means
1 1 15.64 .441 1 2 15.64 .456 2 3 17.28 .455 2 4 17.06 .453 3 5 12.36 .461
IBSE
3 6 13.01 .482
15.162
4 7 12.16 .540 4 8 12.37 .480 5 9 13.31 .447 C
ontro
l
5 10 11.80 .459
12.410
Table 7.4 presents the descriptive statistics of the full paper and pencil posttest,
including the multiple-choice questions that appeared on the pretest, the additional
multiple-choice questions that were included on the posttest, and the constructed response
questions given at posttest. The general patterns seen in this table correspond to what was
described above. IBSE students performed, on average, better than control students but
due to lack of power, the effect was not statistically significant.
Table 7.4 Descriptive Statistics of the Results of the Full Post Paper and Pencil Test
School Class
Constructed Response
mean
Constructed Response
SD
Overall Means
Constructed Response
Post MC and CR mean
Post MC and CR
SD
Overall Means
Post MC and CR
1 1 2.61 1.48 23.59 7.09 1 2 2.64 1.54 24.51 5.05 2 3 3.39 1.81 26.68 5.46 2 4 3.00 1.69 25.45 6.55 3 5 1.24 1.44 17.11 5.94
IBSE
3 6 1.03 1.14
2.252
17.18 5.37
21.98
4 7 .74 .98 16.07 3.83 4 8 .77 1.03 15.57 4.70 5 9 1.74 1.09 20.13 4.66 C
ontro
l
5 0 1.14 1.00
1.236
16.95 5.15
18.07
90
Correlations between constructed response questions and other questions (See
Table 7.2) indicate that high scores on the pretest are associated with high scores in the
Constructed-Response-Total. However, the strength of this association varied between
groups with a low significant correlation between Constructed-Response-Total and
Pretest Total in Control (.304), and a moderate significant one in the IBSE group (.513).
A nested analysis of covariance (ANCOVA) with pretest as covariate and
constructed response or full posttest as dependent variables identified differences among
the constructed response scores and the full posttest combining multiple-choice and
constructed response scores. As mentioned above, the design was Class nested within
School and School nested within Treatment (IBSE v. Control).6 There was no treatment
effect when performing this ANCOVA (Table 7.5) or an effect by class within school
within treatment. On the other hand, the school effect was significant for both
assessments. The treatment effect favored the IBSE students in the constructed response
and posttest total assessments with a mean difference of 1.02 and 3.91 respectively
(Table 7.4). The effect sizes were 0.62 for constructed response and 0.57 for the posttest
total assessment. But due to the large school effect, the power of the statistical test was
compromised and the difference, as said, was not statistically significant.
Table 7.5 Results of the Nested ANCOVA with Constructed Response and Posttest as Dependent Variables
Constructed Response Posttest + Constructed Response
F p F p Treatment 2.26(1,3) p>.05 2.18(1,3) p>.05 School(Treatment) 23.56(3,354) p<.05 18.47(3,354) p<.05 Class(School(Treatment)) 1.06(5,354) p>.05 1.93(5,354) p>.05
6 The ANCOVA assumptions were met including normal distribution of histogram, homogeneity of variances (F=2.78; p<.05), linearity between posttest and pretest, homogeneity of regression slopes, independence of covariance and treatments, and we assume the covariate was measured without error.
91
2. Does IBSE and Control students’ science achievement vary depending on the type of
knowledge tested?
Recall that the multiple-choice test was designed to tap into three types of student
knowledge: declarative, procedural, and schematic. The effects of inquiry science were
examined by type of knowledge demanded by the achievement test. Note one might
expect IBSE student to perform better on procedural and perhaps schematic (inquiry)
items than other items. Table 7.6 presents the descriptive statistics of the overall adjusted
means by type of knowledge tested and type of instruction. The results for all types of
knowledge are similar to those reported above. There is no significant treatment effect
although the mean difference is in the predicted direction.
Table 7.6 Descriptive Statistics of the Overall Adjusted Means of Science Achievement Depending on the Type of Knowledge by Treatment Group Declarative Post total
MC+CR Procedural Post total
MC+CR Schematic Post total
MC+CR Mean Standard
Deviation Mean Standard
Deviation Mean Standard
Deviation IBSE 7.21 1.51 6.37 1.40 6.98 1.60 Control 5.74 2.40 5.52 2.23 5.46 2.55
In the declarative items, IBSE students outperformed the Control students by 1.47
points, with a medium effect size of 0.56. The result in the schematic scale is very
similar, IBSE students outperformed the Control students by 1.52 points, and the effect
size is 0.55. However, in the procedural items, even if the IBSE students performed
better (0.85 points), the effect size was smaller (0.40). Figure 7.1 presents the effect sizes
in the different types of knowledge.
92
Figure 7.1. Effect Size by Types of Knowledge.
Table 7.7 presents the correlations between the pre- and posttest items by types of
knowledge. The correlations among types of knowledge for Control students are, in
general, lower than for IBSE students. In general, moderate significant correlations were
seen among the same type of knowledge at pre- and posttest. However, the correlation
between schematic pre- and posttest scores though not as high it is significant in the
Control group. There is a high correlation between declarative post and schematic post in
the IBSE group (.680) versus a moderate one in the Control group (.357). Additionally,
there is a moderate correlation between procedural post and schematic knowledge at
posttest in the IBSE group (.613) versus a low one in the Control group (.248).
!"
!#$"
!#%"
!#&"
!#'"
!#("
!#)"
*+,-./.012+" 3/4,+56/.-""7,8+9.01,"
!""#$%&'()#&
'$*+#,&-"&./0#,&-"&12-3+#45#&(2&%6#&7-,%%#,%&
93
Table 7.7 Correlations of the Results of Science Achievement Depending on the Type of Knowledge
I IBSE
(N=227)
CONTROL (N=138)
Declara-tive Pre
Procedu-
ral Pre
Schema-tic Pre
Declara-tive Post
Procedu-ral
Post
Schema-tic
Post Declarative Pre 1 .361* .363* .474* .372* .466*
Procedural Pre
.141 1 .212* .537* .533* .527*
Schematic Pre .080 .422* 1 .400* .453* .467*
Declarative Post .441* .216* .242* 1 .567* .680*
Procedural Post
.405* .253* .135 .237* 1 .248*
Schematic Post
.400* .207* .187* .357* .613* 1
* Correlation is significant at the 0.05 level (2-tailed).
Three nested analysis of ANCOVAs with their respective knowledge types as
dependent variables and pretest as covariates were run. Table 7.8 presents the results of
these ANCOVAs. Once again, there were no treatment effects and no class within school
effects.
Table 7.8 Results of the Nested ANCOVA for Knowledge Types with Pretest as Covariate*
Declarative Procedural Schematic
F p F p F p Treatment 2.347 p>.05 2.698 p>.05 1.934 p>.05 School(Treatment) 15.965 p<.05 19.744 p<.05 16.450 p<.05 Class(School(Treatment)) 1.487 p>.05 .379 p>.05 1.671 p>.05
*Degrees of freedom = F(1, 354).
3. Does IBSE and Control students’ science achievement vary depending on the
proximity of the assessment used?
94
Recall that the multiple-choice items varied as to how close they were to the science
curriculum and standards in Colombia. Most items were “close” or “proximal” to what
the students were learning. A few items taken from the Colombian assessment of
achievement were distal—they touched on human body systems but were not directly tied
to the students curriculum. This section compares IBSE with Control on test items
differing by proximity (Table 7.9). The pattern of results is just what we’ve seen earlier:
No treatment effect and a big school effect that explains lack of power. Even though the
mean differences between IBSE and Control are in the predicted direction, they are not
statistically significant.
Table 7.9 Descriptive Statistics of the Results of Science Achievement Depending on the Proximity of the Items
School Class
Proximal Posttotal MC+CR
Overall mean
Distal Posttotal MC+CR
Overall mean
Mean SD Mean SD 1 1 15.59 4.17 4.32 1.93 1 2 16.05 3.38 4.38 1.41 2 3 16.95 3.15 4.84 1.33 2 4 16.68 3.48 4.21 1.86 3 5 11.08 3.44 3.46 1.71
IBSE
3 6 11.41 3.14
14.38
3.65 1.72
4.04
4 7 10.74 2.83 3.45 1.50 4 8 10.48 3.06 3.11 1.66 5 9 12.49 2.71 4.13 1.67 C
ontro
l
5 0 10.89 3.65
11.65
4.00 1.67
3.89
Correlations between different questions measuring proximity are presented in
Table 7.10. Correlations in Control results range from low (Distal Pre with Proximal Pre
and Distal Pre with Proximal Post), to moderate. However, all the moderate correlations
are significant. On the other hand, all correlations for IBSE groups are moderate and
95
significant. It is interesting to observe that there are higher correlations between proximal
and distal items, than between distal pre and distal post items in both groups. This is not
the case in correlations between proximal pre and proximal post items, whose correlation
is the largest in both groups. Low correlations between the pre- and post distal items can
be explained by the low reliability of that scale.
Table 7.10 Correlations of the Results of Science Achievement Depending on Proximity
I IBSE
(N=227)
CONTROL (N=138)
Proximal Pre
Distal Pre
Proximal Post
Distal Post
Proximal Pre 1 .302* .641* .594* Distal Pre .100 1 .365* .342* Proximal Post .535* .095 1 .620* Distal Post .403* .244* .474* 1 * Correlation is significant at the 0.05 level (2-tailed).
Table 7.11 Results of the Nested ANCOVA for Proximity with Pretest as Covariate*
Proximal7 Distal
F p F p Treatment 2.597 p>.05 .339 p>.05 School(Treatment) 28.770 p<.05 4.858 p>.05 Class(School(Treatment)) 1.211 p>.05 .721 p>.05
*Degrees of freedom = F(1, 354).
A nested ANCOVA identified mean differences between IBSE and Control on the
proximity posttest using the corresponding pretest as the covariate. The same pattern of
7 It is important to highlight that proximal items were analyzed trying to identify and separate items that were proximal to the IBSE curriculum and proximal to the Control curriculum. However, the curriculum in both groups was practically the same. Both texts cover each system and its connection with other systems. There was no intention to specifically create different items because of this similar curriculum. When the items were split into IBSE and Control scales (perhaps making close calls), the scales were small (n(items) of 6 and 7) and of low reliability. A statistical comparison was done with each of these scales and there was no IBSE or Control effect in the scales by IBSE items or Control items.
96
results were observed: no statistically significant treatment effect due to low statistical
power; the direction of mean difference was as hypothesized (Table 7.11). With respect
to distal items, there were no effects at all.
Part B - Assessing and Comparing Students´ Knowledge of Human Body Systems with
Performance Assessments: Paper Towels and Pulse 4. Does students’ mean achievement scores differ according to their group (IBSE,
Control), achievement on posttest (high medium, low) or content level (rich or lean)
of the performance assessments?
Recall that for each school, ten high, ten middle, and ten low performing students
were selected based on the multiple-choice posttest. These students took both the pulse
and paper towels performance assessments. The descriptive statistics for these
assessments are presented in Table 7.12.
Table 7.12 Descriptive Statistics for the Performance Assessments.
Pulse Total Paper Towels
Total Paper Towels
Process Sc
hool
Cla
ss
Mean SD Overall mean Mean SD
Overall mean Mean SD
Overall mean
1 1 5.95 1.73 5.05 1.39 3.40 .236 1 2 5.54 1.20 5.00 1.53 3.23 .292 2 3 6.60 1.45 4.20 1.42 3.07 .272 2 4 6.31 1.40 5.00 1.46 3.25 .264 3 5 5.85 1.41 4.62 2.14 3.25 .304
IBSE
3 6 5.67 1.54
5.99
4.67 1.95
4.76
3.34 .272
3.26
4 7 3.91 1.22 5.64 1.50 3.36 .318 4 8 3.85 1.07 4.50 1.65 3.14 .282 5 9 4.47 1.13 4.29 1.98 2.71 .282 C
ontro
l
5 10 4.63 0.96
4.21
3.94 1.95
4.59
2.31 .264
2.88
The table presents an additional data set that only takes into consideration those
questions in the Paper Towels assessment that are related with processes. Recall that the
97
paper towels total score depended on using the correct (controlled) processes to
investigate why some types of towels held more water and some held less water as well
as arriving at the correct answer. In what is reported in the table below the data exclude
the last question (result of which paper towel absorbs more or less water) in the
assessment. It turns out students in both groups could answer the last question without
considering the experimentation or from the observations recorded in their notebooks.
Rather, they could touch and see the towels and get the right answer! These data are
relevant since performance assessments provide direct information about procedural and
strategic knowledge, which the excluded question did not test for. This new version of the
assessment will be named Paper Towels Process and its reliability is 0.627.
In Pulse, IBSE students outperform Control students by 1.78 points. In Paper
Towels Total there were not difference between groups. However, in Paper Towels
Process, once again, the IBSE students have a higher score than that of the Control
students.
A nested ANCOVA with ability and treatment between subject variables and
performance assessments as the within subjects variable provided information about the
effects among performance assessments. This analysis also tested the interaction between
Treatment and Ability.8 The treatment effect—IBSE vs. Control—was statistically
significant for Pulse and for Paper Towels Process (Table 7.13). There was no effect for
school or class in any of the assessments (with the exception of Pulse that showed an
effect in School), nor was there a treatment effect for Paper Towels Total. Therefore, in
8 The ANOVA assumptions were met including normality and homogeneity of variances for pulse (F=.999; p>.20) and for Paper Towels Total (F=1.162; p>.20).
98
both performance assessments, IBSE students performed significantly better than Control
students.
Table 7.13 Results of the Nested ANOVA for Performance Assessments
DF Pulse Paper Towels Total
Paper Towels Process
F p F p F p Treatment 1 27.476 p<.05 .590 p>.05 4.435 p<.05 Ability 2 4.433 p<.05 1.846 p>.05 3.102 p<.05 Ability*Treatment 2 .478 p>.05 .697 p>.05 .693 p>.05 School(Treatment) 3 1.886 p>.05 1.703 p>.05 2.348 p>.05 Class(School(Treatment)) 5 .411 p>.05 1.006 p>.05 .352 p>.05
Part C - Comparing Students’ Performance on Paper and Pencil Tests and Performance Assessments
In this section, data are presented to address the last research sub-question: Do
students perform similarly on paper and pencil tests and performance assessments The
question is followed by the related data collected and analyzed from students that took all
the three assessments.
5. Do students perform similarly on paper and pencil tests and performance
assessments?
I made a distinction between the usual paper and pencil tests used to measure
science achievement and performance assessments. I hypothesized that the two types of
measures, while both measuring knowledge and understanding, also tapped somewhat
different aspects especially inquiry aspects of science achievement. If the two types of
measure are highly correlated, my hypothesis doesn’t hold. However, if there are
different patterns of correlation, I may have some relevant evidence for my hypothesis.
99
Table 7.14 presents the correlations between these assessments. The correlations
between the assessments differ depending on treatment condition. There are no
statistically significant correlations among test for the Control groups but several
significant correlations in the IBSE group.
Table 7.14 Correlations of the Types of Test9
I IBSE
(N=92)
CONTROL (N=55) Posttest
Total Pulse Total
Paper Towels Total
Paper Towels Process
Posttest Total 1 .406*10 .236* .248* Pulse Total .187 1 .050 .017 Paper Towels Total .152 .085 1 .828* Paper Towels Process
.179 .186 .883* 1
* Correlation is significant at the 0.05 level (2-tailed).
Table 7.15 presents correlations among types of knowledge and types of tests in
the Control group. There was a low significant correlation in the Control group between
Pulse and the declarative scale. There are no other significant correlations in this group.
On the other hand, there are several low significant correlations in the IBSE group (Table
7.16), where high scores in Pulse are associated with high scores in the three knowledge
types. The Paper Towels assessment is correlated with the schematic and declarative
scales.
9 Correlations are probably artificially high since they do not reflect the variation and covariation of scores in the middle of the joint distribution. This is due to the nature of the design that was used. 10 When correlating the Pulse performance assessment with questions in the posttest more directly related with this assessment (circulatory and respiratory system, relation between exercise and body systems) the correlation between the Posttest Total and the performance assessment increases to 0.474.
100
Table 7.15 Correlations of the Types of Knowledge and the Performance Assessments
CCONTROL (N=55)
Declarative Post
Procedural Post
Schematic Post
Pulse Total .277* .041 .132 Paper Towels Total .200 .062 .101 Paper Towels Process .206 .091 .107
* Correlation is significant at the 0.05 level (2-tailed). Table 7.16 Correlations of the Types of Knowledge and the Performance Assessments
IIBSE (N=92)
Declarative Post
Procedural Post
Schematic Post
Pulse Total .397* .392* .345* Paper Towels Total .223* .150 .262* Paper Towels Process .214* .181 .261*
* Correlation is significant at the 0.05 level (2-tailed).
A nested ANOVA by ability with Pulse or Paper Towels and paper and pencil test
as dependent variables, was performed. The treatment effect—IBSE vs. Control—,
ability, and school were statistically significant for types of test (Table 7.17). There was
no effect for ability*treatment or class.
Table 7.17 Results of the Nested ANOVA for Types of Test
DF Paper Towels Total
Paper Towels Process
F p F p Treatment 1 25.989 p<.05 34.856 p<.05 Ability 2 139.821 p<.05 166.126 p<.05 Ability*Treatment 1.130 p>.05 1.018 p>.05 School(Treatment) 3 7.607 p<.05 8.026 p<.05 Class(School(Treatment)) 137 .778 p>.05 1.260 p>.05
101
CHAPTER 8. CONCLUSIONS
Colombian science education standards and several educational reforms in the
world now ask students to learn to carry out scientific inquiries in addition to facts and
concepts in science. Even though inquiry-based science education (IBSE) has grown in
Colombia in the past ten years, the impact of these programs has not been systematically
evaluated. The objective of this dissertation was to start addressing the question of the
impact of IBSE on student achievement, and specifically to examine achievement
differences between inquiry science education and typical science education in Colombia
for overall achievement, achievement by different types of knowledge tapped (viz.
declarative, procedural, schematic), achievement by the proximity of the achievement
measure to the curriculum, and achievement as measured by performance assessments.
This is the first effort in Colombia to evaluate the impact of inquiry-based science
teaching measured through student achievement; the results are mixed. In general, there
was, on average no statistically significant treatment effects as measured by the paper and
pencil test over all or by multiple-choice or constructed response questions regardless of
whether they tapped different kinds of knowledge or were proximal or distal to the
curriculum. There was, however, a significant treatment effect on the performance
assessments.
IBSE students demonstrated, on average, “medium” overall performance in the
multiple-choice questions, attaining an average of 54% of the possible points. In contrast,
Control students consistently performed on average 10% lower than IBSE in these types
102
of questions. There was a similar trend for IBSE students outperforming Control students
on the constructed response items. However, performance of both groups was lower on
these items, with IBSE students attaining an average of 37.5% of the possible points and
Control students 20.6%. Students’ low performance on these questions may be due to a
limited ability to communicate scientific knowledge potentially caused by lack of guided
exposition to these types of questions or general literacy problems recognized in
Colombian education (eg PISA and ICFES state exam results).
Even though there was no statistically significant treatment effect as measured by
the variety of paper and pencil tests, IBSE students consistently outperformed Control
students on these different measures of science achievement with a substantial effect size
of 0.6.
When looking at these results more closely, and comparing achievement by types
of knowledge (declarative, procedural, and schematic), I found that the results follow a
similar trend. IBSE students achieve higher scores than Control students, even though
there are no significant differences. There is a greater effect size (around 0.55) on the
declarative and schematic scales than that in procedural scale (0.40). This last finding
strikes me as interesting, since one of the main criticisms to inquiry is that it does not
develop conceptual frameworks since it is focused on hands-on activities and procedures.
Additionally, since the conception of inquiry observed in the literature focuses on the
hands-on or experimental component, one would expect a greater effect size for IBSE
students’ on the procedural-knowledge scale and a smaller effect size on the declarative-
or schematic-knowledge scales. However, in this study, IBSE students had a better
performance on the conceptual facet.
103
One possible explanation for this last finding comes from an analysis of the types
of items used to tap procedural knowledge. Due to the format of the test, the procedural
questions focused on skills such as reading tables and graphs, and the control of
variables, but did not measure in its full depth the procedural facet that is developed
through inquiry (see Chapter 2 on inquiry teaching). This is one of the limitations that
multiple-choice questions present when measuring the procedural aspect (Baxter &
Shavelson, 1994).
IBSE students also outperformed Control students on proximal items (effect size
.65). It is important to consider that both groups used the same human body curriculum.
From a content perspective, what differentiated the curriculum was the teaching method.
IBSE students should able to go beyond the concepts through their experimentation,
while Control students are limited to the content in the book.
On the other hand, when comparing the two groups on distal items, IBSE students
performed only slightly higher (effect size 0.1). This strikes me as unusual given the
comparatively lower performance of IBSE students on distal items than on proximal
items. The source of these differences might be the low reliability in the scale, since there
were only a few distal items in the assessment. Furthermore, distal items are generally not
worked on in the classroom and it is probable that some students guessed when
answering the items, thereby lowering the reliability.
Two performance assessments measured students’ skills in doing scientific
inquiries. These assessments also provide additional information given the limitations of
multiple-choice items mentioned above in regards to measuring the procedural
component of inquiry-based teaching. The first assessment –pulse—was proximally close
104
to the content taught to both groups, although teaching method varied, while the second
assessment –paper towels—focused on students’ scientific inquiry skills. There was a
significant treatment effect on both the content rich performance assessment (pulse-effect
size 1.11) as well as in the content lean assessment (paper towels process-effect size
0.36).
IBSE students are expected to do inquiries better than Control students as
measured by both assessments. IBSE students performed “medium-high” on both
performance assessments, receiving about 66% of the total possible points. Control
students, on the other hand, had a lower yet different performance on both assessments,
correctly answering 46% of questions on pulse and 56% on paper towels.
There are stronger correlations in the IBSE group between the three types of
knowledge —declarative procedural, and schematic—and pulse. The pulse performance
assessment has procedural elements that are similar to those of the paper and pencil test,
since students register information and search for patterns in a table. This performance
assessment also has declarative and schematic elements in that students are asked to
provide explanations for their pulse data. Consequently IBSE students have the inquiry
knowledge needed to do the performance task and could rely on the declarative,
procedural and schematic knowledge. The correlations for control students were
consistently low. They simply did not know how to carry out science investigations and
they worked these tasks in an erratic manner, with no specific types of knowledge
associated with their performance. Hence the low correlation with the knowledge-type
measures.
105
The paper towels performance assessment does not require the skills of a typical
activity in the Human Body Systems unit. The fact that IBSE students performed
significantly better than Control students implies that the former students developed
general investigative and problem solving skills that could be related with strategic
knowledge, and therefore with inquiry skills as a result of the HBS unit.
The significant higher performance of IBSE students on performance assessments
corresponds to the general conception that students who participate in IBSE programs
develop the investigative skills in greater depth than students who don’t. These results are
different from those found in other studies (Pine et al., 2006) where there was no
significant treatment results in three out of four performance assessments. In my study,
there was an effect of inquiry in both measures including Paper Towels, where Pine et al.
(2006)11 found none.
This study provides yet another albeit not conclusive result about inquiry-based
science teaching. However, and even though there was no statistical difference in some of
the measures, IBSE students consistently outperformed Control students in all measures
with a medium to large effect size. Additionally, there was a significant effect in the
performance assessments. In large part, then, the lack of statistical significance could be
traced to the low power of these tests based on the nature of the nested design where
schools nested within treatments varied in the achievement they produced in their
students, in both treatment conditions.
11 Pine et al. (2006) treatment groups studied three or four ‘‘units’’ for 6 – 8 weeks each during a school year.
106
Limitations and Challenges
Previous studies that compared science inquiry teaching to other approaches
showed diverse limitations including inconsistent measures of achievement, lack of
observation of the treatment, differences in the treatments, and a lack of a clear definition
of inquiry. This dissertation aimed at reducing several of those limitations by providing
diverse measures including the paper and pencil tests with the multiple-choice and
constructed response questions and the performing assessments, a detailed description of
the treatment and the control including the observation of classes and interviews with the
teachers, a detailed construction of items that tap into different types of knowledge, and a
clear framework for what inquiry means in the study.
Perhaps one of the most important limitations of previous studies is the lack of
classroom observations in order to pair the conception of inquiry teaching with what gets
enacted in the classroom. This study was able to carry out a detailed revision of the types
of practices that occur in the science classroom of both IBSE and the Control groups.
Nevertheless, this study has several limitations including the selection of schools,
sample size, and nested design. Schools were selected by convenience and availability
and by trying to match them on a set of characteristics including type of administration
(concession), socio-economic status of the community, and results on standardized
exams. Nevertheless the schools and students differed at pretest. Therefore, the data
analysis focused on ways in which this difference could be addressed by using a nested
design and the pretest as covariate.
Even though the number of students participating in this study is 365, the sample
size, using a nested design, was sufficient, the number of schools participating was not
107
(cf. Pine et al. ,2006). It was a challenge to find both Control and IBSE schools that were
willing to participate in the study. Additionally, within the interested IBSE schools,
teacher rotation is a major factor that in this case led to the exclusion of two IBSE
schools from the study. Furthermore, it is difficult to follow trained IBSE teachers closely
to insure fidelity of implementation of the treatment. Control schools were difficult to
find since it was challenging to identify similar concession schools that were willing to
participate in the study and were teaching the same unit. Because of these reasons, and
other logistical ones (money and time), this study only included 2 Control schools and 3
IBSE schools. The small school sample size affects the statistical power of the design and
results in low power to detect the consistent mean differences in favor of IBSE students.
Some of the difference could also be due to chance among the five schools. This
interpretation is bolstered when the design for the performance tasks is considered. Here
we matched students within each school on posttest (low, medium, and high) and in this
ability x treatment nested design, we found statistically significant differences.
Reflections and Directions for Future Research
The results obtained in this dissertation can serve as a stimulus for further
research on the impact of inquiry-based science teaching. There are several lessons that
can be applied in future research. For instance, the effects of inquiry-based science
teaching is directly linked with the teacher. In order to have a better representation of
inquiry teaching in studies, it is very important to carry out the classroom observations
and interviews during the selection process. Additionally, not just the number of students
108
matters. But the number of classes within a school and the number of schools need to be
sufficiently large to provide powerful tests of the IBSE effect.
The development of the paper and pencil assessment was a very rich, yet long
process. The final instrument used in this study can now serve as the basis for future
research with greater number of students, classes and schools. Incorporating aspects such
as types of knowledge and proximity provides the possibility to carry out more in-depth
analysis of aspects where inquiry teaching can have an effect. Additionally, including
other types of questions and assessments such as constructed response questions and
performance assessments provides greater information about student science
achievement. Therefore, the use of multiple measures was an important aspect of this
study and allowed a greater understanding about the impact of inquiry teaching. Future
studies should continue to include multiple-achievement measures.
Even though the results of this research provide inconclusive evidence about the
impact of inquiry-based science teaching, there are several possibilities for further
research.
First, additional research can also shed light on other ways in which to measure
scientific inquiry skills. Since there were a few procedural items that tap into inquiry
skills, the paper and pencil test showed limitations in measuring some inquiry skills that
were tapped with performance assessments. However, there might be additional ways in
which to tap into different types of knowledge with pencil and paper tests that are taught
through inquiry.
Second, IBSE students showed higher gains in schematic knowledge when
compared to Control students. Further research could provide data on the types of skills
109
that students acquire with inquiry, which allow them to go deeper into each concept and
apply it to new situations. A question that can be addressed with additional research
might focus on the effect of inquiry in student learning abilities.
Last, additional studies can address the difference found between achievement in
proximal versus distal items. Even though IBSE students always outperformed Control
students, the effect size on the distal items was considerably lower than on proximal
items. Measures that include a greater number of distal items can provide further
information about what inquiry students are able to achieve. The use of national
assessment data, including results of Colombian students’ achievement at grades 5
(SABER 5) and 9 (SABER 9), can shed light in this topic. If these data sets are used to
compare inquiry students with control students, further information can be collected
regarding the nature of the differences among teaching approaches. Of course the
challenge will be to assure that teachers labeled inquiry teachers are actually teaching
with inquiry!
A Final Note…
The question of how to measure student achievement is very important. This
dissertation addressed this question by using different measures to try to get a better
picture of what each student knows and can do with scientific knowledge and skills. The
initial stages of this research synthesized a framework for inquiry-based science
education. This specific study mapped teachers’ practices to all the facets included in the
framework. However, there are aspects within the framework that are not necessarily
addressed by the measures used in this study such as the design of experiments or the
social facet. Further research can provide additional ways in which to measure student
110
achievement in all the components that make up inquiry-based science teaching. This
way, what a student learns through inquiry teaching can be more accurately represented.
Beyond the specific test taking skills or performance abilities, inquiry aims to
provide a framework for students to think and approach the natural world in a more
systematic and deep way. It goes beyond specific concepts and leads students to be able
to schematically and strategically use available information. In the long run, research can
provide information of a key question: “What is the role of inquiry-based science
education in 21st century skills?”
111
LITERATURE CITED Alianza Educativa. 2008. Proyecto Educativo Institucional. Alianza Educativa. Retrieved
December 23, 2010, from http://www.alianzaeducativa.edu.co/images/documentos/pei_2008.pdf
Alianza Educativa. 2010. ¿Quiénes Somos? Alianza Educativa. Retrieved December 23,
2010, from http://www.alianzaeducativa.edu.co/iquienes-somos.html American Association for the Advancement of Science (1990). Science for All
Americans. New York: Oxford University Press. Baxter, G.P., & Shavelson, R.J. (1994.) Science performance assessments: Benchmarks
and surrogates. International Journal of Educational Research, 21, 279-298.
Barrera-Osorio, F. (2006). The impact of private provision of public education : empirical
evidence from Bogota's concession schools. IDEAS: Economics and Finance Research. Retrieved December 21, 2010, from http://ideas.repec.org/p/wbk/wbrwps/4121.html
Berg, C. A. R., Bergendahl, V. C. B., Lundberg, B. K. S., & Tibell, L. A. E. (2003).
Benefiting from an open-ended experiment? A comparison of attitudes to, and outcomes of, an expository versus an open-inquiry version of the same experiment. International Journal of Science Education, 25(3), 351-372.
Bredderman, T. (1983). Effects of Activity-Based Elementary Science on Student
Outcomes: A Quantitative Synthesis. Review of Educational Research, 53(4), 499-518.
Bruner, J. S. (1961). The Act of Discovery. Harvard Educational Review, 31(1), 21-32. Chang, C.-Y., & Mao, S.-L. (1999). Comparison of Taiwan Science Students' Outcomes
with Inquiry-Group versus Traditional Instruction. Journal of Educational Research, 92(6), 340-346.
Colsubsidio. (2010). Colsubsidio. Retrieved December 10, 2010, from
http://www.colsubsidio.com/porta_serv/educacion/formal.html CSE: Insights: An Inquiry-Based Elementary School Science Curriculum. (2011). EDC's
Center for Science Education (CSE) Home. Retrieved March 30, 2011, from http://cse.edc.org/curriculum/insightsElem/insights6.asp
Duschl, R. A. (2003). Assessment of inquiry. Everyday assessment in the science
classroom, 41-59.
112
Duschl, R. A., Schweingruber, H. A., & Shouse, A. W. (2007). Taking Science to School:
Learning and Teaching Science in Grades K-8: National Academies Press. Editorial Santillana - Casa Ciencias 5. (n.d.). Grupo editorial Santillana, Colombia.
Retrieved March 30, 2011, from http://santillana.com.co/docentes/index.php?player_init/Q2FzYV9DaWVuY2lhc181/c3R1ZGVudA==/
El Tiempo. (2008, September 6). Proyecto de colegios en concesión cumple 9 años en
funcionamiento. El Tiempo. Retrieved December 23, 2010, from www.eltiempo.com/archivo/documento/CMS-4505006
Furtak, E. M. (2006). The Problem with Answers: An Exploration of Guided Scientific
Inquiry Teaching. Science Education, 90(3), 453-467. Furtak, E. M., & Seidel, T. (2008). Recent Experimental Studies of Inquirey-Based
Teaching: A Conceptual Review and Meta-analysis. Paper presented at the National Association of Research in Science Teaching Conference.
Furtak, E. M., Seidel, T., & & Iverson, H. (2009). Recent Experimental Studies of
Inquiry-Based Teaching: A Meta-analysis and Review. Paper presented at the European Association for Research on Learning and Instruction. August 25-29, Amsterdam, Netherlands.
Furtak, E. M., Shavelson, R. J., Shemwell, J. T., & Figueroa, M. (2009). To Teach or Not
to Teach Through Inquiry: Is that the question? Paper presented at the From Child to Scientist: A festschrift to honor the scientific and educational contributions of David Klahr.
Geier, R., Blumenfeld, P. C., Marx, R. W., Krajcik, J. S., Fishman, B., Soloway, E.
(2008). Standardized Test Outcomes for Students Engaged in Inquiry-Based Science Curricula in the Context of Urban Reform.
IAP Science Education Programme. (2006). Report of the Working Group on
International Collaboration in the Evaluation of Inquiry-Based Science Education (IBSE) programs. Retrieved December 12, 2009 from: http://www.ianas.org/Santiago_Report_SE.pdf
Instituto para la Investigación Educativa y el Desarrollo Pedagógico, IDEP, 2010 Insights: an elementary hands-on science curriculum. Human Body Systems. (Teacher's
guide, 2nd ed.). (20032007). Dubuque, Iowa: Kendall/Hunt.
113
Klahr, D., & Nigam, M. (2004). The Equivalence of Learning Paths in Early Science Instruction. Effects of Direct Instruction and Discovery Learning. Psychological Science, 15(10), 661-667.
Li, M., Ruiz-Primo, M.A., & Shavelson, R.J. (2006). Towards a science achievement
framework: The case of TIMSS 1999. In S. Howie & T. Plomp (Eds.), Contexts of learning mathematics and science: Lessons learned from TIMSS. London: Routledge, Pp. 291-311.
Li, M., & Shavelson, R. J. (2004). Validating the links between knowledge and test items
from a protocol analysis.Unpublished manuscript. Manual de Convivencia. (2007). PAGINA OFICIAL DEL GIMNASIO SABIO CALDAS.
Retrieved December 23, 2010, from http://sabiocaldas.edu.co/Manual_convivencia.html
Ministerio de Educación Nacional (MEN). (2004). “Estándares básicos de competencias
en Ciencias Naturales y Ciencias sociales.” Ministerio de Educación Nacional. Julio 2004.
Minner, D. D., Levy, A. J. and Century, J. (2010), Inquiry-based science instruction—
what is it and does it matter? Results from a research synthesis years 1984 to 2002. Journal of Research in Science Teaching, 47: 474–496. doi: 10.1002/tea.20347
National Research Council. (1996). National Science Education Standards. Washington,
D.C.: National Academy Press. National Science Foundation (1997). The Challenge and Promise of K-8 Science
Education Reform. Foundations: A monograph for professionals in science, mathematics, and technology education, 1.
Patrinos, H. (2005, October 5). Education Contracting: Scope of Future Research. World
Bank. Retrieved December 23, 2010, from citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.168.5608&rep=rep1&type=pdf
Pine, J., Aschbacher, P., Roth, E., Jones, M., McPhee, C., Martin, C., et al. (2006). Fifth
graders' science inquiry abilities: A comparative study of students in hands-on and textbook curricula. Journal of Research in Science Teaching, 43(5), 467-484.
Rodriguez, A., & Hovde, K. (2002). CiteSeerX — The Challenge of School Autonomy:
Supporting Principals. CiteSeerX. Retrieved December 21, 2010, from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.124.1536
114
Ruiz-Primo, M. A., & Furtak, E. M. (2004). Informal Formative Assessment of Students' Understanding of Scientific Inquiry: National Center for Research on Evaluation, Standards, and Student Testing, Center for the Study of Evaluation, Graduate School of Education & Information Studies, University of California, Los Angeles.
Ruiz-Primo, M. A., & Shavelson, R. (1996). Rhetoric and Reality in Science
Performance Assessments: An Update. Journal of Research in Science Teaching. 33(10), 1045-1063.
Ruiz-Primo, M. A., Shavelson, R. J., Hamilton, L., & Klein, S. (2002). On the evaluation
of systemic science education reform: Searching for instructional sensitivity. Journal of Research in Science Teaching, 39(5), 369-393.
Ruiz-Primo, M. A., Wiley, E., Rosenquist, A., Schultz, S., Shavelson, R. J., Hamilton, L.
(1998). Performance Assessment in the Service of Evaluating Science Education Reform.
Schneider, R. M., Krajcik, J., Marx, R. W., & Soloway, E. (2002). Performance of
Students in Project-Based Science Classrooms on a National Measure of Science Achievement. Journal of Research in Science Teaching, 39(5), 410-422.
Secretaría de Educación del Distrito (SED). 2009. Boletín estadístico sector educativo
Bogota 2009. SEDBOGOTA - Inicio. Retrieved December 20, 2010, from http://www.sedbogota.edu.co//index.php?option=com_content&task=view&id=33&Itemid=174
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2001). Experimental and Quasi-
experimental Designs for Generalized Causal Inference: Houghton Mifflin. Shavelson, R.J., Baxter, G.P., & Pine, J. (1991). Performance assessments in science.
Applied Measurement in Education, 4, 347 – 362. Shavelson, R. Ruiz-Primo, M.A., Li, M. and Ayala, C. C. (2003) Evaluating New
Approaches to Assessing Learning. Center For Research On Evaluation, Standards, And Student Testing CSE Report 604, UCLA.
Shemwell, J. Fu, A., Figueroa, M., Davis, R & Shavelson, R. 2008. Assessment in Schools – Secondary Science in McCulloch, G., & Crook, D. The Routledge international encyclopedia of education (pp. 300-310). London: Routledge.
Schwab, J. J. (1962). The teaching of science as enquiry. In J. J. Schwab & P. F.
Brandwein, The teaching of science, Cambridge, MA: Harvard University Press. Solano-Flores, G., & Shavelson, R.J. (1997). Development of performance assessments
in science: Conceptual, practical, and logistical issues. Educational Measurement:
115
Issues and Practices, 16, 16–25. Tamir, P., Stavy, R., & Ratner, N. (1998). Teaching science by inquiry: assessment and
learning. Journal of Biological Education, 33(1), 27-32. Uribe, C., Murnane, R., & Willet, J. (2003). Why do students learn more in some
classrooms than in others? Evidence from Bogota. Graduate School of Education. Retrieved December 10, 2010, from gseacademic.harvard.edu/.../Uribe_Murnane_Willett_2003.pdf
Villa, L., & Duarte, J. (2002). Los colegios en concesión de Bogotá, Colombia: Una
experiencia innovadora de gestión escolar reformas o mejoramiento continuo. Comisión Vallecaucana por la Educación. Retrieved December 10, 2010, from Http://www.cve.org.co/pdf/nuevos2004/colegiosconcesion.pdf
Von Secker, C. E., & Lissitz, R. W. (1999). Estimating the Impact of Instructional
Practices on Student Achievement in Science. Journal of Research in Science Teaching, 36(10), 1110-1126.
Walker, D. F., & Schaffarzick, J. (1974). Comparing Curricula. Review of Educational
Research, 44(1), 83-111.
116
APPENDICES12
12 The formatting of some appendices was changed to fit in this dissertation.
117
Appendix A. Interview Instrument for Teachers Participating in the Study.
Información de los profesores Esta encuesta busca tener información sobre las características de los profesores que participaron en la investigación sobre la enseñanza de las ciencias. Esta encuesta es de carácter confidencial. Muchas gracias por su colaboración.
Nombre: ________________________________________________________________
Institución: ______________________________________________________________
Celular: ___________________________________
Correo: ___________________________________
1 ¿Cuántos años tiene?
Encierre solo una opción
Menos de 25 ---------------------1 25–29---------------------------2 30–39---------------------------3 40–49---------------------------4 50–59---------------------------5 60 o más ------------------------6
2 Sexo
Encierre solo una opción
Mujer--------------------------1 Hombre------------------------2
3
Labores que desempeña en la institución:
__________________________________
4 Grados en que dicta (marque con una X):
1. ______ 2. ______ 3. ______ 4. ______ 5. ______ 6. ______ 7. ______
5 Cursos que dicta (ej. 5A)
# de estudiantes por curso (ej. 42)
6 Jornada (marque con una X): Mañana ______ Tarde ______ Única ______
7 ¿Cuánto tiempo lleva enseñando en la institución actual?
____________________________ Número de años
8 Después de obtener su diploma, ¿cuántos años lleva enseñando en total?
____________________________ Número de años enseñando tiempo completo
118
9 ¿Cuál es el mayor grado de estudio alcanzado?
Encierre solo una opción
No terminó el bachillerato -----------------1 Completó el bachillerato ------------------2 Usted es normalista superior ---------------3 Obtuvo un título de licenciado o pedagogo ------------------- 4 Obtuvo un título de pregrado ---------------5 Cual: ______________________ Obtuvo un título de especialización ----------6
Cual: ______________________ Obtuvo un título de un programa en Maestría--7
Cual: ______________________
10 A. Mientras hacía su educación universitaria, ¿cuál fue el nivel de su licenciatura?
Encierre solo una opción
a) Educación – Preescolar ---------------1 b) Educación – Básica primaria ------------ 2 c) Educación – Secundaria ---------------- 3 d) Educación - Otra --------------------- 4 B. ¿Cuál fue su énfasis de su licenciatura?
Si aplica encierre más de una
a) Matemáticas -------------------------1 b) Biología ----------------------------2 c) Química ----------------------------3 d) Física ------------------------------4 e) Humanidades ------------------------5 f) Lenguajes ---------------------------6 g) Artes -------------------------------7 h) Otra área ----------------------------8
Enseñanza de Ciencia
119
11 ¿Qué tan bien preparado se siente para enseñar en las siguientes áreas de ciencia?
Encierre solo una opción por fila
Muy bien preparado Algo preparado
No muy bien preparado
No aplica
A. Ciencias de la vida a) Estructuras principales del cuerpo y sus funciones
en los seres humanos y otros organismos (plantas y animales) --------------1 -- 2 -- 3 --4 b) La reproducción y el desarrollo de las plantas y animales (transmisión de
características generales, los ciclos de vida de organismos conocidos)------- 1 -- 2 -- 3 --4 c) Conexiones entre los diferentes sistemas del cuerpo humano ------------- 1 -- 2 -- 3 --4 d) La salud humana (ejemplo: la transmisión, prevención de las
enfermedades transmisibles los signos de salud/enfermedad , dieta, ejercicio) - 1 -- 2 -- 3 --4 B. Ciencias Físicas a) Clasificación de objetos o materiales sobre la base de propiedades físicas
(Ejemplo: la masa, forma, volumen, color, dureza, textura, calor / conductividad eléctrica. Atracción magnética) --------------------------------------------- 1 -- 2 -- 3 --4
b) Las Fuentes comunes de energía y las formas y sus usos prácticos (ejemplo: el viento, el sol, la electricidad, el agua, alimentos)--------------- 1 -- 2 -- 3 --4
c) La luz (ejemplo: las Fuentes y el comportamiento) ----------------------- 1 -- 2 -- 3 --4 d) Circuitos eléctricos ----------------------------------------------- 1 -- 2 -- 3 --4 e) Propiedades magnéticas -------------------------------------------- 1 -- 2 -- 3 --4
12 Los estudiantes de quinto grado, ¿cuántas horas
de ciencias reciben a la semana?
120
______________________________(Horas)
13
En clase de ciencias con qué frecuencia los estudiantes hacen lo siguiente:
Encierre solo una opción por fila
Todas o casi todas las clases Aproximadamente la mitad de las clases
Algunas clases Nunca
a) Observan los fenómenos
naturales tales como el clima o el crecimiento de una planta y describe lo que ve ---------- 1 -- 2 -- 3 --4
b) Me observan haciendo un experimento de ciencia- 1 -- 2 -- 3 --4
c) Hacen experimentos o proyectos ---------- 1 -- 2 -- 3 --4
d) Diseñan o planean experimentos o proyectos ---------- 1 -- 2 -- 3 --4
e) Trabajan en pequeños grupos en experimentos o proyectos-------- 1 -- 2 -- 3 --4
f) Leen sus libros de texto u otros recursos ------ 1 -- 2 -- 3 --4
g) Memorizan factores y principios -------- 1 -- 2 -- 3 --4
h) Dan explicaciones sobre algo que se está estudiando--------- 1 -- 2 -- 3 --4
i) Relacionan lo aprendido de ciencia con su vida cotidiana ------ 1 -- 2 -- 3 --4
j) Trabajan individualmente a su propio ritmo ---- 1 -- 2 -- 3 --4
14
A finales del año escolar pasado (2010), ¿qué porcentaje del tiempo de enseñanza usó en cada una de las siguientes áreas?
Escriba el porcentaje
El total debe sumar 100% a) Ciencia de la vida (incluye
cuestiones ambientales) --------_____% b) Ciencia física (incluye temas
de física y química) -----------_____% c) Ciencia de la tierra (incluye
la tierra y el sistema solar)-------_____% d) Otro, por favor, especifique:
________________________-----____%
Total --------------------------- 100% 15
A. ¿Qué libros de texto o guías utiliza para enseñar ciencias a los estudiantes de quinto grado?
______________________________________________________ _______________________________________ _______________________________________ B. ¿Cómo utiliza los libros para enseñar ciencias a los estudiantes de quinto grado?
Puede encerrar ambas
Como base para mis clases --------------1 Como recurso adicional-----------------2
16
En los últimos dos años, ¿En qué talleres ha participado sobre estos temas?
Encierre solo una opción por fila
No Si
a) Contenido en ciencias (disciplinar) --1--2 b) Enseñanza en ciencias (metodología) 1--2 c) Currículo en ciencias -------------1--2 d) Integración de tecnologías
de información en la ciencia--------1--2 e) Mejorar el pensamiento critico
o habilidades de investigación ------1--2 f) Evaluaciones en ciencias ----------1--2 g) Enseñanza de la ciencia
basada en indagación ------------1---2 h) Otros.
____________________________________ _______________________ _______________________
121
17 Describa en un párrafo como se describirá usted como profesor de ciencias, puede dar ejemplos de las actividades que realiza al igual que contar sobre los textos y materiales que utiliza. _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________
Muchas gracias.
122
Appendix B. Final Version of the Pretest
GRADO 5º
PRE-PRUEBA DE CUERPO HUMANO
Nombre: _____________________Apellido:___________________________________ Nombre del colegio: _________________________________________Curso: ______ Nombre del profesor(a) de ciencias: ____________________________________________ Eres: Niño Niña ¿Desde qué grado estás en este colegio? 0 1 2 3 4 5
A continuación vas a encontrar unas preguntas que debes contestar con mucho cuidado.
Instrucciones: Para contestar las siguientes preguntas de opción múltiple sigue estas instrucciones: 1. Lee cuidadosamente cada pregunta y elige la opción correcta. 2. Encierra con un círculo la respuesta correcta. 3. No hay límite de tiempo. Tienes el tiempo suficiente para contestar todas las preguntas.
123
EJEMPLOS Contesta estos ejemplos para que practiques. Encierra con un círculo la respuesta correcta. EJEMPLO 1
¿Cuál es el resultado de sumar 25 y 25? A. 30. B. 40. C. 50. D. 60. ASÍ DEBIÓ QUEDAR ENCERRADA TU RESPUESTA: EJEMPLO 1
¿Cuál es el resultado de sumar 25 y 25? A. 30. B. 40. C. 50. D. 60.
124
EJEMPLO 2
¿En qué mes se celebra la independencia de Colombia? A. Septiembre. B. Julio. C. Octubre. D. Enero. ASÍ DEBIÓ QUEDAR ENCERRADA TU RESPUESTA: EJEMPLO 2
¿En qué mes se celebra la independencia de Colombia? A. Septiembre. B Julio. C. Octubre. D. Enero.
AHORA DETENTE Y ESPERA A QUE TE DEN LAS INSTRUCCIONES PARA
PODER EMPEZAR LA PRUEBA.
125
CHOMP01
1. ¿De qué están compuestos todos los seres vivos?
CHOMP02
2. ¿Cuál es la función del corazón?
CHOMP03
3. ¿Cuál es el mejor lugar para sentir el pulso?
CHOMP04
4. ¿Qué sistemas trabajan juntos durante el proceso de respiración?
CHOMP05
A. Órganos B. Sistemas C. Células D.
Tejidos
A. Bombear sangre. B. Combatir enfermedades. C. Intercambiar gases. D.
Regular la temperatura.
A.
B. C. D.
A. Respiratorio y reproductor. B. Circulatorio y respiratorio. C. Respiratorio y digestivo. D. Nervioso y respiratorio.
126
5. ¿Cuál es la función del sistema digestivo?
CHOMP06
6. ¿Qué sistema incluye el corazón y los vasos sanguíneos?
CHOMP07
7. ¿Qué pasa con tu pulso y tu ritmo respiratorio cuando corres muy rápido?
CHOMP08
8. ¿Cuál de las siguientes partes del cuerpo participa en la digestión mecánica?
CHOMP09
9. Las personas respiran agitadamente cuando hacen ejercicio. Esto se explica porque al hacer esfuerzo físico
CHOMP10
A. Transformar los alimentos que entran al cuerpo. B. Llevar oxígeno a todo el cuerpo. C. Transportar los nutrientes a todo el cuerpo. D. Regular la temperatura del cuerpo.
A. Sistema digestivo. B. Sistema nervioso. C. Sistema circulatorio. D. Sistema respiratorio.
A. Tu pulso y tu ritmo respiratorio aumentan. B. Tu pulso aumenta y tu ritmo respiratorio disminuye. C. Tu pulso y tu ritmo respiratorio disminuyen. D. Tu pulso disminuye y tu ritmo respiratorio aumenta.
A. El riñón. B. Los dientes. C. El corazón. D. El pulmón.
A. disminuye la necesidad de nutrientes en el cuerpo. B. aumenta la necesidad de oxígeno en el cuerpo. C. disminuye la necesidad de flujo sanguíneo en el cuerpo. D. aumenta la necesidad de dióxido de carbono en el cuerpo.
127
10. El latido del corazón es un movimiento
CHOMD011
11. Un investigador quiere saber si la cantidad de glóbulos rojos en la sangre de las mujeres embarazadas es igual o diferente al de las mujeres adultas que no están embarazadas. Para averiguar lo anterior, ¿de cuál de los siguientes grupos de mujeres le sugerirías al investigador que extrajera y examinara la sangre?
CHOMP12
A. aprendido. B. involuntario. C. controlado. D. voluntario.
A.
B.
C.
D.
128
12. ¿Qué proceso ocurre en la boca para que comience la digestión química de un alimento?
CHOMP13
13. ¿Cuál de las siguientes opciones es un ejemplo de un movimiento voluntario?
CHOMP14
14. Tomás quiere medir sus pulsaciones en un minuto. Para eso él debe poner sus dedos en algún lugar donde sienta el pulso, y contar las pulsaciones durante
A. Masticación B. Salivación C. Ingestión D. Peristaltismo
A. Sudar B. Aplaudir C. Estornudar D. Digerir
A. 15 segundos y multiplicarlas por 2. B. 20 segundos y multiplicarlas por 4. C. 15 segundos y multiplicarlas por 4. D. 20 segundos y multiplicarlas por 2.
129
CHOMP15
15. ¿En qué lugar de la siguiente figura se realiza la conexión entre el sistema circulatorio y el digestivo?
CHOMP16
16. ¿Para qué sirve un estetoscopio?
A. Esófago B. Estómago C. Intestino grueso D. Intestino Delgado
A. Ver la parte de adentro de la oreja. B. Mirar la pupila en el ojo. C. Medir las pulsaciones en el brazo. D. Oír las respiraciones de los pulmones.
130
CHOMP17 17. ¿Cuáles pasos seguirías para observar células de cebolla en el microscopio?
D.
A.
B.
C.
131
CHOMP18
18. Daniel puso una bolsa semipermeable con colorante en una botella con agua limpia como se muestra en la siguiente figura:
¿Cuál de las siguientes botellas muestra lo que pasó al otro día?
A. B.
C. D.
CHOMP19
19. Los glóbulos rojos permiten que la sangre
A. transporte nutrientes a todo el cuerpo. B. transporte desechos a todo el cuerpo. C. transporte oxígeno a todo el cuerpo. D. transporte agua a todo el cuerpo.
132
CHOMD02
20. En la huerta hicieron el experimento que se muestra a continuación:
Con este experimento se quiere investigar el efecto de:
CHOMP21
21. Ana quiere saber si la harina tiene almidón. ¿Qué debe agregar Ana a la harina para saber si tiene almidón?
A. B. C. D.
A. el tipo de suelo en el crecimiento de las plantas. B. la forma de las hojas en el crecimiento de las plantas. C. el tamaño del tallo en el crecimiento de las plantas. D. el sol en el crecimiento de las plantas.
133
CHOMP22
22. La figura de abajo representa un modelo de caja torácica y sus diferentes partes. ¿Qué representan la primera y segunda parte del tubo plástico?
CHOMP23
23. ¿Cuándo digieres comida, dónde la utiliza tu cuerpo?
CHOMP24
24. ¿Cómo trabajan juntos el corazón y los pulmones?
A. La primera parte es el bronquio y la segunda parte es la tráquea. B. La primera parte es la tráquea y la segunda parte es el diafragma. C. La primera parte es la tráquea y la segunda parte es el bronquio. D. La primera parte es el diafragma y la segunda parte es el bronquio.
A. Sólo en la sangre de tu cuerpo. B. Sólo el estómago de tu cuerpo. C. Sólo los pulmones de tu cuerpo. D.
En las células de tu cuerpo.
A. El movimiento de los pulmones ayuda al corazón a bombear sangre. B. Los pulmones dan oxígeno a la sangre que el corazón bombea a través del cuerpo. C. El corazón y los pulmones trabajan juntos para ayudar a digerir la comida. D. El corazón bombea la sangre, y los pulmones circulan la sangre a través del cuerpo.
134
CHOMP25
25. ¿Cuál de los siguientes diagramas describe el funcionamiento de los riñones?
CHOM P26
26. En el intestino las vellosidades (pelitos) permiten que se difundan
CHOMP27
27. ¿En cuál de los siguientes procesos del cuerpo humano ocurre difusión? A. Masticación de alimentos en la boca.
CHOMD03
A.
B.
C.
D.
A. menos nutrientes porque la superficie disminuye. B. más nutrientes porque la superficie aumenta. C. menos nutrientes porque la superficie aumenta. D. más nutrientes porque la superficie disminuye.
B. Transformación de alimentos en el estómago. C. Absorción de nutrientes en el intestino delgado. D. Bombeo de sangre desde el corazón.
135
28. Unos estudiantes midieron durante ocho meses la temperatura ambiental y la temperatura de unos papagayos; con los resultados elaboraron la siguiente gráfica:
Teniendo en cuenta la información de la gráfica. ¿Cuál de las siguientes conclusiones es la más acertada? A. La temperatura ambiental influye sobre la temperatura de los papagayos.
B. Los papagayos mantienen constante la temperatura del cuerpo. C. Los papagayos cambian su temperatura a lo largo del año. D. La temperatura ambiental en el zoológico es constante.
136
Appendix C. Final Version of the Posttest.
GRADO 5º CUADERNILLO I
PRUEBA DE CIENCIAS
Nombres: _____________________Apellidos:___________________________________ Nombre del colegio: _________________________________________Curso: ______ Nombre del profesor(a) de ciencias: ____________________________________________ Eres: Niño Niña ¿Desde qué grado estás en este colegio? 0 1 2 3 4 5
A continuación vas a encontrar unas preguntas que debes contestar con mucho cuidado. Esta
prueba tiene dos partes, una de opción múltiple y la otra en donde debes escribir tu respuesta.
137
EJEMPLOS Contesta estos ejemplos para que practiques. Encierra con un círculo la respuesta correcta. EJEMPLO 1
¿Cuál es el resultado de sumar 25 y 25? A. 30. B. 40. C. 50. D. 60. ASÍ DEBIÓ QUEDAR ENCERRADA TU RESPUESTA: EJEMPLO 1
¿Cuál es el resultado de sumar 25 y 25? A. 30. B. 40. C) 50. D. 60.
138
EJEMPLO 2
¿Cuál grado estas cursando? A. Cuarto de primaria. B. Quinto de primaria. C. Sexto de bachillerato. D. Séptimo de bachillerato. ASÍ DEBIÓ QUEDAR ENCERRADA TU RESPUESTA: EJEMPLO 2
¿Cuál grado estas cursando? A. Cuarto de primaria. B Quinto de primaria. C. Sexto de bachillerato. D. Séptimo de bachillerato.
AHORA DETENTE Y ESPERA A QUE TE DEN LAS INSTRUCCIONES PARA
PODER EMPEZAR LA PRUEBA. NO HAY LÍMITE DE TIEMPO. TIENES EL
TIEMPO SUFICIENTE PARA CONTESTAR TODAS LAS PREGUNTAS.
PRIMERA PARTE
139
CHOMP02
1. ¿Cuál es la función del corazón?
CHOMP01
2. ¿De qué están compuestos todos los seres vivos?
CHOMP03
3. ¿Cuál es el mejor lugar para sentir el pulso?
CHOMP04
4. ¿Qué sistemas trabajan juntos durante el proceso de respiración?
CHOMP05
5. ¿Cuál es la función del sistema digestivo?
A. Intercambiar gases.
B. Combatir enfermedades.
C. Bombear sangre.
D. Regular la temperatura.
A. Órganos B. Sistemas C. Células D. Tejidos
A.
B. C. D.
A. Respiratorio y reproductor. B. Circulatorio y respiratorio. C. Respiratorio y digestivo. D.
Nervioso y respiratorio.
A. Transformar los alimentos que entran al cuerpo.
140
CHOMP06
6. ¿Qué sistema incluye el corazón y los vasos sanguíneos?
CHOMD06
7. El maestro Carlos y mezcló comida y líquido con químicos. Esta mezcla fue filtrada para que los nutrientes y el agua fueran removidos. ¿Cuál de los siguientes sistemas relacionarías con el experimento?
CHOMP07
8. ¿Qué pasa con tu pulso y tu ritmo respiratorio cuando corres muy rápido?
CHOMP08
9. ¿Cuál de las siguientes partes del cuerpo participa en la digestión mecánica?
B. Llevar oxígeno a todo el cuerpo. C. Llevar los nutrientes a todo el cuerpo. D. Regular la temperatura del cuerpo.
A. Sistema digestivo. B. Sistema nervioso. C. Sistema circulatorio. D. Sistema respiratorio.
A. Nervioso. B. Digestivo. C. Circulatorio. D. Respiratorio.
A. Tu pulso y tu ritmo respiratorio aumentan. B. Tu pulso aumenta y tu ritmo respiratorio disminuye. C. Tu pulso y tu ritmo respiratorio disminuyen. D. Tu pulso disminuye y tu ritmo respiratorio aumenta.
A. El riñón. B. Los dientes.
141
CHOMP09
10. Las personas respiran agitadamente cuando hacen ejercicio. Esto se explica porque al hacer esfuerzo físico
CHOMP10
11. El latido del corazón es un movimiento
C. El corazón. D. El pulmón.
A. disminuye la necesidad de nutrientes en el cuerpo. B. aumenta la necesidad de dióxido de carbono en el cuerpo. C. disminuye la necesidad de flujo sanguíneo en el cuerpo. D. aumenta la necesidad de oxígeno en el cuerpo
A. aprendido. B. involuntario. C. controlado. D. voluntario.
142
CHOMD07
12. El doctor Pérez lleva un registro del ritmo de la respiración de las personas cuando están descansando. El hizo la siguiente tabla:
Ritmo de la respiración Persona Respiración por minuto
El bebé Pedro 38 Niña de 7 años 25 Niño de 7 años 25 Niño de 10 años 20 Mamá 16
La tabla sugiere que E. Los niños respiran más rápido que las niñas. F. Las personas mayores respiran más rápido que las menores. G. Las niñas respiran más rápido que los niños. H. Las personas menores respiran más rápido que las mayores.
CHOMD08
143
13. Cecilia realizó el siguiente experimento: en un plato con una servilleta mojada puso cuatro fríjoles y en otro plato lleno con agua puso otros cuatro fríjoles, luego colocó los dos platos al borde de una ventana y observó lo que sucedía. Unos días después, Cecilia observó que en el plato con una servilleta mojado los fríjoles germinaron, mientras que en el plato con agua no sucedió nada.
Lo que tiene que hacer Cecilia para comprabar los resultados de su experimento es A. repetir exactamente el mismo experimento. B. usar el plato con una servilleta húmeda. C. usar dos platos cada uno cubierto con agua. D. repetir el experimento usando otro tipo de semillas.
CHOMD01
144
14. Un investigador quiere saber si la cantidad de glóbulos rojos en la sangre de las mujeres embarazadas es igual o diferente al de las mujeres adultas que no están embarazadas. Para averiguar lo anterior. ¿De cuál de los siguientes grupos de mujeres le sugerirías al investigador que extrajera y examinara la sangre?
CHOMP11
A.
B.
C.
D.
145
15. ¿Qué proceso ocurre en la boca para que comience la digestión química de un alimento?
CHOMP12
16. ¿Cuál de las siguientes opciones es un ejemplo de un movimiento voluntario?
CHOMP13
17. Tomás quiere medir sus pulsaciones en un minuto. Para eso, él debe poner sus dedos en algún lugar donde sienta el pulso, y contar las pulsaciones durante
CHOMP15
18. ¿Para qué sirve un estetoscopio?
CHOMP14
A. Masticación
B. Salivación
C. Peristaltismo
D. Ingestión
A. Sudar
B. Aplaudir
C. Estornudar
D. Digerir
A. 15 segundos y multiplicarlas por 2. B. 20 segundos y multiplicarlas por 4. C. 15 segundos y multiplicarlas por 4. D. 20 segundos y multiplicarlas por 2.
A. Ver la parte de adentro de la oreja. B. Mirar la pupila en el ojo. C. Medir las pulsaciones en el brazo. D. Oír las respiraciones de los pulmones.
146
19. ¿En qué lugar de la siguiente figura se realiza la conexión entre el sistema circulatorio y el digestivo?
CHOMP16
A. Esófago B. Estómago C. Intestino grueso D. Intestino Delgado
147
20. ¿Cuáles pasos seguirías para observar células de cebolla en el microscopio?
D.
CHOMP17
A.
B.
C.
148
21. Ana puso una bolsa semipermeable con colorante en una botella con agua limpia como se muestra en la siguiente figura:
¿Cuál de las siguientes botellas muestra lo que pasó al otro día?
A. B.
C. D.
CHOMP18
22. Los glóbulos rojos permiten que la sangre transporte
CHOMD02
A. nutrientes a todo el cuerpo. B. desechos a todo el cuerpo. C. oxígeno a todo el cuerpo. D. agua a todo el cuerpo.
149
23. En la huerta hicieron el experimento que se muestra a continuación:
Con este experimento se quiere investigar el efecto de:
CHOMP19
24. Luis quiere saber si la harina tiene almidón. ¿Qué debe agregar Luis a la harina para saber si tiene almidón?
A. B. C. D.
CHOMP20
A. el tipo de suelo en el crecimiento de las plantas. B. la forma de las hojas en el crecimiento de las plantas. C. el tamaño del tallo en el crecimiento de las plantas. D. el sol en el crecimiento de las plantas.
150
25. La figura de abajo representa un modelo de caja torácica y sus diferentes partes. ¿Qué representan la primera y segunda parte del tubo plástico?
CHOMP21
26. ¿Cuándo digieres comida, dónde la utiliza tu cuerpo?
CHOMP22
27. ¿Cómo trabajan juntos el corazón y los pulmones?
CHOMP23
A. La primera parte es el bronquio y la segunda parte es la tráquea. B. La primera parte es la tráquea y la segunda parte es el diafragma. C. La primera parte es la tráquea y la segunda parte es el bronquio. D. La primera parte es el diafragma y la segunda parte es el bronquio.
A. Sólo en la sangre de tu cuerpo. B. Sólo el estómago de tu cuerpo. C. Sólo los pulmones de tu cuerpo. D. En las células de tu cuerpo.
A. El movimiento de los pulmones ayuda al corazón a bombear sangre. B. Los pulmones dan oxígeno a la sangre que el corazón bombea a través del cuerpo. C. El corazón y los pulmones trabajan juntos para ayudar a digerir la comida. D. El corazón bombea la sangre, y los pulmones circulan la sangre a través del cuerpo.
151
28. ¿Cuál de los siguientes diagramas describe el funcionamiento de los riñones?
CHOM P24
29. En el intestino las vellosidades (pelitos) permiten que se difundan
CHOMP25
A.
B.
C.
D.
A. menos nutrientes porque la superficie disminuye. B. más nutrientes porque la superficie aumenta. C. menos nutrientes porque la superficie aumenta. D. más nutrientes porque la superficie disminuye.
152
30. En el siguiente dibujo se comparan un pedazo de tela roja con un pedazo de hoja de un árbol.
Al mirar la hoja y la tela te das cuenta de que una está viva y la otra no.
¿Cuál de las siguientes características te permite afirmar que la hoja está viva y la tela no?
A. El material de la tela es ordenado y el de la hoja es desordenado.
B. La hoja está compuesta de células y la tela de fibras.
C. El color de la tela es rojo y el de la hoja es verde.
D. La superficie de la hoja es suave y la de la tela es áspera.
CHOMP25
31. ¿En cuál de los siguientes procesos del cuerpo humano ocurre difusión? A. Transformación de alimentos en el estómago.
CHOMD03
B. Masticación de alimentos en la boca. C. Absorción de nutrientes en el intestino delgado. D. Bombeo de sangre desde el corazón.
153
32. Unos estudiantes midieron durante ocho meses la temperatura ambiental y la temperatura de unos papagayos; con los resultados elaboraron la siguiente gráfica:
Teniendo en cuenta la información de la gráfica. ¿Cuál de las siguientes conclusiones es la más acertada? A. La temperatura ambiental influye sobre la temperatura de los papagayos.
B. Los papagayos mantienen constante la temperatura del cuerpo. C. Los papagayos cambian su temperatura a lo largo del año. D. La temperatura ambiental en el zoológico es constante.
154
RESPONDE LAS PREGUNTAS 33 Y 34 DE ACUERDO CON LA SIGUIENTE INFORMACIÓN Javier quiere investigar la forma de vida de las tijeretas y para esto puso tierra húmeda sin luz en un lado de la caja y tierra seca con luz al otro lado de la caja; luego metió ocho tijeretas. El siguiente dibujo muestra el experimento.
CHOMD04
33. ¿Qué pregunta se puede responder a partir de este experimento? A. ¿Cuánto tiempo vive una tijereta?
CHOMD05
B. ¿Cómo se reproducen las tijeretas? C. ¿Dónde viven las tijeretas? D. ¿Qué comen las tijeretas?
155
34. Javier llegó a la conclusión de que las tijeretas prefieren la tierra húmeda y la oscuridad. ¿Cuáles datos le permitieron a Javier llegar a esta conclusión? A. Las 8 tijeretas se quedaron en la caja de madera.
¡FELICITACIONES! YA TERMINASTE LA PRIMERA PARTE DE LA PRUEBA. AHORA, RESPONDERÁS TRES PREGUNTAS DONDE DEBES ESCRIBIR TU RESPUESTA. CONTINÚA CON LA SEGUNDA PARTE DE LA PRUEBA.
SEGUNDA PARTE CHOMAP01
35. ¿Cuál es la función de la sangre en tu cuerpo? _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
CHOMAP02
36. ¿Cómo están relacionados el sistema digestivo y el sistema circulatorio? ______________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________
B. Las tijeretas se distribuyeron en los dos lados de la caja. C. De las 8 tijeretas, 7 se fueron a la tierra húmeda y sin luz. D. Las 8 tijeretas pueden vivir en las condiciones del experimento.
156
CHOMAD01
37. ¿Qué necesitan las células de tu cuerpo humano para sobrevivir y de dónde obtienen esas necesidades?
157
Appendix D. Implementation Manual for the Paper and Pencil Test.
MANUAL DE ADMINISTRACIÓN DE LA PRUEBA DE CUERPO HUMANO
Objetivo de la prueba: Evaluar los aprendizajes en cuerpo humano de los niños y niñas de diferentes instituciones. Información de la prueba: El cuadernillo está constituido por preguntas de opción múltiple y preguntas abiertas. Aplicación de la Prueba: Todo lo referente a la aplicación de la prueba se dirá durante el simulacro. Información para utilizar antes de ir a la institución a aplicar la prueba. Asegúrese de tener completa la siguiente información:
Nombre de la institución: ________________________________________________ Dirección de la institución: _______________________________________________
Contacto de la institución: _____________________ Celular: ____________________
Información para utilizar antes de la aplicación de la prueba. Materiales 1. Revise el paquete de pruebas que le fue asignado teniendo en cuenta los siguientes criterios:
! Debe tener una hoja que corresponde al rótulo que debe fijar en la puerta con cinta y dice "Favor No Molestar, Administración de Prueba en Progreso".
! Debe tener una hoja llamada "Formato de Administración Prueba de Sistemas del Cuerpo Humano ". En esta hoja debe anotar las irregularidades y aprendizajes que ocurran durante la aplicación.
! Debe tener un esfero con cinta y un marcador de tablero. Estudiantes 1. Cuando llegue al salón salude a la persona encargada de los estudiantes y a los estudiantes. 2. Revise que la ubicación de los estudiantes sea la adecuada para la aplicación de la prueba. Es decir, los estudiantes deben estar lejos los unos de los otros de tal manera que no puedan hacer copia. 3. Asegúrese que los estudiantes tenga lápiz y borrador. En caso de no tener lápiz, pueden utilizar esfero.
4. Comprometa a los estudiantes con la prueba y dígales lo siguiente: ! Esta prueba la van a presentar muchos niños en el país, agradecemos su participación y esperamos su
mejor esfuerzo. ! Pídales que contesten todas las preguntas y que pongan asterisco frente al número de la pregunta cuando
no conozcan las respuesta. ! Dígales que usted no les podrá ayudar a contestar las preguntas durante la aplicación de la prueba. ! Enfatice a los estudiantes que la prueba la deben realizar de manera individual y en silencio.
Información para utilizar durante la aplicación de la prueba.
158
1. Entregue los cuadernillos con la carátula hacia abajo y dígale a los estudiantes que no lo empiecen a diligenciar hasta que se les informe.
2. Diligencie las dos primeras páginas con ellos y antes de comenzar la prueba pregúnteles si tienen preguntas sobre la aplicación de la prueba.
3. Camine silenciosamente en el salón y revise que los estudiantes estén contestando las preguntas. Asegúrese que los niños estén contestando las preguntas de la manera correcta.
4. Diligencie el formato de administración (hora de entrega del primer y último estudiante). Anote irregularidades y aprendizajes en caso de ser necesario.
5. Si tiene preguntas de los alumnos, siempre contésteles usando algunas de las siguientes respuestas: Lee nuevamente la pregunta; si no puedes contestarla, pasa a la siguiente.
Contéstala lo mejor que puedas. ¿Cuál crees que sea la respuesta correcta? Selecciona entonces esa respuesta.
6. No dé ningún tipo de retroalimentación a los estudiantes, ni ninguna ayuda mientras contesten las preguntas. No clarifique las preguntas. No responda a preguntas de contenido. No interactúe con los estudiantes a menos que estén interrumpiendo el óptimo trascurso de la aplicación de la prueba. 7. Antes de recoger cualquier cuadernillo dígale al estudiante que revise si contestó todas las preguntas y anote en la parte superior derecha la hora de finalización de la prueba. 8. Pregunte a la profesora qué debe hacer con los estudiantes que finalicen la prueba.
• Información para utilizar después de la aplicación de la prueba. 1. Guarde todos los cuadernillos en el orden en que los recibió, junto con el formato de administración de la prueba y el rótulo de la puerta. 2. Agradézcales a los estudiantes y a la profesora por su colaboración, así como nosotros agradecemos su colaboración.
MUCHAS GRACIAS POR SU ADMINISTRACIÓN DE LA PRUEBA
159
Appendix E. Items included in each scale.
Pre-Test
Total Declarative Procedural Schematic Proximal Distal 1 X X 2 X X 3 X X 4 5 X X 6 X X 7 X X 8 X X 9 X X 10 X X 11 X X 12 X X 13 X X 14 X X 15 X X 16 X X 17 X X 18 X X 19 20 X X 21 X X 22 X X 23 X X 24 X X 25 X X 26 X X 27 X X 28 X X
160
Post Total Declarative Procedural Schematic Proximal Distal
1 X X 2 X X 3 X X 5 X X 6 X X 7 X X 8 X X 9 X X 10 X X 11 X X 12 X X 13 14 X X 15 X X 16 X X 17 X X 18 X X 19 X X 20 X X 21 X X 22 23 X X 24 X X 25 X X 26 X X 27 X X 28 X X 29 X X 30 X X 31 X X 32 X X 33 X X 34 X X 35 X X 36 X X 37 X X
161
Post-Test Total Control IBSE Pulse_Items
1 X X 2 X 3 X 4 5 X 6 X X 7 X 8 X 9 X 10 X X 11 X 12 X 13 14 15 16 17 X X 18 X 19 20 21 X 22 23 24 X 25 X 26 27 X 28 X 29 30 31 32 33 34 35 X 36 X 37
162
Appendix F. Description of the paper towels performance assessment including student notebook.13
TOALLAS DE PAPEL
CIFE-PQC-DA-2009-XXX: PRUEBA DE DESEMPEÑO
1. INSTRUCCIONES DE ADMINISTRACIÓN DE LA PRUEBA DESCRIPCIÓN: Los estudiantes deben descubrir cuál toalla de papel puede absorber la mayor y la menor cantidad de agua.
TIEMPO: Otorgue 50 minutos para su realización.
MATERIALES POR ESTUDIANTE • 1 cuadernillo (1 formato observador, 1 formato instrucciones, 1 cuaderno de apuntes científicos)
• 1 gotero
• 3 cajas de petri con tapas
• 1 tijeras
• 1 pinza
13 CIFE-PQC-DA-2009-XXX: Esta prueba fue desarrollada por Richard Shavelson, con una subvención de la U.S. NationalScience
Foundation, y traducida al español por el equipo de PeqCien, con la debida autorización.
Para mayor información, consultar el sitio Web del Stanford Education Assessment Laboratory: http://www.stanford.edu/dept/SUSE/SEAL/
163
• 1 lupa
• 1 recipiente medidor de 250 ml
• 1 recipiente medidor de 100 ml
• 1 balanza para comida
• 1 bandeja (de plástico o de aluminio, aproximadamente 2cm de profundidad, 30cm de largo, 18cm de ancho)
• 1 regla de 30cm
• 1 embudo
• 3 vasos plásticos transparentes (de aproximadamente 1 taza)
• 1 recipiente con agua
• 3 tipos de toallas de papel (papel de cocina): cada una con características (por ej., una blanco, una con dibujos azules, etc.). Son 3 pedazos de cada tipo de toalla (en total 9)
PREPARACIÓN: Organice los materiales en cada mesa de manera que los estudiantes puedan verlos todos: construya un semi-círculo frente al estudiante, tal como se muestra en la foto (arriba). Todo el material debe estar al alcance del estudiante, ninguna pieza debe sobresalir. En otras palabras, se debe evitar sugerir al estudiante que realice la investigación de una manera determinada. Se debe indicar con un letrero el nombre de cada uno de los materiales.
NOTA: Asegúrese de contar con todos los documentos necesarios para la administración y evaluación de la prueba, a saber
• Formato del Observador
• Cuadernillo Estudiantes-Instrucciones
• Cuaderno de Apuntes Científicos
164
TOALLAS DE PAPEL Nombre: __________________ Apellido: __________________________ Curso: ____________________ Colegio: __________________________ Esta es una prueba y tienes dos retos: 1. Averiguar cuál toalla de papel puede absorber más agua.
2. Averiguar cuál toalla de papel puede absorber menos agua.
Para hacer esta prueba, sigue estas indicaciones: A. Observa cada pieza del material.
B. Piensa cómo podrías utilizar algunas piezas de este material para hacer un experimento y resolver los retos.
C. Recuerda que no necesitas usar todo el material.
D. Utiliza el otro lado de esta hoja para hacer tus anotaciones.
E. Tienes máximo 50 minutos para completar la prueba.
Tienes alguna pregunta?
POR FAVOR, DA VUELTA A LA HOJA PARA INICIAR LA PRUEBA
165
HOJA DE ANOTACIONES RESULTADOS: Marca la opción correcta según tus observaciones. Asegúrate de marcar tanto la toalla que más absorbe, como la toalla que menos absorbe.
1. La toalla de papel que absorbe más agua es:
2. La toalla de papel que absorbe menos agua es:
! Blanca ! Blanca ! Azul ! Azul ! Rosada-verde ! Rosada-verde
166
CUADERNO DE APUNTES CIENTÍFICOS Nombre: __________________ Apellido: __________________________ Curso: ____________________ Colegio: __________________________ A. A partir de tu experimento, ¿cómo supiste cuál toalla de papel absorbe más agua y cuál toalla de papel absorbe menos agua? ¿Cómo supiste que la toalla _____________ absorbía más agua? ¿Por qué? ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ¿Cómo supiste que la toalla _____________ absorbía menos agua? ¿Por qué? ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________
POR FAVOR, DA VUELTA A LA HOJA
167
B. Aquí hay algunas preguntas sobre tu experimento. Responde cada pregunta con “sí” o “no”. Las preguntas están relacionadas con el experimento que te ayudó a saber cuales toallas absorbían más o menos agua. 1. ¿Todas las toallas de papel eran del mismo tamaño? 2. ¿Todas las toallas de papel estaban completamente mojadas? 3. ¿Utilizaste la misma cantidad de agua para mojar cada toalla de papel? C. Carolina cree que todas las toallas de papel deben estar completamente mojadas antes de decidir cuál absorbe más agua y cuál absorbe menos agua. Luis no cree que las toallas de papel deban estar completamente mojadas. ¿Qué piensas tú y por qué? _____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_____________________________________________________________________________________
_______________________________________________________
FIN DE LA PRUEBA
¡GRACIAS!
168
Appendix G. Description of the pulse performance assessment.
PRUEBA DE DESEMPEÑO EN CIENCIAS
Investigando tu pulso
Nombre: ______________________ Apellido: _________________________
Curso: ____________ Colegio: ____________________________________
Esta es una prueba en la que quieres investigar cómo cambia tu ritmo cardíaco al hacer
ejercicio.
Para esto, tienes los siguientes materiales:
• Un cronómetro
• Un escalón en el cual puedes subir y bajar.
Tu tarea es:
Encontrar cómo cambia tu pulso después de subir y bajar un escalón, para eso
vas a tomar y contar tus pulsaciones.
169
Para cumplir con tu tarea: 1. Encuentra tu pulso y asegúrate que sabes cómo tomarlo.
2. Cuenta tu pulso por 10 segundos. 3. Anota la cantidad de pulsaciones contadas en 10 segundos en la tabla de abajo, al frente
de la casilla 0 minutos/en reposo.
4. Completa la tabla contando tus pulsaciones, cada minuto, durante 10 segundos mientras subes y bajas un escalón durante 5 minutos.
Recuerda que debes detenerte después de cada minuto para escribir el número de pulsaciones que contaste en la correspondiente columna de la tabla.
Número de minutos Número de pulsaciones contadas en 10 segundos
0 minutos/En reposo
Al minuto 1
Al minuto 2
Al minuto 3
Al minuto 4
Al minuto 5
Después de completar la tabla contesta las siguientes preguntas:
170
1. Al observar la tabla, ¿qué puedes decir sobre el cambio de tu pulso durante el ejercicio? _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ 2. ¿Por qué crees que tu pulso cambió de esta manera? ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ _________________________________________________________________________ 3. ¿Qué aprendiste en la actividad? ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ _________________________________________________________________________
171
Appendix H. Format used by observers to record student actions in the performance assessments.
3. REALIZACIÓN DE LA PRUEBA -OBSERVACION
Nombre: __________________ Apellido: __________________________ Curso: ______________ Colegio: __________________________________________
NOTAS Paso Descripción Observación 1
2
3
4
5
6
7
8
9
172
NOTAS Paso Descripción Observación
10
11
12
13
14
15
16
17
18
19
20
173
Appendix I. Grading matrix used for the pulse performance assessment.
Calificación Pulso
Item 1 - Measure “at rest” pulse rate and record in table. Pulse beats are plausible: 7 to 25 counts per 10 seconds (40 to 150 counts per minute). Total Possible Points: 1
VER TABLA Tema 1 - Medida "en reposo" del pulso y el registro en la tabla. Los latidos del resultado en reposo son plausibles: de 7 a 25 latidos por cada 10 segundo (o 40 a 150 latidos por minuto). Total de puntos posibles: 1
Item 2 - Measure “after exercise” pulse rates and record in table. i) Records pulse at least 4 different times during the exercise (in addition to “at rest” measurement). ii) Pulse rates are plausible: 7 to 25 counts per 10 seconds at the beginning (40 to 150 counts per minute). iii) Pulse rate increases with exercise (may level off or slow near the end). Total Possible Points: 3
VER TABLA Tema 2 - Medida "después de hacer ejercicio" del pulso y registro de la tabla. i) Registro del pulso de por lo menos 4 veces diferentes durante el ejercicio (diferente a la medida de “reposo”). ii) Las frecuencias de pulso son convincentes: 7 a 25 latidos por 10 segundos al comienzo (40 a 150 pulsos por minutos). iii) Hay aumento del pulso con el ejercicio (se pueden estabilizar o disminuir cerca del final). Total de puntos posibles: 3 NOTA: Los datos pueden subir y bajar, porque los estudiantes hacen ejercicio con diferente intensidad entre minutos.
Item 3 - Describe how pulse changes during exercise. i) Description consistent with data presented. ii) Description includes identification of the trend or pattern in the data. Total Possible Points: 2
VER PREGUNTA 1 Tema 3 - Describe cómo cambia el pulso durante el ejercicio. i) Descripción consistente con los datos presentados. ) Descripción incluye la identificación de la tendencia o patrón en los datos. HACE MENCION A LOS DATOSii Total de puntos posibles: 2
Item 4 - Explain why pulse changes. Includes the following three elements relating to physiological needs during exercise: i) role of muscle action (exercise results in need for more energy and oxygen in the muscles); ii) role of blood (more oxygen or food supplied by an increase in blood flow); iii) connection with heart action or pulse rate, (heart is pumping faster to supply more blood). Total Possible Points: 3
VER PREGUNTA 2 Tema 4 - Explica por qué los cambios de pulso. Incluye los siguientes tres los elementos relativos a las necesidades fisiológicas durante el ejercicio: i) el papel de acción de los músculos (resultados del ejercicio en la necesidad de más energía y oxígeno en los músculos), ii) el papel de la sangre (Hay más oxígeno o alimentos suministrados por una aumento del flujo sanguíneo), iii) la conexión con la acción del corazón o del ritmo cardíaco, (Corazón bombea más rápido para suministrar más sangre). Total de puntos posibles: 3
TOTAL puntos posibles: 9
Estudiante
Reposo (1 pto) Tabla (3 ptos) Pregunta 1 (2 ptos) Pregunta 2 (3 ptos)
XXXX 1 pto si registra datos coherentes en reposo
1 pto por escribir al menos 4 datos
1 pto si los valores son coherentes
1 pto si hay una tendencia en los datos
1 pto si lo que describe de los datos es consistente con los datos de la tabla
1 pto si identifica patrón en datos
1 pto por músculos
1 pto por sangre
1 pto por relación corazón y ritmo