journal of pre-college engineering tirupalavanam ganesh ...mnathan/... · 2 j. l. chiu, m. c. linn...

Journal of Pre-College Engineering Education Research (J-PEER)

ISSN: 2157-9288

http://docs.lib.purdue.edu/jpeer/

About this JournalThe graduates of today and tomorrow enter into a world that

requires them to be engineering-literate and technologically savvy. The integration of engineering education in grades P-12 will better arm students with essential tools and skills to enter into the workforce or postsecondary education. Additionally, due to a 20 percent slip in the number of engineers graduat-ing from U.S. institutions and with more than half of the U.S. workforce in the sciences and engineering approaching retire-ment age, the need for a diverse group of students interested in and prepared to study engineering in college is ever growing.

It is essential that young engineers from the U.S. be in-volved in the next generation of innovative ideas that sup-port our society’s needs. This interest and drive to participate in engineering must be fostered at an early age. The Journal of Pre-College Engineering Education Research (J-PEER) is dedicated to addressing the downward trends in engineering interest, preparedness, and representation; to transforming P-12 education to include engineering; to preparing a globally competitive engineering workforce; and ultimately to creating a society of engineering-literate citizens.

J-PEER is issued twice a year electronically and serves as a forum and a community space for the publication of research and evaluation reports on areas of pre-college STEM educa-tion, particularly in engineering.

J-PEER targets scholars and practitioners in the new and expanding field of pre-college engineering education. This journal invites authors to submit their original and unpublished work in the form of (1) research papers or (2) shorter practi-tioners reports in numerous areas of STEM education with a special emphasis on cross-disciplinary STEM approaches in-corporating engineering.

Broadly the topics include but are not limited to research articles on elementary and secondary students learning, cur-ricular and extracurricular approaches to teaching engineering in elementary and secondary school, professional development of teachers and other school professionals, comparative ap-proaches to curriculum and professional development in engi-neering education, parents’ attitudes toward engineering, and the learning of engineering in informal settings.

Editorial BoardEditor: Johannes Strobel, Purdue UniversityEditorial Assistant: Maria Granic-White, Purdue UniversityMonica Cardella, Purdue UniversityRobin Clark, Aston University, United KingdomDavid Crismond, City College of New YorkChristine Cunningham, Boston Museum of Science and

TechnologyLyn D. English, Queensland University of Technology,

Australia

Tirupalavanam Ganesh, Arizona State UniversityJan Hansen, University of St. Thomas, MinnesotaStephen Krause, Arizona State UniversityRich Lehrer, Vanderbilt UniversityMarcia Linn, University of California, BerkeleyJack R. Lohmann, Georgia Institute of TechnologyIngelore Mammes, Paderborn University, GermanyMitchell Nathan, University of Wisconsin-MadisonAnthony Petrosino, University of TexasSenay Purzer, Purdue UniversityBob Sherwood, Indiana University

Submission Guidelines

Who Can Submit?Anyone may submit an original article to be considered for

publication in Journal of Pre-College Engineering Education Research (J-PEER) provided he or she owns the copyright to the work being submitted or is authorized by the copyright owner or owners to submit the article. Authors are the initial owners of the copyrights to their works (an exception in the non-academic world to this might exist if the authors have, as a condition of employment, agreed to transfer copyright to their employer).

General Submission RulesSubmitted articles cannot have been previously published,

nor be forthcoming in an archival journal or book (print or elec-tronic). Please note: “publication” in a working-paper series does not constitute prior publication. In addition, by submitting material to Journal of Pre-College Engineering Education Re-search (J-PEER), the author is stipulating that the material is not currently under review at another journal (electronic or print) and that he or she will not submit the material to another journal (electronic or print) until the completion of the edito-rial decision process at Journal of Pre-College Engineering Education Research (J-PEER). If you have concerns about the submission terms for Journal of Pre-College Engineering Edu-cation Research (J-PEER), please contact Maria Granic-White at [email protected].

Formatting RequirementsJournal of Pre-College Engineering Education Research

(J-PEER) has no general rules about the formatting of articles upon initial submission. There are, however, rules governing the formatting of the final submission. See http://docs.lib.purdue .edu/jpeer/styleguide.html for details. Although bepress can provide limited technical support, it is ultimately the respon-sibility of the author to produce an electronic version of the article as a high-quality PDF (Adobe’s Portable Document Format) file, or a Microsoft Word, WordPerfect or RTF file that can be converted to a PDF file.

It is understood that the current state of technology of Ado-be’s Portable Document Format (PDF) is such that there are no, and can be no, guarantees that documents in PDF will work perfectly with all possible hardware and software configura-tions that readers may have.

Available online at http://docs.lib.purdue.edu/jpeer

Journal of Pre-College Engineering Education Research 1:1 (2011) 1–14

This material is based upon work supported by the National Science Foundation under grants No. ESI- 0334199, and ESI- 0455877. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors appreciate helpful comments from the Technology- Enhanced Learning in Science research group and thank the teachers and students involved in the projects.

Knowledge Integration and Wise Engineering

Jennifer L. Chiu

University of Virginia

M. C. Linn

University of California, Berkeley

Abstract

Recent efforts in engineering education focus on introducing engineering into secondary math and science courses to improve science, technology, engineering, and math (STEM) education (NAS, 2010). Infusing engineering into secondary classrooms can increase aware-ness of and interest in STEM careers, help students see the relevance of science and math in their everyday lives, and increase STEM literacy. This paper describes how the knowledge integration framework provides research- based guidelines to help secondary students develop and connect science and engineering concepts. Results from technology- enhanced curriculum units demonstrate how instruction based on knowledge integration principles and patterns using the Web- based Inquiry Science Environment (WISE) can infuse engineering into existing secondary science classrooms. This paper explores how the knowledge integration framework can guide curriculum develop-ment and assessment of engineering concepts and habits of mind.

Keywords: curriculum design, technology- enhanced instruction, integrating science and engineering, assessment

In a recent speech announcing a new educational initiative to “Change the Equation,” President Obama declared, “[L]eader ship tomorrow depends on how we educate our students today–especially in science, technology, engineering, and math” (Obama, 2010). In addition to the President’s initiative, much effort is needed to improve science, technology, engi-neering, and math (STEM) education (NAS, 2010). Introducing engineering into secondary classrooms has the potential to make science and math relevant to students, increase STEM literacy of students, increase awareness of STEM professionals, and increase interest in STEM careers (Katehi, Pearson & Feder, 2009). With these possibilities in mind, the National Acad-emy of Engineering (NAE) convened a workgroup to explore national K- 12 engineering standards to accompany math and science standards (NAE, 2010). However, the final report did not suggest specific standards. Citing a lack of engineering ex-perience in K- 12 settings and lack of evidence regarding the impact of similar standards- based reforms, the report concluded

2 J. L. Chiu, M. C. Linn / Journal of Pre-College Engineering Education Research

that standards are not the solution. Instead, the report called for infusing engineering ideas into existing K- 12 courses, investigating core ideas in engineering appropriate for K- 12 learning, creating guidelines for K- 12 engineering educa-tion materials, and conducting research on learning that can inform engineering education (Table 1).

To achieve these goals, the field needs coherent research on how K- 12 curricula can affect learning of science and math principles as well as engineering concepts and hab-its of mind such as systems thinking, creativity, optimism, collaboration, communication, and attention to ethical con-siderations (Katehi, Pearson & Feder, 2009). This article describes how instruction based on the knowledge integra-tion (KI) framework using the Web- based Inquiry Science Environment (WISE) can help students develop and inte-grate science and engineering ideas. Knowledge integration offers a unified framework of research- based guidelines for curriculum design and assessment that can help connect and clarify K- 12 engineering education efforts. We draw on two technology- enhanced curriculum units in WISE, Airbags: Too Fast, Too Furious? and Chemical Reactions, as examples to describe how curriculum designed with KI principles can help students connect and learn science and engineering concepts.

Through these two examples, we demonstrate how engi-neering can be infused into existing K- 12 classrooms. We draw upon these findings to suggest core concepts of en-gineering appropriate and accessible to secondary science students. We discuss how research using the KI learning perspective can inform engineering education research, and identify guidelines for teaching and learning engineering design based on the KI framework.

Two Examples

This article highlights two curricular units, Airbags: Too Fast, Too Furious and Chemical Reactions, to describe in detail how the KI framework in combination with the WISE platform helps students connect engineering principles and science content. Airbags guides students through an investi-gation of airbags safety in car collisions (McElhaney, 2010; McElhaney & Linn, 2008). The project encourages students to think as engineers by conducting experiments to explore how the designs of cars and airbags can keep passengers

safe on the road. Students connect these ideas to physics and math concepts by integrating their understanding of motion and graphs with car safety. Airbags uses a series of scaffolded dynamic visualizations to help students explore the relationship of one- dimensional motion to characteris-tics of position and velocity graphs. Students experiment with visualizations that provide simultaneous graphical rep-resentations and animations of airbag and passenger mo-tion. The results of these experiments serve as evidence for students to suggest improvements to the design of airbags and cars (Figure 1).

In Chemical Reactions, students investigate how energy and chemical reactions relate to climate change, and use these chemistry concepts to recommend solutions to de-crease carbon dioxide emissions on a global scale. Students explore the greenhouse effect and combustion reactions using visualizations and molecular simulations. Students connect ideas such as conservation of mass, stoichiomet-ric ratios, and limiting reactants to everyday ideas such as driving and electricity use. Students distinguish math and chemistry ideas such as coefficients and subscripts and link these chemical symbolic representations to what they mean on a molecular scale. Students use their chemistry under-standing to choose a particular solution to mitigate carbon emissions and create a policy brief to submit to their local congressperson (Chiu, 2010; Chiu & Linn, 2008).

Knowledge Integration and WISE

The KI learning framework builds upon decades of empirical studies on student and teacher learning in K- 12 science and engineering classrooms (Linn, 1995; Linn & Eylon, 2006, in press). KI is a tested, research- based per-spective that brings together recent trends in developmen-tal, constructivist, sociocultural, and cognitive perspectives on learning. According to KI, learners build understanding by adding, sorting, evaluating, distinguishing, and refining ideas from classes, everyday experiences, and cultural ex-pectations. KI is based upon a large literature demonstrating that learners come to class with rich, intuitive ideas about phenomena developed from their varied experiences, intel-lectual efforts, and interpretations of the natural world (i.e., Mulford & Robinson, 2002; Nicoll, 2001; Osborne, & Cos-grove, 1983; Ozmen, 2004; Pfundt & Duit, 1991). These

Table 1Research Questions to Be Investigated to Improve K- 12 Engineering Education

• How do children come to understand (or misunderstand) core concepts and apply (or misapply) skills in engineering?• What are the most effective ways of introducing and sequencing engineering concepts and skills for learners at the elementary, middle, and high

school levels?• What are the most important synergies in the learning and teaching of engineering and mathematics, science, technology, and other subjects?• What are the most important considerations in designing materials, programs, assessments, and educator professional development that engage all

learners, including those historically underrepresented in engineering?• What are the best settings and strategies for enabling young people to understand engineering in schools, informal education institutions, and after-

school programs?

From Standards for Engineering Education (2010).

J. L. Chiu, M. C. Linn / Journal of Pre-College Engineering Education Research 3

diverse ideas serve as a basis for students to make sense of science.

The KI perspective encourages learning by creating op-portunities for students to compare, contrast, critique, and distinguish these ideas as well as the new ideas they en-counter in instruction. Research on KI shows that students can refine their understanding by considering all their ideas. When students integrate their own views with new ideas they develop reasoning processes that will serve them well throughout their lives.

Typical instruction often focuses on adding ideas but not on helping students integrate new and existing ideas. As a result, students are prone to isolating the new ideas in the context of the science classroom rather than applying new ideas widely. For example, students can learn about projec-tile motion in physics classrooms and quadratic equations in math classes without any connection between the two. Students can also choose to be cognitive economists, decid-ing when and where to pay attention or resolve conflicts of ideas (Linn & Hsi, 2000). This happens frequently in STEM

classrooms when students do not see the relevance or im-portance of sorting out their ideas. If students can complete homework assignments and earn passing grades, they may see no benefit to ensuring that their ideas about scientific phenomena are coherent.

To guide instructional designers seeking to promote integrated understanding, researchers have synthesized research findings into the KI instructional pattern. The KI instructional pattern (see Figure 2) identifies the learning processes that are essential for supporting students as they make connections among ideas and develop coherent un-derstanding. The pattern emphasizes several aspects of stu-dent learning that are often overlooked in instruction.

Eliciting Ideas

Promoting learning through the KI instructional pattern includes eliciting student ideas. Eliciting existing ideas rec-ognizes the individual backgrounds and experiences that students bring to learning contexts and enables learners to

Figure 1. Screenshot of the Airbags Curriculum.

!

"#$%&'!()!*+'!,-./0'1$'!#-2'$&32#.-!#-42&%52#.-30!6322'&-!'-5.%&3$'4!42%1'-24!2.!73,'!

5.--'52#.-4!37.-$!2+'#&!#1'34!89!'0#5#2#-$:!311#-$:!1';'0.6#-$!5&#2'&#3:!3-1!4.&2#-$!#1'34!

Figure 2. The knowledge integration instructional pattern encourages students to make connections among their ideas by eliciting, adding, developing criteria, and sorting ideas.


make connections from new instruction to their existing ideas. For example, in a curriculum focused around design of fuels, instruction can prompt students to elicit their ex-isting observations and everyday ideas about energy and chemical reactions.

Adding New Ideas

The KI pattern emphasizes adding new ideas that help students make sense of the topic and connect to their existing ideas. Instruction traditionally places a great deal of focus on adding ideas and concepts through lecture, text, videos, and lab activities. For example, students can add ideas using a molecular visualization of a combustion reaction. Ideally, new ideas fit the criteria of being pivotal cases in that they encourage reconsideration of existing ideas. Pivotal cases are carefully designed comparisons that connect to the be-liefs of learners and spur students to seek integrated and consistent accounts of scientific phenomena (Linn, 2005). Pivotal cases are robust over time, help students integrate their understanding in various contexts, stimulate students to apply the cases to different contexts and examples, and help students reason about future investigations and observations.

Distinguishing Ideas

Adding ideas, even pivotal cases, can result in isolated, separate, unresolved, conflicting, and incomplete networks of ideas. To help learners see how their existing ideas relate to, conflict with, or extend these new, normative ideas added during instruction, the KI instructional pattern encourages learners to distinguish among their ideas. For example, stu-dents may look at a visualization of combustion and think it is consistent with their view that bond breaking and for-mation happens instantaneously. Activities to help students distinguish their existing ideas from the new ideas might include prompting students to explain how the molecular view relates to their existing ideas about energy and chemi-cal reactions, posing critique questions, or asking students to make drawings of their observations. To distinguish ideas, students need to develop criteria for evaluating ideas. These criteria can be deliberately and intentionally developed by self- aware learners, socially constructed in class or commu-nities of learners, or developed by contrasting alternatives. Students need to develop and then to apply their criteria to the group or individual ideas. They will generally need to refine their criteria as well as their ideas about the topic they are studying. For example, when students use their criteria to compare their own ideas to the visualization of combus-tion they might need to refine their criteria about chemical bonds. They may also refine their ideas about combustion.

Sorting Out Ideas

Finally, the KI pattern encourages learners to sort out and refine their knowledge based on these evaluations. This in-cludes supporting learners to reflect upon their knowledge,

to find gaps or discrepancies in their understanding, and to act to remedy these situations. For example, when asked to write a narrative explaining bond breaking and forma-tion, students might realize that they initially thought that making bonds required energy, but when they added energy in the visualization, chemical bonds were broken. Because their criteria included the relationship among energy and bonding, students might realize that they have conflicting ideas and go back to refine and sort out their understanding. In addition, students might be asked to reflect on the design of an effective fuel. This question might motivate them to reconcile their ideas about bonding and energy with their ideas about the design of fuels. Encouraging learners to en-gage in the full KI pattern supports students to connect their ideas across domains and settings.

Web- based Inquiry Science Environment

The Web- based Inquiry Environment (WISE) has been developed and refined using the KI framework to provide pedagogical features for teachers, researchers, and students to support implementation of the KI pattern (Linn, Davis & Bell, 2004; Slotta & Linn, 2009). WISE is an open- source digital learning platform that supports student inquiry in middle and high school classrooms. Free to the public, WISE enables anyone to develop curriculum and author content such as online brainstorms and discussions, explana-tion scaffolding, model building, drawing, and online jour-nals (Figure 3). WISE offers a library of tested curricula to implement in classrooms, as well as the ability for teachers, researchers and developers to take the curricular modules and easily customize them to particular contexts. WISE en-ables teachers to interact, give feedback and monitor student work using teacher tools. Teachers can grade student work for a particular step or for a specific student group. Teach-ers can look on a class dashboard to see individual groups’ progress through the project. If teachers see particularly in-teresting work from certain students, they can check a box to anonymously “flag” the work and put it up on a class screen. WISE provides functionality to researchers such as logging student interactions with the environment at different levels. Embedded assessments enable researchers to capture stu-dent thinking during the process of inquiry and design.

Engineering Concepts and Skills Using WISE

The WISE supports for guided inquiry make it possible to incorporate complex engineering concepts such as sys-tems and optimization and associated habits of mind into the units. Dym et al. (2005) describe crucial engineering design skills such as:

• viewing design as inquiry or as an iterative loop of divergent- convergent thinking;

• keeping sight of the big picture by including systems thinking and systems design;

• handling uncertainty;


• making decisions; • thinking as part of a team in a social process; • thinking and communicating in the several languages

of design

This list aligns with the NAE habits of mind: systems think-ing, creativity, optimism, collaboration, communication, and attention to ethical considerations (2009). Related de-sign skills include defining the problem, specifying require-ments, decomposing systems, generating solutions, creating representations, and experimenting and testing (Petrosino et al., 2008).

WISE is ideal for incorporating engineering concepts and methods in part because the emphasis on engineering concepts reflects the goal of making science relevant, as aspect of KI design. In this article we discuss how these and other engineering habits of mind are being infused into WISE units.

For example, powerful visualizations embedded in WISE curriculum encourage students to engage in systems thinking (Figures 1 and 3). Research suggests that simula-tions can foster systems thinking and emergent properties in K- 12 students (Levy & Wilensky, 2008; Wilensky & Reis-man, 2006). The MySystem steps in the Thermodynamics curriculum unit enable students to construct their own sys-tem maps of energy at various levels (Figure 4; Svihla et al., 2010). Within the Improving Your Communities’ Asthma Problem, students use visualizations to investigate how the immune system and respiratory system create an asthma attack, and design community- based solutions to improve local asthma problems (Tate, 2009). The Photosynthesis unit illustrates energy transfer and transformation using vi-sualizations and virtual experiments (Ryoo & Linn, under review). These and other WISE projects introduce the big idea of systems thinking in the context of standards- based science topics.

In addition, WISE encourages students to collaborate with each other through steps such as online brainstorms and

discussion that can be tailored to scaffold students’ knowl-edge integration. These steps enable students to share and build off of each others’ ideas in ways that can encourage participation from typically underrepresented populations (Hsi & Hoadley, 1997). For example, in the Probing Your Surroundings unit, students create principles to describe pat-terns in collected temperature data from objects in the room. Based on these created principles, students are grouped in specific online discussion groups to encourage communica-tion and refining of ideas (Clark & Sampson, 2007).

WISE encourages students to develop communication skills in different modalities. In addition to the MySystem concept mapper, WISE drawing tools enable students to make quick and easy animations of their ideas using pre-determined pictures or “stamps.” The WISE journal allows students to keep an ongoing record of their ideas, incor-porate screenshots or animations, and share these journal pages with other students in their class. WISE notes allow students to write explanations of their ideas and revisit and revise these same explanations as reoccurring notes as they progress through the curriculum.

Figure 3. WISE guides students’ explorations with the inquiry map and uses various step types and tools, such as visualizations. WISE offers tools for teachers and researchers to monitor student work, give feedback, customize and author instruciton.

Figure 4. The MySystem step in the Photosynthesis WISE project supports students to make models of energy systems.


Engaging students in relevant, meaningful, and accessi-ble inquiry projects enables students to think like engineers and learn engineering concepts and design skills. Airbags and Chemical Reactions illustrate these kinds of projects in WISE and demonstrate how students learn science and engineering content.

Airbags

Airbags was designed, iteratively refined, and tested with a partnership of teachers, researchers, and content specialists as part of the Technology- Enhanced Learning in Science (TELS) National Science Foundation Center for Learning and Technology (Kali, Linn, & Roseman, 2008; McElhaney & Linn, 2010). Airbags was designed using KI design principles and patterns (Kali, 2006; Linn & Eylon, 2006). The KI principles (Linn & Hsi, 2000) are guidelines for encouraging coherent understanding in STEM:

Making content accessible encourages learners to build on previous knowledge, connect new knowledge to existing knowledge, and appreciate the relevance of STEM concepts to their everyday lives. The Airbags unit makes content ac-cessible by situating force, motion and position and velocity graphs within the everyday context of driving cars.

Making thinking visible helps students integrate their understanding by modeling how ideas are connected and organized in new knowledge networks. Providing multiple representations of scientific phenomena and highlighting how features of the phenomena interact can make thinking visible. In Airbags, students experiment with visualizations that simultaneously present animated and graphical repre-sentations of airbag and driver motion. These visualizations also provide students with an experimental history in table format so that their previous trials are visible (Figure 5).

Helping students learn from others encourages students to develop criteria for and refine their own understanding by

confronting students with the ideas of others. In Airbags, as in all WISE projects, students are encouraged to work in dyads to promote collaboration and peer discussion about the instructed concepts. Grouping students in pairs has been found to be particularly beneficial for the exchange of ideas (Gerard et al., 2009; Madhok, 2006). Students with different levels of expertise work together to help each other learn. A student with less prior knowledge about the targeted concepts often has quite proficient computer skills, or interacts with visualizations and notices different features than their partner with more prior knowledge. Students often ask each other to explain concepts or visualizations that they do not under-stand. This explanation process helps both the explainer and the explainee learn and reinforce the targeted concepts. This kind of peer collaboration fosters knowledge integration.

Promoting autonomy and lifelong learning helps stu-dents refine their understanding by encouraging monitoring and reflection upon ideas. Airbags promotes reflection by having students construct a report about the design of cars based on the results of their experiments and investigations with the visualizations. Students reflect on their ideas by refining these design recommendations.

Core Engineering Ideas in Airbags

In Airbags students encounter systems concepts includ-ing: knowing how individual parts or processes within a system work together to carry out a particular function, knowing how to break systems down into subsystems to gain insight into the function and performance of particular parts to the whole, and knowing about the boundaries and interactions between subsystems and system or systems and the environment.

Airbags guides students by breaking down the overall system into its constituent parts. Students first investigate simulations of the airbag and its motion. Subsequently, stu-dents explore simulations of the motion of the driver. Stu-dents then experiment with a simulation of a driver and the airbag to determine how the two systems interact and safety implications of these interactions (Figure 6). Students are guided to discover different types of relationships among these variables that govern the risk that the driver will be injured from an inflating airbag. These relationships include covariation and thresholds. Driver height, speed of colli-sion, and size of crumple zone all influence the amount of time from impact to the time when the driver and airbag collide. Low speed collisions with tall drivers in cars with large crumpling will be more likely to hit an inflated airbag (more safe). High speed collisions with short drivers in cars with small crumpling will be more likely to hit an inflating airbag (unsafe). However, there are also threshold values for position and time. Short drivers who sit within the airbag’s range of deployment will always hit an inflating airbag, and where the crumpling time is greater than the airbag deploy-ment time, tall drivers, will always hit an inflated airbag. The project guides students to make these kinds of insights about the relationships among variables.

Figure 5. The airbag visualization makes students’ thinking visible by pro-viding tables with their experimental history.


Optimization involves maximizing effectiveness of a process or system by manipulating variables and taking into consideration tradeoffs, available resources, social norms, and physical laws. Airbags requires students to consider multiple variables, tradeoffs, social norms, and physical laws to make the best decisions in their final reports. The in-quiry around airbags provides a rich context for discussing constraints. Airbags must deploy very quickly within a cer-tain amount of time within a very finite space between the passenger and the steering wheel. Students consider these variables to make recommendations to the design of cars and airbags to decrease injuries and fatalities from airbags.

In Airbags, the overall driving question and inquiry project engages students in generating solutions and mak-ing decisions. Students decide whether black boxes should be designed to produce position or velocity graphs, and explain their choice using what the have learned through-out the project about position and velocity graphs and how these graphs relate to personal safety.

The context of Airbags encourages students to collab-orate with each other as a team on a problem with social and global implications. Additionally, students learn com-munication skills through different forms of representation. Students construct graphs of the airbag and driver’s motion using drawing tools within WISE and compare these graphs to ones in the simulations.

Students develop experimentation and testing skills by in-teracting with scaffolded visualizations in Airbags. Students use the visualizations to investigate questions about the role of the height of the driver, speed of collision, and crumpling

ability in relation to the driver’s risk of injury. These ques-tions align with the variables that students can manipulate in the visualization (position of the driver, velocity of the driver after impact, and time between impact and driver’s initial motion towards wheel). Students conduct trials to test their hypotheses by first selecting an investigation question from a drop- down menu. This menu also includes a choice for just exploring so that students can familiarize themselves with the visualization. Having students choose a particular question encourages students to be more mindful with their trials and focuses them on the inquiry goals. After students run the trial, they judge whether the trial was safe or unsafe. This, along with the variable settings, is visible within the experimentation history of the visualization. The experimen-tation history enables students to see patterns within the data and compare multiple trials to facilitate analysis of data and student monitoring of their experimentation.

Airbags Learning Outcomes

The design partnership for Airbags developed, refined and validated assessment items that measured connec-tions among students’ normative ideas (Lee, Linn, Varma & Liu, 2009; Linn et al., 2006; Liu et al., 2008; Liu, Lee & Linn, 2010). Embedded, pretest and posttest assessments, as well as year- end benchmark assessments consist of open response items that require students to explain, graph, and draw their understanding. Student responses were scored according to the number of normative ideas and the num-ber of elaborated links among those ideas. The overall KI

Figure 6. Airbags breaks the visualizations into airbag and passenger systems before students experiment with airbag and passenger visualization.


rubric assigns a score of 0 for irrelevant or blank answers, a score of 1 for non- normative or invalid ideas, a score of 2 for normative ideas lacking connection, a score of 3 for a valid and elaborated link among two normative and relevant ideas, and a score of 4 for complex links among three or more normative and relevant ideas (Table 2).

Students participating in Airbags made significant learn-ing gains. Across diverse schools and settings with various levels of student prior knowledge, students make large gains from pretests to posttests assessments (McElhaney, 2010). Students participating in the Airbags curriculum made con-nections among graphical representations and motion con-cepts, and made significant improvements in their ability to design and interpret valid experiments. These students also outperform similar cohorts of students on year- end tests (Lee et al., 2009). These results demonstrate that curricula engaging students in engineering thinking in science class-rooms can foster integrated understanding of both engineer-ing and science concepts.

Chemical Reactions

The Chemical Reactions project was designed, im-plemented and iteratively refined using the same TELS partnership model as Airbags. The project also used KI metaprinciples to guide design of the curriculum. For ex-ample, Chemical Reactions makes content accessible by sit-uating the curriculum within the context of climate change, energy use, and greenhouse gases. Chemical Reactions

makes thinking visible by providing interactive visualiza-tions of chemical reactions and coordinating these visuals with other representations of chemical reactions, such as videos of hydrogen balloons combusting or symbolic rep-resentations (Figure 7). Students make their thinking visible by creating their own models of chemical reactions and the greenhouse effect. Chemical Reactions helps students learn from each other through online discussions, where students discuss climate change and are guided to comment on other students’ posts. Students then view a video and subsequently refine or add posts to the online discussion. Students also critique each other’s final reports and use the feedback to revise their own reports at the end of the project. Chemi-cal Reactions promotes lifelong learning by supporting stu-dents to reflect upon their learning. Reflective prompts ask students to explain their understanding before and after the students encounter the visualizations. Additionally, students are prompted to reflect upon their understanding at the end of each activity by revisiting their explanations and notes that build towards the final report to their congressperson.

Core Engineering Ideas in Chemical Reactions

Chemical Reactions encourages systems thinking by using NetLogo (Wilensky, 1999) simulations of the green-house effect where students break down the greenhouse effect into different interacting components (Figure 8). Stu-dents use a scaffolded visualization that includes sunlight, heat, infrared radiation and a temperature output to gain

Table 2Example Knowledge Integration Scoring Rubric for Pretest and Posttest Items

Question: If a grey circle represents hydrogen, a white circle represents oxygen, and a bond is represented with a line, draw a molecular picture of the following balanced equation: 2H2 + O2 → 2H2O.(Possible ideas to integrate: Conservation of mass, molecular understanding of subscripts and/or coefficients, dynamic nature of reaction)

Score Description Student Example

4 Complex link: Two or more scientifically valid links among ideas.

3 Full link: Complete connection among ideas. Students understand how two scientific concepts interact.

2 Partial link: Partial connections among ideas, students consider relevant ideas but not consistent throught response (i.e. correct molecules but incorrect number)

1 No link: Students have non-normative links or ideas in a given context.

0 No/Irrelevant answer: Students do not engage in given science context. I don’t know


a basic understanding of how sunlight can heat the Earth, and how the Earth in turn emits infrared radiation. They are asked to predict what happens to the Earths’ temperature after running the simulation over a period of time. Many students predict that the temperature will continue to rise, or that the temperature will level off but do not understand why. The curriculum guides the students to realize that there is a dynamic equilibrium of energy from the Sun absorbed by the Earth and IR emitted from the Earth. From there, stu-dents investigate how other factors such as carbon dioxide, albedo, and population impact this process.

To promote understanding of optimization, students cre-ate reports based on the physical processes of the green-house effect that take into consideration the social norms and tradeoffs of various solutions. For instance, students in-vestigate the benefits and tradeoffs of switching to alterna-tive fuels, such as hydrogen. Students realize that although hydrogen combustion does not contribute carbon dioxide to the atmosphere, it takes energy to make and store hy-drogen fuel, and these sources of energy contribute carbon dioxide to the atmosphere. Students compare these kinds

of solutions to other solutions such as raising gas- mileage standards in light of social and scientific efficiency. In these cases, students make decisions about various solutions that do not have a defined right or wrong answer. Students ex-plore problems that have implications to students’ everyday lives, like energy use, and connect to social and global is-sues such as climate change. Students realize that what they are learning in chemistry class can contribute to decisions and recommendations in larger social contexts.

Chemical Reactions engages students in iterations of divergent- convergent thinking as they go through specific activities that culminate in an overall proposal to their con-gressperson. In each activity students explore a specific topic and relate it back to the overall goal of finding a way to reduce carbon emissions. For example, students inves-tigate hydrocarbon reactions in an activity and use those concepts to understand current sources of energy for cars. In another activity students learn about hydrogen combus-tion and alternative fuels as possible alternatives to hydro-carbon use. In each activity, students converge on specific topics but then diverge at the end of the activity to relate the

Figure 7. Chemical Reactions features molecular dynamic visualizations and supports for students to distinguish their ideas.

Figure 8. NetLogo greenhouse effect visualization used in Chemical Reactions.


principles or concepts that they just learned to the overall investigation.

This interplay between the specific concepts and the overall inquiry helps students to maintain sight of the big picture and overall system while learning subsystems and related concepts. Explicitly referencing back to the overall goals of the project gives students a support structure and frame for them to place specific knowledge and fit how sub-systems interact with the larger systems. This helps students maintain sight of the big picture during these inquiry proj-ects. This approach builds upon the success of with previ-ous K- 12 inquiry design curricula (Kolodner et al., 2003).

Chemical Reactions promotes collaboration skills by on-line brainstorms of their existing ideas. The project guides students to read and comment on other groups’ postings, then make another comment of their own. The project en-courages communication skills by having students write multiple- paragraph reports that synthesize and convey their understanding. Students create models of the greenhouse effect using drawing tools and construct models of chemical reactions by manipulating atoms and molecules (Figure 9).

Chemical Reactions Learning Outcomes

The design, refinement and validity testing of assess-ments with Chemical Reactions followed the same partner-ship and iterative refinement model as Airbags. Researchers developed pretest, posttest, and delayed posttest assess-ments and analyzed the data in accordance to the KI frame-work. Students across the country in various high schools with various levels of students gained significantly from pretests to posttests, compared to students with traditional instruction (Linn et al., 2006). Students participating in

the Chemical Reactions unit made connections among concepts such as conservation of mass, limiting reactants, heat, and molecular motion as well as connections among representations. Students also connected and distinguished ideas about chemical reactions, the greenhouse effect, and distinguished the greenhouse effect from climate change. Evidence suggests that these learning gains are robust over time; even though the unit takes only 4–5 hours of instruc-tional time, students outperform their peers on extended posttests months after instruction and in some cases outper-form themselves from posttest to extended posttest (Linn et al. 2006; Lee & Linn, 2008).

By designing, testing and refining these units using the KI framework and assessments, students made significant progress in integrating their ideas, outperformed their peers and remembered these concepts months after instruction.

The results from both Chemical Reactions and Airbags along with other TELS projects provide evidence that cur-ricula using the KI pattern can help all students learn sci-ence and engineering concepts. Both projects were tested at very diverse settings with wide ranges of students. Students not only learn, but also retain their understanding. This sug-gests that the KI pattern can be a particularly powerful way to introduce engineering concepts into science classrooms. The outcomes also provide evidence that the KI assessment framework is a valuable and reliable tool to measure links among engineering and science ideas.

Guidelines for Engineering Education Curriculum Design

To infuse engineering ideas into the K- 12 curriculum, designers need to select contexts for investigation that

Figure 9. Both Airbags and Chemical Reactions leverage drawing tools within WISE to support students creating representations.


illustrate complex, realistic situations. Successful activities should require students to use scientific ideas to solve prob-lems in these contexts. To ensure that new scientific ideas are integrated into coherent understanding, activities need trial and refinement in classroom settings. The KI pattern, used to design Airbags and Chemical Reactions, has proven effective for guiding the design process.

Iterative Design and Refinement

Iterative design and refinement based on student learning evidence is essential to ensure that units meet their goals. Both Airbags and Chemical Reactions were designed and iteratively refined by a partnership of researchers, classroom teachers, technologists, and content specialists. Ideally part-nerships will include experts in engineering to certify the validity of the engineering concepts, practices, and habits of mind. The design partnership ensures that these curriculum units are educationally sound, accurately represent impor-tant science and engineering concepts, and succeed in au-thentic classrooms with real teachers and students.

For example, iterative refinement studies of Chemical Reactions demonstrated overall learning gains for both honors and regular chemistry students. Studies of the first version revealed that students in regular chemistry were less successful than honors students in interpreting some of the visualizations. The design of the curriculum surround-ing these visualizations implemented the predict, observe, explain pattern found to be successful in various science classes (Gobert & Pallant, 2004; Krajcik, Blumenfeld, Marx, & Soloway, 1994; Tien, Teichart, & Rickey, 2007). Students predicted what a chemical reaction would look like on a molecular scale, interacted with the visualization, then described what happened in the visualization.

To refine the instruction around the visualizations, the partnership used the KI pattern and added a focus on distin-guishing ideas (e.g., Linn et al, 2010). Visualizations within Chemical Reactions were refined to help students compare and distinguish their ideas. This converted the to predict- observe- explain pattern into predict, observe, distinguish and reflect. Students were asked to distinguish ideas about how the chemical formulas related to the chemical reactions. They also considered what symbolic representations do and do not represent about reactions. Students then assessed and reflected upon their explanations. This change required modifying the context of the visualization from the design of rocket fuels to the use of hydrogen fuel to make the visu-alizations more relevant to the overall inquiry (Chiu, 2010).

This is an example of design- based research where evi-dence of student learning is used for refinement of classroom interventions and also advances theoretical understanding of learning (e.g. Design- Based Research Collective, 2003). Due to the complexity of authentic learning in classroom environments, interventions need to be tested and carefully engineered with the complete system of teachers, students, and classroom culture to reveal insights into cognition in

classroom settings (Brown, 1992). These kinds of design experiments with WISE modules can both improve class-room learning and contribute to learning theory.

In another example, students were randomly assigned to two different versions of Airbags—one version explicitly prompted students to isolate and compare variables, while another version explicitly prompted students to connect variables with the underlying concepts (McElhaney, 2010; McElhaney & Linn, 2010). On posttest assessments of overall understanding, students in the connecting concepts condition outperformed the isolate and compare variables condition. This study clarified research on experimentation to demonstrate that merely isolating and comparing vari-ables correctly may not result in greater understanding of concepts. Using design- based experiments, Airbags was able to contribute to learning theory, provide meaningful and tested instruction to students, and use the results to make refinements to the instruction and future experiments.

The Knowledge Integration Instructional Pattern

The NAE recommends that engineering education should emphasize the process of engineering design. NAE states that “the design process, the engineering approach to identifying and solving problems, is (1) highly iterative; (2) open to the idea that a problem may have many possible solutions; (3) a meaningful context for learning scientific, mathematical, and technological concepts; and (4) a stim-ulus to systems thinking, modeling, and analysis” (p. 4). Dym, Agogino, Eris, Frey & Liefer (2005) define engineer-ing design as “a systematic, intelligent process in which de-signers generate, evaluate, and specify concepts for devices, systems, or processes whose form and function achieve cli-ents’ objectives or users’ needs while satisfying a specified set of constraints” (p. 104). The KI instructional pattern of-fers a research- based design guide for creating science units that also emphasize engineering design. WISE provides a learning environment and set of features to turn these prin-ciples into practice.

The KI pattern aligns well with the engineering design process. It is composed of four processes: eliciting ideas, adding ideas, distinguishing ideas, and sorting out ideas.

Eliciting ideas. Starting by eliciting ideas enables stu-dents to build from their prior knowledge. Eliciting a full range of ideas helps students make connections across contexts and disciplines instead of isolating ideas. In the engineering design process, students elicit their ideas by brainstorming and generating a wide range of possible so-lutions to a design problem. KI research demonstrates that tools such as the WISE online brainstorming tool encour-age participation from students who may not traditionally participate in engineering (Hsi & Hoadley, 1997). Holding online brainstorms also enables all students to see every-one else’s ideas and revisit these brainstorms throughout the project. Student ideas are visible to both students and teachers.


Adding ideas. The next element of the KI instructional pattern, introducing new, normative ideas through carefully designed instruction has the goal of helping students build upon their existing ideas and make connections among these new ideas. In the engineering design process, after students come up with a wide range of possible solutions by brainstorming their ideas, students need to seek out ad-ditional knowledge and information about their proposed solutions, including related math and science concepts. In WISE, powerful visualizations enable students to learn about scientific concepts and experiment with ideas. WISE also gives students the freedom to learn about ideas as they see fit. Students can choose different topics to learn about in a just- in- time manner. For example, in the Designing House project, students choose to become experts in walls, roofs, or windows. Students can then either jigsaw into groups with different expertise or come back to these topics as they need to during the design process (Cuthbert & Slotta, 2004).

Distinguishing ideas. As mentioned, a crucial aspect of the KI framework is to help students develop criteria and distinguish among their ideas. Eliciting and adding new ideas can result in links and connections among ideas or concepts that may or may not be productive. Instruction that guides learners to evaluate their ideas using powerful cri-teria is needed to help students learn. When students pick a certain design solution, they need to evaluate their solu-tions or ideas using design criteria or set of constraints. Stu-dents can use WISE assessment tools to assess and evaluate their own understanding of concepts (Chiu & Linn, 2008; Davis & Linn, 2000). WISE online discussion tools enable students to post designs and offer feedback on each others’ designs according to negotiated or given criteria. These discussions can be seeded, or students can be grouped into predetermined topics or levels of expertise, or based on se-lections that students make.

Sorting out ideas. After learners evaluate their ideas, they need support to reflect, refine, and sort out the con-nections among their ideas. In the engineering design process, after students evaluate their design, they need to reflect upon their initial design and the given evaluations and refine and redesign their solution. These reflective pro-cesses have demonstrated benefit to engineering education (Adams, Turns & Atman, 2003). WISE journal tools en-able students to make refinements to their designs and log changes between previous experiments and new proposed experiments/designs.

This pattern can guide the iterative design process. Com-bined with specific design principles (Kali, 2006) and the features of WISE (Slotta & Linn, 2009), this process can help designers create effective precollege activities that fea-ture engineering design concepts and practices.

Discussion

The KI pattern and WISE features provide a way to le-verage the natural connections between engineering and

science inquiry (NAE, 2010). This article shows how the KI framework can bridge engineering and science topics to support inquiry. For both Airbags and Chemical Re actions we illustrate ways to showcase engineering principles in science units. Both units resulted in student learning of sci-ence content and engineering skills.

Other curriculum materials built for science inquiry have a similar potential. For example, Model- It (Spitulnik, Krajcik & Soloway, 1999; Stratford, Krajcik & Soloway, 1998), Virtual Solar System (Barab, Hay, Barnett & Keating, 2000) and ThinkerTools (White & Frederiksen, 1998) ask students to make, test, and revise models to explain scientific phenomena. To fully succeed, these and other inquiry mate-rials are most successful when they engage students in using the full KI instructional pattern (see Linn & Eylon, in press).

Instruction that engages students in design tasks, such as Learning by Design (Kolodner et al., 2003), design- based science (Fortus et al., 2004), and Learning- for- Use (Edel-son, 2001) have been successfully implemented in K- 12 set-tings. These environments also have the potential of guiding students through eliciting, adding, distinguishing and sort-ing ideas but often depend on a talented teacher to succeed (Linn & Eylon, in press).

KI provides a unified framework based on research in learning and cognition that aligns learning theory, curricu-lum design, and assessment. The KI patterns and principles can provide guidance for the emerging field of K- 12 engi-neering education. Current work with KI and WISE illus-trate the power of the KI pattern and suggest ways to refine instruction to promote coherent understanding.

References

Adams, R. S., Turns, J., & Atman, C. J. (2003). “Educating effective en-gineering designers: The role of reflective practice,” Design Studies, 24(3), 275–294

Bernhard, J., Carstensen, A.- K., & Holmberg M. (2007). Design- based educational research and development of engineering education: Some examples from courses in mechanics and in electrical engineering. Paper presented at ASEE Global Colloquium on Engineering Educa-tion, Istanbul.

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2(2), 141–178.

Chiu, J. L. (2010). Supporting students’ knowledge integration with technology- enhanced inquiry curricula (Doctoral dissertation). Re-trieved from Dissertation and Theses database. (UMI No. AAT 3413337)

Chiu, J. L. & Linn, M. C. (2008). Self- Assessment and self- explanation for learning chemistry using dynamic molecular visualizations. In In-ternational Perspectives in the Learning Sciences: Cre8ting a Learning World. Proceedings of the 8th International Conference of the Learn-ing Sciences (Vol. 3, pp. 16–17). Utrecht, The Netherlands: Interna-tional Society of the Learning Sciences, Inc.

Clark, D. B., & Sampson, V. (2007). Personally seeded discussions to scaf-fold online argumentation. International Journal of Science Education, 29(3), 253–277.

Clark, D. B., Varma, K., McElhaney, K., & Chiu, J. L. (2008). Structure and design rationale within TELS projects to support knowledge in-tegration. In D. Robinson & G. Schraw (Eds.), Recent innovations in


educational technology that facilitate student learning (pp. 157–193). Charlotte, NC: Information Age Publishing.

Cobb, P. (2001). Supporting the improvement of learning and teaching in social and institutional contexts. In S. M. Carver and D. Klahr (Eds.) Cognition and instruction: Twenty- five years of progress.(pp. 455–478). Mahwah, NJ: Erlbaum.

Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O. Shea (Eds.), New directions in educational technology (pp. 15–22). New York: Springer- Verlag.

Cuthbert, A. & Slotta, J. (2004). Designing a Web- based design curriculum for middle school science: The WISE Houses in the Desert project. International Journal of Science Education, 24(7), 821–844.

Davis, E. A., & Linn, M. C. (2000). Scaffolding students’ knowledge in-tegrations: Prompts for reflection in KIE. International Journal of Sci-ence Education, 22(8), 819–37.

The Design- Based Research Collective. (2003). Design- based research: An emerging paradigm for educational inquiry. Educational Re-searcher, 32(1), 5–8.

Dym, C., Agogino, A., Eris, O., Frey, D., & Leifer, L. (2005). Engineering design thinking, teaching, and learning. Journal of Engineering Educa-tion, 94(1), 103–120.

Gerard, L. F., Tate, E., Chiu, J., Corliss, S. B., & Linn, M. C. (2009). Col-laboration and knowledge integration. In International Perspectives in the Learning Sciences: Proceedings of the 8th International Confer-ence of Computer Supported Collaborative Learning (pp. 188–193). Rhodes, Greece: International Society of the Learning Sciences, Inc.

Gobert, J. & Pallant, A. (2004). Fostering students’ epistemologies of mod-els via authentic model- based tasks. Journal of Science Education and Technology, 13(1), 7–22.

Hmelo- Silver, C., Duncan, R., & Chinn, C. (2006). Scaffolding and achievement in problem- based and inquiry learning: A response to Kirschner, Sweller, and Clark (2006). Educational Psychologist, 42(2), 99–107.

Hsi, S, & Hoadley, C. (1997). Productive discussion in science: Gender equity through electronic discourse. Journal of Science Education and Technology, 6(1). 23–36.

Kali, Y. (2006). Collaborative knowledge building using the Design Prin-ciples Database. International Journal of Computer Support for Col-laborative Learning, 1(2), 187–201.

Kali, Y., Linn, M. C., & Roseman, J. E. (2008). Designing coherent science education. New York: Teachers College Press.

Katehi, L., Pearson, G. & Feder, M. (Eds.). (2009). Engineering in K- 12 education: Understanding the status and improving the prospects. Washington, DC: National Academies Press.

Kolodner, J. L., Camp, P., Crismond, D., Fasse, B., Gray, J., Holbrook, J. Puntambekar, S., & Ryan, M. (2003). Problem- based learning meets case- based reasoning in the middle- school classroom: Putting learn-ing by design into practice. Journal of the Learning Sciences, 12(4), 495–547.

Krajcik, J., Blumenfeld, P., Marx, R., & Soloway, E. (1994). A collab-orative model for helping middle grade science teachers learn project- based instruction. Elementary School Journal, 94(5), 483–497.

Lee, H.- S., & Linn, M. C. (2008, March 25). Investigating the longterm impact of technology- rich interventions on knowledge integration. Paper presented at the annual meeting of the American Educational Research Association, New York, NY.

Lee, H.-S., Linn, M. C., Varma, K., & Liu, L. (2009). How do technology- enhanced inquiry science units impact classroom learning? Journal of Research in Science Teaching, 47(1), 71–90.

Levy, S. T. & Wilensky, U. (2008). Inventing a “mid- level” to make ends meet: Reasoning through the levels of complexity. Cognition & In-struction, 26(1), 1–47.

Linn, M. C. (1995). Designing computer learning environments for engineer-ing and computer science: The scaffolded knowledge integration frame-work. Journal of Science Education and Technology, 4(2), 103–126.

Linn, M. C., Chang, H.- Y., Chiu, J., Zhang, H., & McElhaney, K. (2010). Can desirable difficulties overcome deceptive clarity in scientific

visualizations? In A. Benjamin (Ed.), Successful remembering and successful forgetting: a Festschrift in honor of Robert A. Bjork (pp. 239–262). New York: Routledge.

Linn, M. C., Clark, D. & Slotta, J. D. (2003). WISE design for knowledge integration. Science Education, 87, 517–538.

Linn, M. C., Davis, E. A. & Bell, P. (2004). Internet environments for sci-ence education. Mahwah, NJ: Erlbaum.

Linn, M. C., & Eylon, B.-S. (2006). Science education. In P. A. Alexander & P. H. Winne (Eds.) Handbook of Educational Psychology, 2nd edi-tion. Mahwah, NJ: Erlbaum.

Linn, M. C., & Eylon, B.- S. (in press). Science learning and instruction: Taking advantage of technology to promote knowledge integration. New York: Routledge.

Linn, M. C., & Hsi, S., (2000). Computers, teachers, peers: Science learn-ing partners. Mahwah, NJ: Erlbaum.

Linn, M. C., Lee, H.-S., Tinker, R., Husic, F., & Chiu, J. L. (2006). Teaching and assessing knowledge integration in science. Science, 313(5790), 1049–1050.

Liu, O. L., Lee, H. S., Hofstetter, C., & Linn, M. C. (2008). Assessing knowledge integration in science: Construct, measures, and evidence. Educational Assessment, 13(1), 33–55.

Madhok, J. (2006). The longitudinal impact of an eighth- grade inquiry cur-riculum on students’ beliefs and achievements in science. (Doctoral dissertation). Retrieved from Dissertation and Theses database. (UMI No. AAT 3228414)

McElhaney, K. W. (2010). Making controlled experimentation more infor-mative in inquiry investigations (Doctoral dissertation). Retrieved from Dissertation and Theses database. (UMI No. AAT 3413549)

McElhaney, K. W., & Linn, M. C. (2008). Impacts of students’ experi-mentation using a dynamic visualization on their understanding of mo-tion. In International Perspectives in the Learning Sciences: Cre8ting a Learning World. Proceedings of the 8th International Conference of the Learning Sciences (Vol. 2, pp. 51–58). Utrecht, The Netherlands: International Society of the Learning Sciences, Inc.

McElhaney, K., & Linn, M. C. (2010). Helping students make controlled experiments more informative. In K. Gomez, L. Lyons & J. Radin-sky (Eds.), Learning in the disciplines: Proceedings of the 9th Inter-national Conference of the Learning Sciences (Vol. 1, pp. 786–793). Chicago: International Society of the Learning Sciences.

Mulford, D. R., & Robinson, W. R., (2002). An inventory for alternate conceptions among first- semester general chemistry students. Journal of Chemical Education, 79(6), 739–744.

National Academy of Engineering (2004). The engineer of 2020: Visions of engineering in the new century, Washington, DC: The National Acad-emies Press.

National Academy of Engineering (2010). Standards for K- 12 engineering education? Washington, DC: The National Academies Press.

National Academy of Science (2010). Rising above the gathering storm, revisited: Rapidly approaching category 5, Washington, DC: The Na-tional Academies Press.

Nicoll, G. (2001). A report of undergraduates’ bonding misconceptions. International Journal of Science Education, 23(7), 707–730.

Obama, B. (2010). Remarks by the president at the announcement of the “Change the Equation” initiative. Retrieved from the White House Web site: http://www.whitehouse.gov/the- press- office/2010/09/16/remarks - president- announcement- change- equation- initiative

Osborne, R., & Cosgrove, M. M. (1983). Children’s conceptions of the changes of state of water. Journal of Research in Science Teaching, 20(9), 825–838.

Ozmen, H. (2004). Some student misconceptions in chemistry: A literature review of chemical bonding. Journal of Science Education and Tech-nology, 13(2), 147–159.

Petrosino, A. J., V. Svihla, & Brophy, S. (2008). Engineering skills for un-derstanding and improving K-12 engineering education in the United States. Presented at the National Academy of Engineering/National Research Council workshop on K-12 Engineering Education. Wash-ington, DC.


Pfundt, H., & Duit, R. (1991). Students’ alternative frameworks (3rd Ed.). Federal Republic of Germany: Institute for Science Education at the University of Kiel.

Shear, L., Bell, P., & Linn, M. C. (2004). Partnership models: The case of the deformed frogs. In M. C. Linn, E. A. Davis, & P. Bell (Eds.), Internet environments for science education (pp. 289–311). Mahwah, NJ: Erlbaum.

Silk, E. M. & Schunn, C. (2008). Core concepts in engineering as a basis for understanding and improving K-12 engineering education in the United States. Paper presented at the National Academy of Engineering /National Research Council workshop on K-12 Engineer-ing Education, Washington, DC.

Slotta, J., & Linn, M. C. (2009). WISE Science: Web- based inquiry in the classroom. New York: Teachers College Press.

Smith, J. P., diSessa, A. A., & Roschelle, J. (1993). Misconceptions recon-ceived: A constructivist analysis of knowledge in transition. Journal of the Learning Sciences, 3(2).

Svihla, V., Gerard, L., Ryoo, K., Sato, E., Visintainer, T., Swanson, H., et al. (2010, June 29–July 2). Energy across the curriculum: Cumulative

learning using embedded assessment results. Paper presented at the In-ternational Conference of the Learning Sciences, Chicago.

Tate, E. (2009). Asthma in the community: Designing instruction to help students explore scientific dilemmas that impact their lives (Doctoral dissertation). Retrieved from Dissertation and Theses database. (UMI No. AAT 3383554)

Tien, L., Teichart, M., & Rickey, D. (2007). Effectiveness of a MORE lab-oratory module in prompting students to revise their molecular- level ideas about solutions. Journal of Chemical Education, 84(1), 175–181.

Wilensky, U. (1999). NetLogo [Computer software]. Evanston, IL: Center for Connected Learning and Computer Based Modeling, Northwestern University. http://ccl.northwestern.edu/netlogo.

Wilensky, U., & Reisman, K. (2006). Thinking like a wolf, a sheep, or a firefly: Learning biology through constructing and testing compu-tational theories—An embodied modeling approach. Cognition and Instruction, 24(2), 171- 209.

Yarroch, W. L. (1985). Student understanding of chemical equation bal-ancing. Journal of Research in Science Teaching, 22(5), 449– 459.



This work was funded by a grant from the National Science Foundation # EEC-0648267, entitled “Aligning Educational Experiences with Ways of Know-ing Engineering (AWAKEN)” to the University of Wisconsin-Madison.

How Professional Development in Project Lead the Way Changes High School STEM Teachers’ Beliefs

about Engineering Education

Mitchell Nathan, Amy K. Atwood, Amy Prevost, L. Allen Phelps

University of Wisconsin–Madison

Natalie A. Tran

California State University, Fullerton

Abstract

This quasi- experimental study measured the impact of Project Lead the Way (PLTW) instruction and professional development train-ing on the views and expectations regarding engineering learning, instruction and career success of nascent precollege engineering teach-ers. PLTW teachers’ initial and changing views were compared to the views exhibited by a control group of high school STEM teachers. The primary instrument was the Engineering Beliefs and Expectation Instruments for Teachers (EEBEI- T), which included Likert scale items, contextualized judgments about fictional student vignettes, and demographic items. Teachers’ baseline survey responses, on aver-age, revealed the importance academic achievement on teachers’ decision making about who should enroll in future engineering classes and their predictions of who would be most likely to succeed in an engineering career. When making implicit comparisons between students who differed by SES, teachers generally favored enrollment and predicted more career success of high SES students. SES was excluded as a factor in the judgments of all participating teachers when explicitly probed, however. Preexisting group differences showed that budding PLTW teachers reported on STEM integration in their classes with greater frequency than control teachers, while control teachers agreed more strongly about the prerequisite role of high scholastic achievement for engineering studies. Finally, an analysis of teachers’ changing views indicated that nascent PLTW teachers increased their reporting of effective STEM integration over time, above and beyond preexisting group differences and retesting effects. In light of these data we explore the challenges of implementing effective STEM integration in high school classrooms, examine issues of attracting underrepresented students to engineering, and discuss some of the inherent tensions of engineering education at the K- 12 level.

Keywords: Diversity in engineering, K- 12 engineering education, STEM Integration, Teacher beliefs.

16 M. Nathan, A. K. Atwood, A. Prevost, L. A. Phelps, N. A. Tran / Journal of Pre-College Engineering Education Research

As United State high schools respond to calls for im-proving student learning in science, math, and technology and precollege engineering (NRC, 2007) and confront the increasing availability of funding opportunities such as Race to the Top, greater numbers of K- 12 educators are par-ticipating in professional development activities for STEM (science, technology, engineering, and mathematics) educa-tion. Consequently, there is a growing need to understand K- 12 STEM teachers’ knowledge and beliefs, effective-ness and instructional decision making (Fink, Ambrose, & Wheeler, 2005). Education research shows that instruc-tional practice and teacher decision making are influenced by teachers’ beliefs about learning and instruction (Borko, Livingston, & Shavelson, 1990; Brophy & Good, 1974; Grossman, 1990; Nathan & Koedinger, 2000; Rosenthal & Jacobson, 1968). Furthermore, the educational experience for students is dependent on the quality and effectiveness of teachers, more than perhaps any other single alterable fac-tor (Leinhardt & Greeno, 1986; Nye, Konstantopoulos, & Hedges, 2004; Rowan, 2004). For example, teachers’ views have serious implications for the perceived place and pur-pose of engineering in the K- 12 curriculum, as noted in a recent report from the National Academy of Engineering (Custer & Daugherty, 2009). Furthermore, as professional development programs in precollege engineering prolif-erate, there is additional need to understand the nature of changes we can expect to see in teachers’ beliefs and expec-tations about engineering instruction and learning as teach-ers learn more about engineering and ways to teach it.

There have been recent studies on teacher knowledge, beliefs, and instructional practices for engineering educa-tion. Cunningham (2009) showed changes in elementary teachers’ reports about their content knowledge, pedagogy, and student engagement as a result of participating in the Engineering is Elementary (EiE) professional development workshops. Student outcome measures showed greater gains associated with the EiE teacher training.

Yasar and colleagues (2006) surveyed K- 12 teachers’ knowledge and perceptions of engineers and engineering practice. The authors argue that understanding teachers’ views in this area is a necessary step toward developing long- range plans to better integrate technology and design into K- 12 education. Shulman (2005), directing research of the Carnegie Foundation for the Advancement of Teaching, doc-umented how universities prepare students for professional practice in areas of law, nursing, the clergy, and engineering. The “signature pedagogy” for engineering is shown to dem-onstrate “a lovely juxtaposition between the formal require-ments entailed in learning math and science and the creative challenges that accompany ‘messing with the world’” (p. 11). Still, the editors of Journal of Engineering Education rightly point out, there is still little known about the “engineering teaching culture” (Steering Committee of the National Engi-neering Education Research Colloquies, 2006).

To address this growing area of interest and importance, we set out to examine already- practicing teachers’ beliefs

and expectations about engineering instruction and student learning as it occurs at the high school level, and document how these views change as teachers become newly trained to teach an engineering education curriculum. We examined teachers’ changing beliefs in the context of their initial ex-periences teaching courses from the Project Lead the Way (PLTW) program. Although some selection bias is inherent in a study of this nature (we are not currently at liberty to as-sign who will and will not teach PLTW), causal inferences are supported by quasi- experimental design that examines changes in teachers’ views before and after their first PLTW course above and beyond those changes exhibited by a con-trol group of STEM teachers who did not participate in the training or teaching of a PLTW course.

Measuring STEM Teachers’ Beliefs About Engineering Education

Previous research (Nathan, Tran, Atwood, Prevost, & Phelps, 2010) has shown the Engineering Education Be-liefs and Expectations Instrument for Teachers (EEBEI- T; pronounced “eebee tee”) to be a valuable instrument for measuring teachers’ views as they relate directly to precol-lege engineering education, preparation for future studies in engineering, and expectations for success in engineer-ing careers. The EEBEI- T was originally given to 143 high school STEM teachers located in a moderately large urban city in the midwestern United States. Part one of the in-strument included a set of Likert scale items that contained seven highly reliable constructs (α ≥ .70). Reliability of the constructs was replicated with a second administration to a national sample of STEM teachers (N = 82).

In findings about STEM instruction, most teachers report using students’ interests, cultural and family backgrounds, and prior academic performance to guide their teaching prac-tices. A minority of teachers reported that they adequately integrate math and science concepts with engineering activi-ties and concepts. With regard to engineering preparation, teachers generally agreed that it takes place in multiple con-texts, including academic and technical education courses, as well as at home, and in community and workplace set-tings. Teachers generally believed that to become an engi-neer students must show high academic achievement in their science, math, and technology, and technology courses. Teachers also believed, on average, that having a parent as an engineer increases a student’s likelihood of becoming one, as does being male and either white or Asian. However, student socioeconomic status (SES) was not reported as an important consideration by the teachers when determining student preparation using the Likert scale items.

Prior results also showed the EEBIE- T to be sensitive to group differences between teachers who focused primarily on engineering education within career and technical educa-tion programs and those STEM teachers in the sample pri-marily focused on instruction in college preparatory math and science. Statistically significant differences (p < .05)

M. Nathan, A. K. Atwood, A. Prevost, L. A. Phelps, N. A. Tran / Journal of Pre-College Engineering Education Research 17

between these two groups of STEM teachers were identified in three areas. First, math and science teachers as a group were less likely to identify sources of support for engineer-ing in their schools than engineering teachers. Second, on average, math and science teachers more strongly supported the view that an engineer needs to show high levels of aca-demic achievement in science, math, and technology, and technology. Third, engineering teachers collectively were far more likely to contend that their classroom instruction effectively integrates engineering activities with concepts from math and science.

Content validity (Cronbach, 1971) of the instrument, sometimes also referred to as “face validity,” was also es-tablished. First, the survey corroborated the expectation that current and prospective engineering teachers would be more aware of the engineering resources offered and were more likely to be in schools that offer such resources. Second, as expected, the survey found that teachers of academically oriented science and math courses that typically serve a col-lege preparatory function, rather than providing technical skills, will regard excellence in academic performance as paramount to success in engineering.

In part two of the instrument, teachers read vignettes that profiled four fictional high school students with differing academic, gender, and socioeconomic descriptions in order to further reveal teachers’ views during contextualized ad-vising and decision making tasks. Analyses of teachers’ responses compared across the vignettes showed that teach-ers relied a great deal on a student’s prior academic perfor-mance when deciding on whether to endorse the students for enrollment in engineering courses, and when predicting the student’s likelihood of success in a future engineering career. Teachers’ decisions were also apparently influenced by family SES of the student. Specifically, teachers tended to support enrollment in engineering classes and predict higher rates of career success for students from more privi-leged family circumstances. Teachers were not consciously aware of these influences, however, as indicated by their re-sponses to other survey questions.

Having demonstrated the reliability, validity, and utility of the EEBEI- T, the next logical step is to use it to measure changes in teachers’ views as a result of their professional development and teaching experiences in engineering edu-cation. This would provide insights about the impact that these new teaching experiences can have on teachers’ views. Such findings contribute to our understanding of the nature of high school engineering instruction and teacher change during a critical stage of the engineering pathway.

Precollege Engineering Education: The PLTW Curriculum and Teacher Training Program

We chose to examine teacher belief change in the context of a specific, well- regarded engineering program, Project Lead the Way (PLTW). PLTW is one of the most widely used precollege engineering curricula in the United States.

The program has been adopted by more than 2,700 schools (2000 high schools and 700 middle schools; Katehi, Pear-son, & Feder, 2009), and is present in all 50 states (Walcerz, 2007). PLTW was singled out in Rising Above the Gather-ing Storm (NRC, 2007) as a model curriculum for provid-ing the kind of rigorous K- 12 materials needed to improve math and science learning and increase America’s techno-logical talent pool. Thus, findings based on PLTW have far- reaching implications.

PLTW is designed to integrate engineering, science, math, and technology, and technology into the students’ academic program of study at the middle and high school levels. The high school program Pathway to Engineering™ offers seven high school courses including three one- year foundation courses (Introduction to Engineering Design, Principles of Engineering, and Digital Electronics) as well as specialization courses (Aerospace Engineering, Biotech-nical Engineering, Civil Engineering and Architecture, and Computer Integrated Manufacturing). These courses can be used for credit at accredited colleges and universities. In addition, there is an engineering research capstone course, Engineering Design & Development (PLTW, 2004).

As a precondition to teaching any one of the PLTW courses, teachers must attend an extensive professional development program, including training provided by the PLTW network of affiliate colleges and universities. This training aims to make teachers proficient in content knowl-edge and project- and problem- based instruction. National affiliates offer graduate credit for teachers.

A recent international review of research on professional learning for educators by Linda Darling- Hammond and col-leagues (2009) reports that strategically designed, intensive, and sustained professional learning can have a powerful in-fluence on teacher skills and knowledge, and ultimately lead to improvements in student learning. Prevost and colleagues (2009) examined the PLTW teacher professional develop-ment training documents, training activities, teacher pro-jects, and teacher self- assessment and self- reflection items for the PLTW foundations courses. The authors described the trainings as academically intense programs tailored to the respective student course, localized to a two- week sum-mer course. The focus on the PLTW summer training insti-tute is for teachers to gain mastery of the curriculum content they will teach, including familiarity with the design and measurement tools typically used by engineers such as drafting, CAD, and tools for physical and virtual dimen-sioning (Introduction to Engineering Design); knowledge of simple machines, thermodynamics, free body diagrams, kinematics, and ballistic devices (Principles of Engineer-ing); and coverage of laws of physics and principles of engineering design as they apply to analog and digital elec-tronics, such as Ohm’s law, truth tables, Karnaugh maps, Boolean algebra, use of the computer program MultiSims, the basic electronic robot Basic Stamp, combinational and sequential logic design, and how to create and troubleshoot breadboard circuits, including mastering the use of a logic


probe and multimeter (Digital Electronics). Analysis of the teacher materials show it is rich with math and science con-cepts that were often explicitly integrated into the engineer-ing activities, particularly for later courses in the curriculum sequence. Little in that analysis, however, was revealed about the impact these training experiences had on teach-ers’ beliefs, knowledge, and instructional practices.

Research Goals

Several research goals drove this investigation. First, in an effort to better understand the “engineering teaching cul-ture” at the high school level, we set out to measure precol-lege STEM teachers’ beliefs about engineering education. Specifically, we examined teachers’ baseline views in areas such as how they prepare students, which student factors influence their instruction, who teachers thought should have access to engineering courses, and which student traits teachers believed predicted a successful engineer. Since a portion of the teachers in our sample would subsequently participate in a formal professional development program for engineering education, as a second goal we wanted to assess preexisting differences that may exist between STEM teachers that went on to teach engineering courses and those that did not.

Third, we set out to document the changes in beliefs that arose as a consequence of becoming a newly minted pre-college engineering teacher. As noted, we chose to do this in the context of a specific, representative program, PLTW, because of its wide use nationally and its reputation for achieving rigor in STEM education (NRC, 2007). To ob-tain a more realistic sense of the impact of the intervention, we measured the combined impact on teachers’ beliefs of the PLTW professional development training and the initial PLTW teaching experience. We re- administered the beliefs survey to STEM teachers who did and who did not partici-pate in the PLTW training program and go on to actually teach a PLTW course. Together this approach led to a 2 fac-tor (time 1 vs. time 2) by 2 factor (summer institute, SI vs. control, CO) quasi- experimental design (Shadish, Cook, & Campbell, 2002) that examined changes in pre- and pos-tintervention survey responses for SI teachers above and beyond changes exhibited by those CO teachers who did not elect to train and teach a PLTW course within the time period of the study. Finally, we documented teachers’ deci-sion making about specific (fictional) students as portrayed in student vignettes.

Method

Participants in the initial administration of the EEBEI- T survey were high school science, mathematics, and techni-cal education teachers (N = 182; see Table 1 for the popula-tion demographics, where column Ns differ from the total sample size because some participants opted to not respond to some demographics items). Teachers were recruited

by email through state departments of instruction and the PLTW affiliate colleges. Most respondents were white (95.9%) and male (59.8%). None of the teachers had taught PLTW at the time of the first survey administration or taken part in the PLTW teacher summer institute training. Dur-ing summer 2008 some of the teachers (N = 82) attended a mandatory PLTW summer institute (SI) and became ini-tially certified to teach PLTW engineering courses. The re-maining control (CO) teachers (N = 100) provided control for time and repeated exposure to the survey items.

While there are proportionately fewer female teachers in the SI group (27%) compared to the CO group (51%), similar proportions were observed in previous investiga-tions (29% female engineering teachers across 5 curricu-lum programs, versus 71% male engineering teachers in Daugherty, 2009; and 23% female PLTW teachers versus 51% female non- PLTW teachers in Nathan et al., 2010). The sampling appears to exhibit gender differences that are reflected among the population of engineering teachers and engineers in the workforce, more broadly (Clark, 2009).

When teachers were surveyed again in January 2009, we were able to document changes in their views and expec-tations due to the SI training and one semester of PLTW teaching. At retest, 36 SI teachers and 41 CO teachers completed the second survey. This design allowed us to track both initial differences in the beliefs and expectations among teachers with different teaching assignments, and to document the effects that preengineering professional de-velopment had on newly minted PLTW teachers, control-ling for effects of survey retesting and time.

Each survey was administered online to all partici-pants, using a secure system provided by the University of

Table 1Teacher Demographics Overall and By Comparison Groups

Overall Control SI

No. Years Teaching N = 174 N = 96 N = 78

0–3 15.52% 11.46% 20.51%4–10 24.71% 22.92% 26.92%11–20 36.20% 38.54% 33.33%20 + 23.56% 27.08% 19.23%

Highest Degree N = 173 N = 96 N = 77

BA 36.42% 32.29% 41.56%MA 61.85% 66.67% 55.84%PhD 1.73% 1.04% 2.60%

Gender N = 174 N = 96 N = 78

Male 59.77% 48.96% 73.08%Female 40.23% 51.04% 26.92%

Race/Ethnicity N = 169 N = 92 N = 77

White/Caucasian 95.86% 98.91% 92.21%African- American 2.96% — 6.49%Hispanic — — —Other 1.19% 1.09% 1.30%


Wisconsin. Participants read through and agreed to an IRB- approved consent statement, following federal guidelines for working with human subjects. All participants were of-fered $10 in compensation for their efforts each time they participated.

The EEBEI- T survey (Nathan et al., 2010) is made up of 42 Likert scale items across 7 previously tested constructs, along with 16 demographic questions. Below are two ex-ample survey items. A 5- point Likert scale (with a midpoint of 3) was used to rate teachers’ beliefs about the frequency of occurrence of the events stated in some survey items. Item 8a shows a statement followed by the 5 choices, with the verbal anchors for each frequency scale score shown in parentheses.

8a. The math content being taught in my courses is explicitly connected to engineering.

1 (Never) 2 (Almost Never) 3 (Sometimes) 4 (Often) 5 (Almost Always)

A 7- point Likert scale (with a midpoint of 4) was used for rating teachers’ levels of agreement with statements. Item 6a shows a statement followed by the 7 choices, with the verbal anchors for each agreement scale score shown in parentheses:

6a. To be an engineer a student must have high over-all academic achievement.

1 (Strongly disagree) 2 (Disagree) 3 (Somewhat disagree) 4 (Neutral) 5 (Somewhat agree) 6 (Agree) 7 (Strongly agree)

Teachers visited a web link provided by email and, after giving consent for the study, selected the “radio button” that indicated their rating for each statement that was intended to match their own views. The online system ensured that only the choices provided were selected (no intermediate rating values were possible, for example). Because space on a page was not a factor for the online presentation, every item was accompanied by the complete set of verbal an-chors for every numerical rating choice, minimizing errors due to forgetting or reversing of the scales.

Inclusion of the CO group the following winter allowed us to examine the changes in teachers’ views when control-ling for two important influences beyond just the effects of retesting. First, CO and SI teachers started out with some significant differences in their beliefs and expectations about engineering prior to the intervention. Baseline com-parisons between the CO and SI group made these initial differences apparent and quantified them. It also provided

empirical support for the claim that there very likely is some selection bias between the two samples, since teachers self- select for PLTW instruction. Since we are not in a posi-tion to experimentally randomize something as important and personal as who becomes a PLTW instructor, the base-line data allow us to control for these inherent differences. Second, if changes in views occur over time—as teachers mature, as historical events unfold that influence attitudes about engineering or education (such as a presidential elec-tion, or the release of the Grand Challenges), or simply as a result of retesting—these changes can also be controlled for statistically.

Results

In this section we report and interpret the ratings and selections that teachers gave during each of the survey ad-ministrations, before and after the SI group taught a PLTW course.

Teachers’ Initial Beliefs and Expectations About Engineering Preparation

Table 2 summarizes the seven constructs from the Likert scale portion of the survey that were central to our study. The titles and verbal interpretation shown for each construct are inferred and did not appear anywhere on the survey, but are meant to help the reader understand the overall mean-ing conveyed across the range of items given. In addition, we show the total number of final items used in our analy-ses, followed by whether responses were along a 5- point or 7- point rating scale.

Constructs with a 5- point scale (Constructs A, B, F, & G) assessed teachers’ views of the frequency with which specific conditions or events occurred. Mean ratings above 3 (Table 2) indicate that, on average, teachers believed that these conditions were more common than uncommon. Data from Construct A show that teachers’ views overall were slightly above the midpoint of the scale, indicating that their lessons were sometimes shaped by students’ academic performance. Construct B shows that teachers overall rate right near the midpoint of the rating scale, meaning that, as a group, they sometimes use students’ interests and cultural backgrounds to inform classroom activities (though indi-viduals in the group may be anywhere along the frequency range). The responses for Construct F show that teachers believe that they sometimes make the relation between sci-ence and math content to engineering activities explicit to students. Construct G reveals that teachers, as a group, believe their schools sometimes or infrequently provide re-sources such as career day or internships for students inter-ested in engineering.

Constructs with a 7- point scale (Constructs C, D, & E) assessed teachers’ levels of agreement with the given state-ments. A rating of 1 was used for strong disagreement, and 7 for strong agreement. Mean ratings below 4 indicate that


teachers generally disagreed with the statements. The re-sponses from Construct C indicate that, as a group, teachers strongly agreed that students learn science, math, and tech-nology in out- of- school settings such as the home or com-munity center. Construct D shows that teachers generally believe that high academic performance in science, math, and technology, and technology courses is prerequisite to a career in engineering. Data from Construct E reveal that, on average, teachers believe that one’s cultural or social background (e.g., parents as engineers, or being of Asian descent) is influential in one’s decisions about pursuing a career in engineering.

To account for the indirect nature of survey measures and their inherent subjectivity, we performed internal con-sistency reliability analyses on the survey constructs using Cronbach’s alpha (α), a measure that varies between 0 and 1.0 (Cronbach, 1951). The reliability analysis suggests that the EEBEI- T is a well- designed instrument. The relevant parameters are shown in Table 2 for the original sample (N = 182). First, mean ratings of each construct are near the center value for each scale, indicating that responses to these constructs are not statistically skewed. Second, the es-timated values for Cronbach’s alpha are all above .70, and most are nearly .8 or above, indicating a high reliability es-timate (Black, 1999).

Differences in Teachers’ Initial Beliefs and Expectations About Engineering Preparation

For most of the constructs (A, B, C, E), the differences between the SI and the CO groups were not statistically sig-nificant (see Table 3). However, the results show that the EEBEI- T exposes some statistically significant differences when comparing group means for other constructs (D, F, and G).

Three differences were identified. First, CO teachers were less likely to identify sources of support for engineer-ing in their schools (construct G) than future SI teachers, t(180) = –4.029, p = .000. This result, while interesting, may simply be due to differences in the resources actually offered by schools with lesser and greater commitments to technical education and school- to- work transition programs. It is logical, for example, to imagine that those striving to teach PLTW in the future come from schools that already have a commitment to precollege engineering. It also may signal differences in their awareness of the availability of resources. Of course, the actual presence of resources is not known, and CO and SI teachers might be applying differ-ent criteria when considering the availability of legitimate sources of support. Resolving this more definitively would entail documenting the actual programs available at each school, which, while outside the scope of this investigation, could prove to be a valuable area of future research.

Second, CO teachers agreed more strongly than the fu-ture SI teachers that to be successful in engineering, a stu-dent needs to demonstrate high scholastic achievement in science, math, and technology, and technology (construct D), t(180) = 2.612, p = .010. Here we see that teachers of math and science courses, which often serve a college prepa-ratory function rather than emphasizing technical skills, see excellence in academic performance in a gatekeeper role for engineering. This finding replicates previous results show-ing differences among STEM high school teachers ( Nathan et al., 2009). It also raises the issue about the differing purposes of K- 12 engineering programs and the intended student clientele. Those who expect that high scholastic achievement in science, math, and technology, and technol-ogy is prerequisite to participation in engineering studies

Table 2Summary of Means and Construct Reliability Parameters for EEBEI- T Survey Administration Before a Subsample of Teachers Taught PLTW Courses for the First Time

Survey #1 June 2008 Applicable to All Surveys (N = 182)

No. Scale Construct Title and Interpretation Items [Mid] Mean α

A. Influences on Instruction: 3.11 0.72 Students’ Academic Abilities. 5 1–5 My lessons are influenced by [3] students’ academic performance.

B. Influences on Instruction: 3.01 0.78 Students’ Backgrounds and 7 1–5 Interests. I integrate students’ [3] interests and cultural backgrounds into classroom activities.

C. Beliefs and Knowledge about 5.70 0.79 Student Out- of- School Activities. 5 1–7 Students’ science / math / technical [4] learning takes place in the home and community.

D. Careers in Engineering: 4.86 0.79 Academic Achievement. 6 1–7 To be an engineer a student must [4] have high academic achievement in science, math, and technology, and technology courses.

E. Careers in Engineering: Social 4.35 0.80 Network/Background. The student 8 1–7 whose parent is an engineer, who is [4] male, and either white or Asian, is most likely to pursue engineering.

F. Teaching for Engineering: 3.23 0.91 Academic Courses. The science and 3 1–5 math content taught in my courses is [3] explicitly connected to engineering.

G. Environmental and Structural 2.81 0.79 Support. My school provides 8 1–5 resources for students interested in [3] engineering (e.g., internships, career day, professional development opportunities).


may consider high school engineering as a kind of “preengineering” program that should be reserved for a selective group of students who demonstrate a history of excelling in technical courses and are more likely to pursue a STEM field of study. Those who do not espouse a selective view may see engineering studies as contributing more broadly to the technological literacy of all well- educated students (Katehi et al., 2009). While both the CO and PLTW groups show average views that affirm the essential importance of high academic achievement, the CO group, on average, ex-hibits this view far more strongly, suggesting a potentially important ideological division. This division reflects ideo-logical differences between the science and engineering education communities more broadly (Nathan et al., 2010).

Third, even before teaching PLTW courses, SI teachers were more likely than CO teachers to claim that science and math content taught in their classes was integrated with en-gineering content (construct F), t(180) = –5.936, p < .001. We note that this integration can, in principle, be made in both directions: College preparatory courses may elect to use engineering context to motivate the science and math and to demonstrate its applicability in “real world” problem- solving tasks; and engineering courses may highlight the roles that science- and math- specific topics play in engi-neering design and analysis. This difference between CO and SI teachers suggests that teachers drawn to using the PLTW curriculum are more likely to enact or to see points of integration between their science, math, and technolog-yand engineering content. Since the responses are based on

teachers’ self reports, it is also possible that these different groups of teachers may have different criteria for what it means for math and science concepts to be integrated into engineering education activities. As the National Academy Panel (Katehi et al., 2009) noted, STEM integration, while lauded in national education policy, is elusive.

Changes in Teachers’ Beliefs and Expectations About Engineering Preparation

By January, the main divergence between the groups was that SI teachers attended the two- week PLTW summer train-ing institute and then went on to teach PLTW in their high schools for one term. A second administration of the EEBEI- T was given in January 2009. Out of the original sample, 77 teachers responded to the invitation to take the second survey, including 36 SI teachers, and 41 CO teachers who served as our control subjects. It should be noted that those in the SI group were high school science, math and techni-cal education teachers who, like the control group, had not previously taught in the PLTW program before this study.

Administering the second survey the following winter al-lowed us to investigate changes in teacher views once the new PLTW teachers applied the concepts and skills learned during the summer institute to their classrooms. Since signif-icant psychological traits do not easily change (e.g., over the two week period of the summer institute), this was regarded by the research team as a more authentic way to measure the impact of new PLTW instruction on teachers’ views.

Because of the reduced response rate for the second survey administration, comparisons between groups and from June 2008 to January 2009 are now presented exclusively for only those teachers who provided complete data at both points in time. Comparisons (summarized in Table 4) show change data for CO teachers (N = 41) and SI teachers (N = 36).

As reported on the baseline survey, CO teachers were more likely than SI teachers to believe that high academic achievement in science, math, and technology courses was necessary to become an engineer (Construct D), and this group difference showed no change over time. We also learned that teachers in both groups initially reported that they did not strongly address students’ interests and cultural backgrounds when designing classroom instruction (Con-struct B). At retest, regardless of PLTW training, teachers reported attending to student background and interest less than they reported at time 1, F(1, 75) = 4.04, p = .048. This may well be a general effect in response to the increasing accountability climate of high stakes standardized testing that is driving greater focus on “teaching to the test” (Neill, 2003).

SI teachers started out more positive about the institu-tional support they experienced for engineering at their schools (Construct G) than control teachers, and this dif-ference grew significantly over time. Statistically, we found a significant main effect of group FGroup(1,75) = 20.96, p < .001 (SI higher than CO), a significant main effect for time,

Table 3Differences in Ratings of Teachers Prior to Summer Institute (N=182)

Independent Mean Samples (Standard Deviation) t- Tests

CO SI Construct (N = 100) (N = 82) t p

A. Influences on Instruction: 3.04 3.19 –1.788 .075 Students’ Academic Abilities (.516) (.618)

B. Influences on Instruction: 2.97 3.06 –1.038 .300 Students’ Backgrounds and (.597) (.541) Interests

C. Beliefs and Knowledge 5.64 5.78 –1.275 .204 about Student Out- of- School (.796) (.720) Activities

D. Careers in Engineering: 5.02 4.66 2.612 .010* Academic Achievement (.960) (.890)

E. Careers in Engineering: 4.3 4.42 –.976 .330 Social Network/Background (.782) (.903)

F. Teaching for Engineering: 2.87 3.67 –5.936 .000* Integration of Academic (.909) (.912) Concepts and EngineeringG. Environmental and Structural 2.60 3.07 –4.029 .000* Support (.760) (.807)

* p < .05.


FTime(1,75) = 4.154, p = .045 (more over time), and a signifi-cant group by time interaction, FGroupXTime(1,75) = 6.24, p = .015. Analysis of the specific responses showed that while CO teachers remained essentially constant in their views, SI teachers showed a marked increase after the training and teaching experiences.

Finally, SI teachers initially believed more strongly than CO teachers that the math and science concepts taught in their courses were explicitly connected to engineering (Construct F), F(1, 75) = 34.45, p < .001. The retesting of beliefs showed that for this sample of teachers this gap grew, due both to stronger agreement among SI teachers and stronger disagreement among CO teachers over time, F(1, 75) = 15.78, p < .001.

Measuring Teachers’ Contextualized Judgments Using Student Vignettes

Vignettes of fictional high school students were included in the EEBEI- T to contextualize teachers’ judgments about students and use comparisons between the fictitious pro-files to identify factors that may influence teachers with-out explicitly drawing teachers’ attention to them. The four vignettes (Table 5) used were specifically designed to ad-dress two factors: comparisons between V2 and V4 vary social influences such as students’ SES, while the V1- V3 comparison examines the influence of academic factors such as students’ prior course grades and cumulative GPA. Each pair controls for gender and the other factor of interest,

but does so in an implicit manner. An example vignette is provided in Appendix A.

Teachers were directed to read each vignette and then provide the following responses: (a) recommend whether a student should enroll in a precollege engineering course the following year, (b) specify the criteria the teacher used to make that recommendation (e.g., prior academic perfor-mance, overall GPA, gender, age, SES, family background), and (c) offer a prediction of success for the student’s future as a working engineer. Findings from the Likert scale data (Construct D) lead us to predict that teachers would tend to favor students with high academic performance and there-fore favor enrollment for V1, V2, and V4. Teachers also re-ported that SES had little sway with their decision- making processes and so we should therefore expect that V2 (fe-male with high SES) would not receive any greater support than other high GPA students (the other female, V4, or the male, V1) from lower SES families.

Teachers’ responses to the vignettes were analyzed using an ANOVA with Vignette (4 levels, a within- subjects factor and repeated measure for each of the 4 student profiles), Group (2 levels, a between- subjects factor for SI vs. CO), and Time (2 levels, a within- subjects factor), along with the interactions of these factors. Our dependent variables were the proportion of teachers who: endorse enrollment of a student vignette, report the use of any of several factors in making their endorsement judgment, and predict success in an engineering career track for each vignette.

A number of planned pairwise contrasts were also con-ducted to determine differences between the vignettes,

Table 4Means (and Standard Deviations) of Construct Scores for Those Control (CO, N = 41) and Summer Institute (SI, N = 36) Teachers Who Partici-pated in Both Spring 2008 and January 2009 Survey Administrations

CO SI (N = 41) (N = 36)

Survey Survey Survey SurveyConstruct 1 2 1 2

A. Influences on Instruction: 3.03 3.09 3.11 3.21 Students’ Academic Abilities (.493) (.473) (.531) (.445)

B. Influences on Instruction: 3.06 2.91 2.95 2.86 Students’ Backgrounds and (.594) (.547) (.464) (.507) Interests

C. Beliefs and Knowledge about 5.64 5.61 5.82 5.64 Student Out- of- School (.616) (.686) (.743) (.862) Activities

D. Careers in Engineering: 5.10 5.02 4.75 4.53 Academic Achievement (.816) (.783) (.962) (.892)

E. Careers in Engineering: 4.25 4.39 4.58 4.55 Social Network/Background (.688) (.707) (.796) (.726)

F. Teaching for Engineering: 3.02 2.76 3.61 4.08 Academic Courses (.830) (.778) (.885) (.798)

G. Environmental and 2.55 2.52 3.15 3.42 Structural Support (.752) (.773) (.806) (.716)

Table 5Summary of the Content of the 4 Student Vignettes

Vignette 1 (V1) Vignette 3 (V3)

Gender: Male Gender: Male Grade: 10th Grade: 10thCompares Background: low SES Background: low SESAcademic GPA: 3.85 GPA: 1.35Performance Interests Interests(controlling for Wants to enroll in Assembling body kits SES and Gender) Principles of Engineering; on foreign cars; wants attend college. to attend college.

Vignette 2 (V2) Vignette 4 (V4)

Gender: Female Gender: Female Grade: 11th Grade: 11thCompares SES Background: high SES Background: low SES(controlling for GPA: 3.45 GPA: 3.45Academic Interests InterestsPerformance Wants to enroll in Digital Wants to enroll in and Gender) Electronics; thinks father’s Digital Electronics; work as an engineer is uninterested in her “cool.” parents’ blue- collar jobs.

Vignettes 1 and 3 (row 1) compare academic performance, controlling for student social background, while vignettes 2 and 4 (row 2) compare socio-economic status (SES) while controlling for academic performance.


provided a significant omnibus main effect of Vignette was found. We used Shaffer’s (1979) sequentially rejective mul-tiple test procedure to control the Type I error rate of 0.05 across the pairwise contrasts for each factor. For example, for an effect with six possible pairwise comparisons, con-trasts were ordered by significance and their p- values com-pared to the allowed Type I error rates as follows: .05/3, .05/3, .05/3, .05/3, .05/2, and .05/1 (i.e., the denominator is equal to the number of type I errors which could have still been made). To make the strongest theoretical claims we focused our presentation of results for contrasts to those pairs of vignettes that supported the most direct comparisons (Table 5). Comparisons between V1 and V3 allowed us to compare the effect of academic factors while controlling for gender (both male) and SES (both low). The V2- V4 com-parison allowed us to compare the effect of SES on teachers’ judgments while controlling for gender (both female) and academic performance (both high). The V1- V3 and V2- V4 contrasts will be the focus of the results reported below.

Endorsing Student Enrollment

Teachers were asked of each vignette “Would you en-courage this student to enroll?” Our analyses of teachers’ responses for the omnibus question on endorsement of en-rollment into a preengineering course revealed a significant overall main effect for vignette (p < 0.001). This indicates that the level of endorsement depended on which vignette teachers responded to. As Figure 1 shows, endorsement to enroll was generally high, but substantially lower for V3 (male with low GPA) than the others, and somewhat lower for V4 (female with low SES).

Because of the way the profiles were designed (Table 5), pairwise contrasts between vignettes allowed us to infer the actual (rather than reported) influences of academic and social factors. Both the V1– V3 and V2– V4 contrasts were significant (p < 0.001). This shows that teachers in our sample were influenced by both academic and social factors in making their enrollment recommendations. Spe-cifically, teachers were more likely to encourage those with higher academic performance and those with higher SES to enroll. The influence of academic performance is consistent with the Likert scale findings above (Construct D) showing that STEM teachers tend to agree that to be an engineer a student must have high academic achievement in science, math, and technology and technology courses. The influ-ence of SES is somewhat surprising, however. It may reflect a wide array of views. Our interest, explored in the follow-ing section, is how teachers report on the factors they used to make these decisions.

In addition to the main effect of Vignette, the Enroll-ment measure entered into a significant interaction (p = 0.042) with Time and Group (Figure 2). The interaction highlights opposing shifts in beliefs between CO and SI teachers over time: Our intervention group, SI teachers who only first taught PLTW, decreased their support for student enrollment in engineering classes over time (regardless of which student profile they were considering), while control teachers increased their level of encouragement during the same time period. While the pattern is intriguing, and many plausible reasons spring to mind (e.g., PLTW teachers de-velop a more realistic understanding of the expectations of the PLTW courses, while CO teachers are warming up to the idea from repeated exposure), these data provide little

!

Figure 1. Proportion of STEM teachers endorsing the fictional students in vignettes 1 through 4 to enroll in high school engineering classes.


to uncover the actual basis for the effect. We recommend further research in this area.

Factors that Teachers Report as Influencing Their Endorsements of Student Enrollment

Teachers were given a set of seven factors and asked to indicate which, if any, they used in making their enrollment decisions. Of these omnibus tests, significant main effects of Vignette (p < 0.005) were found for Academic history, GPA, Family, Student Interest and Gender. As is evident in Table 6, Gender received so few supporters that the omni-bus test, while significant, is not that meaningful; the result is due to a small but measurable level of support for the Gender factor for V2 (4% of teachers) and V4 (1%), cou-pled with no support for V1 and V3. While other vignette contrasts showed an effect for Gender, it was because they were comparisons with one of the vignettes that received no support (V1 or V3).

Family was rarely endorsed as a factor, so even oc-casional consideration led to both an omnibus effect and pairwise effects, particularly for V2 versus V4, where V2 expressed that she thought her father’s work as an engineer was “cool.” SES was not significant as a factor in the om-nibus test, but SES is unique in that none of the teachers reported using it explicitly as a factor. This response pattern is also notable since overall higher levels of endorsement for V2 (high SES) compared to V4 (low SES) implicates SES as an implicit factor in teachers’ decision making. The vignette data show that teachers tended to favor enrollment of the higher SES student, but based on their identification of influential factors we see that teachers apparently have no awareness that SES influences these decisions.

Differences between V1 and V3 that signal teachers’ sen-sitivity to academic considerations were found in all three of the remaining influences reported by teachers: student’s current academic performance (Academic), student’s past academic performance (GPA), and student interests (Inter-est). In each case, teachers were more likely to report these as factors influencing endorsement when the vignette’s pro-file indicated higher academic achievement. Said another way, fewer teachers reported weighing students’ academic record when that student had a lower academic record but they tend to use it to justify endorsement decisions for aca-demically strong students. Teachers were also more likely to predict success in engineering for the higher academi-cally performing student (V1), in keeping with the Likert scale data above.

SES differences (evident in significant contrasts between V2 and V4) were significant for GPA and Academic (p < 0.001), but not for students’ interests. Teachers reported these academic factors as contributing to their decisions on enrollment in greater numbers when reviewing the profile for the higher SES student (V2) than low SES students. Additionally, teachers were more likely to predict that the higher SES student would have a successful career in en-gineering, even though there were others with comparable academic track records. In the final section we discuss these differences in the broader context of who should have ac-cess to K- 12 engineering education.

Discussion

In this final section we re- examine our findings in light of the challenges and opportunities that STEM integration in the classroom poses, current efforts to attract a more

Figure 2. The Time by Group interaction showing decreased support for enrollment by SI teachers over time, but increased support by control teachers.

!


diverse set of students and practitioners into engineering, and ways in which differences in teacher beliefs and expec-tations are indicative of broader tensions about the purpose and place of K- 12 engineering education. First, however, we address some of the methodological issues of conduct-ing research on teachers’ initial and changing beliefs, and summarize our findings.

Reflections on the Research Methodology

Investigation of the impact of education programs on a select group of participants is an inherently challenging en-terprise. In some cases, the assignment of participants to treatment and control conditions is entirely under the direc-tion of the team of researchers and school leaders. How-ever, it is also often the case that assignment is not under the researcher’s control, but determined by the participants themselves, perhaps in consultation with parents, teach-ers, and others. Consider the makeup of students who opt to enroll in engineering classes in their high schools. To deny (or even delay) access to suit research faces serious ethical barriers, since it withholds from students and parents their preferences, and could impose serious damage to their scholastic progress and even later academic and workplace opportunities.

In a somewhat similar manner, teachers decide for them-selves whether to participate in or avoid engineering instruc-tion. Manipulating this selection for research purposes also incurs serious professional and ethic issues. This study is a quasi- experimental design in that participants were not ran-domly assigned to either condition; teachers in each group self- selected whether they would become PLTW teachers. With limited ability in public schools to assign teachers to

their classes and the associated professional development experiences, there is a need to document inherent differ-ences that may exist among teachers even prior to the inter-ventions that knowingly distinguish them, and to interpret the impact of training and teaching experiences within the context of preexisting differences. Quasi- experimental de-sign research methodology may not be considered to be the “gold standard” by every deliberating body (Cook, Shadish, & Wong, 2008; Shavelson & Towne, 2002; US Department of Education, 2003), but it is a highly effective method for addressing many of the practical constraints that arise within authentic educational settings (cf. Tran, Nathan, & Nathan, 2010).

Summary of Findings

As a group, responses on the Likert scale items showed that the teachers in our sample agreed strongly that STEM education takes place in a variety of settings, including outside of formal schooling. They tended to believe that academic achievement was a precondition for engineering success, that social network and family history shape who will pursue engineering, and that their schools sometimes or infrequently provide institutional support for engineering.

Consistent with the Likert scale findings, teachers’ re-sponses to the situated vignettes showed the importance of academic achievement on teachers’ decision making about who should enroll in future engineering classes and their pre-dictions of who would be most likely to succeed in an engi-neering career. The vignettes also provided a more nuanced view of the influence of student academic record. While, on average, enrollment was advocated nearly 90% of the time, a breakdown of the criteria teachers used showed that

Table 6Means, Standard Errors (SE), and Effect Sizes (ES) for Teachers’ Responses to Questions Involving the Vignettes

V1 V3 V1 vs. V3 V2 V4 V2 vs. V4

Question Mean (SE) Mean (SE) p ES Mean (SE) Mean (SE) p ES1

Would you encourage this student to enroll? 0.99 (.01) 0.63 (0.04) 0.00* .45 0.99 (0.01) 0.89 (0.02) 0.00* .16

Which criteria were used in your enrollment decision?Academic 0.80 (.04) 0.43 (0.05) 0.00* .45 0.77 (0.04) 0.50 (0.05) 0.00* .28GPA 0.63 (.04) 0.30 (0.04) 0.00* .49 0.55 (0.04) 0.36 (0.04) 0.00* .28Gender 0.00 (.00) 0.00 (0.00) — 2 — 2 0.06 (0.02) 0.05 (0.02) 0.66 .00Family 0.02 (.01) 0.08 (0.02) 0.01* .08 0.33 (0.04) 0.03 (0.01) 0.00* .42Interest 0.92 (.02) 0.60 (0.05) 0.00* .39 0.91 (0.03) 0.84 (0.03) 0.04 .05Age3 0.07 (.02) 0.06 (0.02) N/A N/A 0.06 (0.02) 0.04 (0.02) N/A N/ASES3 0.00 (.00) 0.00 (0.00) N/A N/A 0.00 (0.00) 0.00 (0.00) N/A N/A

Would you predict future success in an engineering career? .63 (.05) 0.08 (0.02) 0.00* .64 0.66 (0.04) 0.18 (0.03) 0.00* .61

* All contrasts marked with an asterisk are significant at the appropriate level using the Shaffer method as described in the text to control Type I error rate for each question/factor to .05. 1. The effect size represented in this table is for partial eta squared. 2. Because no teacher endorsed Gender as a factor for V1 or V3, the V1- V3 pairwise comparison cannot be conducted. 3. Age and SES did not have significant main effects of vignette and therefore results from pairwise comparisons were discarded.


teachers tended to justify their endorsement on academic grounds more often when the student profile showed high performance. When low performance was evident, teachers turned to other criteria to justify their endorsements.

The vignette data also reveal a kind of disconnect be-tween the actual influences on teachers’ judgments and the influences of which teachers are aware. When asked explic-itly, teachers did not cite student SES as an influence on their decisions. However, teachers favored students from high SES families for course enrollment and predicted higher rates of success for those from more privileged cir-cumstances. Little from the data reveals the basis of these influences. This pattern, however, is notable given the cur-rent drive to attract more underrepresented groups to engi-neering studies and careers. This point is explored further in the next section.

One additional pattern of note in the baseline data is that, over time, the levels of endorsement offered by SI and CO teachers diverged. Control teachers increased their level of support of course taking from the first to the second survey, while support from SI teachers decreased. Though the data are limited in explaining this pattern, SI teachers formed a realistic understanding of the demands of the PLTW pro-gram that might have shifted their criteria.

We also found preexisting differences between SI and CO teachers. Specifically, budding PLTW SI teachers were more likely to identify sources of support for engineering in their schools and report that science and math concepts were being integrated with engineering activities during their in-struction. CO teachers agreed more strongly of the prereq-uisite role of high scholastic achievement in science, math, and technology, and technology for engineering studies.

Finally, we were able to identify changes in teachers’ views above and beyond preexisting group differences and changes that naturally occurred over time. Teachers who did the PLTW training and taught it for the first time increased their reporting that STEM curriculum materials were being effectively integrated in their classes. This echoes find-ings from other professional development programs (e.g., Cunningham, 2009). Because of the specialized role that teachers play in determining instruction, their attitudes and perceptions about STEM integration in the classroom is an area of central importance, which is explored more below.

Limitations of the Current Study

The current study has several limitations. First, teachers were not randomly assigned to either of the two condi-tions, but self- selected on the basis of whether they were interested in becoming a PLTW teacher. Consequently, this study employed quasi- experimental design methodol-ogy, whereby it is only possible to account for observable differences between treatment and control groups; any un-observable differences due to sampling biases between con-ditions cannot be addressed by this analysis. A future study that could randomly assign teachers to condition may face

serious practical challenges, but would provide for greater generalizability of the experimental results.

Second, internal validity may be compromised by selec-tion bias for participants who completed the postinterven-tion survey (42% response rate). However, a chi- square test for homogeneity of proportions showed no significant dif-ferences in proportions in each demographic category be-tween the original groups and matched groups for either CO or SI. In fact, the demographic compositions of the summer and winter samples were remarkably similar.

Third, external validity may be limited by the characteris-tics of our teacher sample. A large proportion of the teachers was White, was male, and had more than 10 years of teach-ing experience. Therefore, teachers in our sample may be different from teachers in the general population. A fourth limitation is that the results were derived from self- reported data from teachers. While this was common across all par-ticipants, it is possible that differences between teachers in the SI and CO groups actually reflect differences in their un-derstandings or interpretations of what the survey items con-vey rather than true differences in their beliefs and practices. Other data collection methods such as interviews, classroom observations, ethnographic study of classroom practices, while intrusive, would provide another perspective on the nature of teacher instruction. Finally, while we might antici-pate that this study contributes to an understanding of preengineering education in secondary schools more broadly, we caution the reader that the results have only emerged from study of a specific preengineering curriculum.

Current Initiatives to Broaden the Engineering Pipeline

Engineering educational programs and engineering pro-fessions both face well- entrenched historical patterns that tend to exclude females and a number of non- Caucasian and non- Asian ethnic groups, particularly African Ameri-cans and Hispanics (Wulf, 1998). Consequently the demo-graphics of course enrollments, graduating classes, and the engineering workforce do not match the demographics of the country as a whole. This has several consequences. It withholds from many talented youth the economic opportu-nities that follow from technical degrees and careers. It also shows a lack of “cultural competence,” where engineering presents itself as insensitive to cultural aspects of society and less relevant to members of other cultures (Chubin, May & Babco, May, & Babco,2005). The lack of a diverse workforce also prevents engineering firms from being re-sponsive to the shifting technological needs of a rapidly changing population that is becoming more subject to the demands of a globalized marketplace (Katehi, Pearson, and Feder, 2009). In this vein, we observed implicit views about student SES among STEM teachers in our sample that could perpetuate stereotypes of who should have access to highly rewarding technical education programs and who is likely to succeed in an engineering career. The homogeneity of the engineering workforce reinforces a cycle of exclusion that


is invisible to teachers yet effective in blocking systemic change.

Challenges and Opportunities of Providing STEM Integration

Along with a growing urgency for promoting student understanding of the individual facets of science, technol-ogy, engineering, and mathematics has come a drive to re-conceptualize instruction in terms of STEM integration that would break down traditional curriculum “silos” (Katehi et al., 2009). This comes in part from federal initiatives, such as “Race to the Top” (Chang, 2009), policy documents (Committee on Standards for K- 12 Engineering Education, 2010; NRC, 2007), and learning sciences research aimed at fostering greater transfer of knowledge (e.g., Pellegrino, Chudowsky, & Glaser, 2001). To this end, the 2006 Re-authorization of the Perkins Career and Technical Education Act (Public Law 105- 332, 1998) mandates that technical education and academic math and science topics must be integrated “so that students achieve both academic and oc-cupational competencies” with substantial funds allocated “to provide vocational education programs that integrate academic [math and science] and vocational education.” PLTW and other commercial curricula take up this mandate toward STEM integration. As they state in their marketing materials: “The combination of traditional math and science courses with innovative Pathway to Engineering courses prepares students for college majors in engineering and E/T fields and offers them the opportunity to earn college credit while still in high school” (PLTW, 2009).

It is notable, then, that our main finding is that the PLTW intervention seems to instill in new PLTW teachers a sense that they are better able to meet this mandate and provide instruction that more consistently integrates con-cepts from math and science into the engineering activities in their classrooms. By encouraging an integrative outlook on engineering instruction, PLTW engineering students are expected to make the conceptual connections needed to ground their academic knowledge to real world applica-tions, while at the same time developing a greater under-standing of how the specific ideas and procedures that they encounter in an engineering context will generalize to new problems and application areas.

Schunn (2009) has argued that STEM education implies an integrative curriculum that reveals a synergy that goes beyond the constituent parts (also see Moore, Roehrig, Lesh, & Guzey, in review). He singles out math—“the lan-guage of physical sciences and engineering sciences”—as “critical” to achieving this synergy. However, recent inves-tigations of engineering curricula, classroom instruction, and student achievement point to the challenges of realiz-ing effective STEM integration in K- 12 education (Katehi et al., 2009; Nathan, Oliver, Prevost, Tran, & Phelps, 2009; Nathan, Tran, Phelps, & Prevost, 2008; Prevost et al., 2009; Tran & Nathan, 2010a, 2010b; Welty et al., 2008). Schunn

identified several formidable obstacles for fully conceptu-alizing the integration of math with engineering and other STEM fields: To attract students who are otherwise weak or lack confidence in their math abilities, its presence in techni-cal fields is systematically diminished; teachers in nonmath STEM fields often lack math knowledge and math- specific teaching experience to carry out integration effectively; and the ever- present limits on time in the curriculum.

Schunn (2009) notes that several workable methods can enhance the level of integration; among these, he names using the new topics and contexts from engineering or tech-nology to reinforce mathematical understanding and ap-plication. Moore (2008) adapted Model Eliciting Activities (MEA) from math education to engineering education. She has shown that MEAs serve to elicit student thinking as well as provide a pedagogical structure for the design and imple-mentation of complex, collaborative activities in engineering classrooms that effectively integrate each of the STEM dis-ciplines. Stone and colleagues (2008) achieved student gains on standardized math tests through STEM integration using a professional development program that emphasized ways for teachers to regularly and explicitly integrate mathematics concepts with career and technical education (CTE) lessons.

The desire to advance students’ thinking in multiple STEM areas through integration, coupled with the practical challenges of implementing far- reaching changes in teacher preparation and curriculum design highlights, signal the challenges and opportunities that lay ahead for education reform. It also is a reminder of the important role of teach-ers as change agents for enacting systemic reform initia-tives, and the value of understanding teachers’ expectations, attitudes, and beliefs about engineering education.

Conflicting Purposes of K- 12 Engineering Education

Within K- 12 engineering education there is a persistent conflict between allocating limited educational resources to programs that provide preengineering and focus on iden-tifying and educating promising scientists for technical careers, and those that promote a broader agenda of techno-logical literacy “for all” even when most citizens will never pursue technical fields of study or careers. This division plays out in national policy discussions and in local schools districts throughout the United States (Katehi et al., 2009). The teachers in this sample provide a microcosm of the na-tion in this regard. On one hand, even before the interven-tion, budding PLTW teachers identified greater institutional resources that supported engineering education in their schools than the other STEM teachers (Construct G). On the other, those STEM teachers who did not go on to teach PLTW believed more strongly than PLTW teachers that academic achievement in science and mathematics must be a “gatekeeper” for access to engineering studies (Construct D). Differences in beliefs of this kind have implications for the perceived purpose and place of engineering education (Custer & Daugherty, 2009). The role that an individual’s


science and math knowledge plays in the collaborative and globalized nature of engineering professional practice is not clear cut (Anderson, Courter, Nathans- Kelly, Nicometo, & McGlamery, 2009; Gainsburg, 2006). The survey data sug-gest, however, that whatever the nuanced role is, nascent PLTW teachers come in with more inclusive values about engineering education than the broader STEM teacher pop-ulation even before the intervention, and, by signing on to become an engineering teacher, clearly took concrete ac-tions to extend the reach of engineering into the lives of a broader array of high school students.

Appendix A: Example Vignette

Vignette 4

Janet is enrolled in the 11th grade at your school with an overall GPA of 3.48 on a 4.0 scale. She is qualified to receive federal free/reduced lunch. She is liked by many of her peers and teachers. Janet plans to attend college after she graduates from high school. Janet is uncertain about her career plans and would like to learn more about different career choices. Her mother is a part- time waitress at a local restaurant and her father is a construction worker with 20 years of experience. The jobs her parents hold do not seem interesting to Janet. She expressed her interest in enrolling in a preengineering course called Digital Electronics for the preengineering curriculum purchased by your school through the career technical education program in your dis-trict. Janet is currently enrolled in a preengineering course called Introduction to Engineering Design.

Below is a list of courses she is currently enrolled in this semester along with the midterm grade for each course.

Period 1: English 11—Grade: BPeriod 2: Introduction to Engineering Design—Grade: APeriod 3: Pre- Calculus—Grade: APeriod 4: Economics—Grade: APeriod 5: French 3—Grade: BPeriod 6: Physics—Grade: B

References

Anderson, K. J. B., Courter, S. S., Nathans- Kelly, T., Nicometo, C., & McGlamery, T. (2009). Understanding the current work and values of professional engineers: Implications for engineering education. In Proceedings of the 2009 American Society of Engineering Education Annual Conference. Austin, TX.

Black, T. R. (1999). Doing quantitative research in the social sciences: An integrated approach to research design, measurement, and statistics. London: Sage.

Borko, H., Livingston, C., & Shavelson, R. J. (1990). Teachers’ thinking about instruction. Remedial and Special Education, 11(6), 40–49.

Bottoms, G., & Uhn, J. (2007). Project Lead the Way works: A new type of career and technical program. Southern Educational Review Board.

Brophy, J. E., & Good, T. L. (1974). Teachers’ communication of differen-tial expectations for children’s classroom performance: Some behav-ioral data. Journal of Educational Psychology, 61, 365–374.

Chang, K. (2009, December 12). White House begins campaign to pro-mote science and math education The New York Times. Retrieved from http://www.nytimes.com/2009/11/24/science/24educ.html

Chubin, D., May, G. S., & Babco, E. L. (2005). Diversifying the Engineer-ing Workforce. Journal of Engineering Education, 94(1), 73–86.

Clark, A., (2009, Summer). A different point of view. The Florida Engi-neer, pp. 27- 30. Retrieved from http://www.thefloridaengineer.eng.ufl .edu/issues/0906/adifferentpointofview.php

Cook, T. D., Shadish, W. R., & Wong, V. C. (2008). Three conditions under which experiments and observational studies produce comparable causal estimates: New findings from within- study comparisons. Jour-nal of Policy Analysis and Management, 27, 724–750.

Committee on Standards for K- 12 Engineering Education of the National Research Council. (2010). Standards for K- 12 engineering education? Washington, DC: National Academies Press.

Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.

Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educa-tional Measurement (2nd Ed.). Washington, DC: American Council on Education.

Cunningham, C. M. (2009). Engineering curriculum as a catalyst for change. Paper presented at the NSF/Hofstra CTL Middle School Grades Math Infusion in STEM symposium, Palm Beach, FL.

Custer, R. L. & Daugherty, J. L. (2009). Professional development for teachers of engineering: Research and related Activities, 29(3). Wash-ington, DC: National Academy of Engineering. Retrieved from http://www.nae.edu/Publications/TheBridge/16145/16204.aspx

Darling- Hammond, L., Chung Wei, R., Andree, A., Richardson, N., & Or-phanos, S. (2009). Professional learning in the learning profession: A status report on teacher development in the United States and abroad. Dallas, TX: National Staff Development Council.

Daugherty. J. L. (2009). Engineering professional development design for secondary school teachers: A multiple case study. Journal of Technol-ogy Education, 21(1), 10–24.

Fink, L. D., Ambrose, S., & Wheeler, D. (2005). Becoming a professional engineering educator: A new role for a new era. Journal of Engineering Education, 94(1), 185–194.

Gainsburg, J. (2006). The mathematical modeling of structural engineers. Mathematical Thinking and Learning, 8(1), 3–36.

Grossman, P. (1990). The making of a teacher: Teacher knowledge and teacher education. New York: Teachers College Press.

Katehi, L., Pearson, G., & Feder, M. (Eds.) (2009). Engineering in K- 12 education: Understanding the status and improving the prospects. Washington, DC: The National Academies Press.

Leinhardt, G., & Greeno, J. G. (1986). The cognitive skill of teaching. Journal of Educational Psychology, 78(2), 75–95.

Moore, T. J. (2008). Model- Eliciting Activities: A case- based approach for getting students interested in material science and engineering. Jour-nal of Materials Education, 30(5–6), 295–310. Published on the Na-tional Science Foundation’s National Science Digital Library (NSDL) in the Materials Digital Library (MatDL). Retrieved from http://matdl .org/jme.

Nathan, M. J., & Koedinger, K. R. (2000). An investigation of teachers’ beliefs of students’ algebra development. Cognition and Instruction, 18(2), 207–235.

Nathan, M. J., Oliver, K., Prevost, A., Tran, N., & Phelps, L. A. (2009). Classroom learning and instruction in high school precollege engi-neering settings: A video- based analysis. In Proceedings of the Amer-ican Society of Engineering Education (ASEE) 2009 (Paper no. AC 2009–1577, pp. 1- 23). Austin, TX: ASEE Publications.

Nathan, M. J., Tran, N., Phelps, L. A., & Prevost, A. (2008). The structure of high school academic and preengineering curricula: Mathemat-ics. In Proceedings of the American Society of Engineering Education (ASEE) 2008 (Paper no. AC 2008- 2566: pp. 1–19). Pittsburgh, PA: ASEE Publications.

Nathan, M. J., Tran, N., Atwood, A. K., Prevost, A., & Phelps, L. A. (2009). High school teachers’ beliefs about engineering preparation. Proceedings


of the American Society of Engineering Education (ASEE) 2009 (Paper no. AC 2009- 1715, pp. 1–20). Austin, TX: ASEE Publications.

Nathan, M. J., Tran, N. A., Atwood, A. K., Prevost, A., & Phelps, L. A. (2010). Beliefs and expectations about engineering preparation ex-hibited by high school science, mathematics, and technical education teachers. Journal of Engineering Education, 99(4), 409–436.

National Council of Teachers of Mathematics. (2000). Principles and stan-dards for school mathematics. Reston, VA: Author.

Neill, M. (2003, November). Leaving children behind: How No Child Left Behind will fail our children. Phi Delta Kappan, 85(3), 225–228.

NRC (2007). Rising Above The Gathering Storm: Energizing and Employ-ing America for a Brighter Economic Future Committee on Prospering in the Global Economy of the 21st Century: An agenda for American science and technology, Washington, DC: National Academy of Sci-ences, National Academy of Engineering, Institute of Medicine, Na-tional Academies Press.

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237–257.

Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what stu-dents know: The science and design of educational assessment. Wash-ington, DC: National Academy Press.

Phelps, L. A., Camburn, E., & Durham, J. (2009, June). Engineering the math performance gap. Research Brief. Madison: Center on Education and Work, University of Wisconsin- Madison.

Prevost, A., Nathan, M. J., Stein, B., Tran, N., & Phelps, L. A. (2009). Integration of mathematics in precollege engineering: The search for explicit connections. Proceedings of the American Society of Engineer-ing Education (ASEE) 2009 (Paper no. AC 2009- 1790, pp. 27). Austin, TX: ASEE Publications.

Project Lead the Way (2004). <au: title?> Retrieved from http://www.pltw .org/Engineering/Curriculum/sample- schedule.cfm

Project Lead the Way (2009). <au: title?> Retrieved from http://www.pltw .org/ Engineering/Curriculum/Curriculum- high- school.cfm

Public Law 105- 332 (1998). Retrieved from http://www.cccs.edu/Docs/SBCCOE/PerkinsAct1998.pdfRosenthal R. & Jacobson, L. (1968). Pygmalion in the classroom: Teacher expectations and pupils’ intel-lectual development. New York: Holt, Rinehart & Winston.

Rowan, B. (2004). Teachers matter: Evidence from value- added assess-ments. AERA Research Points: Essential Information for Educational Policy, 2(2), 1- 4. Retrieved from http://www.aera.net/

Schunn, C. (2009, March). What needs to be put back into S, T, and E? Well chosen M. Presentation to the Middle School Math Infusion STEM Symposium, Singer Island, FL. NSF/Hofstra CTL Middle School Grades Math Infusion in STEM Symposium: Author.

Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and Quasi- Experimental Designs for Generalized Causal Inference. New York: Houghton Mifflin Company.

Shaffer, J. P. (1986). Modified sequentially rejective multiple test pro-cedures. Journal of the American Statistical Association, 81(395), 826- 831Shavelson, R. J. & Towne, L. (2002). Scientific research in education. Committee on Scientific Principles for Education Research. Washington, DC: National Research Council.

Shulman, L. (2005). If not now, when? The timeliness of scholarship of the education of engineers. Journal of Engineering Education, 94(1), 11–12.

Steering Committee of the National Engineering Education Research Col-loquies. 2006. The research agenda for the new discipline of engineer-ing education. Journal of Engineering Education, 95(4), 259–261.

Stone, J. R., Alfeld, C., & Pearson, D. (2008). Rigor and relevance: En-hancing high school students’ math skills through career and technical education. American Educational Research Journal, 45(3), 767–795.

Tran, N. A., & Nathan, M. J. (2010a). An investigation of the relationship between precollege engineering studies and student achievement in science and mathematics. Journal of Engineering Education, 99(2), 143–157.

Tran, N. A., & Nathan, M. J. (2010b). Effects of precollege engineer-ing studies on mathematics and science achievements for high school students. International Journal of Engineering Education, 26(5), 1049–1060.

Tran, N. A., Nathan, A. B., & Nathan, M. J. (2010). Hand Matching vs. Propensity Score Matching: An Empirical Comparison of Results from a Quasi- Experiment. (WCER Working Paper Series no, 2010- 7.) Madison: Wisconsin Center for Educational Research. Retrieved from http://www.wcer.wisc.edu/Publications/workingPapers/Working _Paper_No_2010_07.pdf

U. S. Department of Education Institute of Education Sciences. (2003). Identifying and implementing educational pactices supported by rig-orous evidence: A user friendly guide.Washington, DC: Author. Wal-cerz, D. (2007). Report on the third year of implementation of the TrueOutcomes Assessment System for Project Lead The Way. Retrived from http://www.pltw.org/pdfs/AnnualReport- 2007- Public- Release.pdf

Welty, K., Katehi, L., & Pearson, G. (2008). Analysis of K-12 engineering education curricula in the United States: A preliminary report. In Pro-ceedings of the American Society for Engineering Education Annual Conference and Exposition. Pittsburgh, PA.

Wulf, W. A. (1998). The urgency of engineering education reform. In E. W. Ernst and I. C. Peden (Eds.) Realizing the New Paradigm for Engi-neering Education: Proceedings of the 1998 Engineering Foundation Conference (pp. 28–30), New York, NY. (June 3–6, 1998).

Yasar, S., Baker, D., Robinson- Kurpius, S., Krause, S., & Roberts, C. (2006). Development of a survey to assess K- 12 teachers’ perceptions of engineers and familiarity with teaching design, engineering, and technology. Journal of Engineering Education, 95(3), 205–216.



Kelly Hutchinson, Department of Chemistry, Purdue University; George M. Bodner, Department of Chemistry and the School of Engineering Education, Purdue University; Lynn Bryan, Departments of Physics and Curriculum & Instruction, Purdue University, West Lafayette, IN 47907

Correspondence concerning this article should be addressed to George M. Bodner, Purdue University, West Lafayette, IN E- mail: GMBODNER@ PURDUE.EDU

This work was supported, in part, by the NSF- funded National Central for Learning and Teaching (NCLT) in Nanoscale Science and Engineering Education.

Middle- and High- School Students’ Interest in Nanoscale Science and Engineering Topics and Phenomena

Kelly Hutchinson, George M. Bodner, and Lynn Bryan

Purdue University

Abstract

Research has shown that an increase in students’ interest in science and engineering can have a positive effect on their achievement (Baird, 1986; Eccles & Wigfield, 2002; French, Immekus & Oakes, 2005; Schiefele, Krapp, & Winteler, 1992; Schwartz- Bloom & Haplin, 2003; Weinburgh, 1995). Whereas many NSF- funded programs in materials science and nanotechnology have included efforts to develop curriculum materials for use in secondary or tertiary classrooms, relatively little work has been done to determine the topics that increase students’ interest in science, engineering, and technology. As part of the work done by the National Center for Learning and Teaching in Nanoscale Science and Engineering (NCLT, 2008), we examined middle1- school and high- school students’ interest in topics and phenom-ena from the field of nanoscale science and engineering (NSE). Analysis of both quantitative and qualitative data suggested that students were most interested in topics and phenomena that related to their everyday lives, were novel, and involved manipulatives. Conversely, students were least interested in topics and phenomena they viewed as irrelevant to their lives, they believed they had learned previously, and in which they were not actively involved. These results were used to inform the development of curriculum materials for middle school and high school students aimed at enhancing the learning of NSE topics.

It is not surprising that motivational factors that influence an individual’s level of interest in a field or content domain have been shown to play a crucial role in learning and development (Alexander, Jetton, & Kulikowich, 1995). Of potentially greater significance is the fact that an individual’s level of interest has been found to be linked to deep- level learning as op-posed to surface- level learning (Eccles & Wigfield, 2002; Schiefele, Krapp, & Winteler, 1992). Krapp, Hidi, and Renninger (1992) noted that learners use more elaboration and make more connections between concepts as they process information to which they are exposed when interest is triggered. Previous work therefore suggests that students’ attention and learn-ing can be enhanced by situations that promote an increased level of interest, which will subsequently be referred to in this article as “interest.”

K. Hutchinson, G. Bodner, L. Bryan / Journal of Pre-College Engineering Education Research 31

A decline has been found in students’ interest in the sci-ence courses that provide the foundation upon which engi-neering curricula build as these students progress through school; this decline in interest has been correlated with an analogous decline in test performance and achievement (Greenfield, 1997; Haussler & Hoffmann, 2002; James & Smith, 1985; Simpson & Oliver, 1985). Conversely, Wein-burgh (1995) found that as students became more inter-ested in science, their achievement levels increased across all ability levels. It is therefore incumbent upon those who teach science to consider their students’ interest when de-signing lessons for the classroom. Thus, although teachers may be required to teach particular concepts based upon national, state, or local standards, they should contextual-ize lessons on these concepts so that they are tailored to the interests of their students.

In the last several years, the use of examples from NSE (NSE) has been proposed as one means of increasing stu-dents’ interest in science (Roco, 2003; Chang, 2006; Foley & Hersam, 2006). Although NSE typically have not been a part of traditional K- 12 science curricula, they may be of potential interest to students due to the many interesting and novel phenomena that occur at the nano level.

Defining Interest

Interest has been defined as “a person’s interaction with a specific class of tasks, objects, events, or ideas” (Krapp, et al., 1992, p. 8). It is “a psychological state that, in later phases of development, is also a predisposition to re- engage content that applies to in- school and out- of- school learn-ing and to young and old alike” (Hidi & Renninger, 2006, p. 111). Researchers have divided interest into two forms: individual or personal, and situational. Individual interest is person centered and lasts over an extended period of time, whereas situational interest is situation centered but has the potential to develop into individual interest (Eccles & Wigfield, 2002; Hidi & Renninger, 2006; Krapp, Hidi, & Renninger, 1992).

A four phase model of the development of interest that includes both individual and situational interest has been proposed by Hidi and Renninger (2006). This model pro-gresses from triggered situational interest to maintained sit-uational interest to emerging individual interest and finally to well developed individual interest as shown in Figure 1.

Characteristics that Influence Interest

Personal relevance has been found to have a positive ef-fect on students’ interests. When students find that a topic relates to their everyday life or to achieving a goal they have, they are more apt to be interested in the topic being discussed (Haussler & Hoffman, 2002; Sandoval, 1995; Schwartz- Bloom & Haplin, 2003). This increased interest has also been shown to relate to better recall and enhanced learning (Eccles & Wigfield, 2002; Hidi and Baird, 1986; Schiefele, 1999). Schwartz- Bloom and Haplin (2003) found that when high school students are taught science concepts using ma-terial that is interesting and relevant to their own lives, sig-nificant gains in achievement can be made. This is consistent with the suggestion that information to be learned should be entrenched within contexts and applications that are mean-ingful and relevant to the students. The problem with achiev-ing this goal is the difficulty of determining what topics are actually relevant to students at a particular grade level.

Prior knowledge or background knowledge may also have an effect on student interest in a topic (Bergin, 1999; Haussler & Hofmann, 2002). At times, student interest is increased by familiarity with a topic. Bergin (1999), how-ever, has suggested that prior experience can also decrease an individual’s interest in a topic.

The use of activities that involve physical manipula-tives has been shown to have a positive effect on interest and learning (Bergin, 1999; Haussler & Hoffmann, 2002; Stohr- Hunt, 1996) because students become more engaged in the topic. The more involved students become in the task or topic, the higher their interest (Eccles & Wigfield, 2002; Hidi & Baird, 1986; Schiefele, 1999).

Figure 1. The four-phase model of interest development (Hidi & Renninger, 2006).

Phase 1 Phase 2 Phase 3 Phase 4

Triggered Situational Maintained Situational Emerging Individual Welldeveloped Individual Interest Interest Interest Interest

Defined as Short- term changes in Focused attention and Beginning desire to Lasting predisposition to affective and cognitive persistence over time to an re- engage with particular re- engage in a particular topic components activity/ task topics over extended time periods

Supported by Externally supported through Externally supported through Self- supported through Self- generated through positive group work, puzzles, meaningful tasks positive feelings, stored feelings and increased stored computers knowledge, and curiosity knowledge

Initiated by Surprising information, Project- based learning, Value in a particular task that Long lasting interest for a personal relevance cooperative group work, reflects their interest particular topic one- on- one tutoring

32 K. Hutchinson, G. Bodner, L. Bryan / Journal of Pre-College Engineering Education Research

Novelty has also been found to influence interest (Ber-gin, 1999; Eccles & Wigfield, 2002; Haussler & Hoffmann, 2002; Hidi & Baird, 1986; Schiefele, 1999). Prior work suggests that people do not report an increase in interest if the stimulus is too familiar to them, or if the stimulus is too unfamiliar for them to understand (Bodner, 2001).

Methodology

Guiding Research Questions

This study was designed to investigate middle school and high school science students’ levels of interest in a variety of NSE phenomena and concepts. The following research questions guided the design of the study, data collection, and the data analysis.

• What nanoscale science/engineering/technology topics and phenomena do students find the most interesting and the least interesting?

• What are the characteristics of the nanoscale topics and phenomena in which the students’ are or are not interested?

Participants

Participants in this study were Midwestern students in middle schools and high schools from predominantly white, middle class, rural (n = 164) and suburban (n = 96) com-munities and from a culturally diverse urban community (n = 156). Table 1 provides a portrait of the participant demographics.

The high school students in the rural community were enrolled in Chemistry 1, Integrated Chemistry and Physics (ICP), or Advanced Placement Chemistry courses that were all taught by the same teacher. High school students from the suburban community were all enrolled in Chemistry 1 courses taught by the same instructor. The urban commu-nity high school students were enrolled in Biology or ICP courses taught by various instructors.

From this population, 40 students (12 rural, 11 suburban, and 17 urban) were selected for interviews based on their gender and academic ability levels in science as determined by the student’s science teacher. Approximately equal num-bers of male and female students were interviewed. The sample population was also divided into approximately

equal numbers of students who had been identified as low, medium, or high achieving by their instructors. The source of quotations in the remainder of the article will be iden-tified using RHS, SHS, or UHS to identify students from rural, suburban, or urban high schools, respectively, and RMS, SMS, or UMS to indicate rural, suburban, or urban middle schools, respectively. Students also will be identi-fied as either male or female, and as low, medium, or high achieving on the basis of their science teacher’s assessment.

Data Collection

The students were introduced to a variety of nanoscale topics and phenomena through four manipulative activities and a series of nanoscale driving questions. A mixed meth-ods approach (Johnson & Onwuegbuzie, 2004; Tashakkori & Teddlie, 1998, 2003) was used to collect data. Quanti-tative techniques enabled the first author to collect survey data on interest from a large number of students, while the qualitative techniques allowed for a more detailed, in- depth follow- up of the survey data

Quantitative data were collected using a three- point Likert- scale survey developed to evaluate students’ interest in a set of NSE topics and phenomena. The term “phenom-ena” was used in the survey in the sense of describing real- world objects, systems, or events in a variety of contexts to make the key ideas plausible (Smith, Wiser, Anderson, Krajcik, & Coppola, 2004). The survey asked students to rate their level of interest as either not interested, kind of interested, or very interested.

The survey covered four NSE manipulative activities and a set of 11 driving questions designed to measure stu-dents’ interest in learning about NSE or nanotechnology topics. The four manipulative activities that demonstrated NSE phenomena involved a waterproof material, a hopping magnet (Lorenz, Olson, Campbell, Lisenski, & Ellis, 1997), changes in the color of nanoscale gold particles (Mc Farland, Haynes, Mirkin, Van Duyne, & Godwin, 2004), and the ef-fect of a surfactant on the ease of stirring a mixture of zinc oxide and water. These activities were designed as a context in which to determine students’ interests in the phenom-ena, rather than to elicit student knowledge. The first author therefore provided support during the activities, but no ex-planation of the science behind the activities. A description of the four activities is given in Appendix A.

A “driving question” has been defined as a well designed question used in problem based science that is elaborated, explored, and answered by both students and their teacher (Krajcik, Blumenfeld, Marx, & Soloway, 2000). Each of the following driving questions used in this study is introduced by a single term that will be employed in subsequent sec-tions of this article when referring to these questions.

1. Atoms: How do we know atoms exist? 2. Penny: If a penny is made of tiny particles (atoms),

why doesn’t it fall apart?

Table 1Survey participants

Middle High Male Female Total

Rural 74 90 63 101 164Suburban 55 41 40 56 96Urban 19 137 76 80 156

Total 148 268 179 237 416


3. Pencil: What do a pencil, a diamond ring, a car tire, and charcoal have in common?

4. Gecko: How can a gecko walk upside- down on the ceiling?

5. Gold: When will gold no longer be the color gold? 6. Aspirin: How did aspirin stop my headache today and

my fever last week? 7. Machines: What kinds of machines are small enough

to fit inside a living cell? 8. Window: What can be done to keep a window clean,

making sure water and dirt do not stick? 9. Robot: How can we make DNA act like a robot?

10. Common: What do styrofoam, fog, milk, Jell- O, latex paint, and steel have in common?

11. CD: Why does a CD have so many colors on the back? Do these colors have anything to do with the music stored on the CD?

The students were not expected to answer the questions on the survey; they were only asked to indicate the level of their interest in learning and understanding the answers to these questions.

Interviews

To further elucidate the results obtained from the re-sponses to the quantitative survey, one- on- one interviews were conducted with a subset of participants. These inter-views were designed to elicit student discussions about why they found particular topics and phenomena more interest-ing than others. The interviews explored the students’ in-terest level for each item and asked students whether they could explain how the activities worked or whether they knew the answer to the driving questions. Students were also asked how they would change the activities and ques-tions to make them more interesting as well as how to in-crease interest in their current science class. The interviews lasted between 20 and 40 minutes and were audiotaped and transcribed.

Data Analysis

The surveys were coded for level of interest in each phe-nomenon and each driving question. The mean score for each phenomenon and driving question was calculated after assigning a code of 1 to the response of “not interested,” 2 to “kind- of interested,” and 3 to “very interested.” The surveys were also analyzed by determining the percentage of students selecting each driving question as their most or least favorite question.

The interviews were analyzed qualitatively in order to evaluate why students expressed a given level of interest. Through the iterative process of the constant comparative method, several emerging themes that governed student in-terest were identified based on common trends found in the transcripts (Patton, 2002).

Results

Results of the analysis of the most interesting and least interesting of the 4 activities and the 11 driving questions are discussed in this section. The themes that emerged using the constant comparative method (Patton, 2002) are also ex-amined. An analysis is also presented of suggestions stu-dents made during the interviews for changing the questions and activities to make them more interesting and sugges-tions they offered for making their current science course more interesting.

Students’ Selections of the Most Interesting Topics and Phenomena

Overall, the students were most interested in the CD, Gecko, and Machines questions, as shown by the data in Figure 2. This was true for the total sample population for all district types (rural, suburban, and urban) and for the total sample population for both middle- school and high- school students. When the data were analyzed by gender, the same questions were judged as most interesting by the males. Females, however, were more interested in the Aspi-rin question than the Machines question. Of the four manip-ulative activities, the students were more interested in the Waterproof and Easy- Stir activities than the Hopping Mag-net and Changing Color activities across all district types, across all grade levels, and across both genders.

Analysis of the interview data indicated that students found certain activities or questions most interesting be-cause of three factors: (1) they were relevant to their every-day lives, (2) they were viewed as being novel, or (3) they involved physical manipulatives with which the students were actively involved.

Figure 2. Percentage of all students selecting each of the 11 driving ques-tions as the most interesting question.

Most Interesting Question


Interest Due to Relevance to Everyday Lives

The students were interested in questions in which they saw relevance to their everyday lives and to societal issues, or in questions that triggered their curiosity. The following excerpts provide examples of comments that explicitly con-nected the driving question or activity to the student’s ev-eryday life or to society in general:

The ones that I was interested in most . . . are the ones that would actually affect my life and affect the lives of others. Things that, like, I can apply to everyday life are worth talking about. While the other things are things that I already know or . . . they don’t matter so much. (RHS, low male)

[Very interested in the Aspirin] because it’s some-thing that I actually can relate to and how like we have those problems and then like it goes away and we don’t know why. (SHS, high female)

The students also responded favorably to questions or activities that aligned with their personal interests, even though the topic might not be interesting to other students, as seen in the following quotes from the interview data.

[Most interested in Gecko] probably cause I like any-thing to do with animals. And I don’t know, geckos are just cool like how they can stick to anything. Like he can climb up a flat surface and he has toes, but evidently, his toes are small enough that he can find something. Like it kind of reminded me of like a mouse, like how do they do that, you know? (RHS, high female)

[Most interested in CD] because I like technology and you know I like to know how it works. I like to know how a computer works and how you know CD and all that works, just how it gets on the back of the CD and how it’s encoded, how it’s read, and how it works. (SHS, high male)

[Very interested in Window because] I thought that was interesting cause I’d like to have that for my car, you know. You have dirt sticking to it and then the water runs on and you can’t see though it, whereas with that tech-nology, you’d never have to wash it and that’d be nice. So that’s why I liked that one. (RHS, mid- high male)

Some students responded favorably to a particular ques-tion because it sparked a sense of curiosity. This was seen most often with the CD and Gecko questions, and was often the result of a connection between the question and previ-ous personal experiences such as owning compact discs.

[Most interested in the CD] because I’ve actually won-dered why the colors were on CD and I don’t know how what’s stored on there and how they can do that or what contributes to it at all. I’ve always actually wondered [about] that myself. (RHS, low female)

[Most interested in the CD because] I listen to a lot of CDs and so I’ve always wondered why there were a whole bunch of different colors on the back on those, so . . . (RMS, high male)

In general, when the activity or driving question related to a student’s personal interest, general curiosity, or connec-tion to their life, students felt that they would be more apt to want to learn the answer to the driving question or become more involved in the activity to discover an explanation for what was occurring.

Interest Due to Novelty

Students expressed an interest in questions that were perceived as novel primarily because they had not learned about the phenomena prior to this study. This theme pre-sented itself in contexts in which something was not only new to the student, but also demonstrated something that the student had not expected.

[Most interested in CD because] I don’t know very much about that. I’ve never thought about that before. (RMS, mid male)

[Most interested in Gecko because] I didn’t think an animal could walk upside down on the ceiling. (SMS, high female)

[Very interested in Waterproof] . . . ’cause like I’ve never really heard of like waterproof thing—like you can really waterproof something. And, I wanted to find out like how, like how it happens. (SMS, low male)

The combination of novelty and surprise was often found in comments from males who were interested in learning more about the Machines question:

[Very interested in Machines because] yeah, I mean, to me that was amazing how we can get like little cameras that are small enough, and getting a picture, and what they do with it, you know. It’s interesting; it’s really amazing. It’s really interesting how they can get it to work when it’s that small and deal with it, with the en-gine being so small, I find that really interesting. (RHS, mid- high male)

[Very interested in Machines] ‘cause something that small, like cells are the smallest living organism and to fit something, for humans to make something that small with precision and put it inside of a living organism is just kind of like mind- boggling, cause it’s so small and I would just like to know like what those machines would do. (SHS, high male)

[Most interested in Machines because] it makes me curious about. . . . It makes me curious about what kind of . . . what kinds of machines are small enough to fit in-side a living cell. Like, it makes me think, I guess. (RMS, low male)


One student commented that she was not interested in the Machines question because the topic was too novel for her to understand. When discussing the Machines question, she noted that she did not understand how it would work because someone would need to operate the machine and no one is small enough to do that.

[Not interested in Machines because] I didn’t think it was fascinating, because if you get a machine in there, then you’re gonna have to get as small as a living cell to get in there to work it, to work the machine. (SHS, low female)

In general, students stated that they were interested in certain activities or driving questions because they did not know that “something could do that.” However, the degree to which the topic or phenomena is novel must be consid-ered, as indicated by the student who found the Machines question too novel for her to be interested in this topic. Our results suggest that students must have some understand-ing of the topic or phenomena being presented in order to stimulate their curiosity and desire to find out an answer. Educators therefore must find a balance between being too novel and not novel enough to affect students’ interest.

Interest Due to Manipulative Nature of Activity

The students expressed particular interest in activities that involved physical manipulatives in which they were actively involved:

[Very interested in the Easy- Stir because] I thought it was cool because you actually, you actually did something to the other side, so it’s more than them just putting water in it, and keep on doing that, so . . . (RMS, high male)

Although the Changing Color activity was of low inter-est to many students, one student expressed his interest in this activity because he was involved in creating an immedi-ate outcome:

[Very interested in Changing Color because] it was im-mediate and that you could see and you changed some-thing. And that was interesting a lot more than talking about the theoretical things like the atoms of uh, and what they’re composed of, and how much they weigh, etc. And just it seems to me like one of those things for using chemical reactions to produce different colors and produce different reactions. (RHS, low male)

Students also commented on being interested in activi-ties, in general, because they enjoyed taking an active role in the classroom:

[Interested in the activities because they were] very in-volved. I mean, you got to actually got to do something and then see what was going on. Even if you didn’t really

know what was going on behind, I mean what was tak-ing part, what was behind it. It was kind of like wow, because I didn’t know that they had anything like that and it let me find out. (RHS, high female)

It should be noted that many of the students who re-sponded that they were interested in the activities that in-volved physical manipulatives were from a district that, according to the students, did not perform very many activi-ties or experiments in science classes.

In general, students indicated that they were more inter-ested in activities in which they were able to physically ma-nipulate an object and in driving questions for which they believed an activity could exist that would allow them to manipulate an object. They felt that this manipulation al-lowed them to figure out the answers, which would enhance their interest.

Students’ Selections of the Least Interesting Questions/Activities

Overall, tudents were least interested in the Atoms and Window questions and the Changing Color and Hopping Magnet activities (see Figure 3). This was true for all groups of students, regardless of district, grade, or gender. The stu-dents noted that they were least interested in these topics and phenomena either because they did not find them rel-evant to everyday life, or they already knew the answer, or were they not actively engaged in manipulating materials during the activity.

Uninteresting Because Not Relevant to “My”Life

The students were not interested in questions and activi-ties that did not seem relevant to their individual lives or did not trigger their personal interests. The Windows question,

Figure 3. Percentage of all students selecting each of the 11 driving ques-tions as the least interesting question.

Least Interesting Question


for example, was not particularly interesting to the students because they did not believe that this new technology would benefit them in their lives:

[Least interested in the Window because] umm . . . I don’t know, I just felt that was kind of, I mean it would be neat if we had windows where you don’t have to dust them and clean them for fingerprints, but at the same time, I could just get water and wash it off myself, so I wouldn’t really need to know that, or feel the need to explore that, I guess. (SHS, low- mid male)

[Not interested in Window because] like what can be done to a window to make sure water and dirt don’t stick, it’d kind of be something to see, but it’s not really useful to have something like that. (RHS, high female)

[Least interested in Machines because] I mean it’s in-teresting, but not as interesting as what I personally like and stuff like that. (RMS, low female)

Overall, students were not interested in a topic or phe-nomenon if they did not find it relevant to their daily lives or helpful in the future. The less a topic or phenomenon was viewed as being related to everyday life, the less the students were interested in discovering an explanation of an activity or an answer to a driving question.

Uninteresting Because I Already Knew the Answer

The students were not interested in questions or activi-ties in which they believed they already knew an answer, or had been taught the answer to the question previously. This category also contained statements by students believ-ing the question or activity was an “old type” of science, rather than something that was new or novel. This became particularly evident in the reasons that students provided for why they were not interested in the topic of Atoms.

[Not interested in Atoms because] I think it’s just be-cause since I’ve sat in Chemistry class and we’ve talked about atoms and atoms and atoms, just after talking about them for so long, and then doing labs and discov-eries with them; not too fond of them. (SHS, low female)

[Not interested in Atoms because] umm, last year we did a whole thing on atoms and I just thought, you know, I already know that atoms exist. I know mostly a lot of stuff about those, so, I just didn’t find anything exciting about it. (RMS, high male)

[Least interested in Atoms], yeah, ‘cause you kinda just know they’re there. So it’s not, it’s not like umm, oh, what does everything mean. You just kind of learn so that, and it’s not really that exciting because you know they’re there and what they do. (RMS, mid female)

[Not interested in the Waterproof because] well, I pretty much already know how that works, kinda. (RHS, mid male)

In general, the students’ level of interest declined when they believed they had heard the information before and were not interested in learning about it again or exploring it in more detail. The more the students felt they knew about a topic, the less interested they were in exploring the topic in more detail. A fine line seemed to exist between knowing just enough information for the students to want to learn more, and students believing that they already know the in-formation they needed, in which case they lost interest in the topic.

Uninteresting Because I Am Not Very Involved

The students were not interested in activities in which they perceived they would not be directly involved, either because they did not see much happening or because they did not have much to do during the activity.

[Not interested in Changing Color because] I just didn’t find it that interesting ‘cause, I don’t know, it didn’t re-ally do much, like it just kind of, like the color change. (RMS, high female)

[Not interested in Changing Color because] it went from a red to a like a purple or a little darker and, I don’t know, it just didn’t seem like much happened. (RHS, mid- high male)

[Not interested in Changing Color because] well re-ally all I saw was a color change and there’s a lot of dif-ferent experiments that, you know, have a different color change, so I wasn’t really sure what was going on, but I just saw a color change. (SHS, high male)

[Not interested in Easy- Stir] because all we had to do was stir it. It wasn’t like . . . exciting. (RMS, low male)

Students who were not interested in the Changing Color activity expressed the opinion that the color change was not exciting or interesting because it was not drastic. One stu-dent stated that although he was not exactly sure what was going on, it was just another color change and he had seen numerous experiments with color changes.

Students suggested that they would be more interested in the activities that were completed during class if they had been more involved in manipulating the materials being used. They also indicated that they wanted to observe dras-tic changes rather than subtle changes during the activities.

More Interesting If . . .

During the interviews, the students often commented on ways to make them more interested in the questions and activities described in this study, or in their science class, in general. Based on the results discussed so far, it is not surprising that they called for topics that were more relevant to their everyday life and involved more experimentation or hands- on activities.


They were specifically interested in everyday things that were relevant to people in their age group.

[More interested in the questions if you] work them so like I could interact with them in everyday life that I do as a normal, however you define normal, human being. Like, put that into my everyday life then I might be a little bit more interested in it . . . the more (questions) relate to our everyday life, the more we’re gonna be willing to pay attention and learn about them cause we can interact with it more than just going to class, sitting in class, and doing homework, like we can put it to our lives. (SHS, mid- low male)

[More interested if you] basically just relate it more to like our age and like things that we know. (SHS, high female)

[More interested in science if] umm, I’d like to see, it annoys me about all the stuff we talk about, like the electrons and stuff, it all seems so flimsy, so theoretical, that like and umm, and I think I’d like to see a lot more of the practical application of chemistry. Like what you can use to advance your life. (RHS, low male)

The students also noted that they were more interested in learning about topics when they were able to see and interact with the phenomena, through experimentation or hands- on activities, rather than just to talk about the topic and to do mathematical problems.

I don’t know, I like hands- on stuff, so maybe if we did a little more like got deeper into the subjects and you know tested out what the different components or what-ever, that might be fun. (SHS, low- mid male)

lots of labs that actually apply to what we’re really doing and that aren’t very time consuming. . . . It’s kind of learning from all the class work and actually get to see. There’s a lot that are involved in labs too. Like you have to read and follow directions, and it gets you think-ing more, than like homework, or whatever when you just tune out. Like the hope is, I want to find out what happens and I’ve got to do it in order to find out what happens. (RHS, high female)

The students expressed the belief that there would be more thinking and learning if the amount of experimentation was increased because they would be active learners who would be more involved in figuring out what is actually happening rather than passive learners who were being told the answer.

Discussion

Three major themes emerged in this study; these themes characterize students’ interest in the introduction of NSE topics and phenomena into the middle school and high school curricula.

Assertion #1: Students Are More Interested in Nanoscale Science and Engineering Topics and Phenomena that Are Relevant to Their Everyday Lives.

The students in this study were more interested in NSE- related activities and questions if and when they were able to see a connection to their personal interests or to their every day lives. They believed that relating topics, in gen-eral, to their everyday lives would significantly increase their interest in their current science courses. This result is consistent with prior work that suggests a relationship between the extent to which students can relate to a topic and their interest in and willingness to learn material being presented (Brooks & Brooks, 1993; Haussler & Hoffmann, 2002; Sandoval, 1995; Schwartz- Bloom & Haplin, 2003). Our results suggest that contextualization, which places material to be learned within the context of the students’ everyday life, personal interests, and/or general curiosity, can lead to an increase in the students’ interest in learning.

Assertion # 2: Students Are Interested in Nanoscale Science and Engineering Topics When the Topics Are Novel, Rather than Topics About Which They Perceive that They Have Prior Knowledge.

The students in this study tended to be interested in NSE- related questions and activities that seemed to be novel to them, which is consistent with research on characteristics that have an effect on students’ interest (Bergin, 1999; Eccles & Wigfield, 2002; Haussler & Hoffmann, 2002; Hidi & Baird, 1986; Schiefele, 1999). It should be noted that the activity worked best when the topic was something that was neither too familiar to the students nor too foreign.

This study suggests that sense of novelty is related to prior knowledge. Students with a high degree of prior knowledge of a topic were not interested in further investigation of this topic. These results are consistent with the work of Bergin (1999), who argued that the amount of interest a person will exhibit decreases as the amount of prior experience with the topic increases. Novel topics and phenomena from the field of NSE have the advantage that they are not likely to be material with which students are overly familiar.

Assertion #3: Students Are Interested in Nanoscale Science and Engineering Topics When They Actively Experience the Topics and Phenomena Using Physical Manipulatives.

The students in this study noted that they would be more interested in questions, activities, and science classes in general, if there were more hands- on activities and ex-periments. The students did not just want to manipulate materials; however, they said that they would be more inter-ested in a topic if they had to think about it and visualize the process first. The relationship between the students’ level of interest and the extent to which they were involved in a


question or activity is consistent with prior work that sug-gested a direct connection between the level of students’ in-terest in a topic and the level of their activity (Hidi & Baird, 1986; Schiefele et al. 1992; Eccles & Wigfield, 2002).

Conclusion

In general, students in this study were more interested in a question or activity if they viewed it as relevant to their everyday life or to their personal interests. Questions or activities that were novel also triggered interest for many students, who wanted to learn about something with which they were unfamiliar or that might, at first glance, seem un-likely or impossible. Novelty was strongly related to prior knowledge, inasmuch as the students were less interested in learning about topics with which they were familiar from prior classes. The students were also most interested in hands- on activities that required them to do more than just follow directions or manipulate equipment or chemicals. They wanted to have to think and figure out explanations, as opposed to following directions or being told the answers in a lecture. By taking these themes into account, curriculum designers and teachers may be able to create curriculum materials that increase student learning as a result of trig-gering student interest in the material.

The results of this study suggest that students’ interest in science might be increased by incorporating examples from NSE into the classroom. The advantage of NSE topics and phenomena for increasing student interest might be due to a combination of their prominence in today’s society in the form of consumer products, advertising, popular media and books; the perception that these topics are novel; and the fact that students are unlikely to view these topics as having been learned in previous coursework.

Appendix A

Description of Manipulative Activities

In the Waterproof Material activity, students compared a traditional pair of khaki pants with a pair of Nanotex® khaki pants. After liquid was poured on each pair, students were asked to respond by describing what they saw. They observed that the old pair of pants absorbed the liquid, while the Nanotex® pants repelled the liquid.

In the Hopping Magnet activity, one side of a flat refrig-erator magnet was cut off, and then dragged across a larger magnet in two directions. The students then cut a piece of the refrigerator magnet off the bottom and dragged it across the larger magnet in two directions. The students were asked to describe what they observed. They then dragged the piece from the bottom of the refrigerator across the larger magnet in a third direction to have the piece of refrig-erator magnet “hop.”

In the Color Changing activity, students were given a vial that contained 13 nm gold nanoparticles in an aqueous

solution. They were then asked how they could change the color of this red solution. About 2 mL of water was added to the vial by the students, which produced no change in the color of the solution. The students then added sodium chloride to the solution, which changed the color of the gold nanoparticles from red to blue.

In the Easy- Stir activity, students were given a small amount of zinc oxide powder in a paper cup and asked to figure out how to make it look like paint. The students added about 2–3 mL of water to the cup and stirred to form a “clumpy” suspension. Half of the students then added 6 more drops of water, while the other students added 3 drops of the surfactant Darvan C- N. Students who added more water saw no change, while those who added the surfac-tant were able to make a suspension that looked more like “paint.”

References

Alexander, P. A., Jetton, T. J., & Kulikowich, J. M. (1995). Interrelation-ship of knowledge, interest, and recall: Assessing a model of domain learning. Journal of Educational Psychology,87, 559–575.

Bergin, D. A. (1999). Influences on classroom interest. Educational Psy-chologist, 34(2), 87–98.

Bodner, G. M. (2001). Why lecture demonstrations are “exocharmic” for both students and their instructors, University Chemistry Education, 5, 1–5.

Brooks, J. G., & Brooks, M. G. (1993). In search of understanding: The case of constructivist classrooms. Alexandria, VA: Association for Super vision and Curriculum Development.

Chang, R. P. H. (2006). A call for nanoscience education. Nanotoday, 1(2), 6–7.

Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132.

Foley, E. T., & Hersam, M. C. (2006). Assessing the need for nanotech-nology education reform in the United States. Nanotechnology Law & Business, 3, 467–484.

French, B. F., Immekus, J. C., & Oakes, W. C. (2005). An examination of indicators of engineering students’ success and persistence. Journal of Engineering Education, 94, 419–425.

Greenfield, T. A. (1997). Gender- and grade- level differences in science interest and participation. Science Education, 81, 259–276.

Haussler, P., & Hoffmann, L. (2002). An intervention study to enhance girls’ interest, self- concept, and achievement in physics classes. Jour-nal of Research in Science Teaching, 39, 870–888.

Hidi, S., & Baird, W. (1986). Interestingness—A neglected variable in dis-course processing. Cognitive Science, 10, 179–194.

Hidi, S., & Renninger, K. A. (2006). The four- phase model of interest de-velopment. Educational Psychologist, 41(2), 111–127.

James, R. K., & Smith, S. (1985). Alienation of students from science in grades 4- 12. Science Education, 69, 39–45.

Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26.

Krajcik, J., Blumenfeld, P., Marx, R., & Soloway, E. (2000). Instructional, curricular, and technological supports for inquiry in science class-rooms. In J. Minstrell & E. H. van Zee (Eds.), Inquiring into inquiry: Science learning and teaching (pp. 283–315). Washington, DC: Amer-ican Association for the Advancement of Science Press.

Krapp, A., Hidi, S., & Renninger, K. A. (1992). Interest, learning, and development. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 3–25). Hillsdale, NJ: Erlbaum


Lorenz, J. K., Olson, J. A., Campbell, D. J., Lisensky, G. C., & Ellis, A. B. (1997). A refrigerator magnet analog of scanning- probe microscopy. Journal of Chemical Education, 79, 1032A–1032B.

McFarland, A. D., Haynes, C. L., Mirkin, C. A., Van Duyne, R. P., & God-win, H. A. (2004). Color my nanoworld. Journal of Chemical Educa-tion, 81, 544A- 544B.

NCLT (2008). National Center for Learning and Teaching in Nanoscale Science and Engineering. Retrieved from http://www.nclt.us/.

Patton, M. Q. (2002). Qualitative research & evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.

Roco, M. C. (2003). Converging science and technology at the nanoscale: opportunities for education and training,” Nature Biotechnology, 21, 1247–1249.

Sandoval, J. (1995). Teaching in subject matter areas: Science. Annual Re-view of Psychology, 46, 355–374.

Schiefele, U., Krapp, A., & Winteler, A. (1992). Interest as a predictor of aca-demic achievement: A meta- analysis of research, Hillsdale, NJ: Erlbaum.

Schwartz- Bloom, R. D., & Halpin, M. J. (2003). Integrating pharmacology topics in high school biology and chemistry classes improves perfor-mance. Journal of Research in Science Teaching, 40, 922–938.

Simpson, R. D., & Oliver, J. S. (1985). Attitude toward science and achievement motivation profiles of male and female science students in grades six through ten. Science Education, 69, 511–525.

Smith, C, Wiser, M., Anderson, C. W., Krajcik, J., & Coppola, B. (2004). Implications of research on children’s learning for assessment: Matter and atomic molecular theory. Paper commissioned by the Committee on Test Design for K- 12 Science Achievement Center for Education, National Research Council.

Stohr- Hunt, P. M. (1996). An analysis of frequency of hands- on experience and science achievement. Journal of Research in Science Teaching, 33, 101–109.

Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches (Vol. 46). Thousand Oaks: SAGE Publications.

Tashakkori, A., & Teddlie, C. (2003). Handbook of mixed methods in so-cial and behavioral research. Thousand Oaks, CA: Sage.

Weinburgh, M. (1995). Gender differences in student attitudes toward sci-ence: A meta- analysis of the literature from 1970 to 1991. Journal of Research in Science Teaching, 32, 387–398.



John Chandler is Co- Director of the TTU Center for Engineering Outreach and TTU T- STEM Center. Dr. Chandler currently directs development of cur-riculum, teacher training, and academic resources to help K- 14 institutions and educators engage students in project- based learning activities using the engineering design process as a framework for applying STEM concepts and skills. Chandler and Fontenot developed the TTU Precollege Engineering Academy program and Applied Math and Engineering Magnet Programs—that serve Title 1 schools with majority populations of children from low- income African American and Hispanic families. Dr. Chandler also directs the Texas Tech T- STEM Center Summer Training Institute for Teachers to provide professional development opportunities in engineering, science, technology, and mathematics for K- 12 teachers. A. Dean Fontenot is an Adjunct Professor teaching Professional Communication for Engineers, Sr. Director TTU T- STEM Center, and Sr. Director, Center for Engineering Outreach. Dr. Fontenot has twenty- five years’ experience teaching written and oral communication and twelve years’ experience working with K- 12 institutions to educate teachers and students about engineering. Dr. Fontenot works with the Texas High School Project to reform K- 12 education and concentrates on integrating engineering design into K- 12 classrooms. She sits on numerous councils and boards including the Texas Alliance for Minorities in Engineering (TAME) board of directors, the NASA RASC- AL steering committee, and the secondary- level, engineering- oriented profes-sional development symposium.

Problems Associated with a Lack of Cohesive Policy in K- 12 Pre- college Engineering

John Chandler, A. Dean Fontenot, and Derrick Tate

Texas Tech University

Abstract

This article identifies a number of issues associated with current STEM education reform efforts, especially with regard to efforts to in-tegrate engineering education into the K- 12 curriculum. Precollege engineering is especially problematic in STEM reform since there is no well established tradition of engineering in the K- 12 curriculum. This discussion aims at identifying some of the issues and problems which serve to impede implementation of engineering education in the K- 12 environment. Historically, engineering education has been the purview of higher education, and the epistemology of engineering education has not evolved to specifically inform the exigencies of K- 12 education. There also is little in the way of cohesive standards that establish appropriate precollege engineering knowledge and skills and to provide a framework for shared understandings, cooperative partnerships across institutional boundaries, curricular development and implementation, and teacher preparation and professional development. The lack of standards and an epistemic foundation and tradition in K- 12 engineer-ing results in significant gaps in experience and knowledge to inform implementation, which is proceeding in schools despite these glaring obstacles––driven by legislative mandate, STEM funding initiatives, workforce demand, and other compelling forces. The lack of systemic infrastructure and support mechanisms for preengineering––such as are found in the sciences, mathematics, and other academic disciplines already participating in K- 12 education––have resulted in a situation in which there is no clear, generally agreed upon standards and defini-tion of a body of engineering knowledge, skills, and activities that constitute appropriate curricular content for teaching and learning in K- 12 education.

Key Words: STEM reform, precollege engineering, National Academy of Engineering, precollege engineering curriculum.

J. Chandler, A. D. Fontenot, D. Tate / Journal of Pre-College Engineering Education Research 41

Overview

In the decade since release of the Glenn Commis-sion report, Before It’s Too Late: A Report to the Nation (2000), we have seen a remarkable proliferation of STEM education reform initiatives at the national, state, and local levels. Alarmed by declining student performance in math-ematics and science, coupled with a continuing trend of decreasing enrollments and poor retention in STEM de-gree programs––the nonpartisan Glenn committee voiced grave concern about whether our educational system could produce the diverse scientific and technical workforce necessary for the United States to remain competitive in a global economy that is increasingly driven by techno-logic development and innovation. Recognizing that com-petitiveness also requires a citizenry capable of mastering the scientific and technical concepts and skills to function in work and home environments requiring ever- increasing technological sophistication, the Commission also advo-cated teaching STEM subjects as interrelated concepts and skills to more closely reflect how they are applied in the workplace.

The Glenn report and more recent studies by the National Academies (2007, 2009) indicate wide consensus that bet-ter preparation of K- 12 teachers and a more rigorous K- 12 curriculum are necessary to improve student performance in STEM subjects at college and career levels. The Glenn Commission also recognized that raising the pay and pro-fessional status of teachers will be necessary to attract and retain high quality teachers capable of affecting the changes in STEM education that the committee advocated.

Precollege engineering is especially problematic in STEM education reform since there is no well- established tradition of engineering in the K- 12 curriculum, or as part of teacher preparation and certification processes. The re-sult––most K- 12 teachers and administrators are typically ill prepared to adequately advise students about engineer-ing careers, much less introduce engineering knowledge and skills into the classroom. While there is a growing appreciation that engineering may be a positive vehicle to motivate K- 12 student study of other STEM subjects (AeA, 2005; NAE, 2009; NSB, 2007), some emerging re-search indicates that there are circumstances in which this position may not be entirely valid (Tran & Nathan, 2010). However, significant gaps in experience with engineering in the K- 12 setting make these kinds of discussions dif-ficult at best.

Establishing the Estacado Precollege Engineering Academy

The release of the Glenn Commission report also coin-cided with the pilot year of the Estacado High School Pre-college Engineering Academy that we helped establish in partnership between Lubbock ISD and the Texas Tech Uni-versity (TTU) Center for Engineering Outreach. Estacado High School has an overwhelming majority population of low- income African- American and Hispanic students. His-torically, the percentage of Estacado High School graduates pursuing postsecondary education perennially has earned it an Underperforming High School classification in Texas. The Precollege Engineering Academy is still in operation, and we are extremely proud that more than80 percent of its students go to college upon graduation. However, the Acad-emy curriculum is very different today than we originally conceived it. For that matter, the TTU Center for Engineer-ing Outreach has also changed significantly. We are now the Texas Tech University T- STEM Center, a component of the Texas High School Project––a statewide STEM ini-tiative which comprises 7T- STEM Centers and 52 STEM Academies, as well as early- college high schools and other innovative education programs.

Some of the changes we have experienced are a result of STEM education reform initiatives, some result from legis-lative and regulatory agency mandate, but all of our current activities are tempered by experience that we have gained along the way––especially with regard to working in an en-vironment with very different institutional objectives and political constraints than are found in higher education. For example, 10 years ago there was little substantive preengi-neering curriculum available, and the courses in the state inventory with engineering in their title were perhaps best characterized as holdovers from a time when Career and Technology Education (CTE) was called Industrial Arts.

Initially, our main strategy to engage students in learning engineering concepts and skills was to shoehorn engineering design projects or other engineering- related content into ex-isting science courses and to sponsor afterschool programs, competitions, and similar learning enrichment experiences. Later we were able to apply for innovative course status for engineering courses that we developed with teachers and administrators at Estacado, when this status was allowed under the Career and Technology Education section of the Texas Essential Knowledge and Skills (TEKS) course stan-dards. Innovative courses are no longer an option in Texas,

Derrick Tate, Assistant Professor of Mechanical Engineering at Texas Tech University, aims to impact society through bringing design thinking to areas of strategic importance: developing sustainable approaches for building systems, transportation, and manufacturing; facilitating mass innovation; and enabling innovation in enterprises. His current projects include development of sustainable wall systems funded by West Texas entrepreneurs, and a US- Tanzania Workshop: Advancing the Structural Use of Earth- based Bricks, funded by NSF. He has been a member of international committees for several conference series: Integrated Design and Process Technology (IDPT), Concurrent Engineering (CE), Axiomatic Design (ICAD), and Design Education Conference (ConnectED).

42 J. Chandler, A. D. Fontenot, D. Tate / Journal of Pre-College Engineering Education Research

but there are now standards for several courses with signifi-cant engineering content, and the Texas Board of Education recently approved TEKS standards for a new capstone engi-neering course: Engineering Design and Problem Solving.

The Topography of STEM Education Reform

While STEM education reform efforts have proliferated and gained traction, resulting in some of the changes that we have observed in the topography of K- 12 education over the past decade, many problems and issues that hampered STEM implementation ten years ago continue to serve as barriers to an integrated STEM curriculum––especially to the integration of engineering content. However, driven by legislative mandate, STEM funding initiatives, workforce demand, and other compelling forces––implementation of precollege engineering and other STEM programs is pro-ceeding in schools despite some glaring concerns and gaps in experience with K- 12 engineering.

The following discussion identifies some issues that the authors believe are significant obstacles that continue to ham-per implementation of engineering in K- 12 education. We ex-pect that our experience and frequent frustration with these issues are not unique, and recognize that some may perceive and experience them very differently from us. Our depiction of the educational landscape is often painted with very broad strokes, because many of the underlying practical issues to STEM integration in the K- 12 classroom are much larger, albeit sometimes deceptively subtle barriers to all human en-terprise––epistemic differences, cultural proclivities, and ter-ritorialism, to name a few. One does not have to look closely at any university campus to conclude that momentum in the construction of knowledge has been toward splintering the scope of larger academic disciplines into smaller fields of specialization. We are not questioning the value or the reasons for this topography. We make this observation in recognition that STEM reform requires a paradigm shift toward integra-tion of disciplinary knowledge and skills against inertia and cultural boundaries existent in our educational system.

In the following section we will discuss issues stem-ming from different epistemic traditions involved in STEM reform, a lack of cohesive standards for preengineering knowledge and skills, and issues related to curriculum re-sources available to schools that prompted us to develop our FRAME engineering design model, which we will discuss in the last section. We do not claim to have solutions for many of the practical questions or problems of integrating engi-neering into the K- 12 STEM curriculum. We simply offer this discussion through the lens of our experience working to implement precollege engineering education in Texas with the hope that it may inform the efforts of others in the field.

Barriers to Implementing K- 12 Engineering

Engineering in K- 12 Education, a report released last year by the National Academy of Engineering (NAE, 2009)

and the National Research Council (NRC) makes a number of convincing arguments for engineering as “a catalyst for a more interconnected and effective K- 12 STEM education system” (p. 1). And in the spirit of true reform, the NAE rec-ognizes that this outcome “will require significant rethink-ing of what STEM education can and should be” (p. 1).

In their review of the NAE report, Rogers, Wendell, and Foster (2010) point out that the committee’s discussion of the potential for precollege engineering education substantially references engineering education research, but that some of the practical issues regarding implementation of precollege engineering education do not receive as much attention in the report. For example, the NAE recognizes that there is often a conceptual disconnect between how engineering is perceived and taught in the K- 12 classroom and the gener-ally accepted disciplinary perspectives and practices within the epistemic traditions of engineering education.

The report also recognizes that this fundamental problem is compounded by a lack of standards for knowledge and skills appropriate to preengineering education, as well as the lack of comprehensive, standards- driven teacher prep-aration mechanisms and curriculum standards. Rogerset al.rightfully argue that summaries and analyses of various curricular resources and reviews of engineering research that examine the impact of engineering curricula on students’ mathematics and science achievement which are undertaken in the report leave many practical questions unanswered.

Issues Related to Epistemology

We should probably question, or at least put into per-spective, the value of purely quantitative examinations in an environment that is poorly defined and understood, and carefully consider what these methods actually tell us about how individual classroom implementations impact the ef-fectiveness of these curricula. We agree with the review-ers that qualitative studies are also needed to capture richer descriptions and experiential narratives to depict more fully and to help understand exigencies of K- 12 engineering ed-ucation as experienced in practice. And there are obvious merits to bringing a wider range of disciplinary knowledge and skills into producing theoretical and practical models, which could better inform implementation efforts, as the reviewers suggest. The engineering education literature overwhelmingly draws upon the experience of university engineering colleges, so attempts to apply lessons from the literature to K- 12 engineering education leave substantial room for skepticism about their power to account for fun-damentally different mandates, institutional perspectives, and functional environments, which separate institutions of higher education and K- 12 education.

We use the term “epistemology” in the original Aristo-telian sense, as a way of reasoning and understanding the things we encounter in the world. Certainly we all bring all of our experience to the table all of the time, but Ar-istotle makes the distinction that training and practice are


the means by which intellect is shaped and developed (Ar-istotle, trans 1998[1925]). The trend toward more special-ized fields of study and practice has resulted in narrowing disciplinary understandings of theory and practice with perspectives that privilege theoretical stances, methods, research results, publications, and other ways of knowing that have emerged from their own specialized traditions and academic pursuits. And while we recognize and concede that there is significant overlap in the topics, literature, and other interests among various disciplinary areas––as well as individual experience, cultural proclivities, institutional structure, shared practice, and a host of other factors, which obscure and blur the boundaries that we are attempting to describe––the Aristotelian distinction that training and prac-tice shape understanding serves our purpose here to account for many of the problems we have experienced in working toward implementing engineering into the K- 12 setting.

The NAE report and others (Busch- Vishniac & Jarosz, 2004) express concern that the profession of engineering is still poorly understood by the public and that miscon-ceptions about what engineers do in practice may actually serve to discourage women and ethnic minority populations from pursuing engineering careers. Prior to STEM reform efforts, there was little incentive for engineering colleges to engage in K- 12 education, and the tendency was to turn inward toward research and teaching the engineering sci-ences. Engineering colleges have historically afforded limited opportunities to develop personal or professional relationships or first- hand experience dealing with very dif-ferent political constraints and other realities of the K- 12 environment––particularly in comparison with the various academic units at universities involved in teacher prepara-tion, which develop strong connections to K- 12 education through the student population they serve and in their teach-ing and research missions.

Funding trends and increased awareness of STEM re-form are having the desired effect of substantially increas-ing participation by engineering educators, practitioners, and professional organizations in efforts toward STEM integration. When we began to develop the Estacado Pre-college Engineering Academy, not only was the lack of understanding about the study and practice of engineering a source of frustration in our efforts, but also our lack of understanding of the structure, practice, and other con-straints of K- 12 education proved an equally significant bar-rier to developing the program. For example, the sequence of course work in mathematics, physics, and the sciences proved an insurmountable barrier to approximating atypical university model for engineering education of first requiring a foundation in these subjects and then teaching students applications for this content knowledge in the engineering sciences.

It is clear that there is a significant learning curve that will have to take place with what can be accomplished and what constitutes appropriate engineering knowledge and skills within the exigencies of the K- 12 environment.

However, it seems reasonable to assume that through de-veloping content standards, programs for certification of precollege engineering teachers, and other mechanisms that will be required to integrate engineering knowledge and skills into K- 12 STEM curriculum, the process will eventu-ally provide a framework for creating authentic connections between K- 12 education, engineering education, and other academic disciplines involved in curricular development and teacher preparation.

Our experience has been that the current literature and traditions in engineering education provide little in the way of a valid epistemic foundation for precollege engineer-ing. Additionally, as STEM reform has gained momentum and opened doors for engineering educator and practitio-ner involvement in K- 12 education, discourse is emerging that considers vertical alignment between prevailing higher education models for engineering education and K- 12 en-gineering, which has potential to significantly improve re-cruitment and retention in engineering degree programs.

Issues Engendered by a Lack of Standards for Precollege Engineering

We commend the NAE committee for undertaking the task of sifting through and summarizing many of the re-sources, activities, and perspectives that have emerged from an area that has experienced explosive growth in both par-ticipation and program development over the past decade. The range of educational resources and activities that claim engineering content or engineering- based learning experi-ences in both formal and informal educational settings can make it difficult at times to see the forest for the trees.

In summarizing and evaluating a large number of these resources and programs it can be tempting to draw conclu-sions that Rogers et al. (2010) describe as painting “a pic-ture of a K- 12 education space already populated with the raw ingredients for both innovative instruction and novel re-search” (p. 179). In the end, however, we have to agree with the reviewers’ conclusion that effective implementation in the K- 12 curriculum requires a systemic, welldefined frame-work for precollege engineering, which we have expanded upon to include specialized programs to educate and certify teachers of preengineering; policy support that includes edu-cation standards and evaluation criteria, shared theoretical and practical models; a robust body of research and litera-ture; and other mechanisms, such as professional organiza-tions to establish professional identity, represent specialized interests and needs, and encourage participation and owner-ship byall the stakeholders in precollege engineering.

We would not necessarily argue against the value of ef-forts to catalogue and evaluate the precollege engineering resources that are increasingly available to schools. How-ever, without benefit of codified standards and policies for the educational infrastructure and support mechanisms, cur-rently missing in K- 12 engineering education- assessment rubrics and metrics will remain as protean and lacking


congruent foundation as the content, programs, and other resources they aim to evaluate. Although schools are in-creasingly introducing engineering as part of the learning experiences they offer students, the process of choosing cur-ricular components and professional development for their teachers is significantly undermined by the current state of affairs in which precollege engineering is too often what-ever the person that writes the book or curriculum, develops the website, or provides the training or equipment says it is.

In 2001, Massachusetts schools were required by legisla-tive mandate to provide engineering education in the K- 12 curriculum for all grade levels. At the time, efforts to meet this requirement underscored the lack of available research based curriculum, professional development, and other components necessary to establish preengineering in K- 12 education. The Texas legislature followed suit eight years later. Interestingly, Jacob Foster works for the Massachu-setts Department of Elementary and Secondary Education, and Rogers et al. (2010) examine the current state of K- 12 engineering in Massachusetts—after ten years of develop-ment the authors describe the implementation of engineering education as being successful yet slow to develop (p. 181).

Both the Texas legislature approval of the new science and mathematics 4x4 high school graduation require-ments, and the Texas Education Agency (2009) revision of TEKS––the content standards for courses in the state inventory––resulted in the approval by the State Board of Education of a new engineering course that counts as a new science category for graduation credit, which students have the option of taking in their junior or senior year. It remains to be seen if this new course creates any significant demand for preengineering curriculum, professional development, and other resources that specifically target the Texas stan-dards. As part of the approval process the TEA contracted education consultants to compare the draft version of the engineering TEKS with the Massachusetts standard for precollege engineering education, and state and national college readiness standards. The State Board of Education established a panel of experts to conduct a similar evalua-tion of the proposed preengineering course standards. Both of these examinations found that the new Texas engineer-ing TEKS meet or exceed the requirements of the standards used for comparison.

A significant difference from Massachusetts’ approach is that Texas is implementing engineering only at the high school level. When we were developing the Estacado Pre-college Engineering Academy 10 years ago, we conducted a telephone survey of administrators in many of the larger districts across the state to determine (1) what engineering coursework, if any, was offered; (2) if any districts offered more substantive engineering programs with a sequence of coursework; or (3) if they offered any other significant en-gineering based learning experiences, such as afterschool programs. More than 30 school districts responded to the survey, and at the time, the number of districts providing en-gineering courses could be counted on one hand, and only

three of these five districts had somewhat more substantive programs. In the ten years that have ensued, precollege en-gineering programs, courses with engineering content, and extracurricular enrichment learning opportunities have be-come common in districts around the state at the elementary, middle, and high school levels. These can be characterized in the same language that Rogers, Wendell, and Foster use to describe precollege engineering in Massachusetts: “much of what has been implemented across the state is widely varied in goals, methods, and quality” (p. 181).

Texas also has established the Mathematics, Physi-cal Science, and Engineering teacher certification (SBEC, 2004); however, until recently there were no undergraduate programs offering engineering coursework to pre- service teachers for this certification, and no established gradu-ate programs for in- service teachers beyond professional development workshops offered for continuing education credits. One obstacle regularly encountered in identifying coursework appropriate for pre- service teachers is that en-gineering courses typically require significant mathematics and science prerequisites, which can pose an obstacle to non- engineering students. Recently the UTeach program at the University of Texas in Austin was awarded $12.5 mil-lion by the National Science Foundation (NSF) to develop engineering tracks for both in- service and pre- service K- 12 teacher certification in which students in UTeach engineer-ing cohorts will earn engineering degrees. Texas Tech is offering a new interdisciplinary degree and certification program in which pre- service and graduate students choose among a number of engineering courses that are not laden with prerequisite requirements. These courses generally emphasize the engineering design process, or other require-ments in the new engineering TEKS, such as ethical or so-cial responsibilities of engineers, or various aspects related to career paths in different engineering disciplines.

Issues Related to Precollege Engineering Curriculum

Project Lead the Way and Infinity Project are two cur-ricula that are being widely adopted in Texas schools, but there has been little substantive research that demonstrates how, or if, these curricula help students to develop the “ habits of mind” that the NAE identifies as an engineering skill set with potential to contribute to a technically profi-cient citizenry for the 21st century (p. 5), or if these cur-ricula are effective cross disciplinary vehicles for teaching standards based concepts in science, math, technology, and other academic subjects, as the NAE also suggests.

Project Lead the Way was developed through a consor-tium effort with the participation of a number of universities and the Infinity Project was developed as a collaborative effort between Texas Instruments and Southern Method-ist University. Both of these curricula require schools to make a significant upfront capital investment in proprietary lab equipment and technology. One drawback to this ap-proach is that this reliance upon proprietary technology and


laboratory equipment can be extremely intimidating for teachers who are trained to use the equipment and teach engineering during sessions lasting from one to two weeks during the summer. Our experience with the TTU Pre- college Engineering Academyand other schools has been that teacher turnover often renders the equipment useless until another teacher can be trained. Proprietary technology also has a limited life span requiring an ongoing replace-ment cycle that can be financially burdensome.

This discussion is not intended to critique the design or value of these curricula, because they have a history of large scale implementation and are well recognized for providing students with learning experiences involving engineering concepts and skills. It is important to note, however, that no matter how widely adopted these curricula are in Texas, they currently are not automatically accepted for transfer credit among high schools, and many universities do not consider them for admission. In Texas, because these cur-ricula are not directly tied to specific course standards in the TEKS, they have either been offered as Innovative Courses in Texas––which will no longer be allowed under the re-vised Public Education Information Management System (PIEMS)––or for local transcript credit.

Since they are offered as local credit courses, there is no guarantee they will be accepted for transfer or for enrollment by postsecondary institutions. Brophy, Klein, Portsmore, and Rogers (2008) and Rogers et al. (2010) indicate that ac-creditation problems and a lack of acceptance of precollege engineering coursework for admission to universities is a pervasive problem and suggest this lack of acceptance re-flects how engineering is viewed by universities themselves in relation to the sciences and mathematics. Whether this value assessment of universities is real or inferred, the dis-cussion suggests that accreditation is a significant obstacle to integrating engineering into K- 12 education.

Development of the TTU FRAME Model

The Texas Tech T- STEM Center is part of the Texas STEM Initiative, which is a key component of the Texas High School Project (THSP), a $180 million public- private initiative committed to increasing graduation and college enrollment rates in Texas. Partners include the Texas Edu-cation Agency (TEA), Bill & Melinda Gates Foundation, Michael & Susan Dell Foundation, Communities Founda-tion of Texas, and industry partners. Resources dedicated to the THSP support new and redesigned high schools, educa-tor training and development, and specific college prepara-tory programs. Some goals and outcomes addressed by this STEM initiative are as follows:

• Establish 50 Texas STEM Academies in areas of high need across the state, each year producing 3,500 Texas high school graduates from diverse backgrounds prepared to pursue careers in STEM related fields

• Create 7 Texas STEM Centers to support the trans-formation of teaching methods, teacher preparation, and instruction in STEM fields with researchdriven methods and resources

• Establish a statewide best practices network for STEM education to promote broad dissemination and adoption of promising practices to improve math and science performance of all Texas students.

Each of the T- STEM centers supports STEM educa-tion in Texas schools through professional development for teachers, developing researchdriven STEM curriculum, and other research and support activities. The centers also work closely with the T- STEM academies to help provide unique STEM learning experiences to students with project based instruction, teaching across the curriculum approaches, and other innovative methods for teaching and learning in STEM areas. Academies are required to select their students by a lottery system that ensures a demographic cross section of students in the district. All of the curriculum developed by the centers and implemented by the academies must in-corporate a hands on, project based approach to engage stu-dents in learning. Each of the centers is expected to develop an area of specialized research and development. The Texas Tech T- STEM Center specializes in precollege engineering and has committed significant effort and resources to ad-dressing many of the problems discussed here and in the literature regarding K- 12 engineering.

Description of FRAME

Instead of developing an engineering curriculum tied to certain equipment or specific science and mathematics content, our approach was to develop the TTU engineer-ing design FRAME model which provides teachers with tools to manage design projects and use project lifecycle conventions for documentation and various project phase activities to assess and evaluate student learning, as well as a framework to teach course content. One advantage of this approach is that it provides a framework for developing de-sign problems which specifically address required content for any course under the TEKS. Projects may also be devel-oped to engage students in designing solutions to real world problems, or as a service project within their communities. The model establishes overarching questions, activities, and outcomes and goals for each project phase to give students a structured approach to resolving poorly defined or open- ended problems.

One problem with project based learning is that because relatively few teachers are exposed to project lifecycle con-cepts as part of their education, they often manage hands on projects by allowing students to begin construction of an artifact without modeling or other proof of concept activi-ties that characterize the engineering design process. The FRAME model requires students to articulate and justify all of their design choices, as well as predict the performance


of any product or artifact that is developed during the course of the project––before anything is built.

A unique feature of the FRAME model is that it employs a heuristic guide to help teachers and students engage in a more complete consideration of constraints and issues that must be addressed during each phase of the project. The heuristic model also helps them understand the role of project documentation and presentations using conventions appropriate for each phase of the project lifecycle. The doc-umentation not only helps students articulate a history and rationale for their design decisions, but schools can submit project documents for feedback from Texas Tech faculty and staff to offer ongoing support to classroom teachers.

While this approach is significantly more difficult to im-plement, it has a number of advantages with regard to many of the issues discussed here. It provides more flexibility to the kinds of design projects available to provide engaging teaching and learning experiences and still allow the teacher to teach the content required by the course standards. More significantly, the FRAME model directly addresses the TEKS established for engineering.

One of the authors, John Chandler, served on the com-mittee that wrote the engineering standard adopted by Texas at the request of the State Board of Education (SBOE). Other members of the committee were K- 12 teachers and administrators, engineering practitioners from industry, and other representatives of higher education. The standard de-veloped by the committee was the result of many passionate discussions about the kinds of content, skills, and processes that we could agree were essential knowledge and skills for engineering education––and which also could be offered within the existing structure, conventions, and capabilities of the school system. There was much discussion regarding the inclusion of rigorous academic content in both mathe-matics and science, developing a structure that reflected the conventions of university engineering education, and how various engineering concepts fit within the conventions and constraints of K- 12 education.

The resulting standard is used not only to define academic content for K- 12 engineering education in Texas for the next ten years, but will also guide development of curriculum re-sources and textbooks that will be adopted by Texas schools. The committee was forced to disregard many deeply held beliefs and expectations for precollege engineering educa-tion when faced with realities of the K- 12 environment and how education standards are implemented in practice. The TEKS emphasize knowledge and skills specific to engineer-ing design process and excluded specific academic content. Instead the assumption was that students coming into the course would have already taken 2–3 years of math and sci-ence courses under the Texas 4×4 plan for graduation. The course is intended to provide a capstone experience for stu-dents to apply previously learned academic content.

The committee members came to recognize that any academic content it might require could have the effect of limiting the kinds of design projects that would be possible.

For example, if the standard included proficiency with cer-tain biology concepts, then design projects that emphasized physics––a rocketry project, for instance––might not meet the requirements of the standard. The standard was written so as not to limit the type and scope of design projects avail-able for project based teaching and learning, but to require classroom implementations to adhere to a rigorous standard for the process itself. As the committee worked through is-sues regarding appropriate content for precollege engineer-ing education and began to focus more on a design process approach, it began to realize that wording of the standard would have to accommodate various design process mod-els, because of problems arising from the lack of substantive K- 12 epistemic experience, as suggested by Tate, Chandler, Fontenot, and Talkmitt (2010):

The literature suggests two basic approaches for rep-resenting engineering design: a phase- based, lifecycle- oriented approach; and an activity- based, cognitive approach. While these approaches serve various teach-ing and functional goals in undergraduate and graduate engineering education, as well as in practice, they tend to exacerbate the gaps in P- 12 engineering efforts, where appropriate learning objectives that connect meaning-fully to engineering are poorly articulated or understood. This is not to suggest that the realms of higher education and industry are immune from conflicting perspectives and agendas regarding engineering education. However, epistemology provides a common lens with which the topographies of various stances can be brought into focus and examined; whereas, no such context exists in P- 12 engineering.

The committee also recognized several disconnects be-tween the structure and expectations for K- 12 and higher education. For example, a common engineering education experience requires almost two years of prerequisite course work in the sciences (usually with emphasis on physics) and a mathematics course sequence through differential equa-tions calculus. It is also not uncommon for engineering education to emphasize theoretical understandings of ther-modynamics, statics, fluids, and other courses commonly referred to as the engineering sciences. In some programs, students may only encounter the engineering design process during a capstone class in their senior year.

Conclusions

This discussion aims at identifying some of the issues and problems which may impede implementation of engineer-ing education in the K- 12 environment. Specifically, the lack of a tradition for engineering in the K- 12 curriculum results in significant gaps in standards and policy, experience with classroom implementation––as well as support infrastruc-ture that exist for academic disciplines which historically have been part of the K- 12 experience. Among the systemic


components that would provide more consistent and effec-tive K- 12 engineering implementation are the following:

• cohesive standards and policies to provide a frame-work for systemic development of educational resources for precollege engineering, including standards for assessment and evaluation

• cohesive efforts across institutional boundaries for collaboration and as a means to address problems with conflicting agendas and perceptions

• infrastructure mechanisms and standards for pre-engineering teacher certification and professional development, including professional organizations for teachers

• research and a body of literature for preengineer-ing with methods and epistemic tradition suited to exigencies of K- 12 education

Until we begin to address these gaps in a systemic fash-ion, the quality of the educational experience we provide to students and the content knowledge and pedagogies of teachers in preengineering will remain extremely uneven, with no researchdriven, generally accepted basis for as-sessment and evaluating effectiveness of implementations. The current state of precollege engineering has no epistemic foundation to provide the common language, shared con-cepts and historical perspective found within the traditions of science, mathematics, and other disciplines that are well established in K- 12 education.

Efforts to view the growing experience with K- 12 en-gineering only through the lens of engineering education have been inadequate because these approaches typically fail to meaningfully account for exigencies specific to K- 12 education. Rogers et al. (2010) suggest researchers adopt a range of methodologies and theoretical lenses from among the many disciplinary traditions conducting K- 12 research. Given the conceptual disconnects between colleges of en-gineering and K- 12 education discussed here, a growing trend toward teaching STEM as interrelated knowledge and skills, an interdisciplinary approach has obvious merit––not just to inform research, but also because many disciplin-ary interests intersect in the K- 12 STEM classroomas an increasingly integrated STEM paradigm emerges. There is growing conviction that substantive STEM reform must be inclusive, allowing participation and ownership by all the stakeholders. This emerging STEM paradigm emphasizes the interrelatedness of concepts from science and mathe-matics, which find application in engineering and underpin the technologies used by in technical workforce and among the citizenry as a whole. The potential of engineering edu-cation in the K- 12 curriculum has created almost unprec-edented development and participation in education reform by a wide- range of stakeholders representing both public and private interests. Perhaps we should heed the notion put forth by the NAE that precollege engineering could serve as a catalyst for significantly changing the way we educate our

children, and that if done right, might precipitate rethinking the whole system.

The attention being paid to putting the E in STEM at the K- 12 level may very well result in questioning the con-ventions of higher education and result in more cohesion between secondary and postsecondary education, possi-bly creating new educational pathways into the technical workforce. New collaborative relationships are emerging in Texas to develop degree and certification programs for preengineering teachers at the University of Texas and at Texas Tech University. Both of these required collaborative development between the respective colleges of education and engineering of an appropriate sequence of courses for preengineering teacher preparation. In the past there had not been much reason for engineering colleges to work with colleges of education or other academic units engaged in teacher preparation. Instead, engineering colleges often had a tendency to focus inward on research and teaching the en-gineering sciences. Developing a cohesive set of standards for preengineering provides incentive that has heretofore largely been missing for engineering colleges to participate in teacher preparation or in other areas of K- 12 service.

Certainly, there are many issues that policy and stan-dards alone cannot effectively address. Studies conducted by the NAE (2002) indicate that a number of commonly held perceptions about engineering contribute to declining enrollments and interest in engineering programs especially among minority populations and women who are needed to increase diversity in the ranks of engineers. Massachusetts and Texas education systems have developed standards- based engineering education in their schools by mandate by their respective legislatures. Ten years ago when Massachu-setts began to introduce engineering into K- 12 education, the process of developing implementations in schools was hampered by a lack of curricular development, teacher train-ing, and other resources. Nine years later, significantly more curriculum and other resources for precollege engineering are available, but many of these preengineering resources are not standards- based and do not meet accreditation re-quirements. There are also no widely accepted definitions of what activities, knowledge, and skills are appropriate for teaching and learning engineering in the K- 12 environment, which complicates making informed choices regarding the quality and suitability of various curricular resources and professional development for teachers.

References

AeA, Business Roundtable, Business- Higher Education Forum, et al. (2005) Tapping America’s potential: The education for innovation ini-tiative.[businessroundtable.org]

Aristotle. The Nicomachean Ethics. David Ross, trans. J. L. Ackrill and J. O. Urmson, revisions. Oxford World’s Classics paperback.(1998). Oxford: Oxford University Press.

Brophy, S., Klein, S., Portsmore, M., & Rogers, C. (2008, July). Advanc-ing engineering education in P- 12 Classrooms. Journal of Engineering Education, pp. 369–387.


Busch- Vishniac, I., & Jarosz, J. (2004). Can diversity in the undergradu-ate engineering population be enhanced through curricular change? Journal of Women and Minorities in Science and Engineering, 10(3), 255–282.

National Academies Committee on Science, Engineering, and Public Pol-icy. (2007). Rising above the gathering storm: Energizing and employ-ing America for a brighter economic future. National Academies Press. Retrieved from http://www.nap.edu/catalog/11463.html

National Academies Committee on Science, Engineering, and Public Pol-icy. (2009). Rising above the gathering storm two years later. National Academies Press. Retrieved fromhttp://www.nap.edu/catalog/12537 .html

National Academy of Engineering. (2002). Diversity in engineering: Man-aging the workforce of the future. National Academies Press. Retrieved from http://www.nap.edu/openbook.php?isbn=0309084296

National Academy of Engineering and National Research Council Com-mittee on K 12 Engineering Education. (2009). Engineering in K- 12 education: Understanding the status and improving the prospects. Re-trieved from http://www.nap.edu/catalog.php?record_id=12635

National Commission on Mathematics and Science Teaching for the 21st Century. (2000). Before it’s too late: A report to the nation. Retrieved from http://www.ed.gov/american accounts/glenn/report.pdf

National Science Board. (2007). Moving forward to improve engineering education. Retrieved from http://www.nsf.gov/pubs/2007/nsb07122/index.jsp

Rogers, C., Wendell, K., & Foster, J. (2010, April). A review of the NAE Report, Engineering in K- 12 education.Journal of Engineering Educa-tion, pp. 179–181.

Tate, D., Chandler, J., Fontenot, A. D., & Talkmitt, S. (2010). Matching pedagogical intent with engineering design process models for pre-college education. AI EDAM 24 (03), pp. 379–395. Available on CJO 12 July 2010

Texas Education Agency. (2009). Chapter130.373. Engineering design and problem solving. Texas essential knowledge and skills. Retrieved from http://ritter.tea.state.tx.us/rules/board/adopted/0709/ch130o- two.pdf

Texas Examination of Educator Standards.(2009). Mathematics/ physical science/engineering 8–12 exam. Retrieved from http://www .texesexampracticetests.com/174- TExES- Mathematics- Physical - Science- Engineering- 8- 12- Exam.html

Texas State Board of Educator Certification. (2004). Engineering stan-dards. Retrieved from http://www.sbec.state.tx.us/SBEConline/standtest/standards/8- 12engin.pdf

Tran, N., & Nathan, M. (2010, April). Precollege engineering studies: An investigation of the relationship between precollege engineering stud-ies and student achievement in science and mathematics. Journal of Engineering Education, pp. 143–157.



Nicole Weber, School of Engineering Education, Purdue University; Daphne Duncan, College of Education, Purdue University; Melissa Dyehouse, School of Engineering Education, Purdue University; Johannes Strobel, Schoole of Engineering Education and INSPIRE, Purdue University; Heidi A. Diefes- Dux, School of Engineering Education, Purdue University Nicole Weber is a Postdoctoral Research Associate in the School of Engineering Education at Purdue University. She received her B.S. degree in Ecology, Evolution and Behavior from the University of Minnesota, St. Paul. At the University of Massachusetts Boston, she received her Ph.D. in Envi-ronmental Biology with an emphasis in Science Education. Her current research is working in “sustainable engineering” education, specifically in creating awareness of engineering as a “caring” discipline,where engineers develop technology incorporating the ecological footprint, keeping in mind the social and ecological impacts. Daphne Duncan is a Ph.D. candidate in the College of Education at Purdue University. She received her B.S. in Elementary Education from Florida State University, her M.S. in Human Resources Management from Troy State University, and her M.Ed. in Curriculum and Instruction from North Carolina State University. Prior to returning to school full- time, she taught third grade for five years. Since 2006, she has been working for the Institute for P- 12 En-gineering Research and Learning (INSPIRE) in the area of teacher professional development. Her research interests include engineering education, gifted education, and instrument development. Melissa Dyehouse is a Postdoctoral Research Associate in the School of Engineering Education at Purdue University. She received her M.S.Ed. and Ph.D. in Educational Psychology from Purdue University. She has conducted research on middle school students’ perceptions of engineers and scientists and the effectiveness of interventions to improve students’ perceptions and attitudes about STEM fields. Her current research focuses on the learning and teaching of engineering as a “caring” discipline in the context of environmental and ecological concerns.

The Development of a Systematic Coding System for Elementary Students’ Drawings of Engineers

Nicole Weber, Daphne Duncan, Melissa Dyehouse, Johannes Strobel, Heidi A. Diefes- Dux

Purdue University

ABSTRACT

The Draw an Engineer Test (DAET) is a common measure of students’ perceptions of engineers. The coding systems currently used for K- 12 research are general rubrics or checklists to capture the images presented in the drawing, which leave out some of the richness of students’ perceptions, currently only captured with an accompanying student interview. The purpose of this study is to build a reli-able coding system, which first establishes an inventory of pictorial elements irrespective of their potential relationship with engineering and second captures aspects of students’ engineering perceptions inductively (from the ground up) while at the same time incorporating categories from previous research. The coding system will be used to help researchers understand how young students’ perceptions of engineering, engineers, and the work of engineers evolve and are impacted by interventions. The longterm goal of this project is to create a standalone measure that can be broadly applied to diverse populations, and to create a large multi- institution student database, with both K- 12 and university populations represented. This database would provide a rich dataset for better understanding common misconceptions about engineering and thus enabling the development of methods to address them.

Key Words: engineering, children, drawings, social constructivism, assessment

50 N. Weber, D. Duncan, M. Dyehouse, J. Strobel, H. Diefes-Dux / Journal of Pre-College Engineering Education Research

With the integration of engineering into the elementary classroom (NAE, 2009) and an increase of research in early and preengineering, the engineering education research community is faced with the same question as the science education community: How can we adequately explore young children’s understanding of engineering? Punch (2002, p. 322–323) describes in her comparison of methods of research how children differ in the following assumptions: (a) children are different from adults and as such should be researched with ethnographic methods; (b) children are simi-lar to adults with different competencies; (c) children are dif-ferent from adults merely in regards to ethical considerations such as consent and confidentiality. This article is situated be-tween assumption (a) and (b), stating that research with chil-dren, particularly young children, is different from research with adults (James, Jenks, &Prout, 1998) due primarily to the differing competencies of expression (Nesbitt, 2000). Acknowledging the differences in expression between chil-dren and adults, our research is focused on improving what Punch calls “methods which are based on children’s skills” (2002, p. 322). One of the most described skills of children is drawing (Nesbitt, 2000) and the combination of a draw-ing and writing test<au: this is not a “skill”; is something missing?> (Backett- Milburn & McKie, 1999; France, Bend-elow, & Williams, 2000; Pridmore & Bendelow, 1995). Our context to research children’s understanding of engineering is a revised schema for the existing “Draw an Engineer Test” (Knight & Cunningham, 2004).

Theoretical Framework

Following our assumptions, while most research ap-proaches with children are described and tested as solely methods and/or techniques, our approach is theoretically grounded as well. We will examine the process of young children’s drawings through a social constructivist theo-retical framework, a Vygotskian (Vygotsky, 1962) lens. In a social constructivist context, experiences are shared to construct meaning, where the knowledge is co- constructed by the combination of prior and new knowledge (Brooks, 2004). A learner constructs meaning and understanding through the surrounding sociocultural environment (Vy-gotsky, 1978). Vygotsky theorized a connection between thought and speech and the development of verbal thought,

and the forms to communicate this might include symbols, algebraic systems, art, writing, diagrams and language (Brooks, 2004, 2009; Vygotsky, 1962). The significance of children’s drawings is described by Brooks (2004) who states that “when we consider children’s drawing to be a form of communication and a meaning- making tool, then the social, the cultural and the historical relationship with this meaning- making process demands careful consider-ation” (p. 1). Therefore, when we use children’s drawings, we are not merely utilizing an artistic form of expression, but a unique language. This lens allows us to utilize chil-dren’s drawings as speech- acts (Bretherton & Beeghly, 1982), which express what a child understands about en-gineering and engineers. The task to code and analyze chil-dren’s drawings analogically then becomes a translation task similar to translating from another language.

Literature Review

Researchers have been studying children’s drawings for decades in an attempt to put words to the marks of crayons, markers, pens, and pencils left on paper by children when asked to draw a particular object (Kosslyn, Heldmeyer, & Locklear, 1977). “Piaget argued that a child’s drawing per-formance reflected the child’s cognitive competence. He did not consider drawing to be a special domain of development but merely a window into the child’s general cognitive de-velopment (Brook, 2009; Piaget & Inhelder, 1967).”

Children’s drawings have been used in a variety of settings as a means of assessment and as a method of gathering infor-mation in a non- threatening way. Children’s drawings have been used as an effective pre/post assessment (Bowker, 2007; Weber, 2008) and to see differences in children’s perceptions (Barraza, 1999; Bowker; Weber). Drawings have also been used to assess attitudes and misconceptions about scientists and engineers (Chambers, 1983; Knight & Cunningham, 2004). These studies demonstrate the basis for our study.

Understanding students’ perceptions of engineers and the work they do is important, as these perceptions can influence students’ understanding and beliefs about the profession, and their consideration of pursuing the profes-sion as a career (Knight & Cunningham, 2004). Assessing attitudes and knowledge about engineering and engineers has often been observed through the Draw an Engineer Test

Johannes Strobel is the Director of INSPIRE and an Assistant Professor of Engineering Education & Educational Technology at Purdue University. Johannes received his B.Ph. (Philosophy) at Munich School of Philosophy, his B.S. (Information Science) and B.A. (Religious Studies) at Saarland Univer-sity, Germany, and his M.Ed. and Ph.D. in Information Science and Learning Technologies at the University of Missouri. His research is centered around teachers’ concerns and innovations integrating engineering into their curriculum, the environmental awareness of engineering students and engineering in the workplace. Heidi A. Diefes- Dux is an Associate Professor in the School of Engineering Education at Purdue University. She received her B.S. and M.S. in Food Science from Cornell University and her Ph.D. in Food Process Engineering from the Department of Agricultural and Biological Engineering at Purdue University. Since 1999, she has been a faculty member within the First- Year Engineering Program at Purdue. She is currently the Director of Teacher Professional Development for the Institute for P- 12 Engineering Research and Learning (INSPIRE). As such, her research interests center on the integration of engineering into the elementary curriculum. This work was made possible by a grant from the National Science Foundation (DRL 0822261). Any opinions, findings, and conclusions or recom-mendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

N. Weber, D. Duncan, M. Dyehouse, J. Strobel, H. Diefes-Dux / Journal of Pre-College Engineering Education Research 51

(DAET; Knight & Cunningham, 2004), which grew out of the Draw a Scientist Test (DAST; Chambers 1983). In the DAET, children are asked to draw a picture of an engineer and then asked a series of questions about engineering. Pioneering this assessment tool, Knight and Cunningham (2004) administered the DAET to 384 students in grades 3–12 and found that most young students believed that en-gineers “build buildings and fix car engines” (p. 7).

Taking this a step further, Cunningham, Lachapelle, and Lindgren- Streicher (2005) used the results of the DAET to create a 16- image survey called, “What is an Engineer?” where students are asked to circle the pictures where engi-neering is represented and then answer the question, “An engineer is a person who ______.” The second part of this assessment is a 16- image survey called, “What is Technol-ogy” where students are asked to circle the pictures where technology is represented and then answer the question, “How do you know if something is technology?” Cunning-ham and her fellow researchers at the Museum of Science, Boston, continued their research, moving towards a more quantitative method of assessing students’ understanding of engineering (Lachapelle, Cunningham, Oware, & Battu, 2008). Research using the DAET reappeared in 2008, when Oware conducted a study using the drawing assessment with an accompanying individual interview to examine elemen-tary students’ perceptions of engineers, and as a result used the two vantage points of data to create a detailed coding rubric (Oware, 2008). Fralick, Kearn, Thompson, and Lyons (2008) then developed a checklist for cataloguing items in a DAET drawing for a middle school student population.

Researchers at The Institute for P- 12 Engineering Re-search and Learning (INSPIRE) were interested in build-ing on the previous research by developing a detailed coding system of children’s drawings that could be used reliably without the need for additional data such as stu-dent interviews. Coding systems that are currently in use rely on interview data to provide a complete representation of student perceptions, which can be time consuming and expensive. This coding system was developed to provide a rich description and inventory of pictorial elements in students’ drawings first, without evaluating their relation-ship to engineering and secondly to score the drawing in order to gain a more complete understanding of students’ perceptions of engineers and engineering, and subsequently incorporating many of the components included in previous rubrics (Oware, 2008) and checklists (Fralick et al, 2008). This study describes the process of developing the detailed coding system that can be used reliably as a standalone measure of students’ perceptions of engineering, building on previous research.

Research Rurpose

The purpose of this study was to develop a rich coding system that could be used to reliably assess elementary stu-dents’ responses to the DAET. This coding system would

then be used to evaluate an educational intervention aimed at integrating engineering into an elementary curriculum. The primary goal of this intervention was to increase engi-neering literacy in young students. The coding system will also be used to help researchers understand how young stu-dents’ perceptions of engineering, engineers, and the work of engineers evolve and are impacted by interventions. The longterm goal of this project is to create a standalone mea-sure of the DAET drawing that can be broadly applied to diverse populations, to create a large multi- institution stu-dent database, with both K- 12 and university populations represented, to better understand common misconceptions and develop methods to address them.

Methodological Considerations

DAET Administration Components

The participants in the DAET were the 2nd through 4th grade students whose teachers had received an INSPIRE engineering intervention. The intervention was a week- long summer engineering academy where the elementary teachers learned ways to integrate engineering into their existing curriculum. Because it was important to represent students’ understanding of engineering as knowledge was constructed, teachers administered the DAET as a pre- post assessment, both before and after engineering instruction. Student participants represented ethnically diverse popula-tions from both urban and suburban elementary schools, in-cluding 10 participating classrooms from one school district in the south central United States.

The teachers first attended an engineering academy, fa-cilitated by INSPIRE in their home school district. During the academy, the teachers participated in several engineer-ing activities covering topics such as engineering design, mathematical modeling, engineering professions, scien-tific inquiry as a basis for engineering, and technology as a product of engineering. When the teachers returned to their classrooms in the fall, they agreed to teach the engineering lessons/curriculum they had learned in the academy, and they agreed to administer the DAET both before any engi-neering instruction took place (pre) and after all engineer-ing instruction had taken place (post).

In the administration of the DAET assessment, each stu-dent participant received the DAET form and a writing uten-sil of their choice (e.g., color crayons, pencils, or markers). Students were told, “In the box on your piece of paper, draw an engineer doing engineering work.” They were then al-lowed to draw freely for 20 minutes. They also answered the writing prompt, “What is the engineer doing?” Both the drawing and the written answers to the question were ana-lyzed in the creation of the DAET coding system.

Each student drawing was assigned an identification number (sample shown in Figure 1). Identification items such as gender, ethnicity, and/or grade were removed prior to coding in an effort to reduce coder bias, forcing coders


to judge drawings based only on the content provided by the students and not on any identifying student information. Additionally, the identification number prevented the coder from knowing if the drawing being evaluated was a pre or a post drawing, thus further reducing bias.

Interrater Reliability

The coding system needed to have an acceptable interra-ter reliability (i.e., 80% using liberal measurements) before it could be used to assess the student drawings. A more con-servative measure of interrater reliability takes into account all sources of unreliability, including chance agreements (e.g., Krippendorff’s α, Scott’s π), while a liberal measure simply calculates the percentage of agreement or correla-tion between raters (e.g., Pearson’s reliability coefficient, agreement coefficients; Krippendorff, 2009). While more liberal criteria (e.g., .70 agreement coefficient) are typi-cally used for indices that are more conservative or for ex-ploratory research (Lombard, Snyder- Duch, & Campanella Bracken, 2002), Neuendorf’s (2002) review of typical cut-offs for interrater reliability found that .90 is an acceptable criteria for all types of situations, and that .80 or greater is acceptable for most situations.

Results

Coding System Development

The INSPIRE DAET Coding System was developed using a grounded theory approach (Corbin & Strauss, 1990), using the students’ drawings, written answers, and

interview transcripts in initial iterations, with open coding used to develop initial categories. A total of 476 drawings were used to develop the coding system. A breakdown of the number of drawings used in each iteration of develop-ment is shown in Table 1. During the initial code develop-ment, all occurrences of objects and ideas represented in the students’ DAET were recorded so as to not miss anything in the children’s drawings. Axial coding was then used to condense and refine the codes. As the coders looked for patterns among the codes recorded during open coding, the codes were collapsed into categories of like ideas or codes. Later, the ideas were merged and given variable labels with specific code instructions (see Appendix A.1). Through-out the coding system development, we continued to fol-low a grounded theory approach (Corbin & Strauss, 1990) and incorporate previous essential research (Fralick et al., 2008; Knight & Cunningham, 2004; Oware, 2008; Prabha & Garg, 2000; Weber, 2008). Throughout several itera-tions of the coding system, codes were refined, collapsed, added, and discarded based on their presence in drawings, students’ written answers, researcher discussions, and prior research findings.

The INSPIRE DAET Coding System emerged from the students’ drawings and written descriptions via six coding iterations. Following the final round of coding, the coding system had seven major classifications: Humans, Human- Engineered Objects, System, Environment, Vibe, Engineer-ing Field Portrayed, and Engineering Understanding (see Table 2). What follows is a description of the evolution of

Table 1Specific Sample Size per Iteration

Iteration Total Drawings

Iteration One 180

Iteration Two Pre 93 (Pre 42, Post 51)

Post

2nd Grade 14 17 313rd Grade 15 18 334th Grade 13 16 29

Iteration Three 79 (Pre 37, Post 42)


Iteration Four 84 (Pre 41, Post 43)


Iteration Five Changes were made to the existing coding system through literature review and team discussions; no student drawings were analyzed

Iteration Six 20

Coding System Verification 20

Figure 1. Student DAET Drawing, with example identification number.


the coding system. The evolution of the coding system is diagrammed in Appendix A.2.

Coding Iteration OneThe first iteration, of 180 drawings, resulted in four

major classifications: Humans, Objects, System, and En-gineering. Under each of the major classifications there were categories, and then subcategories. Humans had two categories: Engineer as Person, and Other Human Beings. Objects had four categories: Natural Objects, Human- Made Objects, Tools, and Engineering Artifacts. System had three categories: Process Present, Activities of Engineer, and In-tention of Engineering. Engineering had one category: How Sophisticated. Within this iteration, data were collected through a description, evidence (picture, text, both, inter-view), and certainty (Likert Scale 1–5) for each category. Coders would be asked to provide a description of the cat-egory being observed (e.g., if the coder saw natural objects in the picture s/he would note a description of those natural objects). The coder would then be asked to indicate how s/he knew that the category was present in the picture (e.g., Did the child draw the natural object, write about the natu-ral object, both draw it and write about it, or speak about it in an interview?). Finally, the coder would be asked to indicate, via a Likert scale (1–5) how certain s/he was that that category was actually present in the picture (e.g., How certain are you that the natural object is what you think it is?). For the Engineer as a Person category of the Humans classification, the description section was pre- set as “male,

female, or multiple” meaning that the student drew a male human, a female human, or multiple humans.

Coding Iteration TwoAfter reviewing 93 student drawings we identified three

main areas for refining the coding system. First, within the Humans classification, we noticed that some students referred to themselves as the engineer, so a new category Student as Engineer was added. Students also drew a non- human engineer (e.g., a car engine), so we needed to add the Engineer as Non- Human category to the Humans clas-sification as well. As we considered the engineer to be de-picted as a human initially, the new realization that a student could consider the engineer as a non- human was important to monitor, and here it was important to keep both concepts connected, thus keeping within the same initial Human classification. The overall classification may be changed in the future, once a better fit is determined.

Second, within the Objects classification, we also felt that the Human- Made Objects category was too broad and needed to be broken into two independent subcategories: “Intention of Engineering” and “Engineering.” The former included four focal areas: “Vehicles,” “Machines,” “Tools,” and “Structures.” Items in these subcategories would be coded if a student intended for an item to be engineering, but leaned more towards another profession (e.g., me-chanic). Within the Human- Made Objects category, the “Engineering” subcategory had the same focal areas (Ve-hicles, Machines, Tools, Structures) plus an additional

Table 2Classification Descriptions

Code and Description Text Hierarchy: Classification—Category—“Subcategory”—Focal Areas Initial Introduction

1. Humans: The student draws an engineer as a human (defined as either a female/male/ or ambiguous). Or, the Iteration 1 student draws the engineer as a non-human (defined as an object).

2. Human- Engineered Objects: The student draws objects used by, created by or thought to be created by Iteration 1 engineers such as vehicles, machines, tools, structures, and/or engineering artifacts. 2–as split category 3–stand-along category 5–name change

3. System: The student indicates a process (such as the engineering design process), or that the engineering in the Iteration 1 drawing is taking place for a purpose or benefit. Additionally, the student lists verbs (correct or incorrect)

associated with engineering.

4. Environment: The student indicates where the drawing is taking place (natural elements, human-managed Iteration 2 elements, and detail of location).

5. Vibe or Affect: The student’s drawing is determined to have a positive/neutral/negative atmosphere based on the Iteration 2 items contained within the drawing and the written description of the drawing. 5–name change

6. Engineering Field Portrayed: The student’s drawing portrays an engineering field (e.g. mechanical, electrical). Iteration 5 The student adds details to the engineer represented, such as clothing or objects, and may indicate attitudes/ dispositions associated with engineers/engineering. The student’s drawing is judged on the ability to match the engineering profession to the engineer drawn.

7. Engineering Understanding: The student’s understanding of engineering is judged to be plain if the student Iteration 1 has 0–3 engineering details drawn (i.e. clothing, objects, attitudes, match between engineer/profession, 5–name change occurrences in the field), and detailed if the student has more than 3 engineering details drawn.


area of Engineering Artifacts. Items in these subcategories would be coded if a student drew an object that represented engineering. We separated the subcategories of “Vehicles,” “Machines,” “Tools,” and “Structures” because of the in-tention behind the students’ drawings. If a student drew a picture where one of these subcategories was represented, we evaluated the drawing to determine if the student had drawn a picture intended to be engineering, or if s/he had drawn a picture that was truly representative of engineering. For example, if a student drew a car, s/he would receive a “Vehicle” code. If the car was represented by an engineer fixing the car, the “Vehicle” code would be coded under the “Intention of Engineering” subcategory. However, if the student drew an engineer designing a car, the “Vehicle” code would be coded under the “Engineering” subcate-gory. Since “Intention of Engineering” was moved into the Human- Made Objects category, it was removed from the System classification.

Third, new standalone classifications were necessary to capture more detail in two areas: Environment (the place where the drawing is taking place) and Affect/Disposition (e.g., smiling in the picture, worried faces). In addition, five new categories were created within the Engineering classifi-cation: Engineers as Other Professions (e.g., when students represent engineers as professions other than engineering, such as firemen or teachers), Engineering as Science (engi-neering and science are the same thing), Who Benefits from Engineering, Clothing (level of detail), and Problems As-sociated with Engineering (negative aspects of the career). Lastly, since the purpose of this coding system is to be used on drawings alone, we were at the point that the interview data could be removed, and eliminated “interview” as an evidence option.

Iteration ThreeFor the third iteration, we reviewed 79 student drawings.

Here within the Humans classification, we noticed that we were unable to describe some of the humans represented as either male or female, so we added an “ambiguous” cod-ing subcategory to the Engineer as a Person category. Also within the Object classification, differentiating between the two Human- Made Objects categories was also difficult, so we consolidated the two categories back into one, and gave the category six subcategories: “Vehicles,” “Machines,” “Tools (Physical Labor),” “Tools (Office),” “Structures,” and “Engineering Artifacts.”

Iteration FourIn the fourth iteration, we reviewed 84 student drawings.

In the Humans classification, some students shaded their drawings, so we added a “shaded/not shaded” subcategory. Secondly, some students included stick figure drawings of humans, while others contained a more detailed repre-sentation, so the subcategory “stick/partially developed/developed” was added to describe the human(s) drawn, to specify where the energy is spent in the drawing. Lastly,

some students were assigning a proper name to their engi-neers (e.g., Tom, Jamie); to capture this we added a Name the Engineer category.

There were additional areas of clarification, first within the System classification, where we added the Intention of Engineering category to indicate why the engineering was taking place. Second, in the Engineering classification, we removed both the Engineering as Science and Problems As-sociated with Engineering as they were difficult to capture reliably. More broadly, we modified the Affect/Disposition classification to be just Affect (e.g., smiling in pictures, worried faces), and we added the category of Attitudes/Dispositions to the Engineering classification in an effort to capture students’ perceptions of the feelings of and towards engineering (e.g., “I love engineering!”). Additionally, we started noticing that some students were mentioning spe-cific engineering disciplines and the work associated with those engineering disciplines, which prompted us to add the classification of Engineering Fields and the category of Work Associated with that Engineer replacing the original category of Engineers as Other Professions.

Iteration FiveAt this time, we brought a new team member on board,

who completed a doctoral dissertation in children’s draw-ings and coding systems. After team discussions, we made three initial changes to help align our current coding system with the coding system implemented in her dissertation. First, we added Vibe to the classification of Affect, along with the following three categories: Negative, Positive, and Neutral. The second change was within the Environment classification, where there were three overall categories; Environment (where), Natural, and Human- Managed. The first referred to where the drawing was taking place, first discussed in iteration two. The new categories Natural and Human- Managed (referred to as Man- Made in Prabha & Garg, 2000 and Human- Managed in Weber, 2008) were added to achieve a deeper description of the drawing and to align the coding system with previous work in the area of environmental awareness. Human Managed contained five subcategories: “Religion,” “Social”, “Education,” “Politi-cal,” and “Science and Technology.” The Natural category catalogued the many aspects of nature captured in the chil-dren’s drawings. The six subcategories of the Natural cat-egory were: “Hydrosphere,” “Lithosphere,” “Atmosphere,” “Plant,” “Animal,” and “Humans.” Third, to determine how detailed the students’ drawings were, we added the category of Detail (also from the above dissertation work: Weber, 2008), with two subcategories: Plain (0–3 variables included) and Detailed (more than 3 variables included). Changes were as noted in the existing coding system as well, within the Humans, Objects, System, and Engineer-ing classifications. In addition, both a codebook and scoring sheet were developed in an effort to streamline the coding process, with variable code names and coding instructions assigned for each of the subcategories.


Within the Humans classification, “stick or developed figure” category was removed. The Objects classification included six corresponding categories: Vehicle, Machine, Physical Labor Tools, Office Tools, Engineered Structures, and Engineering Artifacts. Engineered Structures was a category that underwent some revision to encompass struc-tures that are final products of civil engineering design. Ad-ditionally, Engineering Artifacts was a category that also underwent revision to include objects associated with the planning stages of engineering design. The System classi-fication contained seven categories with reformatted defini-tions: Process (a process is represented), Engineering Verbs (writes verbs that are associated with engineering, even if that conception of engineering is not a correct conception), Why (provides an explanation for why the engineering is taking place), Benefit (people or organizations portrayed that will benefit from the engineering happening in the drawing), and the written description representation coun-terpart for each. Who Benefits was then removed from the Engineering classification.

Last, the Engineering Field Portrayed classification had 6 categories: Engineering Field, Match, Clothing, Objects, Attitudes/Dispositions, and Attitude provided (written). The category of Engineering Field included codes for specific engineering disciplines, as did the categories of Clothing and Objects that were both adapted from previous research (Fralick et al., 2008; Knight & Cunningham, 2004; for more detail please see Appendix A.1) For the category of Match, the student received a number based on their ability to match the work of the engineer with the explicit discipline they described. We also replaced the How sophisticated cat-egory with Engineering Concept category, because making the judgment on the level of engineering “sophistication” in a student’s drawing (scale of 1–5) was proving to be diffi-cult when attempting to gain agreement among coders. The new category included three levels: no understanding, some understanding and understands.

Iteration SixFor the sixth and most recent iteration of the INSPIRE

DAET Coding System, we retained most of the properties of the fifth iteration with only minor changes. Under the Humans classification, the subcategory of “Gender” was in-cluded on the initial coding systems, however was inadver-tently left off of Iteration five and as a result re- included on the sixth iteration. Name changes included a modification within the same classification, the Other Humans category was renamed Group, where a number system was not re-quired. The Object classification was changed to Human- Engineered Objects, due to the Natural Objects category being moved to the Environment classification. Also, within the Environment Classification, the two subcategories of the Environment category were each renamed “Location” to ease coding. Finally, a new Detail in the Engineer category was added to the Engineering Field Portrayed classification, where the level of detail in the engineer would be determined

by adding the number of occurrences in the field, clothing, object, and attitude subcategories, removing the “stick or developed figure” as a subcategory in the Human classifica-tion. As previously, a drawing with 0–3 occurrences is con-sidered plain, while a drawing with 4 or more occurrences is considered a detailed drawing of an engineer. We removed the Match category, as it was too difficult to determine.

Coding System Verification

Initial verification of the coding system took place after the sixth iteration, where two researchers independently coded 20 drawings (see example in Figure 2). For example, the pre- drawing is coded 1 for a human, 1 for a shaded face, 2 for the male gender, and 1 for a vehicle present in the drawing. Zeros were given for the other categories in the drawing because there was no evidence for their pres-ence (e.g., the setting was not in an office). In contrast, the post drawing is coded for the presence of a human, a shaded face, the male gender, an office setting(person is seated at a table), and artifacts (blueprint/drawing). We elected to use critical incident sampling (Patton, 2002), choosing 20 of the most difficult cases to code. The drawings were a mixture of both pre and post drawings, and a balance of 2nd through 4th grade students’ work. The coders were unaware of the grade level of the student and of the pre/post status of the drawing as they coded the drawings. After the critical in-cident drawings, the initial interrater reliability was calcu-lated to be 81.7%. Minor changes were made to the DAET coding system, specifically in the System, Engineering Field Portrayed, and Engineering Understanding categories.

The DAET coding system was then refined through a series of sessions where two coders (including one new coder) independently scored a set of 10 DAET student re-sponses and then met to discuss areas of disagreement, for two rounds while continuing to refine the coding system by discussing areas of disagreement. The interrater reliability was calculated to be 80.1% for the first round and 82.8% for the second round (Table 3). Then for the last round, we scored 10 DAET student- drawing responses with 4 coders, resulting in an average interrater reliability of 79.5% and an overall reliability between the four coders at 79.9%.

In an overall comparison, each rater was compared against the codes of the entire group, allowing for trouble spots to become more visible (Table 3). Having four coders review (or analyze) the same data can shed light on what is really going on with the rubric; however, this level of in-formation could potentially be lost if not compared across the group. Traditionally, each data point is considered an agreement or disagreement between two coders indepen-dently from the group, without seeing where real issues lay or where there may be simple mistakes by someone not fully understanding the concept or even rushing through the data (see Table 3: Personal Disagreements/Researcher 2). The overall average is calculated by looking at each data point across the coders and seeing how each coder has rated


that component compared to the others. In this overall com-parison average, three values were used, including: a) full agreement (0), b) full disagreement (1), and c) minimal dis-agreement (.25). For the full agreement, all coders had the same score for that data point. For a full disagreement, there was a fifty percent (two or more) disagreement on the score

provided. When one person was different from the group, a minimal disagreement was counted, which is a percent based on the total number of coders (4). Table 4 shows an example of how the overall reliability was calculated based on the coders’ level of agreement.

Interpretation, Discussion and Future Steps

With the lens of social constructivism, we utilized chil-dren’s drawings as a means to elicit information about their perceptions about engineers and engineering. To assess these perceptions, the purpose of this study was to develop a coding system to score students’ drawings of engineers. The major strength of the INSPIRE DAET Coding System is that it provides a detailed account of students’ drawings of engineers and engineering. This detailed account will allow researchers to investigate students’ perceptions of engineering, engineers, and the work that engineers do. Ad-ditionally, this coding system, once validated, will serve as a standalone measure of students’ perceptions of engineers

Table 3Interrater reliability

Round Inconsistent Total Percent (Coders/Drawings) Scores Scores Agreement

Round 1: (2/ 10) Coder 1 vs. Coder 2 83 361 80%Round 2: (2/ 10) Coder 1 vs. Coder 2 75 437 83%Round 3: (4/ 10) Coder 1 vs. Coder 2 87 405 79% Coder 1 vs. Coder 3 83 407 80% Coder 1 vs. Coder 4 77 395 79%Traditional Average 80%

Round 3: (4/ 10) Overall Comparison 81.75 406.75 80% Average

Figure 2. Application of Coding System


for the DAET assessment, thus eliminating the necessity for an accompanying student interview.

As with any coding or cataloguing system, some of the richness present when an interview accompanies the draw-ing may be lost. Additionally, there may be times when items in the drawing are interpreted incorrectly by the coder. These weaknesses of the coding system are mitigated, how-ever, by the fact that a coding system allows a much greater number of drawings to be analyzed than would be possible if interviews were required to interpret the drawings.

The next phase is to validate the coding system as a standalone measure of student perceptions of engineers and engineering by triangulating a student DAET cod-ing results with his or her interview. To do this, INSPIRE researchers will be integrating supplemental questions targeted at different components of the drawings in the stan-dard DAET post- interview protocol. These questions will be used during the interview to verify that the coder reliably sees the same components the student describes that s/he drew in the picture, to ensure that the student’s perception

Table 4Overall Reliability Example

Coder Survey ID: Human or Non- Human Gender Structure as Final Product Process Represented

1 39 Person Female No Mention No Mention

2 39 Person Female No Mention Mentioned

3 39 Person Ambiguous Mentioned Mentioned


1 22 Person Ambiguous No Mention No Mention




1 6 Person Female Mentioned No Mention

2 6 Person Female Mentioned Mentioned

3 6 Person Female Mentioned No Mention







2 42 Person Male Mentioned Mentioned

3 42 Person Ambiguous Mentioned No Mention


Total Number 20 5 5 5 5

Disagreements 6 0 2.5 1.5 2

[(.25 ×2) +2] [(.25 ×2) +1] [(.25 ×4) +1]

Personal Disagreements

Coder Total

1 0 0 0 0 0

misunderstanding of concept 2 5 0 1 0 4

3 2 0 1 1 0 4 1 0 0 1 0


of the drawing and the researcher’s perception of the draw-ing align. To do this, the researchers will compare drawings (coded without the aid of an interview) to student interview responses (independent of the coder looking at the actual drawing).

This research is part of a larger project aimed at assess-ing students’ understanding of engineering, and this tool can be used as a pre/post inventory around an engineering intervention to show where students are. Researchers can then use results from the DAET to modify and improve pro-fessional development, curriculum, and instruction to better meet the needs of students, and ensure that students develop more informed perceptions of engineers and engineering. The development of a coding system for the DAET will en-able researchers to assess children’s general understanding of engineering and how these perceptions change as a result of exposure to engineering.

References

Backett- Milburn, K. and McKie, L. (1999). A critical appraisal of the Draw and Write technique. Health Education Research 14, 387–98.

Barman, C. R., & Ostlund, K. L. (1996).A protocol to investigate students’ perceptions about scientists and relevancy of science to students’ daily lives. Science Education International, 7(4), 16–21

Barraza, L. (1999). Children’s drawings about the environment. Environ-mental Education Research, 5(1), 49–65.

Bowker, R. (2007). Children’s perceptions and learning about tropical rainforests: An analysis of their drawings. Environmental Education Research, 13(1), pp. 75–96.

Bretherton, I., and Beeghly, M. (1982).Talking about internal states: The acquisition of an explicit theory of mind. Developmental Psychology, 18, 906–992

Brooks, M. (2009). What Vygotsky can teach us about young children drawing. International Art in Early Childhood Research Journal, 1(1), 1–13.

Chambers, D. W. (1983). Stereotypic images of the scientist: The Draw- a- Scientist test. Science Education, 67(2), 255–265.

Corbin, J., & Strauss, A. (1990). Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative Sociology, 13, 3–21.

Cunningham, C., Lachapelle, C., & Lindgren- Streicher (2005). Assessing elementary school students’ conceptions of engineering and technol-ogy. Paper presented at the ASEE Annual Conference and Exposition.

Fralick, B., Kearn, J., Thompson, S., & Lyons, J. (2008). How middle schoolers draw engineers and scientists. Journal of Science Education and Technology, 18(1), 60–73.

France, A., Bendelow, G. and Williams, S. (2000). A “Risky” business: Re-searching the health beliefs of children and young people. In A. Lewis

and G. Lindsay (Eds.) Researching children’s perspectives, pp. 150–162. Buckingham, UK: Open University Press.

James, A., Jenks, C. & Prout, A. (1998). Theorizing childhood. Cambridge, UK: Polity Press.

Krippendorff, K. (2009). Testing the reliability of content analysis data: What is involved and why. In K. Krippendorff & M. A. Bock (Eds.), The content analysis reader. Thousand Oaks, CA: Sage.

Knight, M., & Cunningham, C. M. (2004). Draw an engineer test (DAET): Development of a tool to investigate student’s ideas about engineers and engineering. Paper presented at the ASEE Annual Conference and Exposition.

Kosslyn, S. M., Heldmeyer, K. H., & Locklear, E. P. (1977). Children’s drawings as data about internal representations. Journal of Experimen-tal Child Psychology, 23, 191–211.

Lachapelle, C., Cunningham, C., Oware, E., & Battu, B. (2008). Engineer-ing is elementary: An evaluation of year 5 field testing. Retrieved from http://www.mos.org/eie/pdf/research/NationalYear5Evaluation.pdf

Lombard, M., Snyder- Duch, J., & Campanella Bracken, C. (2002). Con-tent analysis in mass communication: Assessment and reporting of in-tercoder reliability. Human Communication Research, 28(4), 587–604.

National Academy of Engineering. (2009). Engineering in K- 12 Educa-tion: Understanding the status and improving the prospects. Washing-ton, DC: The National Academies Press.

Neuendorf, K. A. (2002). The content analysis guidebook. Thousand Oaks, CA: Sage.

Nesbitt, E. (2000). Researching 8- to 13- year- olds’ perspectives on their experience of religion, in A. Lewis & G. Lindsay (Eds.) Researching children’s perspectives, pp. 135–149. Buckingham, UK: Open Univer-sity Press.

Patton, M. Q. (2002). Qualitative research and evaluation methods (3rd ed.). Thousand Oaks, CA: Sage.

Piaget, J., & Inhelder, B. (1967). The child’s conception of space. New York: W. W. Norton.

Prabha, & Shilpi Garg. (2000). Environmental awareness of children through spontaneous drawing. Psycholingua, 30(2):131–136.

Pridmore, P. & Bendelow, G. (1995). Images of health: Exploring beliefs of children using the Draw- and- Write technique, Health Education Journal 54, 473–88.

Punch, S. (2002). Research with children. The same or different from re-search with adults? Childhood, 9(3), 321–341

Oware, E. (2008). Examining elementary students’ perceptions of engi-neers.(Doctoral dissertation).Retrieved from Dissertations & Theses @ CIC Institutions. (Publication No. AAT 3344179).

Vygotsky, L. (1962). Thought and language. Cambridge, MA: Harvard University Press.

Vygotsky, L. (1978). Mind and society: The development of higher mental processes. Cambridge, MA: Harvard University Press.

Weber, N. (2008). A comparative analysis of locally based conservation education programs that promote issue awareness and community solutions within Honduras and the United States.(Doctoral Disserta-tion) Retrieved from Dissertation Abstracts International. (v70, (5B). pg 2801, pn: 3360797.)


App

endi

x A

.1. D

AE

T C

odeb

ook

App

endi

x 1:

DA

ET C

odeb

ook

Whe

re F

ound

? De

finiti

ons

SPSS

Var

iable

Nam

e Co

ding

Inst

ruct

ions

Su

rvey

# Su

rvey

ID

surv

ey#

num

ber a

ssig

ned

Hum

an: t

he e

ngin

eer r

epre

sent

ed in

the

draw

ing

is a

per

son

(not

an

objec

t).

Non-

Hum

an:

the

engi

neer

repr

esen

ted

in th

e dr

awin

g is

an

obje

ct (n

ot a

per

son)

. hu

man

0 -N

o m

entio

n 1 -

Per

son

2 -

Non

-Hum

an

99 -

Blan

k St

uden

t as E

ngin

eer:

a st

uden

t mak

es re

fere

nce

to h

im o

r her

self

as b

eing

the

engi

neer

. st

uden

t Na

me:

the

stud

ent g

ives

a p

rope

r nam

e to

the

engi

neer

(e.g

. Jam

es, L

ydia)

na

me

0 -N

o m

entio

n 1 -

Men

tione

d 99

-Bla

nk

Face

: Fac

e of

the

engi

neer

is s

hade

d in

fa

ce

0- N

ot S

hade

d

1- S

hade

d 99

- Bla

nk

Gend

er: h

uman

eng

inee

r was

dep

icte

d as

a m

ale,

fem

ale,

or a

mbi

guou

s (d

rawn

and

/or w

ritte

n)

gend

er

0-N

o M

entio

n 1-

Fem

ale

2- M

ale

3 -

Ambi

guou

s 99

- Bla

nk

HUMA

NS:

Peop

le in

Draw

ing

Grou

p: o

ther

hum

an b

eing

s ar

e re

pres

ente

d in

the

draw

ing

(oth

er th

an ju

st th

e en

ginee

r) gr

oup

0-N

o m

entio

n 1-

Men

tione

d 99

-Bla

nk

Vehi

cle: a

ny ty

pe o

f veh

icle

(e.g

. car

, plan

e)

vehi

cle

Mach

ine:

the

draw

ing

depi

cts

any

type

of m

achi

ne (l

arge

or s

mall

/fou

nd in

any

settin

g and

not

a ve

hicle)

. m

achi

ne

Tool

s (ph

ysica

l labo

r): th

e en

gine

er u

sing

tool

s as

soci

ated

with

phy

sica

l lab

or (h

amm

er, d

rill)

phys

ical

Tool

s (of

fice)

: th

e en

gine

er u

sing

tool

s as

soci

ated

with

offi

ce w

ork

(pen

cil, e

asel)

of

fice

Stru

ctur

es: d

epic

ts a

stru

ctur

e th

at is

a fi

nal p

rodu

ct o

f a c

ivil

engi

neer

ing

desi

gn (h

ouse

, buil

ding,

brid

ge)

stru

ctur

e

Hum

an-

Engin

eere

d Ob

jects

Artif

acts

: dep

icts

obj

ects

ass

ocia

ted

with

the

plan

ning

sta

ges

of e

ngin

eerin

g de

sign

(blue

print

s, dr

awing

s)

artif

acts

Pr

oces

s: a

ste

p-by

-ste

p pr

oces

s re

pres

ente

d, in

clud

ing

at le

ast 2

cle

arly

def

ined

line

ar s

teps

(the

eng

ineer

ing d

esign

pro

cess

, th

e sc

ientifi

c pro

cess

) pr

oces

s

Engi

neer

ing

Verb

s: w

rites

ver

bs th

at a

re a

ssoc

iate

d w

ith e

ngin

eerin

g, e

ven

if th

at c

once

ptio

n of

eng

inee

ring

is n

ot a

cor

rect

co

ncep

tion

(e.g

. fix

cars

, des

ign b

ridge

s) ve

rbs

Engi

neer

ing

verb

s pro

vided

…. w

rite i

n ve

rb2

Why

: pro

vide

s an

exp

lana

tion

for w

hy th

e en

gine

erin

g is

taki

ng p

lace

(int

entio

n wi

th p

rodu

ct de

sign

and/

or cr

iteria

: to

keep

wat

er a

way f

rom

th

e ho

use,

so it

will w

ork)

wh

y

Expl

anat

ion

prov

ided

…. w

rite i

n wh

y2

0-N

o m

entio

n 1-

Men

tione

d 2-

Unc

lear

ly

d

raw

n ite

m

99-B

lank

wr

ite in

…

write

in…

Bene

fit: p

eopl

e or

org

aniz

atio

ns p

ortra

yed

that

will

bene

fit fr

om th

e en

gine

erin

g ha

ppen

ing

in th

e dr

awin

g (a

car o

wner

will

bene

fit fro

m a

car b

eing

fixed

).

bene

fit

0- N

o m

entio

n 1-

Peo

ple/

Fam

ily

2- C

omm

unity

2-

Org

aniz

atio

ns

3- O

ther

99

-Bla

nk

SYST

EM

what

,

why,

wh

o

Bene

fit:

If th

e st

uden

t exp

licitl

y w

rites

who

will

bene

fit fr

om th

e en

gine

erin

g (e

xplic

it), o

r if t

he s

tude

nt im

plie

s w

ho w

ill be

nefit

fro

m th

e en

gine

erin

g (im

plicit

). be

nefit

2 0-

No

men

tion

1-Ex

plic

it 2-

Impl

icit

99-B

lank


App

endi

x A

.1. C

onti

nued

Loca

tion:

Som

etim

es it

is e

xplic

it (w

ritte

n) a

nd s

omet

imes

it is

impl

icit

(impli

ed).

Loca

tion2

0-

No

men

tion

1-Ex

plic

it 2-

Impl

icit

99-B

lank

ENVI

RONM

ENT

Loca

tion:

whe

re th

e dr

awin

g is

taki

ng p

lace

(you

know

that

the

scen

e is

takin

g pla

ce in

a g

arag

e be

caus

e th

e en

ginee

r is f

ixing

a

car,

outsi

de, in

an

offic

e, e

tc.)

loca

tion

Relig

ion:

Chu

rch,

cro

ss, C

hris

tmas

tree

, tem

ple

relig

ion

Socia

l: Hou

se, g

roup

of c

hild

ren,

par

ents

, mar

ket,

gam

es, p

ool

socia

l Po

litica

l: Fla

g, le

ader

po

litica

l Ed

ucat

ion:

sch

ool,

book

, tea

cher

, bla

ck b

oard

, des

k, e

nviro

nmen

tal m

essa

ge

educ

atio

n

H UMA

N MA

NAGE

D

Scien

ce an

d Te

chno

logy

: bus

, car

, TV,

com

pute

r, ho

spita

l, tra

ffic

light

, gun

, chi

mne

y, b

ike

scien

ce

Hydr

osph

ere:

rain

, riv

er, f

ount

ain,

wel

l, st

ream

hy

dro

Lith

osph

ere:

mou

ntai

n, h

ill, ro

cks,

land

, soi

l, pe

ak

litho

NA

TURA

L Ab

iotic

Atm

osph

ere:

clou

ds, m

oon,

sun

, sta

rs, s

ky, b

lack

sm

oke,

gas

es

atm

o Pl

ant:

Tree

, gar

den,

flow

er, p

ark,

law

n, fr

uit

plan

t An

imal:

cow

, dog

, duc

k, a

nim

al s

kull,

bird

s an

imal

Biot

ic

Hum

ans:

hum

an b

eing

s, h

uman

sku

ll (inc

luding

the

engin

eer w

hen

repr

esen

ted

as h

uman

) hu

man

2

0-N

o m

entio

n 1-

Men

tione

d 99

-Bla

nk

DETA

IL

Hum

an M

anag

ed (5

)+ N

atur

al (5

) = #

Pl

ain: le

ss th

an 3

var

iabl

es in

clud

ed in

dra

win

g (0

-3 v

aria

bles

) De

taile

d: m

ore

than

3 v

aria

bles

incl

uded

in d

raw

ing

(4+

varia

bles

)

deta

il 0-

Pla

in

1-

Det

aile

d

99-B

lank

VI

BE

or A

ffect

Feeli

ng o

f Dra

wing

OVE

RALL

: Ne

gativ

e: sa

d fa

ce, “

keep

out

” sig

n Po

sitive

: hap

py fa

ce, “

wel

com

e” s

ign,

env

ironm

enta

l mes

sage

Ne

utra

l: ca

nnot

tell

vibe

0- N

egat

ive

1- P

ositi

ve

2-

Neu

tral

99

-Bla

nk

Field

: any

exp

licit

engi

neer

ing

field

s th

at th

e st

uden

t men

tions

in th

e dr

awin

g or

in th

e co

rresp

ondi

ng s

ente

nces

. fie

ld

See p

age 3

…

Clot

hing

: exp

lore

s th

e cl

othi

ng w

orn

by th

e en

gine

ers

(e.g

. glas

ses,

lab co

at, s

ee fo

llowi

ng p

age)

clo

thin

g Se

e pag

e 3…

Ob

jects

: exp

lore

s th

e ob

ject

s w

ith th

e en

gine

ers

(e.g

. com

pute

r, ro

bot,

see

follo

wing

pag

e)

objec

t Se

e pag

e 3…

At

titud

es/ D

ispos

ition

s: e

xplo

res

the

attit

udes

of t

he e

ngin

eer(s

) ONL

Y. D

oes

the

stud

ent m

entio

n th

e en

gine

ers

feel

ing

a ce

rtain

way

? (e

.g. f

amou

s, m

ad, h

appy

, ner

vous

, nice

) at

titud

e 0-

No

men

tion

1- E

xplic

it 2-

Impl

icit

99-B

lank

At

titud

e pro

vided

: exp

lore

s fe

elin

gs/a

ttitu

des

of en

gine

ers (

fam

ous,

mad

, hap

py, n

ervo

us, n

ice,

)

attit

ude2

wr

ite in

ENGI

NEER

ING

FIE

LD

PORT

RAYE

D

Deta

il in

the E

ngin

eer:

field

+ cl

othi

ng (#

)+ o

bjec

t (#)

+ at

titud

e= #

Pl

ain =

0-3

, Det

ailed

=4+

(s

ame

as a

bove

in “d

etail

”) de

taile

0-

Pla

in

1-

Det

aile

d

99-B

lank

EN

GINE

ERIN

G?

This

cat

egor

y ex

plor

es w

heth

er o

r not

the

stud

ent h

as a

gra

sp o

n th

e co

ncep

t of e

ngin

eerin

g. D

oes

the

stud

ent

unde

rsta

nd w

hat e

ngin

eerin

g is

?

In

dica

tor C

ompo

nent

s: te

amwo

rk, d

esign

pro

cess

, bro

ader

app

licat

ions (

like

teac

hing)

NO U

NDER

STAN

DING

: St

uden

t pre

sent

s no

unde

rsta

nding

of t

he co

ncep

t of e

ngine

ering

?

SOME

: Is t

heir

conc

eptio

n of

eng

ineer

ing st

ill fo

rming

(1 co

mpo

nent

+ so

me

misp

erce

ption

s pre

sent

)?

UNDE

RSTA

NDS:

Doe

s the

stud

ent h

ave

a we

ll-for

med

conc

eptio

n of

eng

ineer

ing (1

+ co

mpo

nent

)

conc

ept

0-N

o m

entio

n 1-

no u

nder

stan

ding

2-

som

e und

erst

andi

ng

3-un

ders

tand

s 99

-Bla

nk


WOR

K (E

NGIN

EERI

NG FI

ELDS

): 0-

N

o M

entio

n 1-

Ae

rona

utic

s an

d As

trona

utic

s 2-

Ag

ricul

tura

l and

Bio

logi

cal

3-

Civ

il 4-

C

hem

ical

5-

C

ompu

ter

6-

Con

stru

ctio

n 7-

El

ectri

cal

8-

Envi

ronm

enta

l 9-

In

dust

rial

10-

Land

Sur

veyi

ng a

nd G

eom

atic

s 11

- M

ater

ials

12

- M

echa

nica

l 13

- Ed

ucat

iona

l 14

- Po

licy

99- B

lank

CLOT

HING

(ada

pted

from

Fra

lick,

Kear

n, a

nd T

hom

pson

200

9):

0-

No

Men

tion

1-

Lab

Coa

t (La

b su

it)

2-

Cra

zy H

air (

Eins

tein)

3-

G

oggl

es/ G

lass

es

4-

Labo

rer’s

Clo

thin

g (o

vera

lls, h

ard

hat)

5-

Busi

ness

Atti

re (s

kirt/p

ants)

6-

C

asua

l (pa

nts/s

hirt)

7-

Oth

er d

etai

ls (h

at, g

loves

, det

ailed

hair

) 99

- Bla

nk

OBJE

CTS

(ada

pted

from

Fra

lick,

Kear

n, a

nd T

hom

pson

200

9):

0-

No

Men

tion

1-

Bu

ildin

g/Fi

xing

Too

l 2-

M

easu

ring

Tool

3-

W

ritin

g To

ol

4-

St

udie

d An

imal

5-

O

ther

Ani

mal

6-

St

udie

d Pl

ant

7-

Oth

er P

lant

8-

R

ock

9-

R

obot

10

- C

ompu

ter

11

- Pa

ssin

g Ve

hicl

e 12

- C

onst

ant V

ehic

le

13-

Flyi

ng V

ehic

le

14-

Roc

ket

15-

Trai

n/tru

ck

16

- O

ther

mac

hine

(tec

hnolo

gy)

17-

Furn

iture

(cha

ir, ta

ble, e

asel)

18

- C

ivil

Stru

ctur

e (m

echa

nic sh

op/g

arag

e, fa

ctory

, fen

ce)

19

- Bo

ok

20-

Sign

s of

thin

king

(thin

king

bubb

le)

21-

Sign

s of

teac

hing

(clas

sroo

m, b

lackb

oard

) 22

- Si

gns

of a

ctio

n (a

rrow

s)

23-

Blue

prin

t/ D

raw

ing

24-

Mod

el

25-

Dip

lom

a

26

- M

ath

(sym

bol, e

tc.)

27-

Che

mis

try (s

ymbo

l, etc.

) 28

- M

edic

ine

(sym

bol, e

tc.)

29-

Met

eoro

logy

30

- Sp

orts

31-

“Dan

ger”

(Kee

p ou

t Sign

, wea

pon)

32-

Oth

er

99- B

lank

NOTE

S:

Eras

ed=

NO

T in

dra

win

g


App

endi

x A

.2. C

odin

g Sy

stem

Pro

gres

sion

journal of pre-college engineering tirupalavanam ganesh ...mnathan/... · 2 j. l. chiu, m. c. linn...

Documents