Figure A. Our definition of the undergraduate STEM community with likely participants of our workshop shaded in gray.

I. Objectives, Need, & Vision

Science in the Classroom,1 a collection of annotated primary research papers and educational resources from the journal Science, is seeking support under the IUSE Exploration and Design Tier for Institutional and Community Transformation to develop educator guides for use in undergraduate Science, Technology, Engineering, and Mathematics (STEM) programs with an emphasis on teaching the nature and practices of science. We will create a workshop for the undergraduate STEM community, defined in this proposal as faculty, pre-service teachers (including participants in teacher-prep programs such as UTeach and PhysTEC, and Learning Assistant programs),2 graduate students, postdocs, and other interested STEM professionals (Figure A), that provides a foundation for teaching the nature and practices of science. The proposal addresses the following needs:

It is necessary to explore additional methods for engaging students in science and engineering practices. How to best teach the nature and practices of science is an active area of STEM education research and policy. Most of this research relates to how students learn in places where they are likely to engage in science and engineering practices, such as laboratories or in the field, and in programs that focus on authentic research experiences. We believe that the incorporation of appropriately annotated, primary scientific literature into science classrooms can be another means for teaching the practices of science.

Alignment to STEM frameworks that emphasize the nature of science should be included in educator guides. The paradigm shift from content memorization and toward promoting a deeper understanding of the nature of science is seen in major STEM learning frameworks. By developing educator guides aimed at placing primary literature in the context of the nature and practices of science, we can increase the utility of annotated primary literature, another means for teaching the practices of science, in undergraduate STEM classrooms.

The benefits of our proposal are three-fold.

1. Through the workshop, participants (Figure A) will learn how to develop high quality educator guides focusing on the nature of science and science practices, a skill that will be useful for STEM education purposes but also transferable to a wide variety of STEM careers.

1 http://scienceintheclassroom.org/2 https://institute.uteach.utexas.edu/ , http://www.phystec.org/webdocs/AboutPhysTEC.cfm , http://fiulearn.fiu.edu/la-pltl-info/

1

2. The educator guides developed by participants (Figure A) will be freely available for use by teachers and students throughout the STEM education community, ultimately increasing the accessibility of annotated primary literature resources that have been vetted by educational experts.

3. Engaging with the nature and practices of science in a teaching context has been shown to contribute to the improvement of essential research skills. Therefore, participants in our workshop (Figure A) may improve upon their own research skills, essentially becoming better scientists.

Intellectual merit: Our proposal is a creative approach for adoption of education research (in the form of STEM education frameworks) into disciplinary teachings. This adoption will take place at two levels. First, members of the undergraduate STEM community will be provided a professional development opportunity in using primary literature to teach the nature and practices of science. We hypothesize that successful completion of our workshop will result in a greater awareness and understanding of the benefits of teaching the nature and practices of science. This method of teaching science is not only important in the classroom, but it is also valuable in mentoring and independent research interactions.

Recent research has shown that when teaching in a context of inquiry-based thinking rooted in the nature of science, graduate students and postdocs reinforced their own learning and, by extension, became better researchers (Feldon et al. 2011). Therefore, providing the undergraduate STEM community a foundation in teaching the nature and practices of science will also improve the quality of the research training and science careers participants are involved in. We hypothesize that workshop participants will enhance their own science practices, including experimental design skills and expert-like thinking skills.

The second level of adoption results from the educator guides that will be created as participants complete the workshop (Figure 9). These educator guides will accompany SitC resources and will be freely available to members of the undergraduate STEM community, further allowing for the proliferation of annotated primary literature resources that have been vetted by educational experts.

Broader impacts: Our proposal will increase the diversity of STEM education opportunities and resources available to the undergraduate STEM community. The proposed professional development opportunity will in turn lead to a valuable set of educator guides allowing for a wider participation in the use of primary literature as an educational tool. SitC resources and workshop materials will be freely available online, allowing for widespread participation and distribution throughout the STEM community. Additionally, an open-access, digital community of scientific professionals with a foundation in science education as it relates to the nature of science will be inaugurated and cultivated.

SitC resources are designed for use by both STEM majors and non- majors, allowing for use in a variety of classroom settings. This flexibility greatly increases the potential of SitC resources to influence science literacy among undergraduates and to increase the awareness of the role science has in society for this diverse population.

II. Background Information

Science in the ClassroomScience in the Classroom (SitC), a collection of annotated research papers, makes Science content more accessible to students and educators. Science is a leading journal of original scientific research, global news, and commentary. Science’s position as a place for the world’s best scientists to communicate their results and engage in debate, across all fields, can enable students to see what scientists do. Through reading and deconstructing scientific papers, students can gain an understanding of how scientists design

2

Figure 1. An example of the learning lens tool being used in a SitC annotated resource.

their experiments and present their results, essentially allowing students to experience the logic of getting from a set of data to a new conclusion.

SitC uses the original text of Science articles along with a “learning lens,” designed to provide students tools to use for interpretation. The “learning lens” is used to selectively highlight different parts of the text and is composed of seven headings: Glossary, Previous Work, Author's Experiments, Conclusions, News and Policy Links, Connect to Learning Standards, and References and Notes, which are color-coded to match the corresponding text of the research article. For example, an annotated glossary term, when clicked on, will produce a pop-up box containing the definition of the word (Figure 1).

Annotations provide an educational scaffold that helps students deconstruct scientific papers, giving them an understanding of experimental design and the presentation of results and conclusions. With the scientific language barrier removed, students are able to see how the authors identified a question, how data was collected and analyzed, and how the next question(s) was proposed, introducing them to the non-linear and iterative nature of science.

The nature and practices of science as essential elements of STEM educationIn recent years, Science, Technology, Engineering, and Mathematics (STEM) education has shifted away from content memorization and toward promoting a deeper understanding of the nature of science. This paradigm shift is seen in major STEM learning standards and frameworks, including Vision and Change for Undergraduate Biology Education (V&C), the Next Generation Science Standards (NGSS) and its parent document, A Framework for K-12 Science Education (Framework), and the Advanced Placement (AP) Sciences.3 While these frameworks were developed with different audiences in mind, they share the fundamental goal of engaging students in scientific and engineering practices in order to enhance their understanding of the nature and practices of science.

What is the most effective way to teach the nature of science? Research on this topic relates mostly to how students learn through authentic research experiences, such as CUREs (course-based undergraduate research experiences) and UREs (undergraduate research experiences) (Auchincloss et al. 2014, Linn et al. 2015). Compared to a traditional science laboratory course, CUREs allow for students to engage in iterative work, such as troubleshooting and problem solving, that more closely resembles the nature and practices of science (Auchincloss et al., 2014). Similarly, there are many positive reports of UREs,

3 Vision and Change http://visionandchange.org/; Next Generation Science Standards http://www.nextgenscience.org/; A Framework for K-12 Science Education https://www.nap.edu/read/13165/chapter/1; AP Sciences https://advancesinap.collegeboard.org/stem

3

(A) I am waiting on a decision about a job as editor of a review journal. In the interview, they thought that the SitC program was really cool, and I think it’s probably what got me in the door. So I’m really glad to be a part of it.

(B) Overall we thought it was a very thorough, clear and exciting engagement with the study and that the material will be a valuable educational resource. It was also great to see the experiments suggested by the students some of which we are actually currently conducting.Figure 2. (A) Feedback from an annotator on the annotation experience; (B) Feedback from an author on the experience of having their paper annotated.

including students learning to “think and work like a scientist,” and increased interest among students in pursuing further education or careers in science (Laursen et al., 2010).

However, CUREs and UREs require a lot of time, money, and effort. Therefore, additional tools and opportunities should be considered in order to achieve a wider variety of complementary opportunities for teaching the nature of science and engaging students in science and engineering practices (National Academy of Sciences, 2012). We believe that the incorporation of annotated primary literature in the classroom represents one of these opportunities.

A recent survey of graduate students asked to reflect on their undergraduate experience cited a need for greater exposure to the research process. Realizing that the research process included more than just lab work, one student stated, “I had little if any undergraduate research experience in a lab or with reading scientific papers” (American Institutes for Research, 2011). Consistent with this statement, a growing body of research shows that scientific primary literature is a valuable and useful tool for STEM education and that it can also be used to teach the nature and practices of science (Hoskins et al. 2007, Murray 2014, Round and Campbell 2013). For example, discussing primary literature in a classroom setting engages students in the scientific practice of discussion and debate around interpretations of experimental data (Hoskins et al. 2007). Pedagogically, teaching with primary literature has been shown to promote critical thinking, increase students’ experimental design ability, and increase students’ positive attitudes about science and scientists, all gains similar to what is seen with CUREs and UREs (Hoskins et al. 2011).

III. Prior work in developing the project

Lessons learned from Science in the Classroom

Previous NSF funding is described in detail in section V found on page 13.

Development of SitC resources is a collaborative process across the STEM community. Currently, annotated papers and accompanying materials (including an educator guide) are written by volunteer graduate students, postdocs, and other STEM professionals, and reviewed for accuracy by the original paper’s author(s). Annotators provide the deep science background necessary for annotations, and paper authors make sure annotations are scientifically accurate and the original research is well-represented. SitC staff ensures that the science is communicated at an appropriate, student-friendly level.

Writing annotations is an opportunity for those in the STEM community to gain experience producing educational resources and to develop valuable communication skills. For authors, having their paper annotated offers a chance to communicate their research to a broader audience. Both authors and annotators have expressed that the process has been valuable for their career development and for their own scientific research (Figure 2).

Following the annotation and review process, annotators develop an accompanying educator guide. These guides contain a summary of the research and its conclusions, connections to learning standards, and

4

“The Teacher Guide training needs to be improved; they were always the most difficult things for me to write when I was doing annotations.”

“I didn't find the section on "core learning standards" to be all that helpful, both in class and in terms of the annotations themselves.”Figure 3. (A) Feedback from annotators on their experience with our current educator guide training

discussion questions. However, as annotators are generally not familiar with STEM education standards and frameworks, their experience with our annotator training and the usefulness of the completed guides to teachers varies (Figure 3). To date, we have not actively involved those trained in best practices in STEM education in SitC resource development. We regard this as a missed opportunity and have developed this proposal as a way to address this.

A recent survey of 28 educators (high school and undergraduate level) relating to their use of SitC resources indicated that they choose educational resources based on connections to the curriculum (62% of respondents said they would not use a resource that did not have clear connections to a curriculum). The survey also showed that there is a need to improve out-of-the-box utility of SitC educator’s guides by more explicitly citing alignment to STEM frameworks that emphasize the nature of science and by providing more practical suggestions for classroom use (Figures 4 and 5).

Implementing effective practices of professional development (PD)It is clear that there is a need for a more detailed educator guide to accompany SitC resources. To accomplish this, we need to provide our annotators with robust training in the nature of science vis-a-vis STEM education frameworks. This type of training is not trivial; simply learning about these frameworks is not enough to gain the insight needed to produce high quality educator guides. Effective PD in this area would require active involvement in creating and analyzing educational material (Garet et al. 2001).

Research on educator PD indicates that focusing on specific content, engaging teachers in active learning, and enabling the collective participation of educators results in success (Wilson 2013). We plan to include all of these characteristics in our workshop. First, our workshop will be specifically focused on how to connect the nature and practices of science to the specific content found in primary literature. Active learning and collective participation will take place as participants work together to develop and review the educator guides they create during the workshop.

5

“Correlation guide for the AP Curriculum or other suggested ways to implement papers in curriculum.”

“No ‘canned’ general instructions or blanket statements in TEACHER GUIDE (sic) (‘use as homework’ or ‘give extra credit’)look as if written by someone who doesn’t understand curriculum.

“More in-depth notes in teacher guide that are activity-specific.”Figure 4. Feedback from a recent survey in response to the question, “What, if anything, could be done to improve the annotated papers or their accompanying resources?”

(A) “[To] learn how scientists think and how they analyze and make meaning of the results and link it back to the broader scientific concepts.”

“To show the diversity of methods used by scientists and debunk the version of a single, universal scientific method.”

(B) “Time constraints require every activity that I select to be carefully aligned to curriculum. I teach the process and nature of science through curriculum: few connections waste time..”Figure 5. Feedback from a recent survey in response to the questions (A) “Are there important reasons … why you would use or are using primary literature in your classroom?” and (B) “Would you use a research paper that has very few clear connections to the curriculum? Please briefly explain why or why not.”

The formation of educator communities has been shown to increase the value of teacher PD. Specifically, supportive leadership, an inclusive group dynamic, and trust and respect among members have been shown to lead to a successful educator community (Vangrieken et al. 2017). Additionally, the “anytime, anywhere” availability of online networks allow for discussion and responses of educators’ diverse interests and needs (Trusta et al. 2016 ). As part of our proposal, we intend to bring our participants together into an online community with Trellis,4 a new platform for communication, collaboration, innovation, and experimentation across the entire scientific community (section 1A, page 7).

We propose to leverage the expertise of the STEM education community (Figures A and 9) to develop educator guides for SitC resources. We believe that science education experience provides a value-added component to any level of research training, as recent research has shown that teaching experience can substantially improve gradute students’ and postdocs’ research skills (Feldon et al. 2011). When teaching inquiry-based thinking rooted in the nature and process of science, graduate students and postdocs reinforced their own learning and, by extension, became better researchers (Feldon et al. 2011). Therefore, providing SitC annotators with a more intensive foundation in teaching the nature of science will not only result in the production of educator guides, but may also improve the quality of the research training annotators are involved in.

III. Proposed Plan of Action

1. Design a professional development workshop that provides an introduction to the nature and practices of science in the context of primary literature.

2. Recruit and train participants to form an active community of practice.

3. Produce high quality, classroom application oriented educator guides that increase accessibility to annotated primary literature.

4. Evaluate the workshop and resulting educator guides.

IV. Specific Aims

Specific Aim #1: Design a professional development workshop that provides an introduction to the nature and practices of science in the context of primary literature. We have developed our workshop in conversations with our Advisory Board (AB) (listed in section VII, page 13). We have also drawn on our own experience with training annotators to produce the current version of our educator guides and on results from a recent SitC resource survey conducted with educators.

Although SitC is a resource intended for an undergraduate audience, with a natural connection to the Core Competencies of V&C, there is value in connecting to the AP Science standards and Framework. The AP Science standards correspond to introductory level undergraduate work, as evidenced by universities accepting AP credits, and the practices included in Framework directly parallel the Core Competencies of V&C. Both AP and Framework provide additional background and context for participants to implement information they learn in the workshop, resulting in a more comprehensive training that is nonetheless rooted in undergraduate-level expectations and competencies.

We believe that a workshop involving three different learning frameworks is unique at this time, as most current PD opportunities focus only on individual frameworks. Additionally, to our knowledge, very few

4 https://www.trelliscience.com

6

existing PD opportunities focus on building skills in the context of creating educational resources, with an explicit focus on the nature and practices of science, at the higher education level. 1A. Outline of the training workshop. The workshop will be conducted in the form of recorded sessions available online with additional real-time work in small cohorts that will allow for discussion, peer review, and a more substantive analysis of issues in applying the workshop content to practice. We anticipate each workshop will have at least six participants. Participants will work in pairs on most tasks and will come together as needed for discussions and peer-review opportunities. We anticipate the workshop will take 10-12 hours to successfully complete.

Expert teachers will be recruited (using our AAAS networks) to develop (in discussions with our AB) and present the content in the recorded sessions. The team overseeing the AAAS Career Development Center5 has extensive experience producing this type of online course and has agreed to assist us.

We will connect our workshop training to Trellis,4 a new platform for communication, collaboration, innovation, and experimentation across the entire scientific community. Developed and housed within AAAS, Trellis is open to any member of the scientific community; AAAS membership is not required. Through Trellis, workshop participants will have a central location (known as a group), where recorded sessions can be housed, documents can be shared, and discussions can take place.

In keeping with research on maximizing the effectiveness of PD, our module will focus on specific content, engage participants in active learning, and incorporate collaborative design and analysis (Wilson 2013). We will seek advice from our AB in the production of workshop videos, the compiling of readings, and the development of rubrics for reflective work.

The workshop will begin with a brief introduction of SitC intended to provide participants with an overview of the goals of SitC, an understanding of how SitC resources are developed and produced, and an explanation of why high quality, classroom application oriented SitC educator guides are needed.

The first session of the workshop (Figure 6) will introduce teaching the nature of science through science practices. This moves STEM education toward a type of three-dimensional learning, where students use science practices to understand key concepts. Specifically, an emphasis on the nature of science requires a shift from “learning about scientific ideas to figuring out scientific ideas that explain how and why phenomena occur” (Reiser 2013). This session will also include an overview of the frameworks and standards covered in sessions 2-4, and emphasize what these standards have in common.The session will begin with a video explaining, for example, the difference between simply telling students the fact that all living things are made of cells versus engaging students in conducting an investigation to provide evidence that living things are made of cells (Reiser 2013). The video will be supplemented by assigned readings and reflective work on participants own experience with teaching and learning the nature of science.

Topic Participants will be able to: Activity1 Overview of teaching the

nature and practices of science and STEM frameworks and standards

Explain the difference between teaching science as facts and teaching science as a practice

(1) Video: The nature of science and the current STEM education paradigm

(2) Readings: the nature of science in STEM education

(3) Reflective work: teaching and/or learning the nature of science.

5 https://careerdevelopment.aaas.org/

7

(4) Discussion with another participant.Figure 6. An overview of Session 1.

The next three sessions (Figure 7) will connect the nature of science to the three STEM frameworks we address (V&C, AP Sciences, and Framework), and explain how these have revolutionized STEM teaching and learning. Participants will learn where to find resources and information about each of these frameworks, how they require students to demonstrate their ability to apply science practices in articulating understanding of key concepts, and how these practices contribute to the scientific enterprise.

Topic Participants will be able to: Acvitity

2

Introduction to Vision and Change for Undergraduate Biology Education (V&C)

Locate V&C resources and relate them to the nature of science

(1) Video: V&C Core Competencies and the nature of science

(2) Readings: Selections from the V&C report and research on V&C initiatives

(3) Reflective work: connecting Core Competencies to key concepts

(4) Peer review and discussion

3Introduction to Advanced Placement (AP) Science Practices

Locate AP resources and relate them to the nature of science

(1) Video: AP Science practices and the nature of science

(2) Readings: Selections from published research on AP initiatives

(3) Reflective work: Teaching AP science practices that incorporate key concepts

(4) Peer review and discussion

4Introduction of the Scientific and Engineering Practices

Locate Framework resources and relate them to the nature of science

(1) Video: Framework and the nature of science

(2) Readings: Selections from published research on Framework initiatives, including "A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas"

(3) Reflective work: Engaging students in understanding a phenomenon by using 3-dimensional learning

(4) Peer review and discussionFigure 7. An overview of sessions 2-4.

Each session will begin with a video introducing each framework, followed by a connection to the nature of science. The reflective work activity in session 2 will use modified rubrics developed by the PULSE program.6 Participants will fill in these rubrics with specified ideas or methods for student activities that would introduce key concepts using a Core Competency from V&C. The reflective work activity in session 3 will be similar, with participants selecting a different key concept (ideally outside their discipline) for additional practice in linking key concepts to a practice.

In the reflective work activity in session 4, participants will be asked to write a Learning Performance that will demonstrate student understanding of the key concept. For example, to understand the key concept “the Moon is visible during the day and shows different amounts of the surface lit” using one of the Framework science practices, students would, for example, “develop and use a model of the sun, earth, and moon to describe the cyclical pattern of lunar phases.”

6 http://www.pulsecommunity.org/

8

It is important to note that the practice of developing and using models, used in the previous paragraph as an example, is found in all three frameworks included in this workshop. Learning to use this practice in a variety of settings connected to three different STEM frameworksis what we believe makes our workshop valuable and unique.

In each of sessions 2-4, participants will participate in peer review of their reflective work and discussion with other workshop participants.

The fifth session (Figure 8) will describe how these learning frameworks connect to primary literature, and why primary literature is an ideal tool for teaching the nature of science. This session will begin with a video that introduces the goals of SitC and how our resources are a way to present students with the nature of science. We will also cover several additional ways to use primary literature as a teaching and learning tool (Hoskins et al. 2007, Murray 2014, Round and Campbell 2013).

Participants will design a method to link key concepts and the nature of science using primary literature. This will be similar to the reflective work in sessions 2-4, as an adapted rubric encompassing criteria from all three frameworks, with additional focus on where and how primary literature can be incorporated, will be provided. Finally, trainees will participate in peer review and discussion with other participants.

The final session (Figure 8) will introduce SitC educator guides, and describe how to synthesize information from the first five modules and organize it into SitC’s educator guide template (Figure 10, found on page 14). Participants will connect specific content from Science papers to the nature of science using the frameworks presented in sessions 2-4. This session will ultimately result in the development of new educator guides to be shared with the STEM community.

Topic Participants will be able to: Acvitity

5How do these learning frameworks connect to primary literature?

Use primary literature as a vehicle for teaching the nature of science.

(1) Video: Primary literature and the nature of science

(2) Reflective work: Teaching the nature of science with primary literature

(3) Peer review and discussion

6 Introduction to SitC educator guides

Connect specific content from Science papers to the nature of science using the three frameworks presented earlier.

(1) Video: Creating a SitC educator guide

(2) Resource development: In pairs, participants develop an educator guide

(3) Peer review and discussionFigure 8. An overview of sessions 5 and 6.

1B. How does this workshop advance our knowledge of teaching the nature of science? Our training will provide participants a small amount of pedagogical instruction. We will focus on presenting a background in current STEM learning frameworks and describe how they promote the teaching of the nature and practices of science. Our workshop will be a foundation for STEM professionals who wish to learn more about what effective implementation of STEM education entails. Full evaluation details are found in Specific Aim #4 (page 11).

Specific Aim #2: Recruit and train participants to form an active community of practice.

9

2A. Recruitment of participants. When recruiting volunteer annotators for SitC, we emphasize the opportunity for PD and experience with science education. This opportunity has been well-received (we recruit around 50 volunteers each year, Figure 2), and we anticipate that a similar, more focused opportunity will have a comparable, if not larger, audience.

While the workshop will be open to all members of the STEM community (Figure 9), we anticipate the majority of participants will be pre-service teachers (including participants in teacher-prep programs such as UTeach and PhysTEC, and Learning Assistant programs), graduate students, and postdocs. To recruit participants for our first workshop, we will draw from a pool of around 100 past annotators who have indicated high interest in reviewing, editing, and creating additional SitC resources. We hope to recruit at least 20 of these annotators for the first round of training. Additional recruitment will take place through our AAAS networks, our collaborator networks, and announcements posted to relevant listservs and social media outlets. We will reach out to faculty through our AAAS contacts and relevant professional societies, and to teacher-prep programs such as UTeach and PhysTEC. To be eligible for participation, participants will need a STEM background and an interest in science education.

2B. Incentives for participation. We receive high interest in our current annotator training program and participants are pleased with their time investment (Figure 2). Volunteers who write annotations have their name and website listed on our webpage and feedback suggests that the main motivation for participation is inclusion of this citation on a CV or résumé. In addition to being listed on the SitC website, we will provide successful workshop participants with a certificate of completion from AAAS and Science and provide a modest $100.00 stipend. This stipend level is based on our belief that the majority of participants will be pre-service teachers and members of teacher-prep programs and on the workshop requiring 10-12 hours of work.

Finally, as part of the training is the development of an educator guide to accompany a SitC resource, each participant will have a high-quality, tangible, and classroom-ready deliverable that they will be able to include in a portfolio or teaching packet.

2C. Developing a community of practice. Discussions and peer review of educator guides will take place on Trellis,4 AAAS’s digital communication and collaboration platform for the science community. Workshop participants will be able to discuss each session, share files with each other, and review each other’s work. Not only is Trellis ideal for connecting workshop participants, it will also allow for communication with fellow scientists and educators beyond the scope of the workshop.

Specific Aim #3: Produce high quality, classroom application oriented educator guides that increase accessibility to annotated primary literature.

3A. The creation of “living” resources. A unique aspect of our workshop is that participants will develop a product (an educator guide) as part of their training. The first group of workshop participants will use as examples three prototype educator guides we have created and refined through feedback sessions with our AB (Figure 10, page 14). As we conduct successive workshops, participants will create more educator guides for existing resources, which will be available as examples for future workshops and for use by the STEM community. A schematic of the training flow is pictured in Figure 9.

Currently, we have over 80 SitC resources posted with additional resources going online roughly twice a month. All of these resources will need an upgraded educator guide, as will all future SitC resources being developed. Workshop participants will be able to select an annotated paper of their choice and develop an accompanying educator guide as part of Session 6. As workshop training proceeds, this will

10

create many new examples from which participants can learn as well as provide more completed guides that can be used in the undergraduate STEM community.

Successive iterations of the training will ultimately create a community that can continue to review and refine SitC materials and serve as mentors and reviewers for future workshops. This results in a set of “living” resources that will be continuously improved upon. This model carries the dual benefits of providing practical, collaborative PD for participants and producing high-quality educational resources.

3B. Quality control. Together with our AB, we will as one of our first steps develop a rubric for what we expect the educator guides to include. Before posting, all SitC educator guides will be reviewed, using this rubric, by a master educator. This is to ensure that the educator guide meets SitC requirements and is ready for classroom use. Educator guides that are not approved by the master educators will be send back for editing until they meet SitC requirements.

A cohort of master educators will be recruited for this part of the production process and they will be compensated with a modest stipend ($100.00). We will recruit interested master educators through our AAAS list serves and through our colleague’s recommendations.

Specific Aim #4: Evaluate the workshop and resulting educator guides.

11

Figure 9. An overview of the proposed workshop and how it involves the undergraduate STEM community. The curved arrows indicate places where interactive development of resources and peer review will take place.

4A. Research Plan. To assess the effects of successfully completing the workshop, as well as the utility of the newly created educator guides, our evaluation will focus on the following research questions:

1. How does participating in our workshop influence the teaching philosophies and practices of participants?

2. Does participating in the workshop improve the science practices of the participants themselves?

3. Does participating in the workshop influence participant’s attitudes towards science education?

4. Do the resulting educator guides successfully highlight the nature and practices of science?

4B. How does participating in our workshop influence the teaching philosophies and practices of participants? We hypothesize that participants will be more aware of both the reasoning behind teaching the nature and practices of science as well as how to incorporate this kind of teaching into their STEM education environments. To assess changes in teaching philosophy from the workshop, we will adapt the Teaching Practices Inventory (TPI) to include questions relating to participants’ self-reported perspective of the nature of science in a teaching and learning context (Wieman and Gilbert 2014). Participants will take this modified TPI before beginning the module and again upon completion.

4C. Does participating in the workshop improve the science practices of the participants themselves? We hypothesize that successful completion of this workshop will result in participants becoming better scientists. Therefore, we will collect baseline measurements on participants’ experimental design skills by adapting surveys such as the Biological Experimental Design Concept Inventory (BEDCI) and the Inquiry for Scientific Thinking and Reasoning (iSTAR)7 (Deane et al. 2014). Further, we will collect baseline measurements in expert-like thinking skills using CLASS (The Colorado Learning Attitudes about Science Survey) (Semsar et al. 2011). We will give identical post-surveys to participants upon completion of the workshop.

We anticipate that some participants will be faculty and other advanced scientists and that any changes in these skills (of which they are already experts) may be difficult to measure. We anticipate seeing a much larger change with participants who are early in their scientific careers.

4D. Does participating in the workshop influence participant’s attitudes towards science education? We have a unique opportunity to evaluate how participating in our workshop influences attitudes about science education. We will use the same survey we currently give our annotators that asks about the overall training experience, including changes in their beliefs about the relative importance of various pedagogical techniques as well as changes in their attitude, confidence, knowledge, and commitment to science education. We are especially interested in how workshop participation impacts the pre-service teacher population.

4E. Do the resulting educator guides successfully highlight the nature and practices of science? All educator guides will be reviewed by a master educator using a rubric developed by the SitC team and our AB. Once an approved educator guide is available, we will return to our original survey pool of teachers and ask for their input on the revised guides. Additionally, we will plan a second survey, using similar questions, for a group of teachers who have never worked with SitC resources to receive their input. Survey results will be reviewed by the SitC team and the AB and feedback will be included in any updates of workshop materials.

7 http://www.istarassessment.org/

12

V. Previous NSF fundingSitC has been previously funded by three NSF grants:

1043998, awarded 1/1/2010 for $199,991.00, allowed for the development of a prototype SitC website for beta testing. Our first website was reviewed and evaluated by approximately 200 members of the STEM education community. No publications were produced under this award.

1224661, awarded 08/19/2012 for $474,178.00, allowed for the full development of our website and a sustainable workflow for developing new SitC resources. With this funding, we implemented the website improvements collected during our beta testing, annotated 24 Science papers, and began outreach. This award allowed us to launch http://scienceintheclassroom.org and its sister site http://portal.scienceintheclassroom.org/. No publications were produced under this award.

1525596, awarded 09/21/2015 for $1,408,138.00, allows for further development and expansion of SitC, including the addition of including the STEM education community in the development of annotated papers, beginning to partner with other non-profits and STEM education partners, and wide dissemination of the website and the resources it includes. Additionally, we are improving our annotator training and beginning research on how students and educators are using SitC resources to improve STEM education. We anticipate 2-3 publications to result from this award.

VI. Proposed Timeline (36 months)

Timeline

Year 1 Year 2 Year 3

Month

1-4 5-8 9-12 1-4 5-8 9-12 1-4 5-8 9-12

Develop rubric to evaluate SitC educator guides

Production of training videos

Recruit participants for workshops

Offer workshop

Production of new educator guides

Attend conferences and conduct outreach

Gather feedback on educator guides and workshop

Review and update rubrics and workshop materials

VII. The Science in the Classroom team

AAAS Staff Shirley M. Malcom, PhD, director of the AAAS Directorate for Education and Human Resources

(EHR) Programs, will serve as the Principal Investigator for this proposal. Dr. Malcom brings more than 35 years of experience overseeing and implementing education projects at AAAS which include increasing access to education as well as enhancing public science literacy. As Directorate Head, she also oversees several programs that can be used to expose educators to the SitC initiatives.

Shelby Lake, Program Associate, joined SitC in February 2016. Mr. Lake has overseen the posting of over 35 SitC resources and is familiar with the development process and editing requirements for SitC resources. Mr. Lake has also been involved with volunteer recruitment and management.

13

Project Director, to be named, will be hired in November 2016. The Project Director will manage the day-to-day operations at AAAS and will be responsible for all grant budgeting and reporting.

Florida International University (FIU) Staff Melissa McCartney, PhD, Assistant Professor in the STEM Transformation Institute (starting January

2017), will serve as a Co-PI and Research Coordinator. Dr. McCartney brings 7 years of experience as an Associate Editor for Science and as a Senior Project Director for SitC at AAAS. Dr. McCartney will oversee the development of workshop materials and will conduct relevant evaluations in order to measure the teaching and learning gains taking place. Dr. McCartney will also recruit participants from the teacher prep and Learning Assistant programs at FIU.

Advisory Board Sheri Klug Boonstra, Director of the Mars Education Program, Arizona State University Don Boonstra, STEM Education Consultant, sySTEMec LLC Rebecca Vieyra, K-12 Program Manager, American Association of Physics Teachers Dr. Jo Ellen Roseman, Director for Project 2061, AAAS Dr. Ellie Rice, Director, Quantitative & Science Center, Franklin and Marshall College Jennifer Stancill, Graduate Student, Vanderbilt University

Figure 10. Excerpts from a prototype educator guide showing connections to V&C, AP Sciences, and Framework. This prototype was developed in conversations with our AB and is connected to a specific SitC resource on the GFP protein. The guide includes a learning performance statement, explicit connections to major STEM frameworks and standards, and activities and questions to aid educators in integrating the SitC resource into their classroom.

Learning Performance: Students will evaluate the use of green fluorescent protein as a solution to the real-world problem of tracking the activity of proteins.

AP Science Practices Framework Science and Engineering Practices

Science Practice 4 (SP4)The student can plan and implement data collection strategies in relation to a particular scientific question. (Note: Data can be collected from many different sources, e.g., investigations, scientific observations, the findings of others, historic reconstruction and/or archived data.)

Science Practice 7 (SP7)The student is able to connect and relate knowledge across various scales, concepts and representations in and across domains.

Planning and Carrying Out Investigations (SEP3)Plan an investigation … to produce data to serve as the basis for evidence as part of building and revising models, supporting explanations for phenomena, or testing solutions to problems. Consider possible confounding variables or effects and evaluate the investigation’s design to ensure variables are controlled.

Engaging in Argument from Evidence (SEP7)Evaluate the claims, evidence, and/or reasoning behind currently accepted explanations or solutions to determine the merits of arguments.

Make and defend a claim based on evidence about the natural world or the effectiveness of a design solution that reflects scientific knowledge and student-generated evidence.

Obtaining, Evaluating, and Communicating Information (SEP8)Critically read scientific literature adapted for classroom use to determine the central ideas or conclusions and/or to obtain scientific and/or technical information to summarize complex evidence, concepts, processes, or information presented in a text by paraphrasing them in simpler but still accurate terms.

Communicate scientific information (e.g., about phenomena and/or the process of development and the design and performance of a proposed process or system) in multiple formats (including orally, graphically, textually, and mathematically). (HS-LS4-1).

14

Vision and Change for Undergraduate Biology EducationCore Competencies and Disciplinary Practices

Next Generation Science StandardsUnderstandings About the Nature of Science

Ability to apply the process of scienceDesign scientific process to understand living systems: observational strategies, hypothesis testing, experimental design, evaluation of experimental evidence, and developing problem-solving strategies.

Ability to communicate and collaborate with other disciplinesCommunicate biological concepts and interpretations to scientists in other disciplines: scientific writing, explaining scientific concepts to different audiences, team participation, collaborating across disciplines, and cross-cultural awareness.

Scientific Models, Laws, Mechanisms, and Theories Explain Natural PhenomenaA scientific theory is a substantiated explanation of some aspect of the natural world, based on a body of facts that have been repeatedly confirmed through observation and experiment and the science community validates each theory before it is accepted. If new evidence is discovered that the theory does not accommodate, the theory is generally modified in light of this new evidence.

Scientific Investigations Use a Variety of MethodsScientific inquiry is characterized by a common set of values that include: logical thinking, precision, open-mindedness, objectivity, skepticism, replicability of results, and honest and ethical reporting of findings. (HS-LS1-3)

3. Activities for interactive engagement

Writing an abstractStudents write a new abstract for the article at a grade-appropriate reading level.

SEP8SP7

Locating this study in the larger fieldStudents use the list of references to explain how this research builds on the published work on at least one other independent group of scientists. Students will evaluate whether data from this research supports or contradicts previous conclusions, and reflect on the statement that scientific knowledge is a “community effort.”

SEP7

Results and conclusionsStudents diagram the each of the experiments presented in the study (divided up by figure, if appropriate). They then consider the results depicted in each figure, and how these results support the conclusions of the study.

SEP8SP7

The next stepsStudents design a follow-on experiment to this study that either addresses flaws or unanswered questions, or builds on it to explore a new question.

SEP3SP4

4. Discussion questions1. Why is it important that GFP does not require cofactors or exogenous

substrates? How does this affect its usefulness as a reporter?LS1.A, ETS1.BStructure and FunctionEK3.B.2, EK3.B.1

2. Before GFP, researchers used a class of enzymes called luciferases as reporters. A luciferase catalyzes the oxidation of a protein called luciferin, which produces light. What are the relative advantages of using a reporter like luciferase, which requires an additional step?

SEP7LS1.A, ETS1.BSystems and ModelsEK3.B.2, EK3.B.1

3. Since the discovery that GFP works in other species, many new fluorescent proteins have been engineered for laboratory use. These proteins are similar to GFP in that they do not require cofactors or exogenous substrates, but they emit different colors of light when they are activated. What are some of the advantages of having many colors of fluorescent proteins available?

SEP3ETS1.BSP4

15

16