web viewcheck here if your website addresses the ... contact person and email: ellen metzger,...

SJSU Annual Program Assessment FormAcademic Year 2015-2016

Department: Science Education ProgramProgram: M.A., Science EducationCollege: ScienceWebsite: http://www.sjsu.edu/scied/academic_programs/masters/X Check here if your website addresses the University Learning Goals. Program Accreditation (if any): Contact Person and Email: Ellen Metzger, [email protected] of report: May 20, 2015

Part A1. List of Program Learning Outcomes (PLOs)

1 – To enhance student’s depth and breadth of understanding of selected topics in science education. (See Table 1)

PLO 1.1 Students will be able to synthesize primary literature from science education research and apply how it fits to their project.

PLO 1.2 Students will demonstrate knowledge of at least two areas (e.g. inquiry based instruction, learning theory, assessment) that are related to, or supportive of research for their project.

2 – To enhance communication skills, both written and oral, in science education discourse.

PLO 2.1 Students will present science and science education content in the form of graduate seminars or in the oral defense of their project (also known as the culminating experience).

PLO 2.2 Students will organize and write the results of their project in a manner consistent with standards in professional science education publications.

2. Map of PLOs to University Learning Goals (ULGs)

Program Learning Objectives

ULG 1.1

ULG 2.1

ULG 2.2 ULG 2.3

ULG 3.1

ULG 3.2

ULG 4.1

ULG 4.2

ULG 4.3

ULG 5.1

ULG 5.2* Course

1.1 X X X N/A X X X X X SCI 220

1.2 X X N/A X X X X

SCI 173, SCI 298/299

2.1 X X N/A X X

SCI 220, SCI 298/299

2.2 X X N/A X XSCI 298/299,

X indicates that the ULG is tangentially addressed by the PLO.

3. Alignment – Matrix of PLOs to CoursesTable 1. Curriculum Map of Program Learning Outcomes Addressed by Required Courses for the MA in Science Education. The PLOs are listed along the top of the table and the course(s) that address the PLOs are listed on the side.

Y indicates that the PLO is addressed by the course.

1.1 1.2 2.1 2.2

SCI 220 Y Y Y

SCI 205 Y Y Y

SCED 173

Y

Sci 298/299

Y Y Y

4.Planning – Assessment ScheduleThe matrix below diagrams the Science Education Program’s plans to assess PLOs from present until the 2016-2017 AY. The proposed schedule is based on the planned course offerings in this four-year period (N.B. Science Education is offered every semester; Science 220 [the required introductory course in the M.A. Program] is typically offered every fall, dependent on enrollment. Should recruitment efforts succeed in attracting more students, and the CSU permits spring acceptances, it is likely that Science 220 will be offered more frequently in the future. Still, the Program faculty believes it is prudent to alternate assessment of the PLOs in the fashion illustrated below).

2014-2015

2015-2016

2016-2017

2017-2018

PLO 1.1

Y Y

PLO 1.2

Y Y

PLO 2.1

Y Y

PLO 2.2

Y Y

5. Student ExperiencePLOs and ULGs are communicated to students on Program Websites and green sheets. As a program designed for pre- and in-service science teachers, these benchmarks serve as models for our students, who need to consider developing learning objectives for their own students. Faculty have expressed that there is usually a great deal of discussion and “buy-in” by our students as a result.

Part B6. Assessment Data & resultsAssessment data were collected from the faculty of the PLO-pertinent courses, specifically, SCED 173, SCI 220, and SCI 205. Data were provided by the instructors of the course (2015-2016 AY) & from various course rubrics addressing PLOs 1.2, and 2.2. Rubrics are included in the appendix. We accepted seven new MA students in fall 2015, and graduated four students since spring 2015.

Analysis



SCED 173 PLO 1.2 :

Even though SCED 173 is a required course for MA students, the vast majority of MA students take this course before they are enrolled in the program. Therefore, it’s impossible to predict which students currently enrolled in SCED 173 will be MA students in the future. Therefore, we will assess all students in SCED 173 as if they plan to enroll in the MA program. There were six students enrolled in the program and all of these students showed evidence of making progress toward PLO 1.2.

Students demonstrated knowledge in the areas of inquiry based learning and assessment in SCED 173; during this course students developed their understanding through constructing science focused lesson plans and a unit plan. They also practiced teaching parts of a lesson involving inquiry aspects to their peers. Each teacher candidate successfully modeled various flavors of inquiry in their lessons from the

hook, to the setup of instruction prior to an activity, the activity itself and how to bring a lesson to closure. After each lesson students reflected on the inquiry process - how to emphasize it and enhance it and they received feedback from their peers and the instructor. They revisited lessons to show growth in their teaching skills and to demonstrate how they used feedback to modify their instruction. The second area, assessment, was approached by having students evaluate tests related to their subject areas. In addition, they were taught how to construct a test blueprint. Students practiced constructing test questions for assignments, and at the end of the semester they wrote a test and provided the blueprint as part of their unit plan assignment. All students showed an ability to construct content appropriate questions. The Common Rubric for Signature Assignment, the rubric required by the College of Education’s Teacher Education Department, was used as the primary assessment instrument of SCED 173. The rubric evaluates students’ performance in the following areas: Focus, Assessment, Sequence/Organization, Lessons-Structure, Lessons-Pedagogy, Rationale for Teaching, and Academic Language.

Assessment: The students’ performance on the assessment tools they included in their unit plan were evaluated. 2 of 6 students scored at the Level of 4, the highest score possible, while 4 of 6 students were rated as a Level 3. Level 3 performance involves the following: Opportunities are consistently provided for students to learn what is assessed. AND Assessments allow students to show some depth of understanding or skill with respect to the standards/objectives. AND Assessments include both productive (speaking/ writing) and receptive (listening/reading) modalities to monitor student understanding. AND Students are assessed periodically with planned opportunities to receive formative feedback. A Level 4 is measured by these characteristics: All components of Level 3 plus: Assessment methods and strategies are modified and adapted to allow opportunities for students with diverse needs (e.g. linguistic, cognitive) to demonstrate understandings and skills around the central focus or “big idea” AND Students are assessed periodically with planned opportunities to receive formative feedback, assess their own learning and set goals with the assistance of the teacher.

Pedagogy: Pedagogy as demonstrated in the unit plan was assessed using the following criteria: Level 2 - Lessons make use of multiple instructional strategies that help some students engage with subject matter or discipline-based thinking. Level 3 - Lessons make use of multiple instructional strategies that help most students engage with subject matter or discipline-based thinking. AND Selection of strategies demonstrates attention to the typical challenges students may face with the content of the lesson. Level 4 - Lessons make use of multiple instructional strategies that help all students engage with subject matter or discipline-based thinking. AND Selection of strategies demonstrates attention to the typical challenges students may face with the content of the lesson. AND Instructional strategies selected to challenge students at a variety of levels. Students enrolled in SCED 173 performed as follows: 1 student was rated a Level 2, 2 of 6 students rated Level 3 and 3 of 6 students rated Level 4.

Rationale: Rationale for teaching as demonstrated in the unit plan was also assessed and this item lends insight into the ability of students to provide how their teaching practice demonstrates

connection to principles of theory and research. Performance was as follows: 2 of 6 students were rated at a Level of 2 indicating that they included a rationale that was marginally consistent with principles of theory and research. 3 of 6 students were evaluated as a Level 3, indicating that they included a rationale and they provide some connection to principles of theory and research, and 1 of 6 students performed at a Level 4, indicating that a rationale for teaching practice demonstrated strong connection between theory and research.

2015-2016: Assessment of PLOS 1.2 and 2.2


Students’ learning was assessed as a part of Science Education 220 (Theory & Practice in Science Education) in which students are required to research, synthesize, and author a literature review on a Science Education Topic of their choosing. Student were required to identify the major learning theory(ies) relevant to their projects, as well as the major related research areas in education, science education and learning sciences. The topics were aligned with the interests of the individual students: hence there is a great deal of variability in the nature of the assignments. The rubric for the signature assignment in SciEd 220 is included at the end of this report.


Students’ learning was assessed in two places: 1) as a part of Science Education 205 (Research Methods in Science Education) in which students are required to present a research proposal for their masters project as one of the elements of the course, and 2) as a part of their culminating experience for their master’s plan B projects. In SCI 205, each student is required to write a proposal for the MA project that adhered to APA guidelines and standards in the field of science education (see rubric Appendix A). This proposal includes a revision of their Literature Review from 220 and methods for conducting their project and later can be leveraged in the writing of their final project. In SCI 298 students write a written report of the findings and/or analsysis of their final project.

A total of 8 student records were considered for this report 8 in SCI 220, and 6 in SCI 205. All but one student were successful in completing the Literature review project in SCI 220 assessing PLO 1.2 (see rubric in Appendix A). The final student did not complete her literature review project and earned a failing grade in the class. All 6 Science Education masters students enrolled in SCI 205 (where PLO 2.2 is addressed) successfully completed MA Project proposals (see rubric in Appendix B). One student currently has an incomplete due to medical reasons, but had already met expectations on the proposal prior to the end of the semester. Three MA students, Jessica Potter, Rob Iverson and Jennifer Hinthorne, completed their projects in Spring 2016. All students met expectations for writing results and Jessica’s MA project will be submitted for publication later this summer.

1. Proposed changes and goals (if any)We only have between 5 and 10 students who are accepted and enrolled into the Science Education MA program each year. We would like to increase these numbers, and are in the process of creating better recruitment pipelines. For example, this year we having been working with the College of Education to combine the credentialing program with our MA program to create a full-time 3-year program. We are also actively working to decrease time to degree by intentionally scaffolding our courses in a way that allows students to make progress on their masters project earlier in their course work. It is our goal to assist students in meeting requirements needed for graduation in 2-3 years, well ahead of the 7 year limit (when courses would need to be reauthorized) that has been a problem for our students in the past. These curricular changes were made in fall of 2014, so it is too early to tell if progress has been made, but one student from that cohort has already graduated, which is evidence that our changes may have made a positive impact. Students who have enrolled in the M.A., Science Education program will continue to be advised by a faculty member from the time they enroll, until the time they graduate.

PLOs will continue to be assessed, as per the proposed schedule. described on page 2.

Rubric for Literature Review SCI 220

Appendix A:

Exemplifies expectation

Adequately meets

expectation

Approaches expectation

Does not meet expectation

Describes importance /relevance

Makes a clear, concise, and

important (relevant) claim

that has implications for student success

Makes a claim that is either unclear or does not have relevance

Makes a claim that is both unclear and has no relevance

Does not make a claim

Organization Writing is well organized, has a compelling opening, an informative body and conclusion. Has appropriate paragraph format.

Writing shows a clear beginning middle and end. Generally uses correct paragraph format.

Writing is usually organized, but sometimes off topic.

Writing is aimless and disorganized.

Supporting evidence for a claim(s)

Gives clear and accurate evidence in support of the claim

Gives evidence in support of claim but overlooks important reasons

Gives weak evidence which doesn’t support the claim well or is irrelevant.

Does not give reasons in support of the claim

Connection to Science Education research/Theory

Has at least fifteen peer review citations strongly connected to

Has fewer than fifteen peer review citations or has only tentative

Has fewer than fifteen peer-reviewed citations, and literature

Uses fewer than 10 peer-reviewed citation.

project. Current research is included.Based on/grounded in prior literature

connections to literature.

review is superficial.

Counter opinion /Drawbacks/critique (common argument)

Identifies limitations (boundaries, and/or flaws) of previous research. Reveals conflicting ideas, and states how future or previous efforts can/were made to resolve this.

Identifies limitations or conflicting ideas, but does not suggest how resolution can be made.

Identifies limitations or conflicting ideas, but these are not well integrated into the narrative.

Does not address limits or address conflicting ideas.

Grammar/typos Uses correct grammar, mechanics, spelling, and citations (consistent)

Mostly uses correct grammar and spelling. There are some minor issues that don’t distract reader.

Sometimes uses correct grammar and spelling. There are one or two issues distracting to the reader.

Has multiple distracting grammatical errors

Appendix B

SCI 205 Project/Research Proposal Rubric

Meets expectation Approaching expectation No Evidence

Literature

Review (16 points

)

Motivation

The literature review includes references to appropriate literature

that orients the reader to the importance of or the

motivation for the proposed research. The

motivation is grounded in research not simply

personal interest. Note: Personal motivation is

not required. (4)

The literature review includes a discussion that orients the reader to the

importance of or the motivation for the proposed

research but does not include references to

appropriate literature or is only personally motivated.

Note: Personal motivation is not required. (2)

The literature review does not include a

discussion that orients the reader to the importance of or

the motivation for the proposed research (0)

Theoretical

Framework

The literature review includes the researchers theoretical framework with a minimum of 2

relevant citations (Your theoretical framework may be one of your Big

Ideas in Science Education - See Review

of Literature). (4)

The literature review includes the researchers

theoretical framework with 1 relevant citation. (2)


theoretical framework (0)

Review of

relevant

literatur

The literature review includes a discussion of

at least 2 topics (Big Ideas in Science

Education) associated

The literature review includes a discussion of 1 topic (Big Ideas in Science

Education) associated with research/project and


discussion of relevant topics (Big

Ideas in Science

e with research/project and references a

minimum of 2 relevant pieces of literature for

each of those topics. (6)

references a minimum of 2 relevant pieces of literature

for that topic - OR - the literature review includes a

discussion of at least 2 topics associated with research/project and

references only 1 piece of literature for each of those

topics(3)

Education) associated with the

literature (0)

Research

Question(s)

The literature review includes a list of 2-5 clear comprehensive research

questions and incorporates a discussion of how those questions fill a current gap in the

literature (2)

The literature review includes a list of 2-5

research questions but does not discuss of how those

questions fill a current gap in the literature or the research questions are

vague.(1)

The literature review does not include

research questions, or includes

extremely vague research questions

which are unconnected to the

literature. (0)

Common Rubric for Methods Signature Assignment

Focus

Level 1 Level 2 Level 3 Level 4

The unit plan has no central focus or “big idea” which informs the planning of the learning tasks and the assessment tasks for the unit.

AND

Learning tasks/lesson objectives within and/or between lessons lack coherence.

The central focus or “big idea” for the unit is primarily one-dimensional*

OR

The central focus or “big idea” is vaguely connected to the learning tasks/lesson objectives, without any clear progression towards a central focus or “big idea.”

*As in lower level cognitive demands

There is a clear connection between the central focus or “big idea” for the unit and the progression of learning tasks/lesson objectives and assessment tasks designed for

the unit.

There is a clear connection between the central focus or “big idea” for the unit and the progression of learning tasks/lesson objectives and assessment tasks designed for the unit.

AND

The progression of learning tasks and assessment tasks lead students to deeper understandings of the central focus of the unit.

Assessment


There are limited opportunities provided for students to learn what is assessed

OR

There is a significant mismatch between the assessment methods and the central focus or “big idea”

Opportunities are provided for students to learn what is assessed.

AND

There is a general connection between the assessment methods and the central focus or “big idea”

AND

It is not clear that assessments of students’ understanding go beyond surface-level learning.

AND

Students are assessed periodically over the course of the unit

Opportunities are consistently provided for students to learn what is assessed.

AND

Assessments allow students to show some depth of understanding or skill with respect to the standards/objectives.

AND

Assessments include both productive (speaking/ writing) and receptive (listening/reading) modalities to monitor student understanding.

AND

Students are assessed periodically with planned opportunities to receive formative feedback

All components of Level 3 plus:

Assessment methods and strategies are modified and adapted to allow opportunities for students with diverse needs (e.g. linguistic, cognitive) to demonstrate understandings and skills around the central focus or “big idea”

AND

Students are assessed periodically with planned opportunities to receive formative feedback, assess their own learning and set goals with the assistance of the teacher.

Sequence/Organization


The sequence of lessons has little or no relationship to the overall focus or “big idea” of the unit plan as a whole

The sequence of lessons is vaguely connected to the overall focus or “big idea” of the unit plan as a whole

The sequence of lessons is clearly connected to the overall focus or “big idea” of the unit and build logically to help students move towards meeting the learning objectives of the unit.

The sequence of lessons is clearly connected to the overall focus or “big idea” of the unit, and build logically to help ALL students move towards meeting the learning objectives of the unit. AND

There are planned-for supports for students who do not meet daily learning outcomes to engage with learning and make forward progress.

Lessons—Structure


Lessons lack a clear, well-defined structure (e.g. beginning, middle, end) OR

Lessons lack clear objective(s), are not standards-based, and lack a clear progression of learning tasks leading to an outcome connected to the objective(s) of the lesson.

Lessons have a structure (e.g. beginning, middle, end) AND

Lessons have objective(s), are standards-based, and contain a progression of learning tasks related to an outcome connected to the objective(s) of the lesson.

Lessons have a clear and well-defined structure (e.g. opening hook, procedures, closure) aimed at moving students toward a central focus/big idea

AND Lessons have clear objective(s), are standards-based, and have a clear progression of learning tasks leading to an outcome connected to the specific objective(s) of the lesson.

All components of Level 3 plus: Lessons lead to deeper or extended understanding of the academic content of the lesson AND

The lessons connect to what students have learned earlier and anticipate what students will be learning later. AND

Lessons anticipate challenges with academic content. AND

Lessons align their objectives, activities and assessments.

Lessons—Pedagogy


Across lessons, repetition of same instructional strategies without clear purpose for the repetition

AND

Limited opportunities for students to engage with subject matter and/or discipline-based thinking

Lessons make use of multiple instructional strategies that help some students engage with subject matter or discipline-based thinking

Lessons make use of multiple instructional strategies that help most students engage with subject matter or discipline-based thinking.

AND

Selection of strategies demonstrates attention to the typical challenges students may face with the content of the lesson.

Lessons make use of multiple instructional strategies that help all students engage with subject matter or discipline-based thinking.

AND

Selection of strategies demonstrates attention to the typical challenges students may face with the content of the lesson.

AND

Instructional strategies selected to challenge students at a variety of levels.

Rationale for Teaching


Rationale for teaching practices are inconsistent with principles of theory and research

Rationale for teaching practices is marginally consistent with principles of theory and research

Rationale for teaching practices demonstrates some connection to principles of theory and research

Rationale for teaching practices demonstrates a strong connection between theory and research.

AND

Detailed lesson plans fit within the unit

Academic Language – Developing students’ academic language repertoire1, 2


The candidate gives little or sporadic support to students to meet the language demands of the learning tasks.OR

Language and/or content is oversimplified to the point of limiting student access to the core content[1] of the curriculum.

The candidate uses scaffolding or other

support [2] to

address identified gaps between students’ current language abilities and the language demands of the learning tasks and assessments, including selected genres and key linguistic features.

AND

Candidate articulates why instructional strategies chosen are likely to support aspects of students’ language development.

The candidate’s use of scaffolding or other support provides access to core content while also providing explicit models, opportunities for practice, and feedback for students to develop further language proficiency for selected genres and key linguistic features.

AND

Candidate articulates why the instructional strategies chosen are likely to support specific aspects of students’ language development for different levels of language proficiency.

The candidate’s use of scaffolding or other support provides access to core content while also providing explicit models, opportunities for practice, and feedback for students to develop further language proficiency for selected genres and key linguistic features.

AND

Candidate articulates why the instructional strategies chosen are likely to support specific aspects of students’ language development for the full range of language proficiency and projects ways in which the scaffolds can be removed as proficiency increases.

[1] Core content is the set of facts, concepts, skills, and abilities that are absolutely necessary to participate at least minimally in the learning/assessment tasks in the learning segment.

[2] Such support might include one or more of the following: modeling strategies for comprehending or constructing texts such as lab reports; explicit communication of the expected features of oral or written texts (e.g., using rubrics, models, and frames); use of strategies that provide visual representations of content while promoting literacy development (e.g., graphic organizers); vocabulary development techniques (context cues, categorization, analysis of word parts, etc.); opportunities to work together with students with different language and literacy skills.

web viewcheck here if your website addresses the ... contact person and email: ellen metzger,...

Documents