Using Taskstream for Student Assessment

Dr. Chris Boosalis – Teacher Education

May 13, 2011

Purpose

Demonstrate how Taskstream can be

used for student assessment and

accreditation

• Teacher Performance Assessment

• Signature Assignment Evaluation

• Student Teaching Supervision

• Reporting on Student Learning

Outcomes

SLOs in Teacher Ed

A. Making Subject Matter

Comprehensible

B. Assessing Student Learning

C. Engaging Students

D. Planning Instruction

E. Creating Environments for

Learning

F. Developing as an Educator

Performance Assessment

Signature Assignment

Student Teaching

Accreditation Reporting

Lessons Learned

• Taskstream is useful for archiving

student assessments

• Taskstream is useful for rating

students on SLOs

• Taskstream is useful for displaying

SLO reports to accreditors

Thank You!

Q & A

First Annual

Assessment Spotlight

Friday, May 13, 2011

MSR 130

Welcome

External Review: Nuts & Bolts Jennifer Helzer, Geography

May 13, 2011

Purpose

• Purpose:

To relay experiences and lessons

learned with external review as part

the Academic Program Review

(APR) process.

• Outline:

– Process

– Selecting an external reviewer

– Results

GEOGRAPHY

APR

Process

1. Learn about the process…reach out

and research.

2. Share your knowledge with program

faculty.

3. Address questions and concerns.

4. Select a style of review that works

best for the program

– Campus visit

– No campus visit

– Number of reviewers

5. Develop a timeline that meshes with

the APR calendar.

Process – Selecting an External

Reviewer

1. Identify a pool of candidates

– Previous experience

– Professional conferences/affiliations

– Send out call (professional list-serves, professional

organizations, journals)

– Consider candidates from programs you aspire to

2. More communication with program faculty, Dean.

3. Arrange campus visit:

– Candidates may request to meet with particular

support units and administrators

– Consider including: staff, other departments, IR,

OSRP, OSL, PAC and of course students.

– Develop itinerary

– Send draft APR to reviewer; other relevant

materials.

End Result

21 page report with executive summary: Data Sources

• This external review is based on information from the following sources:

• Examination of the “Self-Study -- Geography Program (2009-2010)” document, including its substantial

• collection of supportive and supplementary appendices

• Visits to the following web pages:

• • Geography Program web pages:

• ◦ http://www.csustan.edu/geography/

• ◦ http://geoimages.csustan.edu/Geowebsite.htm

• ◦ http://www.csustan.edu/academics/CHSS/Geography.html

• ◦ http://www.extendeded.com/GIS/

• • Geography Club's web sites and Facebook pages:

• ◦ http://csustanislausgeographyclub.webstarts.com/index.html

• ◦ http://www.facebook.com/pages/CSU-Stanislaus-Geography-Club/157930007711

• • College of Humanities and Social Sciences web page:

• ◦ http://www.csustan.edu/CHSS/

• • CSU Stanislaus web page:

• ◦ http://www.csustan.edu

• • CSU Stanislaus online catalogues from 2005-06 through 2010-11:

• ◦ http://catalog.csustan.edu/index.php?catoid=3

• • The 2010 WASC report

• ◦ http://www.csustan.edu/wasc/

• • Outside ranking sites:

• ◦ Princeton Review: http://www.princetonreview.com

• ◦ State University: http://www.stateuniversity.com

• • Searches for information on the Geography Program at CSU Stanislaus in local newspaper web

• sites:

• ◦ Modesto Bee: http;//www.modbee.com

• ◦ Turlock Journal: http://www.turlockjournal.com/

End Result

21 page report and

executive summary:

• Mission and Goals

• Assessment

• Curriculum Improvement and

Modernization

• Faculty and Staff

• Faculty Research

• Service to Students

• Institutional Support and

Resources

• Tables and graphs

• Conclusions and

Recommendations

Assessment Spotlight 2011

Direct and Indirect Assessment of Biology Majors Matt Cover, Jim Youngblom, Marina Gerson Department of Biological Sciences

May 13, 2011

Purpose

Existing approach: senior surveys – indirect assessment of learning goals – career goals

Desired approach: senior AND new student surveys – direct assessment of student learning – career goals, expectations,

preparation, motivation, activities

Challenges • Large number of bio majors (~650) •Direct assessment: what test or assignment?

–Should reflect the entire discipline –Year-to-year consistency

• No shared senior-level class •Logistics of administering surveys

Process Direct Assessment

– Campbell and Reece’s Biology: ~4,000 question test bank

– Reviewed by bio faculty, reduced to ~1,400 questions

– Random sample of 20 questions – Knowledge/comprehension – Qualtrics online survey

(I.R.: Angel, Lisa, Kelli)

Process New Student Survey

– Background, motivation, expectations Senior Survey

– Indirect assessment of learning objectives (“Please evaluate how your degree program…”)

• Skill in communication, analysis, research, teamwork…

– Plans, career goals, student experience, activities

# Correct / 200 5 10 15 20

05

101520253035

Results New Student Survey

• 4th week of lab, 1st semester of General Biology (BIOL1050), <1 hour

• Direct assessment (n = 164) – Mean: 6.6 / 20 – SD: 2.5

Results New Student Survey (BIOL1050) 180 responses Bio Major?

– Yes (67%) Why bio?

– Preparation for career (49%) – Familial advice (28%)

Preparation – AP Bio (38%) – Honors Bio (21%)

Results New Student Survey (BIOL1050) Background

– English language learner (11%) – 1st Language: Spanish (32%), Hmong (26%)

Work – None (59%) – 11-20 hrs/week (17%) – >20 hrs/week (19%)

Results New Student Survey (BIOL1050) Planned highest degree

– B.A./B.S. (14%) – M.S. (26%) – PhD (18%) – Professional health degree (41%)

Future career – Medical Doctor (26%) – Pharmacist (8%) – Nurse (8%) – Dentist (6%)

Results Senior Survey • Email students who have applied to

graduate (mid-May) • Outside of class, online survey (honor

system) • Entice with party and prize

Lessons Learned 1. I.R. is a tremendous resource for

departmental assessment 2. Multiple choice makes large-scale direct

and indirect assessment feasible 3. Appreciation for new students'

backgrounds, expectations, motivations 4. System in place for:

– direct assessment of student learning (knowledge)

– changes in career goals, motivation, work… – long-term trends

Library Assessment of Research

Skills Instruction Tim Held Library

May 13, 2011

Purpose

• The library is a support unit that provides

instruction.

• Information literacy forms the basis for lifelong learning. An

information literate individual is able to:

– Determine the extent of information needed.

– Access the needed information effectively and efficiently.

– Evaluate information and its sources critically.

– Incorporate selected information into one’s knowledge base.

– Use information effectively to accomplish a specific purpose.

– Understand the economic, legal, and social issues

surrounding the use of information.

From Association of College & Research Libraries Information

Literacy Standards for Higher Education

Purpose

• Support units need to demonstrate their

contribution to student success.

• WASC says:

“The Library’s Support Unit Review notes that the Library needs

to collect data that will assess student learning as a result of

their instructional programs, and the team encourages the

Library to do so. Based on the documents made available, the

Library has not developed the capacity to assess the library’s

role in student engagement and learning (CFR 4.3).”

Report of the WASC Visiting Team Capacity and Preparatory Review,

Western Association of Schools and Colleges, 2008, p. 11.

Issues

• Assessing information literacy is challenging

because:

– Librarians only see students for 50 minutes.

Issues

• Assessing information literacy is challenging

because:

– Librarians only see students for 50 minutes.

– Evidence of learning goals is in their research papers.

Issues

• Assessing information literacy is challenging

because:

– Librarians only see students for 50 minutes.

– Evidence of learning goals is in their research papers.

– Few instructional programs assume responsibility for

information literacy as a learning goal.

Issues

• Assessing information literacy is challenging

because:

– Librarians only see students for 50 minutes.

– Evidence of learning goals is in their research papers.

– Few instructional programs assume responsibility for

information literacy as a learning goal.

– The library does not have the ability to reach all

students or track their progress from start to finish.

Build Assessment

• Library Instruction Assessment Plan

– Based on academic program assessment plan

– Information literacy learning goals mapped to library

curriculum.

– Each year target one or two goals to assess.

Build Assessment

• Library Instruction Assessment Plan

– Based on academic program assessment plan

– Information literacy learning goals mapped to library

curriculum.

– Each year target one or two goals to assess.

• Assess Research Skills Instruction Sessions

– Typical session

– Typical learning goals

– Survey faculty

• http://library.csustan.edu/

Results of Faculty Survey

• Sessions working well.

Results of Faculty Survey

• Sessions working well.

• Issues with citations as a goal of the session.

Solution includes addressing it in other forms of

instruction.

Results of Faculty Survey

• Sessions working well.

• Issues with citations as a goal of the session.

Solution includes addressing it in other forms of

instruction.

• Previous pilot survey also garnered qualitative

comments from faculty that indicated

satisfaction, though lecture is boring.

Assess the Assessment

• Goals of survey are not flexible, though session

goals are flexible.

• Direct assessment, but aggregated.

• Easy to administer.

Future Improvement

• New expectations from WASC Substantive

Change Proposal to assess library instruction.

• Automate the survey.

• Increase sample size.

• Customize your survey.

• IR is key.

• Integration of assessment with curriculum is key.

• Instructional faculty buy-in is key.

• Library faculty buy-in is key.

Critical Transitions: Using Literacy

Assessment Data to Guide Retention

Efforts for At-Risk Students John Wittman and Alex Janney, English

May 13, 2011

Purpose

• To investigate student matriculation from

ENGL1000: Introduction to Composition to

ENGL1001: First-Year Composition

(skills, perceptions, pedagogy)

• Do skills acquired in ENGL1000 adequately

prepare students for ENGL1001 and writing in

subsequent courses? (transference)

Process/Methods

• Student interviews of ENGL 1000 and ENGL

1001 experiences--coded with MaxQDA

(indirect)

• Rhetorical analysis of student writing (direct)

• ENGL1000/1001 cohort survey (indirect)

• Rhetorical analysis of ENGL 1000 portfolios

(direct)

• Who and When: 24 ENGL 1000 students over

the course of the Spring 2011 semester

Recurring Themes

• Pedagogy: Peer Review, Rubrics, Instructor

Feedback

• Student Experience: Small Class Size, Student-

teacher interaction, Writing Center

• Skills: Reader Response, Writing across Genres

End Result

• Direct impact-curriculum development, staff

training

• Direct impact-assessment protocol developed

for writing program

• Indirect impact-research and assessment

protocol established for future research (i.e. a

longitudinal study to track student matriculation

to graduation--especially the WPST and WP

courses)

Lessons Learned/Advice

Lessons and Advice

• Take advantage of university resources

– (EAP, Office of Assessment, IR, GEAC, ORSP)

• Have both short-term and long-term goals

• Double dip (or make it count as service and

research-IRB process)

To improve

• Continue to refine methodology (grounded

theory)

• Translate qualitative research into administrative

value

Assessing the Impact of

Community-Based Education

David Colnic, Steven Filling, Jennifer Helzer, Julie

Fox, & Anna Epperson, Office of Service Learning

May 13, 2011

Purpose Application of 21st century learning

• To share community-based education’s

assessment strategy and history (via the

Office of Service Learning) .

• Challenge: To create an assessment

rubric that reflects the needs of various

disciplines and provides useful

information.

What is Community Based Education?

• Dr. David Colnic, Politics & Public Administration

• Dr. Steven Filling, Accounting & Finance

• Dr. Jennifer Helzer, Anthropology & Geography

Community-based education (also called service

learning) empowers students to learn through active

participation in meaningful and planned community

experiences that are directly related to course content.

In 2010/11:

109 sections (64 courses)

Approximately 3,078 students participated

Process

• Collaboration – SL Steering Committee, faculty &

community members.

• Formation of focus group of SL faculty.

• Developed 3 student learning objectives (SLOs):

• Apply discipline-specific and/or interdisciplinary knowledge and

critical thinking skills to community issues.

• Demonstrate critical self-reflection of one’s own assumptions and

values as applied to community issues.

• Demonstrate knowledge and sensitivity to issues of culture, diversity,

and social justice as applied to community engagement.

• Rubric finalized for SLO 1 and drafted for SLO 2.

• Pilot of student end-of-the-semester survey:

• 7 courses in fall 2009.

End Result

- Collect annual data on number of SL courses,

faculty, students, and community partners.

- Assess community partners:

- Survey

- Focus groups

- Assess each SLO (one SLO/year).

- Writing Prompt (writing sample) and

student surveys.

Lessons Learned

• Challenging approach to work with a large group

of faculty from all disciplines on campus.

• Challenging to quantify CBER or

transformational experiential learning:

• Impact after graduation.

• Collaboration:

• Take time to let people be comfortable.

• Listen to all points of view.

• Next steps:

• Data collection and analysis.

• SLO #1 Write Up.