college of staten island march 2007 - baruch college - … & using assessment results linda...
TRANSCRIPT
Understanding & Using Assessment Results
Linda Suskie, Vice President Middle States Commission on Higher Education
3624 Market Street, Philadelphia PA 19104 Web: www.msche.org Email: [email protected]
College of Staten Island
March 2007
© Linda Suskie, MSCHE
Today…
1. What is assessment?
2. What is “good” assessment?
3. Why are you assessing?
4. Tips on sharing assessment results
5. What questions do you want your assessment to answer?
6. Using assessment results to improve teaching & learning
© Linda Suskie, MSCHE
1. What is Assessment of Student Learning?
l Deciding what we want our students to learn
l Making sure they learn it!
Jane Wolfson, Director, Environmental Science & Studies Program, Towson University
© Linda Suskie, MSCHE
The TeachingLearningAssessment
Cycle
Learning Goals
Using Results
Learning Opportunities
Assessment
2. What is “Good” Assessment?
© Linda Suskie, MSCHE
Used
Five Dimensions of Good Assessment
•Focus on clear and important goals •Results communicated widely and transparently
© Linda Suskie, MSCHE
Used
Cost effective
Five Dimensions of Good Assessment
•Focus on clear and important goals (36) •Simple (one assignment for several goals)
© Linda Suskie, MSCHE
Used
Cost effective
Reasonably accurate &
truthful results
Five Dimensions of Good Assessment
•Flow from clear and important goals •Use multiple approaches, including direct evidence of student learning
•Developed thoughtfully; perpetual works in progress
© Linda Suskie, MSCHE
Used
Cost effective
Reasonably accurate &
truthful results
Five Dimensions of Good Assessment
Valued
•Results inform important decisions on important goals
© Linda Suskie, MSCHE
Used
Cost effective
Reasonably accurate &
truthful results
Clear & important
goals
Five Dimensions of Good Assessment
Valued
•Flow from & focus on clear & important goals •Inform important decisions on important goals •Have clear, appropriate standards for acceptable & exemplary student performance
© Linda Suskie, MSCHE
3. Why Are You Assessing Your Program or Curriculum?
l Validate it/institutionalize it l Make sure it isn’t slipping l Improve it
© Linda Suskie, MSCHE
Who Cares About Your Program or Curriculum?
l Why do they care about it? l What do they need to know? lWhy?
© Linda Suskie, MSCHE
What decisions will this assessment help us make?
l Learning goals l Are our learning goals sufficiently clear and focused?
l Curriculum l What is the value of service learning? l Should our courses have more uniformity across sections?
l Teaching methods l Is online instruction as effective as traditional instruction? l Is collaborative learning more effective than traditional lectures? l Are we developing a community of scholars?
l Assessments l Have our assessments been useful?
l Resource allocations l Where should we commit our resources first?
© Linda Suskie, MSCHE
Time to Talk!
© Linda Suskie, MSCHE
Tips on Sharing Assessment Results
l Keep it simple! l First aggregate (sum up) data, then drill down into details as needed. l Use short, simple charts & graphs. l Round results.
l Sort results from highest to lowest. l Percentages may be more useful than averages. l As you collect data over time, show trends. l Find someone skilled at finding the stories in reams of data. l Have a statistician identify meaningful vs. insignificant differences.
5. What Questions Do You Want
Your Assessment to Answer?
© Linda Suskie, MSCHE
Michael earned 55 points on the midterm.
l Did he do well on the midterm?
© Linda Suskie, MSCHE
To decide if Michael “did well,” we must compare his 55 against something else.
l Benchmark l Target l Frame of reference l Lens
l Benchmark definition l Something that serves as a standard by which others may be measured or judged
l The “something else” depends on what we want the test to tell us.
© Linda Suskie, MSCHE
Suppose 35 is passing and 80 is a perfect score.
l Standardsbased perspective l aka competencybased l aka criterionreferenced
l Question answered: l Are our students meeting our standards?
l Challenge: l Establishing sound performance standards
© Linda Suskie, MSCHE
Suppose 35 is passing and 80 is a perfect score
on a published exam.
l External standardsbased perspective l aka competencybased l aka criterionreferenced
l Question answered: l Are our students meeting external standards?
l Challenge: l Do the standards match what we think is important?
© Linda Suskie, MSCHE
Suppose the class average is 65.
l Peerreferenced perspective l aka normreferenced
l Question answered: l How do our students compare to peers?
l Challenge: l Identifying appropriate peers & collecting info from them
© Linda Suskie, MSCHE
Suppose the university average is 65, but the average at South University is 75.
l Best practice perspective l aka bestinclass
l Question answered: l How do our students compare to the best of their peers?
l Challenges: l Commitment to improving teaching & learning l Identifying bestpractice peers
© Linda Suskie, MSCHE
Suppose Michael scored 25 a year ago.
l Valueadded perspective l aka growth, change, prepost
l Question answered: l Are our students improving?
l Challenges: l Transfers in or out l Motivating students on pretest l Is growth due to us? l Imprecise assessments mask growth l Is this question relevant?
© Linda Suskie, MSCHE
Suppose class average is 65 now and 40 three years ago.
l Longitudinal perspective l aka improvement
l Questions answered: l Are our teaching & curricula improving? l Are we getting better?
l Challenge: l Using the same assessment
© Linda Suskie, MSCHE
Suppose Michael scored a 65 for knowledge and
a 45 for realworld applications.
l Strengths and weaknesses perspective l Question answered: l What are our students’ relative strengths and areas for improvement?
l Challenge: l Getting “subscores” that are truly comparable
© Linda Suskie, MSCHE
Suppose Michael is tonedeaf.
l Capability perspective l aka potential l aka “A for effort”
l Question answered: l Are our students doing as well as they can?
l Challenge: l Determining potential
© Linda Suskie, MSCHE
Suppose Michael’s 55 cost $400 and Melissa’s 55 cost $300.
l Productivity perspective l Question answered: l Are we getting the most for our investment?
l Challenge: l Calculating cost and benefit accurately l Keeping the focus on effectiveness as well as efficiency
© Linda Suskie, MSCHE
Recap: Nine Questions that Assessments Can Answer
1. Are our students meeting our standards?
2. Are our students meeting external standards?
3. How do our students compare to their peers?
4. How do our students compare to the best of their peers?
5. Are our students improving?
6. Are our teaching & curricula improving?
7. Are our students doing as well as they can?
8. What are our students’ relative strengths & weaknesses?
9. Are we getting the most for our investment?
© Linda Suskie, MSCHE
Which Perspective Should You Use?
l Each has advantages and disadvantages.
l Each gives a somewhat incomplete picture.
l Multiple perspectives give the most balanced picture of student learning.
Time to talk again!
6. Using Assessment Results to Improve Teaching & Learning
When Assessment Results Are Good
Publicize!
Celebrate!
When Assessment Results Are Disappointing
l Goals l Curriculum l Pedagogy l Assessments
© Linda Suskie, MSCHE
Look at your learning goals.
l Are your goals inappropriate or overly ambitious? l Do your goals need to be clarified? l Do you have too many goals?
© Linda Suskie, MSCHE
Look at your curriculum.
l Including placement and developmental education.
l Does the curriculum adequate address each learning goal?
© Linda Suskie, MSCHE
Look at your teaching methods.
l How do student learn best?
© Linda Suskie, MSCHE
Look at your assessments.
l Are they poorly written and misinterpreted? l Do they match your key learning goals? l Are they too difficult for most responsible students?
© Linda Suskie, MSCHE
Isn’t Poor Performance the Student’s Fault?
l Sometimes, but usually a minority l Suskie’s “50% rule”
Time to Think and Discuss!
Questions?