all college day: our role in student success incorporating authentic assessment in the classroom...
TRANSCRIPT
All College Day:Our Role in Student Success
Incorporating Authentic Assessment in the
Classroom
October 2010
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 2
St. Petersburg College
Presenters
Dr. Carol Weideman, Mathematics Professor
Dr. James Coraggio, Director, Academic Effectiveness and Assessment
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 3
Assessment Basics
Why do we assess? To see how well we are doing To confirm what we already know To share our progress with others To see where we can improve and change In some cases to demonstrate what does not
work
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 4
Assessment Basics
Source: http://www.c-pal.net/course/module2/pdf/Week1_Lesson5.pdf
Why do we assess?
J
Incorporating Authentic Assessment in the Classroom 2010
Purpose of an Assessment
“Clearly delineate between those that know the content and those that do not.”
To determine whether the student knows the content, not whether the student is a good test-taker.
Likewise, confusing and tricky questions should be avoided to prevent incorrect responses from students who know the material.
October 26, 2010 All College Day: Our Role in Student Success 5
J
Incorporating Authentic Assessment in the Classroom 2010
Types of Assessments
Objective assessments Authentic assessment
October 26, 2010 All College Day: Our Role in Student Success 6
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 7
Objective Assessments
Measure several types of learning (also levels) Wide content, short period of time Variations for flexibility Easy to administer, score, and analyze Scored more reliability and quickly
C
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 8
Types of Objective Tests
Written-response Completion (fill-in-the-blank) Short answer
Selected-response Alternative response (two options) Matching Keyed (like matching) Multiple choice
C
Incorporating Authentic Assessment in the Classroom 2010
Small Group Assignment
Multiple Choice Assessment
Assignment Objectives Solving Equations using addition and
multiplication principles Solving applied problems by translating to
equations
October 26, 2010 All College Day: Our Role in Student Success 9
C
Incorporating Authentic Assessment in the Classroom 2010
Issues with Objective Assessments
Limited depth of content Not able to reveal student misconceptions Limited ability to test critical thinking skills Students are ‘Test-wise’
October 26, 2010 All College Day: Our Role in Student Success 10
C
Incorporating Authentic Assessment in the Classroom 2010
Test-wise Students
Are familiar with item formats Use informed and educated guessing Avoid common mistakes Have testing experience Use time effectively Apply various strategies to solve different
problem types
October 26, 2010 All College Day: Our Role in Student Success 11
C
Incorporating Authentic Assessment in the Classroom 2010
Authentic Experiences
Is the course aligned with the expectations for the student in the ‘real-world’? Authentic Learning Authentic Assessment
October 26, 2010 All College Day: Our Role in Student Success 12
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 13
Class Without Authentic Experiences
Didactic instruction where students are presented with factual information from a text book
Assessment is primarily multiple choice items where students are expected to regurgitate factual information
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 14
Class With Authentic Experiences
Interactive learning environment where students not only learn facts but the relationship between the facts and the application of that information
Authentic assessment where students are able to model the applications of the discipline through simulations, projects, etc.
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 15
Authentic Assessments
Authentic assessments serve dual purposes of encouraging students to think critically and providing assessment data for measuring improved
student learning.
These assessment techniques fall into three general categories: criterion-referenced rubrics, student reports (reflection or self-assessments), and student portfolios.
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 16
Authentic Assessments
Authentic assessments include… Criterion-referenced rubrics. Complex, higher-order
objectives can be measured only by having students create a unique product, whether written or oral [in-class essays, speeches, term papers, videos, computer programs, blueprints, or artwork] (Carey, 2000).
Student Reflection. Written reflection is espoused to have several important benefits: it can deepen the quality of critical thinking, increase active involvement in learning, and increase personal ownership of the new learning by the student (Moon, 1999).
Student Portfolios. Collections of students’ work over a course or a program and can be an effective method of demonstrating student progress in the area of critical thinking (Carey, 2000).
J
Incorporating Authentic Assessment in the Classroom 2010
Small Group Assignment
Authentic Assessment: Building a Fence Worksheet
Assignment Objectives Solving Equations using addition and
multiplication principles Solving applied problems by translating to
equations
October 26, 2010 All College Day: Our Role in Student Success 17
C
Incorporating Authentic Assessment in the Classroom 2010
Small Group Assignment
You want to fence in part of your backyard. The dimensions of the fenced yard are shown in the diagram below.
October 26, 2010 All College Day: Our Role in Student Success 18
The fence will have a 10ft gate and a vertical support pole every 10 ft.
•10 ft Gate: $100
•Vertical Support Poles: $5 each
•Chain Link Fencing: $2 per foot
C
Incorporating Authentic Assessment in the Classroom 2010
Small Group Assignment
A. Write and graph an equation for the cost of building a fence that has the cost per foot and one gate.
Let y = total cost of the fence
x = cost per foot
B. Using the given information and the equation from Part A, how much will it cost to build the fence?
C. Suppose chain link fencing is sold only in 50 ft length rolls. You don’t want to waste or throw away any fencing. How much wider does the yard need to be to use all the fencing in the rolls?
October 26, 2010 All College Day: Our Role in Student Success 19
C
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 20
Rubrics
What is a rubric? Scoring guidelines, consisting of
specific pre-established performance criteria, used in evaluating student work on performance assessments
C
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 21
Rubrics
SPC currently uses rubrics in such programs as…
College of Education College of Nursing Paralegal Studies Program
C
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 22
Rubric Development Process
1. Re-examine the learning objectives to be addressed by the task
2. Identify specific observable attributes your students should demonstrate
3. Describe characteristics at each attribute 4. Write narrative descriptions for each level of
continuum 5. Collect samples of student work 6. Score student work and identify samples that
exemplify various levels 7. Revise the rubric as needed
Repeat as Needed
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 23
Assignment Profile
Designed to provide consistency and accuracy as well as provide guidelines for the use
Rubric is an evaluation ‘tool’, but for a tool to be effective it must be in the correct situation or ‘job.’ It would be inefficient to use a machete to conduct heart surgery.
Rubric must be aligned to the most appropriate course assignment
The instructor is the assessment instrument not the rubric
J
Incorporating Authentic Assessment in the Classroom 2010
Full Group Assignment
Rubric: STA2023 Sampling Project Assignment Objectives
Identify the sampling strategies commonly employed to collect data
Describe potential biases encountered with sampling strategies used in various statistical applications
Suggest strategies to avoid potential biases when using sampling to collect data for a statistical application
October 26, 2010 All College Day: Our Role in Student Success 24
J/C
Incorporating Authentic Assessment in the Classroom 2010
Full Group Assignment
Management at a retail store is concerned about the possibility of drug abuse by people who work there. They decide to check on the extent of the problem by having a random sample of the employees undergo a drug test. The lawyers for the retail store have assured the management that there are no legal issues with the proposed drug testing as long as the individual test results are not identified to a specific employee. Depending the extent of illegal drugs identified in the drug testing, drug counseling may be offered to all employees under the promise of complete confidentiality.
You have been hired as the statistician who will design the sampling plan. The budget for the drug testing will cover the cost of 40 drug tests.
October 26, 2010 All College Day: Our Role in Student Success 25
J/C
Incorporating Authentic Assessment in the Classroom 2010
Full Group Assignment
Management has proposed several different ideas about the best way to obtain the random sample of 40 employees who will be drug tested. There are currently 500 employees at this retail store. There are four classifications of employees: supervisors, full-time sales clerks, part-time sales clerks and maintenance staff. These sampling possibilities are listed below:
Select one of the employee classifications and sample all employees in that classification. Choose every fourth person who clocks in for each shift. Randomly select 10 employees from each classification. Each employee has a three-digit employee number. Randomly select 40 employees.
October 26, 2010 All College Day: Our Role in Student Success 26
J/C
Incorporating Authentic Assessment in the Classroom 2010
Full Group Assignment
Answer the following questions regarding this scenario:
Define this problem in your own words. Compare and contrast the four proposed sampling plans. Select one of the proposed sampling plan that you feel is most appropriate for
this situation and defend your choice. Describe any weaknesses in your selected sampling plan. Make suggestions on ways to improve/strengthen the sampling plan. You
may include information not described in the scenario above. Reflect on your own thought process after completing the assignment.
“What did you learn from this process?”
“What would you do differently next time to improve?”
October 26, 2010 All College Day: Our Role in Student Success 27
J/C
Incorporating Authentic Assessment in the Classroom 2010
Assessment Basics
Alignment of course objectives Competency, Clarity, Bias, Level of
Difficulty Validity and Reliability
October 26, 2010 All College Day: Our Role in Student Success 28
C
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 29
Assessment Basics
Alignment Everything needs to align (objectives
through assessment)
C
Incorporating Authentic Assessment in the Classroom 2010
Competency
Items should test for the appropriate or adequate level of knowledge, skill, or ability (KSA) for the students.
Assessing lower division students on graduate level material is an ‘unfair’ expectation.
The competent student should do well on an assessment, items should not be written for only the top students in the class.
October 26, 2010 All College Day: Our Role in Student Success 30
C
Incorporating Authentic Assessment in the Classroom 2010
Clarity
Clear, precise item and instruction Correct grammar, punctuation, spelling Address one single issue Avoid extraneous material (teaching) One correct or clearly best answer Legible copies of exam
October 26, 2010 All College Day: Our Role in Student Success 31
C
Incorporating Authentic Assessment in the Classroom 2010
Bias
Tests should be free from bias…No stereotypingNo gender biasNo racial biasNo cultural biasNo religious biasNo political bias
October 26, 2010 All College Day: Our Role in Student Success 32
C
Incorporating Authentic Assessment in the Classroom 2010
Level of Difficulty
Ideally, test difficulty should be aimed a middle level of difficulty. This can not always be achieved when the subject matter is based on specific expectations (i.e, workforce area).
October 26, 2010 All College Day: Our Role in Student Success 33
J
Incorporating Authentic Assessment in the Classroom 2010
Trivial and Trick Questions
Avoid trivia and tricks Avoid humorous or ludicrous responses Items should be straight forward, they
should cleanly delineate those that know the material from those that do not
Make sure every item has value and that it is contributing to the final score
October 26, 2010 All College Day: Our Role in Student Success 34
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 35
Assessment Basics
Does one size fit all? Assessments need to be valid Assessments need to be reliable
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 36
Validity
Does the assessment measure what it is suppose to measure?
“Validation is the process of accumulating evidence that supports the appropriateness of inferences that are made of student responses…” (AERA, APA, & NCME, 1999)
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 37
Types of Validity Evidence
Content Related - the extent to which a student’s responses to a given assessment reflect that student’s knowledge of the content area
Construct Related - the extent to which the responses being evaluated are appropriate indicators of the underlying construct
Criterion Related - the extent to which the results of the assessment correlate with a current or future event
Consequential – the consequences or use of the assessment results J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 38
Reliability
Consistency of the assessment scores
Types of reliability… Interrater Reliability – scores vary from
instructor to instructor. Intrarater Reliability – scores vary from a
single instructor from paper to paper A test can be reliable and not valid,
but never valid and not reliable J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 39
Reliability Concerns
Reliability1. Are the score categories well defined?2. Are the differences between the score
categories clear?3. Would two independent raters arrive
at the same score for a given student response based on the scoring rubric?
J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 40
Improving Scoring Consistency
Provide grading rubrics or scoring criteria to students prior to assessment
Grade papers anonymously Use anchor papers to define levels
of proficiency for reference Use multiple scorers Calculate reliability statistics during
training and grading J
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 41
Assessment Basics
Multiple Measures Always to good to implement
multiple measures when possible Ideally direct and indirect measures
of competency
C
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 42
Indirect Methods
“indirect measures …help deepen the interpretation of student learning” (Maki, 2004).
SSI is a good example of an indirect measure.
C
Incorporating Authentic Assessment in the Classroom 2010
October 26, 2010 All College Day: Our Role in Student Success 43
Questions/Next Steps
All College DayOur Role in Student Success
Incorporating Authentic Assessment in the
Classroom
October 2010