crossing the rubricon - palm beach state college · 2014-10-15 · because we are learning...

Crossing the Rubricon:

Workbook

“Crossing the Rubicon”, Simon Kozhin 2001. Oil on Canvas. http://www.kozhinart.com/

Mindy Yale, PSAV Instructor Palm Beach State College

Professional Development Day, October 2014

Copyright 2014

Crossing the Rubricon | Mindy Yale

Pag

e2

Feedback is a central feature in the world of today’s college students. Student’s give and/or receive feedback on social media, television and business websites. Viewers can tweet, text or email while watching TV shows or broadcast news to share their opinions or experiences. Movie and music sites such as Netflix and Pandora use a gradual but continuous feedback loop, steadily refining the match of recommendations to preferences. Amazon, Yelp, Angie’s list, Travel sites, E-bay, Google and more, offer consumer reviews about products or sellers. Reviews make a considerable contribution to the decision making process of consumers, particularly if positive or negative experiences are reinforced with multiple reviews. As every consumer has an opportunity to contribute to the success or failure of a product or service, positive reviews are seen as essential to the reputation and success of many businesses. Living in a feedback society, our students want to be seen as competent and they want feedback that will help them to achieve their goals. Authentic assessment fulfills the expectation and desire of students for feedback that indicates specifically what is needed to improve on the path towards reaching their goals.

An instrument that measures the degree to which a student is able to demonstrate

the practical application of knowledge based on a set of criteria determined by faculty to be meaningful and useful.

Success on a multiple choice test is not an accurate indicator of whether, or to what degree a student will be able to apply the information beyond the classroom. Rubrics measure the level of a student’s ability to perform the skills or tasks involved in the learning outcome.

While a rubric may be structured in a variety of ways, there are two main types of rubrics: Holistic and Analytical. A Holistic rubric groups all criteria descriptions under levels of performance, where an analytic rubric measures the level of performance of each criteria separately. On the following page, there is a sample of each type of rubric. As students may perform different tasks at different levels, the analytic rubric is a more specific grading and feedback tool.

A Rubric is a type of authentic assessment.

Authentic Assessment:


Pag

e3

Sample Critical Thinking Holistic Rubric Based on the QEP Analytic Rubric Score of 5:

Student response soundly analyzes and interprets relevant information

Student reaches a relevant conclusion and soundly demonstrates that the conclusion is based on a reasoning process

Student response soundly demonstrates the skills required to evaluate and explain Score of 3:

Student response includes analysis and interpretation, but the response is somewhat incomplete or inaccurate

Student reaches a relevant conclusion but the reasoning process is somewhat incomplete or inaccurate

Student evaluates the information and explains the results, but the response is somewhat incomplete or inaccurate

Score of 1:

Student does not analyze or interpret the information

Student does not provide a relevant conclusion nor demonstrate a reasoning process

Student does not evaluate or explain the information

Portion of the QEP Critical Thinking Analytic Rubric (Complete rubric may be found at the end of this workbook and on the College’s QEP website)

QEP / Critical Thinking Rubric To be used for scoring student responses on scenarios to measure QEP Outcomes 1-3.

Unacceptable (1) The student does not demonstrate the skills required for the outcome.

Developing (3) The demonstration of skills required for the outcome is present, but needs improvement.

Exemplary (5) The demonstration of skills required for the outcome is excellent.

Outcome 1 Students are able to analyze and interpret relevant information.

Student does not analyze or interpret the information.

Student response includes analysis and interpretation, but the response is somewhat incomplete or inaccurate.

Student response soundly analyzes and interprets the relevant information.

Outcome 2 Students are able to reach conclusions based on a demonstrated reasoning process.

Student does not provide a relevant conclusion nor demonstrate a reasoning process.

Student reaches a relevant conclusion, but the reasoning process is somewhat incomplete or inaccurate.

Student reaches a relevant conclusion and soundly demonstrates that the conclusion is based on a reasoning process.

Outcome 3 Students are able to evaluate and explain relevant information.

Student does not evaluate or explain the information.

Student evaluates the information and explains the results, but the response is somewhat incomplete or inaccurate.

Student response soundly demonstrates the skills required to evaluate and explain.


Pag

e4

Cover Image: "Crossing the Rubicon" by Triola (Trine Lise Olaussen).

Image credit: Elftown - Triola Portfolio

Summative v. Formative

Rubrics can be summative; to determine a student’s level of performance at the end of a course, or they can be used formatively to chart the student’s development throughout the course. The image on the right shows a rubric that was used at three different stages in a graphic design project. The blue markings indicate that the original design fell at a low level of performance, where the green shows improvement on the following version and the final version, marked in pink, shows significant improvement in some areas, and slight improvement in others. Accordingly, a rubric can offer students a clear path to success, as well as indicating to faculty where student skills are weak, providing a specific focus for instruction of the group or in working with an individual student. List some of the skills that appear throughout your course. These will be useful to keep in mind when designing rubrics for formative assessment.

________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

http://roobrix.com/learn.html

http://en.wikipedia.org/wiki/Rubicon

http://www.elftown.com/_Triola%20Portfolio

https://www.google.com/imgres?imgurl=http://roobrix.com/images/badrubric.gif&imgrefurl=http://roobrix.com/learn.html&docid=bzZRZRlxTU1rbM&tbnid=XIK8sCfqIk3klM:&w=500&h=437&ei=bbEtU9GMLuyuyAGVrYGoCQ&ved=0CAIQxiAwAA&iact=c


Pag

e5

Rubrics provide clarity. By defining criteria and providing a clear description of what is required at each level of performance, students know exactly what tasks are involved in attaining success. This is empowering to students, but it also puts the responsibility for their success directly into their own hands. Has a student ever suggested that his or her poor performance was the result of something you did or did not do? Have you rolled your eyes with exasperation when students declare that you did not make expectations clear (in spite of the fact that you verbalized it, presented it in the syllabus, and underlined the expectation on the assignment prompt)? Have students attempted to negotiate or manipulate for a better grade with more creativity than they used on the assignment? Has a student ever perceived a poor grade to be the result of your personal feelings towards him or her? Now imagine circling a little box on the rubric. No further explanation will ever be needed. Sigh with relief.

REQUIREMENTS 0 1 2

A minimum of three resources are cited.

No resources were cited.

One or two resources were cited.

Three or more resources were cited.

Rubrics provide an objective grading tool for even the most subjective assignments. In the massage department, students have a hands-on massage evaluation. Before rubrics were designed and implemented, instructors would often be accused of not liking the student or even “being out to get” a particular student if he or she did not pass the evaluation. When students take a grade personally, it can create quite a bit of chaos.

Rubrics promote student accountability and responsibility.

With rubrics we are able to:

Clarify requirements and expectations to students prior to assessment

Indicate specific strengths and weaknesses in individual and group performance

Identify at-risk students early on

Assist individual students with specific areas of weakness

Document a clear communication process

Deliver consistent feedback from different instructors

Reduce the likelihood of a poor grade being taken personally by the student.

Chart student progress throughout the course.


Pag

e6

Rubrics can provide the kind of objective detail oriented feedback that is valued by today’s students. Different students may be strong in one area, can use improvement in another or in some cases, resistant or even hostile in others. When it comes to selecting a grading tool, efficiency is enhanced by allowing for the measurement of degrees of variation in skill demonstration. There are biases that we may exhibit while grading, whether we are aware of it or not. Little things that annoy us like spelling atrocities, illegible penmanship, size 8 font, etc, can make us grade more strictly. Rubrics enhance our objectivity by keeping the focus firmly on the criteria being measured. Factors such as fatigue, hunger and time pressure can be unconscious influences affecting the tendency to grade more leniently or more strictly. A well-constructed rubric provides less room for scoring inconsistencies of any kind.

Furthermore, rubrics save a great deal of time (and brain cells) during grading. The more clarity you have and the more precise your descriptions, the more time you cut from grading. Because you are focused on the few competencies you are measuring, you can more readily distinguish between levels of performance across the wide range of papers you receive.

With all of the advantages to using rubrics, why might people resist the idea? Basically, creating and utilizing rubrics requires learning a new skill. People generally feel confident working with the skills they already possess and are reluctant to start playing around with new tools that, in the beginning, feel awkward and uncomfortable. Every semester, we ask students to take risks, to see mistakes as learning opportunities, and to apply themselves in the development of skills that will help them to achieve their goals. Many feel insecure, uncomfortable and vulnerable when taking the risks involved in learning something new. Trying something new can be scary but we reassure them that learning something new is a worthwhile adventure (hint, hint, wink, wink, nudge, nudge…).

1. “Rubrics take too much time to make!” False. When we first start making and utilizing rubrics, it can be time consuming simply because we are learning something new. With practice, rubrics become incredibly simple and quick to design. After making a few, you will be coming up with criteria and descriptions while showering. Bonus: Grading time is significantly reduced!

2. “You can’t make rubrics for math!”

False. You can make rubrics for anything in the universe! Take a look at this one from John Carroll University in Ohio. (Permission to share) http://www.jcu.edu/academic/planassess/pdf/Assessment%20Resources/Rubrics/Science%20and%20math/QuantRubric.pdf

http://www.jcu.edu/academic/planassess/pdf/Assessment%20Resources/Rubrics/Science%20and%20math/QuantRubric.pdf

http://www.jcu.edu/academic/planassess/pdf/Assessment%20Resources/Rubrics/Science%20and%20math/QuantRubric.pdf


Pag

e7

Quantitative

Rubric

Algebraic Graphic Numeric Verbal

4 Exemplary

- Accurately derives, uses, and/or manipulates algebraic

representations of pertinent data

and/or problem elements. - Interprets logical relationships

between problem elements and aptly

characterizes the underlying logic with mathematical symbols.

- Graphic displays accurately and completely represent the data

and/or algebraic relationships

between problem elements, are accompanied by equations from

analysis, and have clear labels.

- Analysis draws appropriate inferences from graphic displays.

- Accurately identifies quantitative information

pertinent to the solution of a

problem. - Uses quantitative

information in a solution

that supports appropriate translations between

different modes of thinking

(algebraic, graphic, and/or verbal) about the problem.

- Succinct explanation presents a reasoned account

of the answer, which may

include pertinent examples or counter-examples.

- Appropriate translations

between different modes of thinking (algebraic, graphic,

and/or numeric) about the

problem are used to establish a sound scholarly explanation

and explicate the underlying

logic of the answer.

3

Competent

- Algebraic representations are

accurate and demonstrate competent

translation of the problem into mathematical symbols.

- Logical interpretations of problem

elements are correct, but are in some ways incomplete to support full

integration of different modes of

thinking (graphic, numeric, and/or verbal).

- Graphic displays are accurate

and completely represent the data

and/or algebraic relationships between problem elements.

- Graphic displays may not be

accompanied with complete and appropriate analytic inference.

- Correctly identifies

quantitative information to

solve the problem. - Numeric information asked

for in the problem is given,

but the solution does not go beyond the question posed.

- Robust interpretation of

the numeric information is not presented.

- The answer is correct and

demonstrates thoroughness

and competence working with the task’s mathematical

concepts and processes.

- The argument may not completely capture

appropriate translations

between different modes of thinking (algebraic, graphic,

and/or numeric) about the

problem.

2 Developing

- It may be unclear what algebraic relationships are used that best

and/or correctly characterize

pertinent data and/or problem elements.

- Graphic displays are incomplete, poorly labeled, and/or hard to

follow.

- Graphic displays are not presented in ways that support

further interpretation of the

elements of the problem.

- Quantitative information is partially correct but

incomplete.

- Quantitative information is presented in ways that do

not lead to other modes of

thinking (algebraic, graphic, and/or verbal) about the

problem.

- The answer may be partially correct, but the argument may

be poorly focused or weak or

poorly conceived. - Major ideas related to the

content may be ignored or

inadequately explored. - Appropriate translations

between different modes of

thinking (algebraic, graphic, and/or numeric) about the

problem may to inadequately

explored or incorrectly reported.

3. “Rubrics are an attempt at standardization!” False. Rubrics provide creative latitude even when measuring a state or departmental learning outcome because the criteria are determined by what YOU value, what YOU think students should be able to do outside of the classroom. You are free to revise the criteria as often as you like. There is room for creativity in both content and structure.


Pag

e8

4. “Rubrics are part of an administrative plot to control faculty, end contracts and crush our spirits!”

False. Rubrics are not another brick in the wall. There is no mandate forcing faculty into a little rubric meat grinder. YOU decide what to measure, how you measure it, and how you want to respond to findings after using it. Rubrics are an excellent and empowering admin-faculty communication vehicle because criteria and results are so specific that no explanation or justification is needed. You can say goodbye to writing dissertation abstracts on your appraisal form. Additionally, there is nothing that could be used punitively. A report to administration basically says: “My students were strong here, and weak here, so I’m gonna mess around with that one a bit by doing such and such… and I’ll let you know how it goes when I assess it a year from now.” In my opinion, if admin really wanted to oppress faculty, they would forbid the use of rubrics.

Now that we’ve cleared that up, let’s CROSS THE RUBRICON!

© Gerald Scarfe. Used with kind permission.


Pag

e9

Subject: the standard or learning outcome being measured. Criteria: Specific tasks involved in demonstrating the learning outcome. Performance levels: Commonly 3-5 levels are established to distinguish between low, moderate and high performers. Descriptors: Clear description of what is required at each performance level for each criterion.

Step 1: Clarify the Subject

Select a topic or assignment and jot down what you want students to be able to do. You can start with a generalized statement containing an action such as “solve algebraic formulae”, “write a persuasive essay” or “identify muscles”.

____________________________________________________

____________________________________________________

____________________________________________________

____________________________________________________

____________________________________________________

You can use the subject you identified above or you can make the subject a little more specific. For example, “accurately identify the muscles involved in a specific movement sequence”.

____________________________________________________

____________________________________________________

____________________________________________________

____________________________________________________

____________________________________________________

Analytic Rubric Components

Subject CriteriaPerformance

levelsDescriptors


Pag

e10

To illustrate the process, we will use the foodnetwork.com description of the performance activity involved in the cooking contest, “Chopped”.

The parts of the following description indicating the subject or learning outcome that will be measured are circled in red.

“Chopped is a cooking competition show that is all about skill, speed and ingenuity. …

The challenge? They have seconds to plan and 30 minutes to cook an amazing course with the basket of mystery ingredients given to them moments before the clock starts ticking! And the pressure doesn't stop there. Once they've completed their dish, they've got to survive the Chopping Block where our three judges are waiting to be wowed and not shy about voicing their culinary criticisms! … Chopped is a game of passion, expertise and skill — and in the end, only one chef will survive the Chopping Block. Who will make the cut? The answer is on Chopped!” (http://www.foodnetwork.com/shows/chopped.html).

Subject: Contestants will prepare a meal in 30 minutes using mystery ingredients,

demonstrating speed, skill and ingenuity.

Step 2: Identify the Criteria

Identify the tasks involved in performing the learning outcome that you just created. What do you value most? What would you want students to be able to do specifically? What do you complain students can ‘t do? If I were to manage a restaurant, I would want my employee to show professionalism. The subject would be: Employee demonstrates professionalism.

Based on the subject, the rubric title could simply be “Professionalism”. Now I make a list of things I believe employees do that demonstrates professionalism.

http://www.foodnetwork.com/shows/chopped.html


Pag

e11

Professionalism Criteria:

Reliable

Courteous

Follows sanitary guidelines

Break down your subject or learning outcome into tasks that you value or tasks that you complain students are not performing or are not performing sufficiently.

Subject: ________________________________________________________________________________________________________________________________________________________________________

Tasks: ________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ Reflecting on the criteria for the “restaurant employee professionalism” rubric, one might note that “follows sanitary guidelines” is a specific and objective task, where “courteous” and “reliable” are sort of vague and subjective. With language, groups of people have shared hidden assumptions but faculty and students are often from different generations and might have a very different idea of what is meant by “courteous” or “reliable”. What tasks reflect reliability and courtesy? The list might look something like this:

Professionalism Rubric Criteria:

Arrives on time (reliability)

Patient and informative when communicating with customers (courteous)

Prompt and pleasant when responding to requests (courteous)

Follows sanitary guidelines

Reflect on your criteria list. Are there words that are vague, subjective or rely on shared assumptions? Refine your list that criteria are specific and more objective tasks.

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________


Pag

e12

In the learning outcome created using the Chopped show description, we know that contestants must demonstrate speed, skill and ingenuity. Tasks that may demonstrate these requirements might hypothetically include:

“Chopped” Rubric Criteria:

Meal was cooked and plated before time ran out (speed)

Ingredients were cooked to the proper degree (skill)

Different tastes were well-balanced and well-blended (skill)

Meal was proper portion size and presentation was aesthetically appealing (skill)

All mystery ingredients were incorporated (ingenuity)

Ingredients were creatively incorporated (ingenuity)

Step 3: Establish Performance Levels

How many performance levels would there be for the criteria? Some criteria may fit well into 3 levels. For example:

1. Poor 2.Developing 3.Outstanding

Ingredients were cooked to the proper degree

An important ingredient was significantly undercooked or overcooked. Inedible.

An important ingredient may be slightly undercooked or overcooked. Edible.

An important ingredient was cooked to perfection.

All mystery ingredients were incorporated

Plate was missing one of the mystery ingredients.

A mystery ingredient was plated without preparing it in any way or integrating it with other ingredients. Thrown in.

All mystery ingredients were prepared and/or integrated with other ingredients.

There are many criteria that do not fall neatly into 3 performance levels. Let’s say someone asks how often you exercise and gives you the following options:

Always Sometimes Never.

“Always” and “never” are extremes and are fairly specific. “Sometimes”, on the other hand, indicates a broad range. Sometimes could be broken down into:

Rarely Sometimes Often.

Look at your criteria and decide if it would be best to work with 3 performance levels or to extend it to five.


Pag

e13

Step 4: Descriptors

Define or describe the specific characteristics of criteria at each performance level. It is often easiest to describe the extremes first. What would outstanding look like? If I were to make a student engagement

rubric, the first criteria might be the following:

Student Engagement Rubric 1: Unacceptable

3: Developing

5. Outstanding

Student management of electronic devices is conducive to student’s engagement.

Some people find it easier to start describing what they don’t want to see, or the worst case scenario. It doesn’t matter if you start with the lowest or highest performance level. For this criteria, outstanding might be described as:

Student Engagement Rubric 1: Unacceptable

3: Developing

5. Outstanding


Electronic devices are always silent and stored unless required for class activity

The opposite extreme would then be described.

Student Engagement Rubric

1: Unacceptable 3: Developing

5. Outstanding


Student makes no attempt to be discreet, texts constantly, phone rings repeatedly, and/or answers phone in class unapologetically.


Describe what it would look like in the middle between a pair of opposites.

Student Engagement Rubric

1: Unacceptable 3: Developing 5. Outstanding


Student makes no attempt to be discreet, texts constantly, phone rings repeatedly, and/or answers phone in class unapologetically.

May discreetly text on ocassion while making effort to hide cell phone, and/or student is embarrassed and apologizes if ringer volume was accidentally left on.



Pag

e14

Use your imagination, picture any predictable scenarios regarding students demonstrating the criterion. What sorts of actions or attitudes might students demonstrate? If someone were to ask me to summarize this step, it would be “write descriptive stuff and then scribble it out when you come up with better stuff.” Descriptions should be written with language that would be understandable to a student or faculty from another discipline.

Use the space below to write a description from one of your criteria of an exemplary or unacceptable demonstration. (Lots of space in the event that you scribble a few times because you are coming up with better stuff.)

Criterion 1- Unacceptable 2- Developing 3- Outstanding

Write the description of the opposite extreme, followed by the middle level. Repeat for each criterion and you have a complete rubric.


Pag

e15

Analytic Rubric Subject/ Learning Outcome ________________________________________________________________________________________________________________________________________________

Rubric Title: __________________________________________

Criterion 1- Poor 2- Developing 3- Outstanding

Reflect on the descriptors. Are the descriptions clear and specific? Do they accurately convey what will be required at each level of performance? During construction we sometimes realize that a competency description contains two tasks that might be best to divide into two separate criteria. Simply expand the rubric vertically to include additional criteria. If the middle performance level seems like it may encompass a broad range, expand the rubric horizontally to increase the number of levels.

Subject or Learning Outcome _______________________________________________________________________________________

Rubric Title________________________________________

Criteria

1- Unaccaptable 2-Emerging 3- Developing 4- Proficient 5- Exemplary

Review

Prior to implementation, the clarity of your rubric can be checked in a number of ways, including:

Reflect to make sure that criteria match what you find useful, meaningful or significant.

Ask a colleague to review and point out anything that could be further clarified

Review with students to make sure that the expectations are clear to them

Reflect on your rubric and use the space below to work on modifications of criteria or descriptions for greater clarity or accuracy.

Scoring is quick when descriptions are clear and accurate. Students’ work will reveal the strengths and weaknesses of the rubric, indicating where specific improvements could be made. For example, if a student’s response does not fit neatly into a level of performance, consider revising the description or breaking the criteria into two separate competencies. This is a natural step in developing rubric construction skills and is expected. Necessary improvements are easy to identify and quick to make.

Scoring student work with a colleague is invaluable. Different graders may see or interpret things differently. When colleagues reach consensus on a score, the rubric likely provides a solid measurement for the competency. If the graders score more than one level apart, a discussion as to why each score was provided will illuminate different points of view and often results in reaching consensus. If colleagues cannot agree, reasonable consideration should be given to revisions in that area.

While rubrics are an objective grading tool, biases may factor into grading. For example, sloppy handwriting, spelling atrocities or even the untidy fringes of ripped out notebook paper may be pet peeves that influence your grading. Self-binding is the removal of the possibility of being influenced by factors that are not relevant to the listed criteria. This technique is an essential part of “The Voice”, a televised vocal competition series on NBC. As the judges know that many aspects of personal appearance give impressions that can influence their judging, they face away from the candidates to focus solely on the vocal qualities.

For self-binding with rubrics, you can:

Have student work numbered so that names are not visible. Have essays typed prior to grading so that handwriting and spelling are not distracting

(unless handwriting and spelling are being assessed). Grade with a colleague and listen objectively to differences of opinion. Grade after eating and when well-rested.

Rubric Scoring


Pag

e19

Rubrics evolve. Revisions or fine-tuning of the rubric are commonly called for but are quick and easy to make. Some of the more common revisions include:

Separating multiple aspects of a task into separate criteria Clarifying descriptors Expanding performance levels

Rubric Scoring Summaries are easy to use for departmental or IRE assessment reporting. With analytic rubrics, it is easy to determine the percentage of students that met the established benchmark standard and develop an improvement plan for implementation with the next group of students. Over the years, massage department improvement plans have included:

Improvements to the assessment tool, including: o Rubric wording modifications for improved clarification o Splitting criterion expectations into two or more distinct competencies o Refining performance level descriptors

Improvements to the assessment process, including: o Using a summative assessment formatively. For example:

Giving the assignment at the beginning of the course, establishing a benchmark to compare with student progress at the end of the course.

Assessing the same skills in the beginning, middle and end of course to chart student progress and to evaluate the efficacy of instructional components given at specific points throughout the course.

Improvements to the assignment being scored by the rubric, including: o Refining assignment descriptions or prompts to clarify requirements or expectations o Modifying the form of the assignment to evoke responses that more accurately reflect

performance levels.

Instructional enhancements, including: o Activities or other enhancements designed to help students develop in the noted area of

weakness. Following is an example of a rubric assessment summary:

Revisions

Scoring Summaries


Pag

e20

After being told that my cooking habits and skills are less than desirable (“kind of suck”), I constructed a rubric based on what my family reports to value in home cooked meals. A complete assessment report follows.

Home Cooked Meal Analytic Rubric Mindy Yale 3/10/14

1- unacceptable 2- Seriously? 3- Developing 4-Acceptable 5-Outstanding

Meals are tasty and satisfying.

Rocks are easier to digest. I eat before I come home and pretend I’m too full or exhausted to eat whatever that is. Totally dissatisfied.

I pretend to like it but I look forward to when we eat restaurant food. Rarely tasty, rarely satisfied.

May or may not be tasty and/or satisfying. May produce mild indigestion or gas. Inconsistent.

Consistently tasty and satisfying. Nice variety.

Gourmet deliciousness. I feel completely satisfied, physically nourished and have a sense of psychological well- being.

Meals are provided.

Never. I am entirely responsible for providing the meal, if I want to eat anything.

Mostly eat out, pick up or order in. Cooks once in a blue moon. I never know when a home cooked meal is coming.

Meals are cooked 1-2 days a week, eat out or take out on other days.

Meals are cooked 4-5 days a week, eat out or take out on other days.

Dinner is on the table at the same time every day, ten minutes following my arrival, 5 days a week minimum.

Presentation is appealing.

Unidentifiable blob. Smell is horrifying. Do not eat.

Unappetizing. Food is recognizable, but not well organized or attractive. Smell is potent and may linger in the curtains for days. Approach with caution.

Looks somewhat appealing but smells a bit strange or smells somewhat appealing but is not attractive. Organization could be improved.

Looks appetizing, aroma is expected given the ingredients, well organized on the plate.

Gourmet presentation. Interesting sauces are skillfully drizzled around the main course, appealing aroma, and attractive to the eye; a work of art.

Portion Sparrows require larger portions or so much food was made that it is disgracefully excessive.

Portion size is way too small or overwhelmingly large.

Portion size may be somewhat small or somewhat excessive.

Portion size is appropriate for the meal, minor imbalances in meal components.

Perfect portion. Meal components are well balanced.


Pag

e21

Assessment reporting period: October, 2013 – October 2014

Home Cooking Assessment Summary

Tasty and Satisfying: 3 Meals Provided: 2

Presentation: 4 Portion: 4

Average Grade: 3.25

Assessment Outcome:

This grade falls below the benchmark standard of 4 or higher, established by the family.

Improvement Plan: The competencies targeted for improvement are:

o “tasty and satisfying” and o number of “meals provided”.

When analyzed, time management seemed to be a significant factor in the number of meals provided. Accordingly, an assignment will be added to create and implement a meal planning schedule. This competency will be reassessed in October 2015. For improving performance in the taste and satisfaction category, instructional materials will be modified to include resources such as allrecipes.com and YouTube tutorials. This plan will be implemented in January of 2015 and reassessed in October of 2015. Additionally, the analysis revealed that the family values two of the competencies more highly than the others. As taste/satisfaction and number of meals provided seem to be more important to a successful demonstration of the learning outcome, a rubric modification will increase the weight of the scores of those two competencies x2.

Prior to assessment, consistent negative feedback received during “testing”, resulted in demotivation and ultimately apathy with regard to cooking. As analytic rubrics are specific and efficient, the formal assessment showed that there were some strengths along with some areas of weakness. This was encouraging and increased motivation because the path to improvement was clear and specific. I felt capable of making improvements in those areas and was re-inspired. The report also allowed for a reasonable time frame in which to implement the improvement plan for re-assessment. The example provided here is a reflection of the experiences working with rubrics to assess all of our Program Learning Outcomes over the past four years. Instructors are motivated to make improvements because we see where students are successful and the course for improvement is clear, tangible and targets specific competencies. We have no frustration or sense of overwhelm when it comes to figuring out how to support students on their path to success.

Silly Sample Analytic Rubric: Husband Home Repairs Mindy Yale 3/10/14 1-Unacceptable 2-Seriously? 3-Developing 4-Satisfactory 5-Outstanding

Demonstrates ability to complete home repair tasks

Repairmageddon. Dramatic exacerbation of the problem and/or repair effort results in a new, bigger problem.

Somewhat handy but work is shoddy and aesthetically offensive. Clean up may involve finding important parts that he claims are “extra”.

May sweat a lot, make grunting sounds or curse but shows determination. May need minor first aid upon completion.

Is efficient with general repairs and genuinely knows “a guy” for all major repair work.

MacGyver. Can repair anything and is unusually resourceful. Work is efficient, tidy and repairs last.

Demonstrates willingness to complete home repair tasks

Demonstrates total unwillingness. Resents task, resents wife for involving him in the matter. Self identifies as lazy and apathetic with regard to home repairs.

Demonstrates strong resistance. May need extrinsic motivation (such as punishment/reward) from wife and/or may require cheerleading throughout process with praise following completion.

Drags self to repair site. May require emotional support in the guise of needing a reader for manufacturer’s instructions and/or may need a gesture of appreciation following completion.

Willingly attends to repairs after being fed, before/after a televised sporting event, or following through with other obligations. May Appreciate a pat on back following task.

Enthusiastically attends to repairs. Takes pride in workmanship and derives a sense of personal satisfaction upon completion. May have to be called away from task for dinner.

Starts and Completes home repairs within a reasonable time frame

Doesn’t start job and does not make phone call to hire professional. Wife must handle all home repairs.

Takes things apart but does not put back together. Leaves project incomplete for 6 months or more before going back to it or deciding to call a professional.

May walk past item for several weeks before starting, but completes jobs within 1-3 months or calls professional within same time frame.

Immediately assesses and completes job within 1 month, depending on pre-existing high priority obligations or calls professional within 48 hours.

Immediately assesses the situation and completes the job following assessment or immediately hires professional following assessment.

Communication regarding home repairs is clear, informative and demonstrates affective listening skills

May say “ok” but avoids repairs even when being reminded passively, passive aggressively and with extreme aggression in that order. Deflects and accuses wife of nagging. Couples Counseling is recommended.

Identifies required resources (“that is going to require a step ladder”) and offers reassurance (“Don’t worry about it honey, It’ll get done”). Reassurance sounds sincere but may be an avoidance tactic. No additional information is provided.

Reassures wife that task will be completed but may not be clear or honest regarding required resources, or completion time with self or others.

Communicates clearly and honestly based on current knowledge about what may be required and/or completion schedule.

Communication is clear honest, thorough and accurate regarding the needed resources and completion time.


Pag

e23

Crazy Muscle Dance Party Analytic Rubric. Mindy Yale, 2008 Following is the actual rubric designed and used for a summative assessment of students’ ability to identify the muscles involved in specific movement sequences.

Assignment Description: In the culminating performance task, students design and present a movement sequence and teach it to the class while identifying the specific actions and the muscles responsible for those actions, in each step of the sequence. Their performance will demonstrate the ability to analyze complex movements, which is a pre-requisite to treating clients with postural imbalances and sports related injuries.

Crazy Muscle Dance Party Super Crazy Cool : 3 Kind of Crazy, Kind of Cool : 2 So Not Cool : 1

Student performed a dance or movement sequence reflecting effort, creativity and preparation.

Performance was unique or very creative (included music, Costume/props) AND was well rehearsed. Sequence is memorized and practiced.

Performance was creative (included music and/or costume, props, or multi media) and/or was somewhat rehearsed. Student may use video to support memory in demonstrating sequence.

Performance was not Creative (no music, Costume, props, etc) and/or Performance was not rehearsed. Student relies entirely on video to demonstrate sequence or is obviously “winging it.”

Student accurately identifies muscle groups involved in the dance or movement sequence.

Accurately identified more than 3 muscle groups (actions) involved in the sequence.

Accurately identified 2 -3 muscle groups (actions) involved in the sequence.

Accurately identified 1 muscle group (action) involved in the sequence or failed to accurately identify muscle groups involved.

Student accurately names the muscles that perform the actions identified.

Accuracy in identifying all of the major muscles involved in those actions and most or all synergists.

Accurately identified some of the major muscles involved, may have missed one or two and identified some synergists.

Unable to accurately identify major muscles involved or makes big mistakes regarding important muscles and actions.

Student taught the sequence step-by-step, combining verbal instruction with physical demonstration.

Able to teach the movement Step-by-step, naming actions accurately while demonstrating.

Able to teach the movement Step-by-step but unable To coordinate verbal instruction With physical demonstration. May say one action but is performing another or fails to name the action while teaching the movement.

Demonstrates movement But unable to break it down step-by-step in physical demonstration with verbal instruction or relies on a video to teach the sequence.


Pag

e24

Palm Beach State College QEP Critical Thinking Analytic Rubric July 2012

QEP / Critical Thinking Rubric To be used for scoring student responses on scenarios to measure QEP Outcomes 1-3.

Unacceptable (1) The student does not demonstrate the skills required for the outcome.

Emerging (2) The demonstration of skills required for the outcome is weak.

Developing (3) The demonstration of skills required for the outcome is present, but needs improvement.

Proficient (4) The demonstration of skills required for the outcome is apparent.

Exemplary (5) The demonstration of skills required for the outcome is excellent.

Outcome 1 Students are able to analyze and interpret relevant information.

Student does not analyze or interpret the information.

Student uses analysis and interpretation minimally.

Student response includes analysis and interpretation, but the response is somewhat incomplete or inaccurate.

Student response proficiently analyzes and interprets the relevant information.

Student response soundly analyzes and interprets the relevant information.

Outcome 2 Students are able to reach conclusions based on a demonstrated reasoning process.

Student does not provide a relevant conclusion nor demonstrate a reasoning process.

Student conclusion is either not relevant or it is not based on a demonstrated reasoning process.

Student reaches a relevant conclusion, but the reasoning process is somewhat incomplete or inaccurate.

Student reaches a relevant conclusion and proficiently demonstrates that the conclusion is based on a reasoning process.

Student reaches a relevant conclusion and soundly demonstrates that the conclusion is based on a reasoning process.

Outcome 3 Students are able to evaluate and explain relevant information.

Student does not evaluate or explain the information.

Student either does not evaluate the information, or poorly explains the results of the evaluation.

Student evaluates the information and explains the results, but the response is somewhat incomplete or inaccurate.

Student response proficiently demonstrates the skills required to evaluate and explain.

Student response soundly demonstrates the skills required to evaluate and explain.

The QEP Rubric was deliberately designed to be applicable across the disciplines. Accordingly, the Learning Outcomes represent categories of skills. This rubric can be used as is or adapted to reflect the associated skills more specifically. On the following page is an example of how the QEP Critical Thinking Rubric was adapted for a massage therapy assignment.


Pag

e25

Massage Therapy 3: Integrative Case Study Rubric Summary

Adapted Palm Beach State College QEP Critical Thinking Learning Outcomes Analytical Rubric

Outcome: 1: Student does not demonstrate the outcome skills.

2: Demonstration of

outcome skills is weak

3: Demonstration of

outcome skills is evident but needs improvement

4: Demonstration of outcome

skills is proficient

5: Demonstration of outcome skills

is exemplary

Analyze and interpret relevant information to draw an appropriate treatment plan.

Student does not gather, analyze or interpret relevant information from verbal or written intake and does not note relevant observations.

Student minimally identifies relevant information through intake and observation, insufficient analysis and interpretation for drawing an appropriate treatment plan.

Analyzes and interprets relevant information received from intake and observation but may be somewhat incomplete or inaccurate.

Analyzes and interprets relevant information received from intake and observation sufficient for developing an appropriate treatment plan.

Analyzes and interprets relevant information from intake and observation; including subtle nuances to develop a specific and appropriate treatment plan.

Treatment plans reflect sound clinical reasoning.

Treatment plans are not based on evidence supplied by observation and/or intake, and/or is inappropriate or contraindicated.

Treatment plans reflect minimal reasoning based on observation and intake and/or barely address client needs.

Treatment plans reflect clinical reasoning but may be somewhat inaccurate or incomplete in addressing client needs.

Treatment plans reflect sound clinical reasoning based on observation and intake and incorporate some modalities appropriately in addressing client needs.

Demonstrates ability to create an individualized treatment plan including the appropriate application of various modalities in addressing client needs beyond the physical or stated indications.

Evaluates and Explains

Does not identify additional or alternate treatment possibilities or solutions and/or does not explain reasoning process.

Poorly explains reasoning process and identification of additional or alternate treatment possibilities is insufficient or inaccurate.

May be somewhat incomplete or inaccurate in explaining reasoning process or in identifying additional or alternate possibilities.

Provides sufficient explanation of reasoning process and identifies acceptable additional or alternate treatment possibilities.

Provides clear explanation, demonstrates creative problem solving through clear and specific identification of additional or alternate treatment possibilities.

Draws reasoned conclusions regarding Professional Boundaries.

Is unaware of boundary issues, violates a professional boundary standard and/or does not see boundaries as relevant to treatment. No consideration of implications or consequences.

Demonstrates minimal awareness of boundary issues as relevant to treatment. Minimal consideration of implications or consequences.

Reflects awareness of professional boundary issues but may have an incomplete or inaccurate understanding of issue or proper professional handling.

Reflects sufficient awareness of professional boundary issues and how to handle them, aware of implications or consequences for self and client. Handling of boundary issues is on course level.

Reflects clear awareness self and client and of professional boundary issues throughout treatment, awareness of implications and consequences from a variety of viewpoints and responds professionally to all circumstances.

Palm Beach State College QEP/ Programs and Educational Support Services

Key words from outcome Other words to describe

Analyze, interpret • gathers information • identifies assumptions, reasons, and claims • accurately determines meaning in context of given

information for the given purpose

Draw conclusions • provides reasons or evidence • considers various points of view • considers implications or consequences of actions • separates fact from opinion • (conclusion: decision, position, solution)

Evaluate, explain • assesses credibility of claims, strengths or weaknesses in given information

• identifies additional information that may be helpful in approaching the situation presented

• identifies alternate interpretations, explanations, or solutions

Affective dispositions • willingness to think critically • systematic, open-minded, confident in reasoning, truth-

seeking, judicious, inquisitive, analytical • explore, evaluate, express, and engage –which ones fit

here?

Alternate descriptions are based on key words that are used to assess each skill. Detailed information about the skills is offered by Insight Assessment, Division of California Academic Press, the publisher of the California Critical Thinking Skills Test (CCTST) and California Critical Thinking Disposition Inventory (CCTDI). To review skills assessed by the CCTST, visit: http://www.insightassessment.com/Products/Critical-Thinking-Skills-Tests/California-Critical-Thinking-Skills-Test-CCTST To review skills assessed by the CCTDI, visit: http://www.insightassessment.com/Products/Critical-Thinking-Attributes-Tests/California-Critical-Thinking-Disposition-Inventory-CCTDI

http://www.insightassessment.com/Products/Critical-Thinking-Skills-Tests/California-Critical-Thinking-Skills-Test-CCTST

http://www.insightassessment.com/Products/Critical-Thinking-Skills-Tests/California-Critical-Thinking-Skills-Test-CCTST

http://www.insightassessment.com/Products/Critical-Thinking-Attributes-Tests/California-Critical-Thinking-Disposition-Inventory-CCTDI




Notes:


Pag

e28

Please take a moment to use the following rubric to score this workbook on rubrics. Analytic Rubric Workbook Scoring Rubric:

Crossing the Rubricon

1 Unacceptable

2 Poor

3 Developing

4 Proficient

5 Outstanding

The advantages of analytic rubrics were presented clearly.

Not at all. I have a poor understanding of the advantages of analytic rubrics

The advantages were presented but could use improvement.

The advantages of analytic rubrics were presented clearly.

The advantages of analytic rubrics were presented very clearly. I totally get it.

All components of a rubric were constructed in class.

No components were constructed or I only got as far as the topic.

I have a topic, learning outcome and a page filled with possible criteria.

I have a topic, learning outcome, criteria and some descriptions

I constructed all components and have 3 levels of performance. Descriptors may be incomplete.

I constructed a complete rubric with 3 or 5 levels of performance. Descriptors are complete.

The constructed Rubric is ready to implement.

I chose not to construct one.

My rubric is prepared to become a paper airplane or origami swan.

I feel that mine needs major improvement before I can implement it.

Minor refinements could be made before implementing

Ready to implement!

Participants are confident that they have enough information to construct and grade rubrics on their own.

The information presented verbally, on PowerPoint and in the workbook was entirely insufficient. I will not cross the Rubricon.

Research beyond the course materials would be required for me to feel confident to construct rubrics on my own. I am not sure if I will be able to cross the Rubricon.

I received enough information but feel somewhat overwhelmed. I will feel more confident when I review the workbook on my own. I will cross the Rubricon eventually.

I feel confident that I can construct a basic rubric and expect my confidence to grow through practice. I am in the midst of crossing the Rubricon.

I have crossed the Rubricon!

Any additional comments or feedback that you would like to share?

Thank You for Participating!

crossing the rubricon - palm beach state college · 2014-10-15 · because we are learning...

Documents