+

Utilizing Rubrics in Audio /Visual ProductionNext Step Plans

Corey Jermaine AndersonEDGR – 601 | Educational Research Concordia University – Portland Oregon

+Research QuestionCan developing and utilizing rubrics with my high school A/V Production students help improve the quality of their films for public service announcements and commercials?

+Overview of Research The majority of the research suggests positive outcomes for students and instructors when rubrics

are included as a means of authentic assessment (Jonsson, 2014; Diller and Phelps, 2008.) In Rubrics as a Way of Providing Transparency in Assessment, Jonsson (2014) researched three different assessment situations in professional education.

Although, there is significant evidence that rubrics may enhance the academic experience by providing some transparency to students in regards to grading and expectations, all researchers don’t agree. Common Structural Design Features of Rubrics May Represent a threat to Validity, focuses on the research of Mark Humphry and Sandra Heldsinger (2008).

Typically, students claim that using rubrics helps them to focus their efforts, produce work of higher quality, earn better grades and feel less anxious about assignments (Jonsson, 2014).

It takes time for the raters to truly understand the rubrics in a consistent way; therefore normalizing activities are needed to bring consistency to the process (Diller & Phelps, 2008). Another drawback is the utilization of a common language between instructors (raters) and students (those being rated). Although one part of the explanation for student’s lack of understanding could be attributed to illegible writing, which seems to be a common problem, the real challenge lies in teachers’ use of academic terminology or technical jargon (Jonsson, 2014).

Research found that structural alignment could create a degree of unintended conceptual overlap and redundancy in descriptions and gradations for some pairs of criteria (Humphry & Heldsinger, 2014).

Having students generate the criteria for assessing the performance can serve several purposes (Wolf & Stevens, 2007). In addition to gaining a deeper understanding of what is being taught, Wolf and Stevens (2007) found when you involve students in the development of the rubrics criteria prior to an assignment or project it can help them make more informed choices, offer insights into the quality of their performance.

+Big Idea #1Pros and Cons of Rubrics When considering whether rubrics

can improve the quality of student work, it is important to become familiar with the benefits and drawbacks of this type of assessment. Researchers Diller and Phelps (2008) suggest reliable evaluation of student work and reflection can be accomplished through the use of an assessment rubric. Furthermore, rubrics are effective for authentic assessment, because they make it easy for educators to close the feedback loop into the learning environment (Diller & Phelps, 2008).

The benefits of utilizing rubrics do not come without consequences or drawbacks. There must be a commitment of time for proper training and developing a clear understanding of the meanings of each criterion. It takes time for the raters to truly understand the rubrics in a consistent way; therefore normalizing activities are needed to bring consistency to the process (Diller & Phelps, 2008).

+Big Idea #2Criteria Selection and Gradations of Quality for Rubrics Humphry and Heldsinger’s (2014)

research found that typically, there is no underlying developmental or learning theory that justifies having precisely the same number of qualitatively distinguishable stages across multiple aspects of a construct. This makes it unlikely that the gradations of quality faithfully capture that which is observed in student performances for each criterion separately from other criteria (Humphry & Heldsinger, 2014).

Kenneth Wolf and Ellen Stevens (2007) suggest since there may be one or two criteria that are valued more than the others and they could be given a higher value when calculating the overall score for the performance or product.

+Big Idea #3Best Practices for Implementing Rubrics in the Classroom Jonsson (2014) cites

research that implies that, in order to educate and improve student’s performance; all tasks, criteria and standards must be transparent to both students and teachers. Learning is a partnership and using rubrics and structured collaboration imposes accountability on the system (Benjamin, 2011).

Training students how to use rubrics will increase transparency and effectiveness of usage (Diller & Phelps, 2008; Jonsson, 2014). It takes time for the raters to truly understand the rubrics in a consistent way; therefore, more normalizing activities are needed to bring consistency to the process (Diller & Phelps, 2008).

+Next Steps: Plan of Attack1. Contact the Librarian to see if I

can get help locating more articles on the benefits of including students in the rubric design process.

2. I have identified a few methods of data collection that would prove beneficial.  Observation, interview, and document and artifact analysis techniques result in a great amount of data (McMillan, 2016). 

3. It seems as though this might take some time, so after my research proposal is accepted, some thought on a timeline for conducting and analyzing data would be a good idea.

+Guiding Thoughts and Questions

Should rubrics be lean and represent a few behaviors or robust and specific?

How long does it take to develop and implement the utilization of rubrics in a program?

How might I accomplish this in the classroom setting student buy-in and respect for the benefits of transparency as delivered through rubrics?

What is the best design for a generic rubric?

What is the best design for a more detailed rubric?

How can I learn more about the implications for students of color?

Developing assessment instruments

The best time to implement this research

Triangulation of data and implications

Talk to colleagues about my research and get feedback

Questions to Explore New or Existing Ideas to Further Pursue

+Continuing and Concluding My Research

I have gained a wealth of information regarding the utilization of rubrics in my A/V Production classes to improve the quality of their commercial and public service announcement videos. It was important for me to know the pros and cons to using rubrics as a learning strategy, the best way to develop criterion and qualitative ratings and the best way to implement them in the classroom. Not only can I use them to help teach technical jargon and assess student work but after developing them together, students can use rubrics as a means of self-assessment or during peer review. Using examples and devoting time to training will also help ensure my students are prepared for Destination 2025 (Shelby County Schools Board of Education, n.d.)

+References Benjamin, S. (2011). Simple leadership techniques: Rubrics, Checklists, and structured

collaboration. Phi Delta Kappan, 92(8), 25–31. doi:10.1177/003172171109200806 Diller, K. R., & Phelps, S. F. (2008). Learning outcomes, portfolios, and Rubrics, oh my!

Authentic assessment of an information literacy program. Portal: Libraries and the Academy, 8(1), 75–89. doi:10.1353/pla.2008.0000

Humphry, S. M., & Heldsinger, S. A. (2014). Common structural design features of Rubrics may represent a threat to validity. Educational Researcher, 43(5), 253–263. doi:10.3102/0013189x14542154

Jonsson, A. (2014). Rubrics as a way of providing transparency in assessment. Assessment & Evaluation in Higher Education, 39(7), 840–852. doi:10.1080/02602938.2013.875117

McMillan, J. (2016). Fundamentals of educational research (7th ed.). VitalSource Bookshelf Online: Retrieved from https://mbsdirect.vitalsource.com/#/books/9780133580747/.

Shelby County Schools Board of Education. Destination 2025. Retrieved March 26, 2016, from http://www.scsk12.org/uf/webadmin/foundation/2025/

Wolf, K., & Stevens, E. (2007). The role of rubrics in advancing and assessing student learning. The Journal of Effective Teaching, 7(1), 3–14. Retrieved from http://www.uncw.edu/cte/et/articles/vol7_1/wolf.pdf