scholarly project: developing a clinical rubric
DESCRIPTION
Scholarly project: Developing a clinical rubric. Erin Kibbey Ferris State University. Objectives. Describe the project goals and process taken in developing a clinical rubric Discuss relevant research and survey results influencing rubric development - PowerPoint PPT PresentationTRANSCRIPT
SCHOLARLY PROJECT:DEVELOPING A CLINICAL RUBRIC Erin Kibbey
Ferris State University
Objectives
Describe the project goals and process taken in developing a clinical rubric
Discuss relevant research and survey results influencing rubric development
Describe a plan for future implementation of clinical rubric
Project Description
Goals Create rubrics for measuring clinical competency of
critical care interns at Munson Implement rubrics for use by preceptors working with the
interns Objectives
Obtain references containing information about the use of rubrics and their development
Collaborate with team and staff educators for input Create an evaluation tool Present rubrics to preceptors Trial rubrics on units Obtain and compile feedback
Procedure/Methodology
“A carelessly planned
project takes three times longer to
complete than expected; a
carefully planned
project takes only twice as
long” ~ Golub's Law
Procedure/Methodology
Develop Rubric
Review Standard
s
Needs Assessm
ent
Research
Procedure/Methodology - Step One
Research Topics Clinical performance evaluation Evaluation strategies Using rubrics as a clinical evaluation
method Rubric development Clinical competency development Adult learning theory Cognitive learning theory
Clinical Evaluation Research
Issues Consistency Subjective Values
Norm-referenced: Compared to others
Criterion-referenced: Compared to pre-determined criteria Known in advance &
used as basis for evaluation
Observing performance & arriving at judgments
Compares data collected about performance to a set of standards
(Bonnel, 2008; Gaberson & Oermann, 2010)
Best Practices for Clinical Evaluation Extensive formative evaluation and periodic
summative evaluation Timely & continuous feedback with ways to
improve Based on preset outcomes, clinical objectives,
or competencies Without a guide to the process = No basis for
evaluation NEED a framework for observing performance and
arriving at judgments Preparation and tools to promote consistency in
evaluation (Bonnel, 2008; Gaberson & Oermann, 2010)
Rating Scales & Rubrics
Creation and use of rating scales or rubrics is supported in a wide range of academic subjects Economics, writing, speech, dentistry, and
chiropractic medicine (McGoldrick & Peterson, 2013; O’Donnell et al., 2011; Rezaei &
Lovorn, 2010; Saxton, Belanger, & Becker, 2012; Xiaohua & Canty, 2012)
Can be used for self-evaluation (Bonnel, 2008)
Despite the lack of consistent use of rubrics in clinical nursing education, rubrics have many benefits (Frentsos, 2013)
What is a Rubric?
Rubrics are “scaled tools with levels of achievement and clearly defined criteria placed in a grid. Rubrics establish clear rules for
evaluation and define the criteria for performance”
(O’Donnell, Oakley, Haney, O’Neill, & Taylor, 2011, p. 1163)
What Does the Research Say?S
evera
l B
en
efits
Support adult learning principles
Provide competency documentation required by regulatory agencies
Can improve quality of care Allow more discrimination in
judging behaviors Can increase knowledge gain(Bonnel, 2012; Frentsos, 2013; O’Donnell et al., 2011;
Walvoord & Anderson, 2010)
Procedure/Methodology – Step Two
The first step to rubric
creation is
reflection
(Stevens & Levi,
2005)
Needs assessment survey Target Population -
Critical care intern preceptors
32 Question Survey Sent to 55 Preceptors
Respondent Demographics
12.5%
37.5%22.5%
27.5%
What department do you currently work
and precept in?
ED A2 A3 ICU
DepartmentYears Precepting in Department
17.5%
25.0%27.5
%
15.0%
15.0%
Number of years serving as a precep-
tor in current de-partment at Munson?
less than 1 year
1-3 years
4-6 years
7-10 years
more than 10 years
Respondent Demographics
ED A2 A3 ICUED A2 A3 ICU
02468
101214161820
5
15
911
5
20
1416
Survey Respondents By Unit
# of Respondants Total # of Surveys Sent
Key Findings/Results 1
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
0
5
10
15
20
25
4
10
21
2 1
I have a tool I consistently use to provide written feedback.
Nu
mb
er
of
Resp
on
ses
Key Findings/Results 2
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
02468
101214161820
2
18
12
24
Interns have a tool they con-sistently use for self-assess-
ment of learning.
Nu
mb
er
of
Resp
on
ses
Key Findings/Results 3
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
0
5
10
15
20
25
6
20
9
2 1
Evaluating progression is in-consistent among preceptors.
Nu
mb
er
of
Resp
on
ses
Key Findings/Results 4
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
02468
1012141618
2
16 16
4
0
My unit uses a uniform method for measuring progression of compe-
tence.
Nu
mb
er
of
Resp
on
ses
Key Findings/Results 5
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
0
5
10
15
20
25
0
12
21
41
My unit uses a tool that accu-rately communicates levels of
progress.
Nu
mb
er
of
Resp
on
ses
Key Findings/Results 6
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
0
5
10
15
20
25
0
12
22
31
The criteria for evaluating the progression of competence is
clear.
Nu
mb
er
of
Resp
on
ses
Key Findings/Results 7
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
0
2
4
6
8
10
12
14
16
5
13
15
3
1
I am familiar with the use of rubrics as an evaluation tool.
Nu
mb
er
of
Resp
on
ses
Key Findings/Results 8
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
02468
1012141618
3
16
3
0
15
Rubrics would be beneficial for use in evaluating progression
of competence.
Nu
mb
er
of
Resp
on
ses
Key Finding/Results 9
Strongly Agree
Agree Disagree Strongly Disagree
Unable to comment
0
5
10
15
20
25
2
20
6
0
9
I am interested in using rubrics for evaluating progression of
competence.
Nu
mb
er
of
Resp
on
ses
Procedure/Methodology – Step 3 How is competence defined?
ANA (2010) Expected level of performance that integrates
knowledge, skills, abilities, and judgment Can be influenced by the nature of the situation
IOM (2002) Five Domains: Patient centered care,
interdisciplinary team, evidence-based practice, quality improvement
QSEN adds to these domains: safety & informatics (Cronenwett, et al., 2007)
Procedure/Methodology – Step 3 Standards of Practice
QSEN - Quality & Safety Education for Nurses ANA – American Nurses Association AACN – American AssociationOf Critical-Care Nurses IOM – Institute of Medicine NCSBN – National Council of State Board of Nursing Joint Commission
Benner’s Novice to Expert Model(ANA, 2010)
Rubric Development
3 Main partsScale, Dimensions, Description of dimensions
(O’Donnell et al., 2011)
Rubric Development
Reflection
Defining Learning Objectives
Grouping Similar Components
Applying dimensions & descriptions
(Kirkpatrick & DeWitt, 2012; O’Donnell et al., 2011; Stevens & Levi, 2005)
Important Design Considerations Ease of use Valid & Reliable Fair Flexible & Adaptable Evaluated
(Bargainnier, 2003; O’Donnell et al., 2011; Stevens & Levin, 2005)
What’s the Next Step?
Feedback & evaluation from you!
Metarubric Reliability and
validity
Rubric evaluati
on
Preceptor
Education
Trial Use
Feedback &
Revision
(Stevens & Levi, 2005; O’Donnell et al., 2011)
Questions and Discussion
References
American Nurses Association [ANA]. (2010). Nursing scope and standards of practice (2nd ed.). Silver Spring, MD: American Nurses Association.
Bargainnier, S. (2003). Fundamentals of rubrics. Retrieved from http://www.webpages.uidaho.edu/ele/scholars/practices/Evaluating_Projects/Resources/Using_Rubrics.pdf
Bonnel, W. (2012). Clinical performance evaluation. In D. Billings & J. Halstead (Eds.), Teaching in nursing: A guide for faculty (4th ed.). (pp. 485-502). St. Louis, MO: Elsevier Saunders.
Cronenwett, L., Sherwood, G., Barnsteiner J., Disch, J., Johnson, J., Mitchell, P., . . . Warren, J. (2007). Quality and safety education for nurses. Nursing Outlook, 55(3), 122-131.
Frentsos, J. M. (2013). Rubrics role in measuring nursing staff competencies. Journal for Nurses in Professional Development, 29(1), 19-23.
Gaberson, K. & Oermann, M. (2010). Clinical teaching strategies in nursing (3rd ed.). NY: Springer Publishing Company.
Hallgren, K. A. (2012). Computing inter-rater reliability for observational data: An overview and tutorial. Tutorials in Quantitative Methods for Psychology, 8(1), 23-34.
Institute of Medicine [IOM]. (2002). Who will keep the public health: Educating health professionals for the 21st century. Washington, DC: National Academy Press.
References continued
McGoldrick, K., & Peterson, B. (2013). Using rubrics in economics. International Review of Economics Education, 12, 33-47.
O’Donnell, J.A., Oakley, M., Haney, S., O’Neill, P.N., & Taylor, D. (2011). Rubrics 101: A primer for rubric development in dental education. Journal of Dental Education, 75(9), 1163-1175.
Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15, 18-39.
Saxton, E., Belanger, S., & Becker, W. (2012). The critical thinking analytic rubric (CTAR): Investigating intra-rater and inter-rater reliability of a scoring mechanism for critical thinking performance assessments. Assessing Writing, 17(4), 251-271.
Stevens, D. D., & Levi, A. J. (2005). Introduction to rubrics: An assessment tool to save grading time, convey effective feedback, and promote student learning . Sterling, VA: Stylus. Retrieved from https://resources.oncourse.iu.edu/access/content/user/fpawan/L540%20_%20CBI/steven-rubrics.pdf
Walvoord, B., & Anderson, V. A. (2010). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey-Bass.
Xiaohua, H., & Canty, A. (2012). Empowering student learning through rubric-referenced self-assessment. Journal of Chiropractic Education, 26(1), 24-31.