degrees of quality
DESCRIPTION
Degrees of Quality. Terrel L. Rhodes, Association of American Colleges and Universities October 19, 2012. 1 . Have you heard of “Swirling”?. “To be blunt: The colleges must redesign their institutions, their mission and their students’ educational experiences - PowerPoint PPT PresentationTRANSCRIPT
Degrees of Quality
Terrel L. Rhodes, Association of American Colleges and Universities
October 19, 2012
1. Have you heard of
“Swirling”?
“The American Dream is imperiled.”
“To be blunt: The colleges must
redesign their institutions,
their mission and their students’
educational experiences
to ensure that they meet the needs
of a changing society.”
http://www.nola.com/newsflash/index.ssf/story/performance-funding-making-its-way-into-higher/150ea4cc13ebc91cae1f6d386a6ca090
The message to higher education leaders is simple:
"If you want more money, prove you deserve it."
"They're frustrated about completions.
They're frustrated about transfers.
They're frustrated that students
are taking six or seven years to graduate!"
The Evidence is Compelling...
Too Many Students Are
Underachieving!
The Evidence is Compelling...
Too Few Institutions Are Responding!
2. Have you heard of
“Badges”??
What Do Employers Want?
20%
20%
59%
Which is more important for recent college graduates who want to pursue advancement and long-term career success at your company?
BOTH in-depth AND broad range of skills and knowledge
Broad range of skills and knowledge that apply to a range of fields or positions
In-depth knowledge and skills that apply to a specific field or position
“Irrespective of college major or institutional selectivity, what matters to career success is students’ development of a broad set of cross-cutting capacities…”
—Anthony Carnevale, Georgetown University
Center on Education and the Workforce
9
Expecting students to complete a significant project before graduation that demonstrates their depth of knowledge in their major AND their acquisition of analytical, problem-solving, and communication skills (62% help a lot)
Expecting students to complete an internship or community-based field project to connect classroom learning with real-world experiences (66%)
Ensuring that students develop the skills to research questions in their field and develop evidence-based analyses (57%)
Expecting students to work through ethical issues and debates to form their own judgments about the issues at stake (48%)
Source: Raising the Bar (AAC&U, 2010)
Employers Assess the Potential Value of Emerging Educational
Practices% saying each would help a lot/fair amount to prepare college students for success
84%
81%
81%
73%
2. Have you heard of the “Completion
Agenda”?
3. Have you heard of “the
DQP”?
What is the Degree
Qualification’s Profile (DQP)?
How is the DQP
different?
CompetencyThat is neither trivial or
snackable...
LEAP Promotes Essential Learning OutcomesA Guiding Vision and National Benchmarks for College Learning and Liberal Education in the 21st Century
High Impact PracticesHelping Students Achieve the Essential Learning Outcomes
Authentic AssessmentsProbing Whether Students Can APPLY Their Learning – to Complex Problems and Real-World Challenges
Inclusive ExcellenceDiversity, equity, quality of learning for all groups of students
High Impact Practices
First-Year Seminars and Experiences
Common Intellectual Experiences
Learning Communities
Writing-Intensive Courses
Collaborative Assignments and Projects
High Impact PracticesUndergraduate Research
Diversity/Global Learning
Service Learning, Community-Based Learning
Internships
Capstone Courses and Projects
ePortfolios
LEAP Principles of Excellence
Principle One Aim High—and Make Excellence
Inclusive
Principle Two Give Students a Compass
Principle Three Teach the Arts of Inquiry
and Innovation
Principle Four Engage the Big Questions
Principle Five Connect Knowledge with
Choices and Action
Principle Six Foster Civic, Intercultural,
and Ethical Learning
Principle Seven Assess Students’ Ability to
Apply Learning to
Complex Problems
Why Did AAC&U Join the DQP Effort?
Focus on Student Performance
Research ProjectsPapers
PerformancesCreative
The DQP Emphasizes....
Integration of learning
Application of learning
The DQPhttp://www.luminafoundation.org/publications/The_Degree_Qualifications_Profile.pdf
1.Specialized Knowledge2.Broad Integrative Knowledge3.Intellectual Skills and
Abilities4.Applied Learning5.Civic Learning
The Lumina Degree Profile – in Brief –
Outlines Competencies Required
for the Award of Degrees
The Need?No “consistent public
understanding” of what degrees mean....
High Standards do not need standardization!
Associates Level
Bachelors Level
Masters Level
An Example...Communication Fluency
Associate Level: The student presents substantially error-free prose in both argumentative and narrative forms to general and specialized audiences
Bachelor’s Level: The student constructs sustained, coherent arguments and/or narratives and/or explications of technical issues and processes, in two media, to general and specialized audiences
Master’s Level: The student creates sustained, coherent arguments or explanations and reflections on his or her work or that of collaborators (if applicable) in two or more media or languages, to both general and specialized audiences
The DQP Strategy
Moving Students’ Actual Work – and
Faculty Judgment – about student progress – to the Center of Attention
Assessments Worthy of Our Mission
The Five Areas are Interrelated, Not Separate
For example: Knowledge and Intellectual skills
are integrated in the context of application
– e.g. research, field-based assignments, projects, and civic problem-solving
Current Realities and Framework Ask Us to Shift from My Work – Each Course is a Silo – to OUR
Work – Intentional Practices that Both Develop and Demonstrate
Students’ Competence
The Work Ahead?
The higher ed market is reinventing what a university is, what a course is, what a
student is, what value is. I don’t know why anyone would think that the online
experience is about reproducing the classroom experience.
Richard A. DeMilloDirector, Center for the 21st Century University
Georgia Institute of Technology
Goal:
Foster graduates who consider
ethics, legal, and socio-cultural issues and use
critical thinking in decision-making.
Measure :
Grades that focus on presentation style,
grammar, and citation form and
number.
We have had our why's, how's, and what's upside-down,
focusing too much on what should be learned, than on how, and
often forgetting the why altogether.
In a world of nearly infinite information, we must firstaddress why, facilitate how, and let the what
generatenaturally from there.
Michael Wesch, “From Knowledgeable to Knowledge-able,” Academic Commons, January 2009 (academiccommons.org)
38
4. Have you heard of
ePortfolios?
ePortfolio Focus
What We Know and Have Evidence to Believe
•Used on over half of American higher education institutions•Improve retention and graduation•Deepen student learning – retain and integrate information, apply information beyond single course
Institutional Usage
Institutional Size
• Fostering reflection• Student responsibility• Tracking growth• Showcasing learning• Self-assessment• Integrative learning
What is most important to individual respondents?
Motivation for ePortfolio
How e-portfolio ideas and practices have improved or could improve
student learning
“Could learning e-portfolios encourage students to have deeper, more reflective learning, and stimulate more significant connections across learning experiences?” – University of Oregon
We found specific changes in learning and pedagogy
occur as a result of participation in
e-portfolios
Essential Portfolio Practices
• Purposeful collection• Multiple measures to track development and
improvement• Self-assessment and reflection to foster analysis,
synthesis, evaluation, etc.• Integrative opportunities/requirements• Build evidence of an empowered, informed,
responsible learner • Can be used a course, program, institutional level
assessment
Positioning Integration in the Student’s Experience
One of the great challenges in higher education is to foster students’ abilities to integrate their learning
across contexts and over time. Learning that helps develop integrative capacities is important because it
builds habits of mind that prepare students to make informed judgments in the conduct of personal,
professional, and civic life; such learning is, we believe, at the very heart of liberal education.
~Mary Taylor Huber and Pat Hutchings
Integrative Learning: Mapping the Terrain (2005)
5. Have you heard of the
VALUE Rubrics?
Rubrics Basics
Criteria
VALUE Rubrics & AssessmentVALUE Rubrics & Assessment
Rubrics Basics
Levels
VALUE Rubrics & AssessmentVALUE Rubrics & Assessment
Rubrics Basics
Performance Descriptors
VALUE Rubrics & AssessmentVALUE Rubrics & Assessment
Reliability Study• 40 Faculty• 4 Traditional Disciplinary Divisions –
Humanities, Social Sciences, STEM, Professions• Three VALUE rubrics – Critical Thinking, Civic
Engagement, Integrative Learning• Common set of student portfolio work• Agreement = .66 without norming; .8 normed• Another set of 5 campuses, using same set of
rubrics with 500 samples of student work – still analyzing
AAC&U Perspectives
• Standardized vs. Standards-based Assessments
• Faculty Developed• Focused on Competence vs. Deficits• Based on Student Work• Demonstrated over time vs. Snapshot• VALUE Rubrics
Building the Evidentiary Base• University of Kansas – Representing Results
Per
cen
t o
f R
atin
gs
Critical Thinking: Issues, Analysis, and Conclusions
Inter-rater reliability = >.8
Building the Evidentiary Base• University of Kansas – Representing Results
Per
cen
t o
f R
atin
gs
Critical Thinking: Evaluation of Sources and Evidence
Building the Evidentiary Base
Perc
en
t of
Rati
ng
s
“VALUE added” for 4 years - writing
Building the Evidentiary Base
• University of Kansas – • “analysis of the data from the AACU VALUE rubrics
affirmed that a team approach to course design can yield larger improvement in some forms of student writing and thinking”
• “We also saw that the rubrics work best when there is close alignment between the nature of the assignment and the dimensions of intellectual skill described in the rubric”
• “Finally, at a practical level we are very encouraged that this process is manageable and sustainable”
A Learning College: Adjusting
for Student Growth &
Development
Faculty Read & Assess Student Work• Faculty provide
appropriate developmental guidance and a final assessment of the work – student learning, pedagogical improvement
Programs Read & Assess Student Work• Programs can make
changes based on what they find & with resources if possible The College Reads
& Assesses Student Work• The college is able to
assess progress in core competencies across the curriculum, not just in a program - accountability.
Faculty
Reviews Assignment & Guides Student
Progress
Adapts Assignment & Makes Changes
Where Appropriate
Program
Program X discovers that oral
communication is weak in an intro
course.
Works with faculty from Oral
Communication to design an
intervention. Intensive module design
supported with small grant.
College
Benchmark Assessment Readings reveal gains in Critical Literacy and Research & Information Literacy but also reveal issues
with the rubrics
Rubrics will be redesigned to better support the kinds of work faculty want to
assign. Additional work in CTL seminars
and in the college around the core competencies.
The Feedback Loop
Assessment & the Learning Portfolio
Two different facets of the ePortfolio
ePortfolios can be used for accountability (summative - supported by student work placed in the ePortfolio
evaluated through rubrics)
ePortfolios are also used for student learning, growth, and development (formative and on-going)
Faculty Collaborati
on
We are Being Asked to Shift...
from My Work
to OUR Work
Assessment Practices
That Verify Achievement