Task
1. How would you describe your experience with Scholarship of Teaching and Learning?
Circle one: Extensive Moderate Hardly-Any None
2. OUTCOMES: List 3 topics or issues you hope to learn more about at this workshop .
Origin of SOTL
What & Why of SOTL
Initiating a SOTL Program
Faculty SOTL Projects
Bridges to Productivity
Questions, Designs & Resources
Continuum of Experience with SOTL
A Metaperspective
Α Ω
A Conceptualization of Teaching Related Activities
Scholarly Teaching: Teaching that entails certain practices of classroom assessment and evidence
gathering; teaching that is informed not only by the latest ideas in the field but by current ideas about teaching generally and specifically in the field; and teaching that invites peer collaboration or review.
Scholarship of Teaching: An act of intelligence or artistic creation becomes scholarship when it possesses
at least three attributes: it becomes public, it becomes an object of critical review and evaluation by members of one’s community, and members of one’s community begin to use, build upon, and develop those acts of mind and creation.
(Definitions after Shulman)
Teaching ScholarlyTeaching
Scholarship of Teaching &
Learning
Teaching
ScholarlyteachingAn excellent deed of teaching
A superior deed of scholarly teaching
An ineffective deed of teaching
Scholarship
Plane of the Venn Diagram
Quality (or “excellence”) of any teaching-related activity is an independent dimension not represented in the plane of the Venn diagram.
Distinction between scholarship of teaching and learning as a domain of academic achievement and as a campus program: SOTL as academic achievement (yellow ellipse) SOTL as a campus program can span all teaching
related activities (leftmost three ellipses).
A SOTL program encourages movement in teaching-related activities toward the right in the diagram.
Teaching Scholarlyteaching
Contemporary forms of research and creative activity
Scholarship
Why SOTL?
• Synthesis
• Building community
• Wisdom of practice
• Generativity
• Reserve capacity
Task
• What reasons, these or others, are most important on your campus?
• What reasons might be irrelevant to faculty on your campus?
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
CA/CR (Angelo & Cross 1993)
New epistemology (Schön 1995)
New American scholar (Rice 1996)
Scholarship Assessed (Glassick et al 1997)
Implementing Strategies
AAHE FFRR
CASTL
- Carnegie Scholars
- Teaching Academy
- Professional Societies
- Knowledge Media Lab
Peer Review of Teaching
Recent Articulations
Shulman, Change 31 (4) 1999
Hutchings & Shulman Change 31 (5) 1999
Kreber & Cranton JHE 71 (4) 2000
Richlin in Kreber (Ed.) New Directions 86 2001
Origin and Evolution of SOTL (In Progress)
What is certain is that most Ph.D.’s become teachers and not productive scholars as well. [A Ph.D. candidate who plans to be a teacher] must know his field and its relation to the whole body of knowledge. It means too that he must be in touch with the most recent and most successful movements in undergraduate education, of which he now learns officially little or nothing. How should he learn about them? Not in my opinion by doing practice teaching upon the helpless undergraduate. Rather he should learn about them through seeing experiments carried on in undergraduate work by the members of the department in which he is studying for the degree”
Robert Maynard Hutchins
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
CA/CR (Angelo & Cross 1993)
New epistemology (Schön 1995)
New American scholar (Rice 1996)
Scholarship Assessed (Glassick et al 1997)
Implementing Strategies
AAHE FFRR
CASTL
- Carnegie Scholars
- Teaching Academy
- Professional Societies
- Knowledge Media Lab
Peer Review of Teaching
Recent Articulations
Shulman, Change 31 (4) 1999
Hutchings & Shulman Change 31 (5) 1999
Kreber & Cranton JHE 71 (4) 2000
Richlin in Kreber (Ed.) New Directions 86 2001
Origin and Evolution of SOTL (In Progress)
“On the high ground, manageable problems lend themselves to solution through the use of research-based theory and technique. In the swampy lowlands, problems are messy and confusing and incapable of technical solution. ...the problems of the high ground tend to be relatively unimportant to individuals or to the society at large, however great their technical interest may be, while in the swamp lie the greatest problems of human concern. ...Shall [the practitioner] remain on the high ground where he can solve relatively unimportant problems according to his standards of rigor, or shall he descend to the swamp of important problems where he cannot be rigorous in any way he knows how to describe?”
Schon, D. “The New Scholarship Requires a New Epistemology,” Change, Nov./Dec. 1995 p. 28
CONCRETE CONNECTED
KNOWING
ABSTRACT ANALYTIC KNOWING
ACTIVE PRACTICE REFLECTIVE
OBSERVATION
R. Eugene Rice. 1996. The New American Scholar. AAHE. p. 14
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
CA/CR (Angelo & Cross 1993)
New epistemology (Schön 1995)
New American scholar (Rice 1996)
Scholarship Assessed (Glassick et al 1997)
Implementing Strategies
AAHE FFRR
CASTL
- Carnegie Scholars
- Teaching Academy
- Professional Societies
- Knowledge Media Lab
Peer Review of Teaching
Recent Articulations
Shulman, Change 31 (4) 1999
Hutchings & Shulman Change 31 (5) 1999
Kreber & Cranton JHE 71 (4) 2000
Richlin in Kreber (Ed.) New Directions 86 2001
Origin and Evolution of SOTL (In Progress)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
CA/CR (Angelo & Cross 1993)
New epistemology (Schön 1995)
New American scholar (Rice 1996)
Scholarship Assessed (Glassick et al 1997)
Implementing Strategies
AAHE FFRR
CASTL
- Carnegie Scholars
- Teaching Academy
- Professional Societies
- Knowledge Media Lab
Peer Review of Teaching
Recent Articulations
Shulman, Change 31 (4) 1999
Hutchings & Shulman Change 31 (5) 1999
Kreber & Cranton JHE 71 (4) 2000
Richlin in Kreber (Ed.) New Directions 86 2001
Origin and Evolution of SOTL (In Progress)
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• Higher Education
• K-12
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• Carnegie Scholars
• Teaching Academy
• Professional Societies
• Related Carnegie Initiatives– Knowledge Media Lab– Resources
http://www.carnegiefoundation.org/CASTL/highered/index.htm
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• Pew National Program for Carnegie Scholars
http://www.carnegiefoundation.org/CASTL/highered/Pewscholars.htm
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• Teaching Academy Campus Program AAHE & Carnegie
– Level I: Campus Conversations
– Level II: Going Public
– Level III: National Teaching Academy
http://www.carnegiefoundation.org/CASTL/highered/teachingacademy.htm
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• AAHE CASTL Campus Program WebCenter
– Campus Reports
– List of "Going Public Grants"
– Resources
http://aahe.ital.utexas.edu
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• Professional Societies
– Networking
– Invitational Small-Grants
– Register & Reports
http://www.carnegiefoundation.org/CASTL/highered
/collaborations.htm
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• Knowledge Media Lab
– Tour
– Exhibition: Scholarship Under Construction
– Gallery
– Multimedia Tutorial for SOTLhttp://www.carnegiefoundation.org/KML/index.htm
CASTL = Carnegie Academy for the Scholarship of Teaching & learning
• [Other] Resources
– Bibliography of SOTL
– Text of Key Papers
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
CA/CR (Angelo & Cross 1993)
New epistemology (Schön 1995)
New American scholar (Rice 1996)
Scholarship Assessed (Glassick et al 1997)
Implementing Strategies
AAHE FFRR
CASTL
- Carnegie Scholars
- Teaching Academy
- Professional Societies
- Knowledge Media Lab
Peer Review of Teaching
Recent Articulations
Shulman, Change 31 (4) 1999
Hutchings & Shulman Change 31 (5) 1999
Kreber & Cranton JHE 71 (4) 2000
Richlin in Kreber (Ed.) New Directions 86 2001
Origin and Evolution of SOTL (In Progress)
An Exemplary Course PortfolioAnd
A Superb Model of SOTLAn Alternative Approach to General
Chemistry
Assessing the Needs of ‘At-Risk’ Students with Cooperative Learning Strategies
Dennis JacobsProfessor of ChemistryUniversity of Notre Dame
Rita Naremore, Simon Brassell, Shanker Krishnan, David ParkhurstConstructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent October 26 2000
Contents of Course Portfolio:• Overview• Rationale:
• Why develop an alternative approach?
• Implementation• Changes introduced to foster learning. • Targeting ‘at-risk’ students.
• Impact• Documentation and assessment of immediate
and longer-term effects.
• Library• Examples of videos of group discussions, tests,
on-line quizzes, questionnaires.
http://kml.carnegiefoundation.org/gallery/djacobs/Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 2 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Course Portfolio Design:
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 4 -
Rationale:• Recognized Problems:
• ‘At-risk’ students (Math SAT ≤630): • dropped out of General Chemistry. • didn’t take any advanced science.• frustrated by large lecture format.
• Alternate Course Design:• Similar requirements and lectures.• Comparable exams.• Various activities involving structured cooperative
learning.• Initial Comments on Proposal
• “Only delaying inevitable failure.”• “Efforts should be focused on the best not the ‘at-risk’
students.”
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 5 -
Documentation:• ‘At-risk’ students
markedly more likely to drop or fail the course in the traditional class format.
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Implementation:• Aim:
• Provide improved learning opportunities for ‘at-risk’ students.
• Develop more effective teaching in large lecture format.
• Alternate Course Design:• Introduced opportunities for structured
cooperative learning including:• discussion of concepts in pairs.• small group in recitation sections.• work as pairs in laboratory.
• Mandatory recitation sections:• more time committed to class. • direct contact with instructors.Constructing and Evaluating a Course Portfolio:
Making Good Teaching Apparent October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 6 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Traditional vs. Alternative Classes:• Similarities:
• Size (250 students), text, chapters.• Lecture time (3 hr), lab time (2.5 hr).• Lecture format (Powerpoint slides and demonstrations).• Exam format and many exam questions.
• Differences in Alternative Section:• Mandatory recitations (1 hr/wk, 20 students);
attendance 95% vs. 10%.• Weekly homework (10 vs. 30, graded).• On-line quizzes (www chapter reviews).• Weekly feedback from homework, group problems, on-
line quiz, in-class questions.• Personal contact with instructor and follow-up if
performance declined.
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 7 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Impact of Alternative Section:• Assessment Strategies:
• Effects on conceptual understanding, problem-solving and self-confidence:
• feedback from students.• evaluation of individual elements of the
cooperative learning activities. • Immediate and long-term benefits:
• retention of ‘at-risk’ students.• success in advanced science classes.
• Data Collection:• Recording in-class learning activities.• Tracking individual grades and progress.• Longitudinal study of ‘at-risk’ students:
• progress in subsequent classes.
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 8 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Measurement of Impact:•Success of ‘At-risk’ Students:
•Better grades in General Chemistry.•Improved retention in class.•Higher success rate in subsequent classes.
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 9 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Measurement of Impact — II:
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 10 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 11 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Measurement of Impact — III:
Conclusions:• Dennis Jacobs’ Course Portfolio:
• Succinct and reflective.• Focused on specific learning objectives.• Uses readily available information:
• grades, course materials, etc.• Documents the positive effects of changes in
instructional methodology.• Benefit for Dennis Jacobs:
• Ready justification for resources to support recitation sections.
• Value of Course Portfolios:• Demonstrable outcomes of teaching practices that can
be peer-reviewed.
Constructing and Evaluating a Course Portfolio:Making Good Teaching Apparent
October 26 2000
QuickTime™ and aGIF decompressor
are needed to see this picture.
- 12 -
An Alternative Approach to General Chemistry Assessing the Needs of At-Risk Students with Cooperative Learning StrategiesDennis Jacobs (University of Notre Dame)
Not a new idea (Hutchins 1923)
“Scholarship of teaching” coined (Boyer 1990)
Reform Concepts
CA/CR (Angelo & Cross 1993)
New epistemology (Schön 1995)
New American scholar (Rice 1996)
Scholarship Assessed (Glassick et al 1997)
Implementing Strategies
AAHE FFRR
CASTL
- Carnegie Scholars
- Teaching Academy
- Professional Societies
- Knowledge Media Lab
Peer Review of Teaching
Recent Articulations
Shulman, Change 31 (4) 1999
Hutchings & Shulman Change 31 (5) 1999
Kreber & Cranton JHE 71 (4) 2000
Richlin in Kreber (Ed.) New Directions 86 2001
Origin and Evolution of SOTL (In Progress)
Task
Think about your own background in SOTL. What additions would you make to the map “Origins and Evolution of SOTL” in each category?
Reform Concepts Implementing Strategies Recent Articulations
OutlineThe IUB Experience
• The Administrative Perspective– Design Principles– Kinds of Resources Needed– Changes to the Faculty Culture
• The Departmental Perspective– Going from Zero to Sixty in 10.2 Seconds
• The Individual Faculty Member’s Perspective
How Did We Set Up a Campus SOTL Program?
The Ingredients:– A dedicated director
– A faculty advisory committee
– A core of committed researchers willing to share their work
– Massive Administrative Support
The Key: Faculty Input
Design Principles
• Get out the crowd to initial events
• Serve food
• Connect with national– Programs– Resources
• Be inclusive
Design Principles (Cont.)
• Involve librarians
• Partner with stakeholders
• Identify needs– Campus conversations– Faculty committees
• Exhibit administrative support
What Resources Were Needed?
• Financial—Money is a great motivator!
• Human—People will go where the money is!
• Structural—Other rewards are important in the academic environment. Integrate SOTL into this reward structure.
Deploying Available Resources: Some Examples
• Expand rewards
• Travel grants
• Presentation/research grants
• Visibility for contributions
ONE EXAMPLE OF A STRUCTURAL CHANGE
IU FACULTY SUMMARY REPORT PRIOR TO ACADEMIC YEAR 1999-2000
TEACHING ACTIVITIESA. Courses taught (weekly contact hours reported by course
number in tabular form).C. Development or major revision of course(s) during the year.D. Dissertation, Research and Field Work CommitteesE. Teaching awards and honors, including those of your students.
FACULTY SUMMARY REPORT [Revised in Academic Year 1999-2000]
TEACHING ACTIVITIESA. Courses taught (weekly contact hours reported by course
number in tabular form).B. Activities directed at improving instruction, learning, or
course administration. (Please describe rationale for/description of innovations, methods/measures for assessing outcomes, and results.)
Please note: Scholarly activity related to teaching and learning (e.g. investigation/research, dissemination/publication of results) should be reported under the section on Research/Creative Activities.
FACULTY SUMMARY REPORT, CONTINUED
C. Development or major revision of course(s) during the year.
D. Dissertation, Research and Field Work Committees
E. Teaching awards and honors, including those of your students.
Effects to Date
1999-2000 Campus SOTL Program of Events• 15 events - 930 total registrants• 53 participants per event• 515 individuals registers for one or more events • 43 requests for videotapes• SOTL community formed and functioning• Developmental process for scholars in formation• Several SOTL-related research projects identified
The Program in Year 2
• 10 Events Scheduled
- 75 Participants per Event (Average over 4 events)
• More Efforts to Link SOTL With Other Initiatives– Preparing Future Faculty– The Concern with Research Ethics– The Course Portfolio Project
Working At the Departmental Level
• In the beginning:– A concern for and
commitment to excellent teaching
– A key group of faculty members willing to begin the conversation about teaching in the unit
• Next?– Collecting data
– Sharing the results
– Designing follow-up to the initial research:
• Involving others
• Extending the questions
An Example of Department-Based Research
• The problem: a group of faculty members concerned that teaching excellence was getting short shrift in the faculty evaluation process
• The solution: appoint a Committee on Teaching to make recommendations
Working Toward a Solution
Issue # 1: a lack of confidence in student evaluations
• Survey the faculty—”What can/should students tell us about our teaching?”
• Develop and test a new evaluation instrument
Issue #2: a lack of good procedures for peer evaluations
• Gain a consensus for how peer evaluation might be made better
• Develop and test a peer evaluation procedure
Issue #1: Student Evaluation
• A departmental form was developed and used along with the traditional university form in all classes for 4 semesters.
• Results using the two forms were compared, and statistical analyses were conduced to determine reliability and validity.
A Brief Summary of Results
• Faculty members who taught 10 required undergraduate courses were ranked in terms of their overall student evaluation scores.
• The differences among the top 5 faculty were very small, but the difference between the top 5 and the bottom 5 was sizeable.
• 86% of variance in the rankings was predicted by responses on one item: I learned a lot in this course.
Issue # 2: Peer Evaluation
• The faculty consensus: traditional classroom visitation is not very revealing of anything useful. What we really need is evaluation of course structure and materials used in the class for teaching and evaluation. We’d like to have periodic external peer review of these materials.
Ongoing Concerns with Peer Evaluation
• The biggest concern: IT TAKES TOO LONG TO PUT ALL THIS STUFF TOGETHER!
• What are we doing? We’re working toward the development of course portfolios, to be done over time and turned in by faculty members every second or third year, not every year.
The Campus Tie
• The departmental course portfolio effort should eventually tie in with a campus-wide effort now underway, conducted by an interdisciplinary team in conjunction with a Pew Foundation funded project directed by Dan Bernstein at the University of Nebraska. For more information about this project, see http://www.unl.edu/peerrev/
TASK: Refining Your Own Campus SOTL Program or Plan
Think for a minute about your own campus. What administrative priorities or structures might help in setting up a campus level SOTL program? What hurdles might need to be overcome on your campus?
Other Campus Models
• Elon College
• The Citadel
• Rockhurst University
• Abilene Christian University
• Notre Dame University
• Middlesex Community College
Elon College
• Multidisciplinary, Multiyear, $72,000 Investment – $6000 projects in each of 3 years – Projects directed by faculty-student research teams – Learning for BOTH student and teacher – Eight projects selected in years 1 and 2
• Create intellectual engagement– New thinking in diverse fields – Application of learning to life – Opening spaces for reflective integration
The Citadel
• Mission– Increased campus
awareness of and participation in SOTL
• Focus – Communication,
Resources and Continuing Education
– Self-selected research projects
• Highlights– Biweekly, participatory meetings with
assignments– http://www.citadel.edu/carnegie
• Effectiveness– 15% of full-time, tenure-track faculty
at bi-weekly meetings– 12% of full-time, tenure track faculty
in classroom research
• Administrative Support– Attendance at functions– Financial support
Rockhurst University
• Beginnings– (Fall 1998) All University Symposium– (Spring 1999) Follow-up Symposium
Year-long Carnegie faculty seminar
– (2000-2001) Carnegie faculty seminar continued
• The Rockhurst “Carnegie Seminar”– Central Questions– Seminar Members– Discussions– Methods
• Products– Formal Letter on SOTL
– Faculty SOTL Projects
• Selected Key Issues and Observations– Obstacles to Discussion
– Interdisciplinary/Collaborative Approaches
– “Where's the beef?”
– “Scholarly Teaching” as best first path
– To be a good consumer of the SOTL
Abilene Christian University
• 19 Faculty doing SOTL Projects
• Strong Institutional Support
– Stipends for materials & resources
– Travel to teaching-related conferences
– Ongoing peer meetings,videoconferences
University of Notre Dame
Initial campus conversations with 90 leaders– SOTL needed support
– RFP resulted in 9 funded SOTL projects
Sample research question: Do new teaching methods in intro engineering affect students’ learning?
Support for SOTL teams includes these elements:– $5,000 per team for student time, equipment, supplies, faculty time
– Consulting with methodology experts
– Group meetings 2x/semester for mutual support
– Help in dissemination of results
– http://www.nd.edu/~kaneb/Carnegie.html
Task: Campus Models for SOTL
List 2 features of the models you have just seen that might be most applicable to your campus.
Working at the Level of the Individual Researcher
Deploying Available Resources: Some Examples
• Expand rewards
• Travel grants
• Presentation/research grants
• Visibility for contributions
Task
"MINUTE PAPER"
1. What were the 2 most important points presented in the first half of this workshop?
2. What 2 things would you most like to learn tomorrow?
.
How Could I do Scholarship of Teaching and Learning?
• Different Genres of SOTL– Reports on Particular Classes– Reflections on Years of Teaching– Larger Contexts—Comparisons– Formal Research– Meta-Analyses
Task
Which 3 of these genres would be easiest for faculty on your campus to do?
Approaches to Research
Classroom vs. Traditional Research
Classroom
Professor’s practice
Obtain knowledge applicable in limited circumstances
Specialized training not essential
Students and Professor
Origin
Purpose
Requirements
Benefits
Traditional
State-of present research
Build or verify theory
Specialized training usually essential
Field and Researcher
Action Research
“Research carried out by practitioners with a view to improving their professional practice and understanding it better.”
Quoted in Borg, Gall & Gall Applying Educational Research 3rd Ed. Longman, 1993 p. 390
Classroom Research
“Classroom Research is not traditional research conducted in or on classrooms. It is a specific methodology designed for discipline oriented teachers without training or experience in the methods of educational research. Classroom Research is ongoing and cumulative intellectual inquiry by classroom teachers into the nature of teaching and learning in their own classrooms. Inquiry into a question about how students learn typically leads to new questions and thus to continual investigations through classroom research.”
Cross, K.P. & Steadman, M. Classroom Research, Jossey-Bass, 1996 p. xviii
IMPROVING TEACHING AND LEARNING VIA CLASSROOM RESEARCH
Improved Teaching and
Learning
Existing Scholarship
Recorded Observation
Summative Assessment
Formative Assessment
Institutional Database
Questions
Goals
Reflection
Analysis
IMPETUS
INFORMATION GATHERING
EFFECT
SCHOLARLYPRODUCTIVITY
Synthesis
APPLICATION
Classroom AssessmentClassroom assessment is systematic and formative• Class is the unit of measurement rather than the individual • Conditions of learning may be assessed rather than student performance. • Right and wrong are not the emphasis. • Unexpected rather than expected responses are often most useful.
Angelo, T. & Cross, K.P., Classroom Assessment Techniques: A Handbook for College Teachers (2nd Ed) Jossey-Bass (1993)
Improved Teaching and
Learning
Existing Scholarship
Recorded Observation
Summative Assessment
Formative Assessment
Institutional Database
Questions
Goals
Reflection
Analysis
IMPETUS
INFORMATION GATHERING
EFFECT
SCHOLARLYPRODUCTIVITY
Synthesis
APPLICATION
Effective GradingPrimary Trait Analysis (PTA)… building scales that make performance criteria explicit in order to categorize/classify student work. – Provides a documentary source for changes in student learning. – Improves grading.
Walvoord, B. and Anderson, V., Effective Grading: A Tool for Learning and Assessment, Jossey-Bass 1998.
Improved Teaching and
Learning
Existing Scholarship
Recorded Observation
Summative Assessment
Formative Assessment
Institutional Database
Questions
Goals
Reflection
Analysis
IMPETUS
INFORMATION GATHERING
EFFECT
SCHOLARLYPRODUCTIVITY
Synthesis
APPLICATION
The Course Portfolio (I)“I was familiar with teaching portfolios … but thinking about teaching as scholarly inquiry began to lead me in the direction of something I had not seen anyone else doing: a portfolio that focused on the course rather than on all of one’s teaching. Being a social scientist, I began to think of each course … as a kind of laboratory - not a truly controlled experiment of course but as a setting in which you start out with goals for student learning, then you adopt teaching practices that you think will accomplish these and along the way you can watch and see if your practices are helping to accomplish your goals, collecting evidence about effects and impact.”
W. Cerbin quoted in Hutchings, P. (Ed.) The Course Portfolio, AAHE 1998
Improved Teaching and
Learning
Existing Scholarship
Recorded Observation
Summative Assessment
Formative Assessment
Institutional Database
Goals
Reflection
Analysis
IMPETUS
INFORMATION GATHERING
EFFECT
SCHOLARLYPRODUCTIVITY
Synthesis
APPLICATION
Questions
CLASSROOM RESEARCH
Improved Teaching and
Learning
Existing Scholarship
Recorded Observation
Summative Assessment
Formative Assessment
Institutional Database
Questions
Goals
Reflection
Analysis
IMPETUS
INFORMATION GATHERING
EFFECT
SCHOLARLYPRODUCTIVITY
Synthesis
APPLICATION
Cross, K.P. and Steadman, M., Classroom Research, Jossey-Bass 1996
Task
Which kind of information gathering (previous slide) is most common on your campus? Which kind is most needed? Why?
Asking the Question
Framing Research Questions(Goal Approach)
1. Define a goal
2. Answer questions about the goal
3. Create a one (or two) sentence summary of the specific goal
4. Ask “what evidence would reveal
-the present state?”
-that the goal is achieved?”
5. Write possible research questions
Framing Research Questions(Issue Approach)
Criteria• Investigable (not necessarily empirical)• Bounded and well-defined• Significant (not necessarily statistically)
Considerations• Length of time needed• Complexity of procedures• Availability of subjects• Availability of support (resources, personnel, funds)
Example
Less framed
Do students who help others learn an academic discipline learn it better themselves?
More framed
Do students in CMSC 250 who tutor students in CMSC 150 perform better on the CMSC 250 final exam than students who do not tutor but have similar grades in CMSC 150?
Making Vague Questions Answerable
• 1. Do students learn more in small classes?
• 1. Do students in sections of M118 enrolling fewer than 50 students perform better on the departmental final exam than students from sections enrolling more than 75 students?
Making Vague Questions Answerable
• What is the optimum number of homework assignments to give in a beginning math class?
• Do students enrolled in M036 who are given a homework assignment every week perform differently on the departmental final exam than students enrolled in M036 who are given homework every day?
Making Vague Questions Answerable
• What is the effect of the number of exams in a course on students’ opinions about the course?
• Do students enrolled in S333 who are given 6 exams per semester evaluate the overall course effectiveness on BEST item #1 differently from students in S333 who have only 1 exam per semester?
Task 11. Write the tentative issue or question to be addressed in your
project.
2. Form a group with two colleagues (groups of three)
3. Discuss your tentative project with your colleagues for the purpose of framing your goal, issue or question in the clearest and most assessable way. Encourage your colleagues to questions you and comment.
4. Write your (clarified and assessable) goal/issue/question again.
5. Assist each of your colleagues in completing steps 3 & 4.
Design Frameworks
What is a research design?
• A plan or protocol for carrying out a research project
• An underlying scheme that governs functioning, developing, or unfolding
Regardless of your definition, a good design promotes efficient and successful data gathering and analysis.
Quantitative or Qualitative Research?
Quantitative• Empirical, statistical
• Goal: hypothesis testing or confirmation
• Design: predetermined, structured
Qualitative• Fieldwork,
constructivist• Goal: hypothesis
generating, making meaning
• Design: flexible, evolving
Comparisons, continued
Quantitative• Sample: large,
representative• Measures: scales,
tests, surveys• Researcher is outsider• Findings: precise,
reliable
Qualitative• Sample: small,
purposeful• Measures: interviews,
observations• Researcher is insider• Findings: rich, deep
Guiding Questions in Choosing Methodology
• What approach fits your research problem?
• Do you have the skills/resources to carry out the methods?
• Will your audience find these approaches acceptable?
Provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University
Choosing the Measures to Answer the Question
Examples of Quantitative Measures
• course exam, project, paper scores
• survey scores (Likert)
• scores on standardized scales and tests
• counts (participation, visits)
• measures of time use
• institutional research data (GPAs, grades, admissions scores, demographics)
Examples of Qualitative Measures
• observations
• interviews
• focus groups
• student projects, essay exams (summative)
• reflective statements (formative)
• reports of others (counselors, etc.)
Illustrations of Qualitative and Mixed Methods
• Qualitative case study• Quantitative study enhanced by qualitative data• Qualitative study enhanced by quantitative data
Provided by Samuel Guskin, Professor Emeritus, School of Education, Indiana University
Task
Design an investigation to address the research question you framed earlier.
Summary of Standards
Clear Goals
Does the scholar state the basic purpose of his or her work clearly? Does the scholar define objectives that are realistic and achievable? Does the scholar identify important questions in the field?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Adequate Preparation
Does the scholar show an understanding of existing scholarship in the field? Does the scholar bring the necessary skills to his or her work? Does the scholar bring together the resources necessary to move the project forward?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Appropriate Methods
Does the scholar use methods appropriate to the goals? Does the scholar apply effectively the methods selected? Does the scholar modify procedures in response to changing circumstances?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Significant Results
Does the scholar achieve the goals? Does the scholar’s work add consequentially to the field? Does the scholar’s work open additional areas for further exploration?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Effective Presentation
Does the scholar use a suitable style and effective organization to present his or her work? Does the scholar use appropriate forums for communicating work to its intended audiences? Does the scholar present his or her message with clarity and integrity?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Summary of Standards
Reflective Critique
Does the scholar critically evaluate his or her own work? Does the scholar bring an appropriate breadth of evidence to his or her critique? Does the scholar use evaluation to improve the quality of future work?
Glassick, C.E., Huber, M.T., Maeroff, G.I. Scholarship Assessed Jossey-Bass (1997) pp. 22-36
Where to Publish and Present
How to Find Potential Sources of External Funding
Closing Evaluation of Workshop