putting cognitive principles into practice using formative assessments and exam wrappers improving...
TRANSCRIPT
Putting Cognitive Principles into Practice Using Formative Assessments and Exam
Wrappers
Improving Student Performance by Addressing Student and Teacher Misconceptions about Learning
Part 2
The Cognitive Challenges to Teaching(that we know about thus far)
1) Student Mental Mindset
2) Metacognition and Self-regulation
3) Student Fear and Mistrust
4) Prior Knowledge
5) Misconceptions
6) Ineffective Learning Strategies
7) Transfer of Learning
8) Constraints of Selective Attention
9) Constraints of Mental Effort and Working Memory
And they all interact with each other
Viewing Pattern
Video 1 Video 2 Video 3 Video 4 Video 50
100,000
200,000
300,000
400,000
500,000
600,000
700,000
800,000
900,000
1,000,000
822,958
388,062
301,851254,782
210,048
Unique Views by Video
Video
# V
iew
s
Think-Pair-Share
• Create a question that requires conceptual understanding or application of a concept, preferably one which also encompasses a common misconception.
• Think: Present the question and have students think of their answer
• Pair: Have students pair up and discuss their answers and reasoning
• Share: Discuss as a class
Think-Pair-Share for Study Videos
Which of the following is an example of poor metacognition?
1. Joe failed an exam because he memorized definitions but his professor tested him over comprehension.
2. Amy felt confident she did well on the exam but was stunned to find out she barely made a D grade.
3. Cindy studied by reading her notes and her textbook over and over again, but still made a bad grade.
4. Sam thought he could learn the material well enough if he just read the chapter summaries, but he ended up failing the exam.
Another One
Which of the following does NOT represent studying at a deep level?
1. As I read, I relate the information to what I already know.
2. As I read, I relate the information to my own personal experience
3. As I read, I think of the key distinctions between this concept and other concepts I’ve learned about.
4. As I read, I repeat the information to myself multiple times.
5. I often close the book and my notes and just try to write out all the information I can remember on my own.
Introductory Video: Developing a Mindset for Successful Learning
• Principle 1: Your level understanding is a direct result of how hard you prepare.
• Principle 2: There are effective and ineffective ways to prepare.
• Principle 3: You have to master the basics before moving on to more complex skills.
• Principle 4: Overconfidence should be avoided at all costs
• Principle 5. Effective preparation requires your total focus; no distractions
Developing a Mindset for Successful Learning (cont’d)
• Principle 6. Successful learning requires planning ahead.
• Principle 7. Feedback helps you get better• Principle 8. Recognize and take advantage of
prime learning opportunities • Principle 9. Improvement involves dealing with
challenges, difficulty, and uncertainty • Principle 10. Find the pleasure or value in what
you are studying in order to do your best
Productive PersistenceSilva & White (2013)
• [P]roductive persistence is the package of skills and tenacity that students need to succeed in an academic setting.
• [Students] quit because they believe they aren’t smart enough to do math, that the class itself has little relevance to their personal or academic goals, and that they don’t really belong in the course or in college at all.
• They assume that classes, while necessary to move ahead in college, will not likely be relevant to their long-term or even short-term goals.
Academic Mindset for Productive Persistence
1. I belong in this academic community
2. My ability and competence grow with my effort
3. I can succeed at this
4. This work has value for me
Formative Assessments
• Brief, low stakes assessments that give students (and teachers) feedback BEFORE exams/high stake grades
Angelo, T. A. and K. P. Cross (1993). Classroom Assessment Techniques: A Handbook for College Teachers, Jossey-Bass.
e.g. Minute papers, muddiest point, think-pair-share
– Brief– Low stakes– Provide feedback to teacher and student– Before summative assessments
When does a clicker question become a formative assessment?
When it is designed using cognitive principles to achieve a desired learning goal
.
Goals of Formative Assessments
• Improve metacognition for students and teachers
• Address tenacious student misconceptions• Illustrate desired level of understanding of
knowledge for students• Promote student learning and
understanding• Model thinking for understanding• Promote rapport and trust
Properties of Conceptests(Mazur, 1997)
A multiple choice question that…• Focuses on a single concept• Requires conceptual understanding to
solve• Has adequate response alternatives
– Ideally the incorrect alternatives should reflect the most common misconceptions
• Be unambiguously worded• Be neither too easy nor too difficult
The ConcepTest General Format
1. Present ConcepTest to class – 1 minute
2. Students given time to think – 1 minute
3. On a given signal, students indicate their answer by number of fingers.
4. Have the students pick someone around them, preferably with a different answer, to discuss their choices – 1- 2 minutes
5. Repeat step three to see how choices have changed
6. Explain and discuss the answer as a class – 2+ minutes
Which of the following statements is TRUE concerning attachment (the powerful emotional bond between infant
caregiver)?
• It develops between both parents and child and is present at birth.
• It takes 7-8 months to develop and may not develop at all.
• It is present at birth between the mother and child, but takes 7-8 months to develop between the father and the child.
• It is usually present at birth, but is disrupted if the parents are not allowed to bond with the child by holding, feeding and comforting the child.
Which of the following statements is TRUE concerning attachment (the powerful emotional bond between infant
caregiver)?
• It develops between both parents and child and is present at birth.
• It takes 7-8 months to develop and may not develop at all.
• It is present at birth between the mother and child, but takes 7-8 months to develop between the father and the child.
• It is usually present at birth, but is disrupted if the parents are not allowed to bond with the child by holding, feeding and comforting the child.
Read
First Poll
Discuss
Second Poll
Misunderstandings About Correlations
• A positive correlation is better than a negative one.
• Correlations imply causation.
ConcepTest for Correlations
A marriage counselor studies four different tests designed to predict marital happiness to see which one is best. She administers the four tests to 80 couples who are about to get married. After two years, she measures the marital happiness of the couples and correlates it with each of the four tests with the following results:
Test 1: r = -.73 Test 2: r = .62Test 3: r = .25 Test 4: r = .10
If the therapist wants to pick the single best test to use in her work, which one should she choose and why?
Testing Understanding of Correlations in Three Different Ways
• Standard multiple choice• Contextual application question• Probe for understanding: Multiple choice
with alternatives that capture common misunderstandings (based on your experience). Require explanation of answer
Standard Multiple Choice
Which of the following is the strongest correlation?
a. r = 0.10
b. r = 0.57
c. r = -0.63
d. r = 0.25
Probe for UnderstandingSamford decides that the ACT and SAT are
obsolete, and decides to find a new college entrance exam to use for admissions. They have incoming freshmen take four new entrance exams. After the end of the first year, the student's GPA is correlated with each of the four scores.
Test A: r = 0.10 Test C: r = 0.55Test B: r = -0.64 Test D: r = -0.50
Which exam should Samford adopt? Explain your answer. To get full credit, you must choose the correct test and have the correct explanation.
Incorrect Answers and Explanations
• Test A because .10 has the highest correlation. This will show the best example of each student's scores.
Incorrect Answers and Explanations
• Test C: r = 0.55; because people who took test C made higher than the others, so their GPA would be higher.
• Test C, since you would want a positive correlation and not negative. Even though the test B correlation is greater, the correlation is negative which would mean their GPA's were low.
Correct Answer and Explanation
• Test B would be the best answer. Even though it has a negative correlation, it is the strongest (strength of correlations are determined in terms of absolute value).
Correct Answer and Incorrect Explanation, and Vice-Versa
• The Test B should be used because it's confounding variables are more close to an absolute value of -.64. The students already have enough stress as it is only the highest scores on the ACT and SAT should determine the status of entrance.
• I think they would pick Test C. Even though .64 is bigger, it is also negative, resulting in a negative correlation. They would want the test scores and the grades to be positively correlated.
Conceptests vs. Control: Factual
2216
62
0
8.33
0
91.67
00
25
50
75
100
A B C** D
Question Alternative
% C
ho
se
n
No Conceptests Conceptests
Conceptests vs. Control: Probe
18.18
29.55
50
2.278.51
53.19
36.17
2.13
0
25
50
75
100
A B** C D
Question Alternative
% C
ho
se
n
No Conceptests Conceptests
Advantages of ConcepTests• They give feedback to both the student and teacher
about the level of student understanding (formative assessment).
• They are highly engaging to students. • They take little preparation or class time. • They can be used with any size class. • They stimulate class discussion. • Students learn from each other as well as the
teacher. • They make students aware of intuitive but incorrect
beliefs they hold about psychology. • They give a preview to the class about the kinds of
questions they can expect on an exam.
ConcepTests are shown to …• Improve student learning through
feedback, questioning, peer learning, engagement, and formative evaluation
• Help overcome tenacious misconceptions• Improve metacognitive awareness for
students and teachers• Model proper thought processes for
understanding
The Critical Components of ConcepTests
• The Quality of the Question– Engaging– Models desired thought process such as
appropriate application or relevant implications
– Acceptable cognitive load
• The Peer Discussion between Polls
What are Exam Wrappers?• Exam Wrappers are an After Exam Review or Exam
Debrief completed by students, usually after their tests have been returned to them for examination.– Developed by Marsha C. Lovett, PhD at Carnegie Mellon
University
• Exam wrappers promote metacognition and self-reflective learning by helping students
• identify areas of strength and weakness to guide further study
• reflect on the adequacy of their preparation and appropriateness of their study strategies
• understand the nature of their errors to find any recurring patterns
Previous Exam Preparation and Performance
• Metacognitive Awareness: – I felt I was really well prepared for the last exam. – I was surprised by how poorly I did on the exam.
• Exam Preparation:– Did you read all assigned reading thoroughly before
the exam? If not…– Estimate the percentage of points you lost due to
each of the following:• The information needed to answer the item was
not in my notes, nor highlighted in reading– Check all the reasons below that may have
undermined your exam performance:• Did not spend sufficient time in reviewing materials
Behavior Change• Watch the “How to Study” Videos• Identify at least three changes you will make
in your preparation for the next exam. Relate the change specifically to the videos. For example:
1. ”Video 1 says that it is a mistake to write out note cards and memorize them because it leads to learning isolated facts. Next time I will create diagrams that show how concepts are related and distinct from other concepts (Video 1, 3, and 4).”
2. Or, “I will remove distractions when I study so I can concentrate better (Video 1).”
Preparation Schedule for Next Exam
• How many days before the exam will you complete reading and viewing all materials for the first time?
• How many days before the exam will you begin reviewing and studying the materials?
• About how many hours each day do you plan to study for the exam?
• How do you plan to minimize distractions while studying?
Study 2: Effectiveness Study• Participants were 42 students in General Psychology.• After the first two exams, students were given the
opportunity to complete the exam wrapper exercise; extra credits provided.
• Completed wrapper/video, n = 26.• Non-participating controls, n = 16.• Two approaches to analysis:• Latent growth modeling of change trajectories from
exam 1 to exam 3 (where students in the two groups were headed)
• End-point analysis: comparison of groups at exam 3, adjusting for previous performance.
Exam 1 Exam 2 Exam 360
62
64
66
68
70
72
74
76
78
80
Growth TrajectoriesWrapper Control
Pe
rce
nt
Co
rre
ct
Students who completed wrappers had typically performed worse on exam 2 than exam 1. Their participation paid off: Their scores increased dramatically on exam 3, surpassing those who did not participate.
A Test of Critical Thinking
In the box, draw a picture of what the dressmaker used to cut the fabric
Obviously Wrong Answers
And the Correct Answer is:
The dressmaker used the scissors to cut the cloth for the dress.
How did critical thinking fail?
• Content was not enough• Just telling students to “think critically” was
not enough• What could you do to ensure critical
thinking?
Formative Assessment
A.
B.
C.
D.
E.
Which of the following would most likely be used by a dressmaker?
If students don’t know what critical thinking looks like, they can’t
accomplish it, even if they are capable of it
Concept Maps
• Concept Mapping: a graphical representation of a topic with concepts as nodes that are linked together to represent connections
• The value is in the effort and reflection as much as the map itself
Mind
ConsciousWhat we are aware of
Preconscious:Easy to makeconscious
UnconsciousWhat we are not aware of
Unconscious:Very difficult to make conscious
By far the largestPart of the mind
Contains powerfulDrives that influenceOur behavior without Our awareness
Concept Map
Student Generated Concept Maps
• A concept map is a graphical, hierarchical, node-link representation of knowledge structures.
Question Generation
• Generate at least three questions over each lecture or each topic you read about
• Aim for higher order questions– How, Why, What if…
• Getting answers is good, but even asking the questions aids learning.
• Easier to do than concept mapping
Practicing Recall and Application
• Close your book and notes and write down or say what you can recall
• Explain it to a classmate or friend• Apply concepts to a novel situation• Try to answer questions about the material
from the book or from other sources• Double check to see what you might have
missed
Cognitive Principles of Effective Teaching
1) Beliefs about Teaching
2) The Cognitive Challenges of Teaching: Mindset, Metacognition, and Trust
3) Prior Knowledge, Misconceptions, Ineffective Learning Strategies, and Transfer
4) Constraints of Selective Attention, Mental Effort, and Working Memory
5) Teachable Moments, Formative Assessment, and Conceptual Change
Take Home Message, Part 2• Pedagogy is theory driven practice to achieve
a learning goal • Faculty can have a huge direct impact on the
ease or difficulty of student learning• Faculty need to help students become better
learners• I’ve tried to give you an understanding of
cognitive principles and how they can be applied to help accomplish these goals