nordic helsinki © john oversby and shu-nu chang 2007 1 mapping affect project (map) promoting links...

Nordic Helsinki © John Oversby and Shu-NNordic Helsinki © John Oversby and Shu-Nu Chang 2007u Chang 2007

11

Mapping Affect Mapping Affect Project (MAP)Project (MAP)Promoting links between Promoting links between

values, judgement of learning values, judgement of learning and cognition in learning and cognition in learning

chemistry in Taiwanchemistry in Taiwan

John Oversby, The University of Reading, UK

[email protected]

Shu-Nu Chang, College of Liberal and General Education, Aletheia University, Taiwan [email protected]

Nordic Helsinki © John Oversby and Shu-Nu Chang 2007

2

My backgroundMy background

► Chemist (UG and PhD): Leeds UniversityChemist (UG and PhD): Leeds University► School teacher (22 years):School teacher (22 years):

AfricaAfrica UK Secondary SchoolsUK Secondary Schools Responsible for ICTResponsible for ICT

► University teacher (15 years)University teacher (15 years) Teacher trainingTeacher training Research in modelling and teacher educationResearch in modelling and teacher education


3

Theme of talkTheme of talk

► Three Taxonomies by Bloom (1956)Three Taxonomies by Bloom (1956) CognitionCognition Motor skillsMotor skills AffectAffect

► Focus on Affect i.e. Values, Social, Focus on Affect i.e. Values, Social, Attitude, Motivation, Relationship, Attitude, Motivation, Relationship, Judgment of Learning (JoL)Judgment of Learning (JoL)


4

Two problems…Two problems…

►Assuring quality learning outcomesAssuring quality learning outcomes Requires data about learning during Requires data about learning during

learning events rather than at their learning events rather than at their conclusionconclusion

►Developing lifelong learnersDeveloping lifelong learners Requires that students can assess their Requires that students can assess their

own learning without formal summative own learning without formal summative tasks, relying instead on self-assessmenttasks, relying instead on self-assessment


5

Why use student confidence in Why use student confidence in learning?learning?

► Informed confidence, or a critical Informed confidence, or a critical awareness of your own capacity, is one of awareness of your own capacity, is one of the primary components of self-efficacythe primary components of self-efficacy

► Judgement of Learning (JoL) is a reliable Judgement of Learning (JoL) is a reliable indicator of test performanceindicator of test performance Even with gender bias (attribution theory) i.e.Even with gender bias (attribution theory) i.e.

► males attribute success to hard work and failure males attribute success to hard work and failure to bad luckto bad luck

► females attribute success to good luck and failure females attribute success to good luck and failure to lack of effortto lack of effort


6

Quality Assurance from JoLQuality Assurance from JoL

► JoL can be assessed rapidly, and during JoL can be assessed rapidly, and during task performancetask performance Some student training is needed to enable Some student training is needed to enable

critical self-appraisalcritical self-appraisal►Frequent evaluation allows timely Frequent evaluation allows timely

intervention to maximise learning intervention to maximise learning outcomesoutcomes

►Records of JoL assessment allow a timely Records of JoL assessment allow a timely and frequent process of assessment for and frequent process of assessment for learning to be documented and formalised.learning to be documented and formalised.


7

Development of Self—Development of Self—efficacyefficacy

……an individual’s estimate of his/her ability to cope with a an individual’s estimate of his/her ability to cope with a situation, and outcome expectancy; an individual’s situation, and outcome expectancy; an individual’s estimate of the likelihood of certain consequences estimate of the likelihood of certain consequences occurring. This combination of assessments of potential occurring. This combination of assessments of potential threat and coping resources determines how anxious an threat and coping resources determines how anxious an individual may become in a given situation.individual may become in a given situation.

Regular feedback regarding the accuracy of JoL Regular feedback regarding the accuracy of JoL should heighten critical capacity and reliabilityshould heighten critical capacity and reliability

Appropriate interventions by Teacher or Learner Appropriate interventions by Teacher or Learner should promote both generic and academic skill should promote both generic and academic skill developmentdevelopment

foundations for independent foundations for independent learninglearning


9

The QuestionsThe Questions

• How would you describe your How would you describe your • understandingunderstanding of the recent lecture content?of the recent lecture content?• ability to use the ability to use the diagram conventionsdiagram conventions??• participation in the participation in the teamworkteamwork aspects?aspects?• ability to ability to communicate ideas and processescommunicate ideas and processes??• ability with the level of ability with the level of symbolssymbols??• methods for methods for problem solvingproblem solving??• approach to asking questions or approach to asking questions or seeking helpseeking help??• approachapproach to this subject i.e. personal confidence? to this subject i.e. personal confidence?


10

Measuring Student Measuring Student ConfidenceConfidence

►Fortune LinesFortune Lines A variety of possible curvesA variety of possible curves

confidence

Does it work with this new evidence?

We didn’t do it like this before

Now I know which idea about acids to choose?

What are these experiments about?

Ah! Now I know about ways of looking at acids


11

The ResponsesThe Responses

►Based on an extension of Bloom’s Based on an extension of Bloom’s taxonomies for the affective (or taxonomies for the affective (or psychomotor) domainspsychomotor) domains Internalised values … ( Characterising Internalised values … ( Characterising

self)self) Organised values … (ranking values)Organised values … (ranking values) ValuingValuing Responding (active)Responding (active) Responding (passive)Responding (passive) ReceivingReceiving

Surface Approach?

Deep Approach?

Achieving orientation?


12

Defining persistent learning Defining persistent learning featuresfeatures

► Long term self-assessment is dependent on Long term self-assessment is dependent on characterisation of persistent learning featurescharacterisation of persistent learning features

► These are at odds with most of our learning These are at odds with most of our learning objectives – short term, fragmented and factualobjectives – short term, fragmented and factual

► Forging new ground in defining higher level Forging new ground in defining higher level generic cognitive aspectsgeneric cognitive aspects

► Perhaps leads to too much innovation and Perhaps leads to too much innovation and hence problems with self-assessment of the hence problems with self-assessment of the new areasnew areas


13

Example of persistent feature - Example of persistent feature - diagramsdiagrams

► Know types of diagrams and their major features e.g. Know types of diagrams and their major features e.g. 2D, 3D (and 2.5D!)2D, 3D (and 2.5D!)

► Know why many diagrams have accepted Know why many diagrams have accepted conventions e.g. colours of atoms in ball-stick conventions e.g. colours of atoms in ball-stick diagramsdiagrams

► Know how to interconvert diagrams e.g. 3D into 2D Know how to interconvert diagrams e.g. 3D into 2D through contours such as electron density diagramsthrough contours such as electron density diagrams

► Know how to relate different representations of the Know how to relate different representations of the same thing e.g. dot cross diagrams with orbital same thing e.g. dot cross diagrams with orbital energy diagramsenergy diagrams

► Know the strengths and limitations of running Know the strengths and limitations of running dynamic diagrams in the minddynamic diagrams in the mind


14

Example: How would you describe your Example: How would you describe your methods for problem solving?methods for problem solving?

► It is IntuitiveIt is IntuitiveI don’t really think about the solution method – it usually seems that I don’t really think about the solution method – it usually seems that

there is an obvious approach to take, even though most of these there is an obvious approach to take, even though most of these problems can be solved in more than one way. Sometimes I take a problems can be solved in more than one way. Sometimes I take a new approach to a problem to see if it will work more quickly or new approach to a problem to see if it will work more quickly or more accurately.more accurately.

► I am adeptI am adeptI understand the solution method for all the tutorial and example I understand the solution method for all the tutorial and example

problems, and know why the different methods were used. I problems, and know why the different methods were used. I usually start with a general approach based on the type of problem usually start with a general approach based on the type of problem and modify the method as I go along so that I get to the solution. and modify the method as I go along so that I get to the solution. Novel problems are interesting in the challenge because they Novel problems are interesting in the challenge because they challenge me to vary my method in some way to get to the challenge me to vary my method in some way to get to the answer.answer.

► I am competentI am competentThe approach for most problems is reasonably straightforward, and The approach for most problems is reasonably straightforward, and

depends on the type of problem. I can usually decide what method depends on the type of problem. I can usually decide what method to use fairly quickly and my choice usually gives me the correct to use fairly quickly and my choice usually gives me the correct result. If I make errors, they tend to be in the working rather than result. If I make errors, they tend to be in the working rather than the choice of method.the choice of method.


15

Training of studentsTraining of students

a)a) Provide criteria for initial discussionProvide criteria for initial discussion

b)b) Discuss the snapshots (following) in the Discuss the snapshots (following) in the light of the criterialight of the criteria

c)c) Discuss the criteria during data collectionDiscuss the criteria during data collection

Hindsight suggests that we did not provide Hindsight suggests that we did not provide strong training in using the criteria. This strong training in using the criteria. This will be amended in phase 2.will be amended in phase 2.


16

Snapshots of IndividualsSnapshots of IndividualsStudent1

0123456

Understanding

Diagrams

Teamwork

Communications

Mathematics

Problem Solving

Help Seeking

Approach

Student2

0

1

2

3

4

5

6Understanding

Diagrams

Teamwork

Communications

Mathematics

Problem Solving

Help Seeking

Approach

Student3

0

1

2

3

4

5

6Understanding

Diagrams

Teamwork

Communications

Mathematics

Problem Solving

Help Seeking

Approach

Student4

0

1

2

3

4

5

6Understanding

Diagrams

Teamwork

Communications

Mathematics

Problem Solving

Help Seeking

Approach


17

Research questionsResearch questions

► Can we construct a valid and reliable method Can we construct a valid and reliable method for assessing feelings about knowing and for assessing feelings about knowing and values of learning of students in a chemistry values of learning of students in a chemistry course?course?

► What aspects of learning are persistent What aspects of learning are persistent features from session to session that enable features from session to session that enable progress in higher level skills to be monitored?progress in higher level skills to be monitored?

► How do self-assessments of feelings about How do self-assessments of feelings about knowing change during the course?knowing change during the course?


18

Methodology IMethodology I► 110 freshmen non-major chemistry students in 110 freshmen non-major chemistry students in

TaiwanTaiwan► 6 weeks collecting data6 weeks collecting data► Attrition rate of 31% - did not attend every sessionAttrition rate of 31% - did not attend every session► Eight features of affect:Eight features of affect:

Use of diagramsUse of diagrams Use of graphsUse of graphs Help seekingHelp seeking Using symbols and equationsUsing symbols and equations Confidence in learning i.e. JoLConfidence in learning i.e. JoL CommitmentCommitment Group workGroup work ExplainingExplaining


19

Methodology IIMethodology II► Use of a six point scale of indicators for each Use of a six point scale of indicators for each

attribute, linked to Bloom’s Taxonomy of Valuesattribute, linked to Bloom’s Taxonomy of Values► Indicators written by John Oversby and Phil Turner Indicators written by John Oversby and Phil Turner

(construct validity)(construct validity)► Collection of named data each week in classCollection of named data each week in class► Two standardised hypothetical Case Studies as Two standardised hypothetical Case Studies as

reference points. Used in second week and sixth reference points. Used in second week and sixth week.week.

► Case studies written by John Oversby and Phil Case studies written by John Oversby and Phil Turner (construct validity)Turner (construct validity)

► Class test results used as reference point for Class test results used as reference point for cognitive abilitycognitive ability


20

Methodology IIIMethodology III► Data collected by SNC and analysed by JO, Data collected by SNC and analysed by JO,

anonymouslyanonymously► Analysis remains work in progress (more to Analysis remains work in progress (more to

be shared)be shared)


21

Interpretation IInterpretation I

►Case studies data shows large range Case studies data shows large range over class (standard deviation)over class (standard deviation)

►The range changes little even after six The range changes little even after six assessments. More experience does assessments. More experience does not provide greater consistencynot provide greater consistency

►May be an artefact of the referencing May be an artefact of the referencing system or difficulty in self-assessmentsystem or difficulty in self-assessment


22

Are students accurate in Judgment of Are students accurate in Judgment of Learning?Learning?

► Apparently not!Apparently not!► Perhaps we are Perhaps we are

expecting too much expecting too much to focus on to focus on persistent features persistent features of learning e.g. new of learning e.g. new thinking about thinking about metacognitive metacognitive aspects of diagramsaspects of diagrams

Use of diagrams

y = 0.0012x + 3.1124

R2 = 0.0002

0

1

2

3

4

5

6

7

0 10 20 30 40 50 60 70 80 90 100

scores

use

of

dia

gra

ms


23

Are students accurate in behaviour self-Are students accurate in behaviour self-assessment?assessment?

► Apparently a little!Apparently a little!► In this case we see In this case we see

that more able that more able students seek help students seek help less.less.

► Perhaps there is a Perhaps there is a plateau effect, plateau effect, where students aim where students aim for good enough. Is for good enough. Is there value in doing there value in doing better than this?better than this?

Help

0

1

2

3

4

5

6

7

0 10 20 30 40 50 60 70 80 90 100


24

Are students accurate in self-Are students accurate in self-assessment of confidence (JoL)?assessment of confidence (JoL)?

► Apparently not!Apparently not!► No difference No difference

between less able between less able and more able and more able students in students in judgement of judgement of confidence in confidence in learning.learning.

► May be caused by May be caused by lack of traininglack of training

Confidence

y = -0.0006x + 3.461

0

1

2

3

4

5

6

7

0 10 20 30 40 50 60 70 80 90 100


25

Are students accurate in self-Are students accurate in self-assessment of commitment?assessment of commitment?

► Apparently only a Apparently only a little but in the little but in the wrong direction! We wrong direction! We expected more able expected more able students to be more students to be more committed!committed!

► May be better May be better analysed in terms of analysed in terms of gender split but we gender split but we do not have this do not have this evidence in phase 1evidence in phase 1

Commitment

y = -0.0069x + 3.5708

0

1

2

3

4

5

6

0 10 20 30 40 50 60 70 80 90 100

scores

com

mit

men

t


26

Do more able students wish to work in Do more able students wish to work in groups?groups?

► Less able students Less able students prefer, a little, to prefer, a little, to work more in work more in groups.groups.

► May be better May be better analysed in terms of analysed in terms of gender split (in gender split (in phase 2)phase 2)

Group

y = -0.024x + 4.7155

R2 = 0.0751

0

1

2

3

4

5

6

0 10 20 30 40 50 60 70 80 90 100

scores

gro

up

wo

rk


27

Successes and areas for Successes and areas for improvementimprovement

► Clarified our own Clarified our own thinking about thinking about persistent learning persistent learning featuresfeatures

► Constructed attribute Constructed attribute lists and tested them lists and tested them for validityfor validity

► Demonstrated the rapid Demonstrated the rapid nature of the self nature of the self assessmentsassessments

► Found some evidence Found some evidence that correlated with that correlated with commitment as commitment as evidenced through evidenced through attendanceattendance

► Under-estimated Under-estimated training needstraining needs

► Under-estimated Under-estimated challenge of self-challenge of self-assessment of assessment of persistent learning persistent learning featuresfeatures

► Need to look at ways of Need to look at ways of sharing the information sharing the information with studentswith students

► Need to look more Need to look more widely at self-widely at self-regulationregulation


28

Where do we go next?Where do we go next?

► Repeat research with Repeat research with more specific training more specific training materials based on our materials based on our pilot studypilot study

► Review instructions for Review instructions for defining attribute levelsdefining attribute levels

► Refine Case Studies Refine Case Studies and check with a and check with a variety of colleagues variety of colleagues for validityfor validity

► Further analysis of dataFurther analysis of data► Involve more colleagues Involve more colleagues

– Slovenia, Poland, – Slovenia, Poland, Finland, Austria, US Finland, Austria, US Virgin Islands, UKVirgin Islands, UK

► Use different groups of Use different groups of learners – pre-service learners – pre-service teachers, undergraduate teachers, undergraduate science students, high science students, high school and junior high school and junior high school pupilsschool pupils


29

Keeping in touchKeeping in touch

► We are constructing a web-site with all the We are constructing a web-site with all the materials:materials: Background research and other materialsBackground research and other materials Papers and PP written on the subjectPapers and PP written on the subject Manuals for doing the project with Beginning Manuals for doing the project with Beginning

Teachers, high school students, teachersTeachers, high school students, teachers Spreadsheet templates for data collection and Spreadsheet templates for data collection and

analysis – leading to sharing of dataanalysis – leading to sharing of data Meetings and conferences on the researchMeetings and conferences on the research

► Write to John Oversby at Write to John Oversby at [email protected] after the end of [email protected] after the end of April to be in the next roundApril to be in the next round


30

ThanksThanks

►To our respective universities for their To our respective universities for their supportsupport

►To the students in Taiwan for sharing To the students in Taiwan for sharing their self-assessments with ustheir self-assessments with us

►To Phil Turner (James Cook University) To Phil Turner (James Cook University) for his ideasfor his ideas

►To ESERA for being the origin of this To ESERA for being the origin of this collaborationcollaboration

►To Shu-Nu Chang for being so reliableTo Shu-Nu Chang for being so reliable


31

Selected referencesSelected references► Bandura, A. (1991). Bandura, A. (1991). Self-regulation of motivation through anticipatory Self-regulation of motivation through anticipatory

and self reactive mechanisms.and self reactive mechanisms. In R. A. Dienstbier (Ed.), In R. A. Dienstbier (Ed.), Perspectives on Perspectives on motivationmotivation. Nebraska Symposium on Motivation. Lincoln University of . Nebraska Symposium on Motivation. Lincoln University of Nebraska Press.Nebraska Press.

► Bandura, A. (1997). Bandura, A. (1997). Self-efficacy: The exercise of controlSelf-efficacy: The exercise of control. New York: W. . New York: W. H. Freeman.H. Freeman.

► Cao L & Nietfield JL Cao L & Nietfield JL Judgment of learning, monitoring accuracy, and Judgment of learning, monitoring accuracy, and student performance in the clssroom context student performance in the clssroom context Current Issues in Current Issues in Education 8(4) 2005 Accessed on 26.8.05 from Education 8(4) 2005 Accessed on 26.8.05 from http://cie.asu.edu/vume8/number4http://cie.asu.edu/vume8/number4

► Gunstone RF and Mitchell IJ (1998) Gunstone RF and Mitchell IJ (1998) Metacognition and conceptual Metacognition and conceptual changechange in Mintzes Jr J, Wandersee JH and Novak JD (Eds) (1998) in Mintzes Jr J, Wandersee JH and Novak JD (Eds) (1998) Teaching for understanding. A human constructivist viewTeaching for understanding. A human constructivist view (pp133-165) (pp133-165) Academic Press, New YorkAcademic Press, New York

► Gunstone RF and Northfield J (1994) Metacognition and learning to Gunstone RF and Northfield J (1994) Metacognition and learning to teach. teach. International Journal of Science EducationInternational Journal of Science Education 16(5) 523-537 16(5) 523-537

► Gunstone R and White R (1992) Gunstone R and White R (1992) Probing understanding Probing understanding Falmer Press, Falmer Press, LondonLondon

► Krathwohl, D., Bloom, B., & Masia, B. (1956). Krathwohl, D., Bloom, B., & Masia, B. (1956). Taxonomy of educational Taxonomy of educational objectives. Handbook II: Affective domainobjectives. Handbook II: Affective domain. David McKay, New York. David McKay, New York

nordic helsinki © john oversby and shu-nu chang 2007 1 mapping affect project (map) promoting links...

Documents

shunu chang

learning events

learning chemistry

learninglifelong learning

taiwan john oversby

combination of selfefficacy

ict university teacher

aletheia university