okairp fall conference nsu @ broken arrow november 4, 2005
TRANSCRIPT
Designing Methods for Determining Educational Gain: Involving Faculty in Assessment
Mark L. Giese
Northeastern State University
Tahlequah, Oklahoma
What are dispositions?
What is the big deal about dispositions?
Dispositions are:Concepts we inherently value as a
profession.Esoteric to the discipline.
What are dispositions in Education?
In Health and Kinesiology, dispositions may include:
Hard workTeamworkOur students or clients can improve with
practice
Why measure?
Looks at the Affective Learning Domain.
In the past, typical areas of measurement have only included knowledge and skills.
Accrediting agencies (learned societies) require measurement in this domain.
Steps in developing a measurement tool for your unit
PlanningReview the literature and ask your friends
what they are doing.Decide on which dispositions to measure
(committee).Decide if you intend to use this exercise as
an exit competency or if you are looking for a change from entry to exit.
Determine an a priori standard.
Steps in developing a measurement tool for your unit
ImplementationDevise a measurement tool (rubric)Collect your dataAnalyze your dataDetermine if an intervention is necessaryREMEASURE!
Examples
Example of a college rubric for measuring dispositions in pre-service teacher education candidates in the COE at NSU.
Examples
Example of a Sociology department rubric for measuring dispositions:
Attempt to measure “Social Distance”
Importance
Measuring dispositions is important because it gives us another piece of information to determine our effectiveness and quality.
Examples of Presentations on Social Distance
“Social Distance as a Disposition of Teacher Candidates.” Presented at AACTE, Chicago, Illinois, February 8, 2004.
Examples of Presentations on Social Distance
“Attitudes Towards Social Distance of Pre-Service Teacher Education Students.” Presented at Higher Education Research Day, Edmond, Oklahoma, November 2003.
Examples of Presentations on Social Distance
“Attitudes Towards Social Distance of Pre-Service Teacher Education Students.” Presented at the Oklahoma Association of Colleges for Teacher Education Conference, Norman, Oklahoma, November 9, 2000.
Measurement Goals
One Time: Graduation requirement etc.
Using a pre/post test design in an attempt to determine program gain. May be done at entry and exit of any major.
Determining Program Gain
Measure at pretest.Measure at post test.Determine the difference.Use hypothesis testing to determine if a
significant difference exists.
How ANCOVA works
The variable under study is the dependent variable and the covariate is the “other” data point. Both must be known for each subject (matched).
How ANCOVA works
The influence of the covariate is removed using a straight-forward linear regression method.
The ANCOVA produces an adjusted post test score for each subject and an adjusted mean as well.
ANCOVA Example
Three Methods of teaching French:Pretest……..instruction……Post testMeasures of intelligence availableRemove the effect of intelligence so the
French method may be the cause of the change from pre/post score.
ANCOVA
Group 1Pre
Post
. .
. .
. .
. .
. .
. .
. .
. .
X X
Group 2Pre
Post
. .
. .
. .
. .
. .
. .
. .
. .
X X
Group 3Pre
Post
. .
. .
. .
. .
. .
. .
. .
. .
X X
Extra Assumptions of ANCOVA
IV does not influence the covariate variable
Equal regression slopes
Linearity
r of .60 (Shavelson, 1996)
Review
Ask faculty if they need/desire to measure dispositions.
Assist with choosing an instrument and measurement model.
Assist in determining gain and the magnitude of the effect.