edit 6900: research methods in instructional technology
DESCRIPTION
EDIT 6900: Research Methods in Instructional Technology. Lloyd Rieber Instructor. Eunjung Oh Graduate Assistant. UGA, Instructional Technology Spring, 2008 If you can hear audio, click If you cannot hear audio, click If you have a question, click. Two Topics for Today. - PowerPoint PPT PresentationTRANSCRIPT
EDIT 6900: Research Methods in Instructional Technology
UGA, Instructional Technology
Spring, 2008
If you can hear audio, click
If you cannot hear audio, click
If you have a question, click
Lloyd RieberInstructor
Eunjung OhGraduate Assistant
Two Topics for Today• Continue Introduction to Quantitative
Research Methods
• Overview of a class activity on how to compute a t statistic to determine if two means (pretest and posttest) are significantly different.
Not This Week
Informal Activity
SDCSystematic Data Collection
• An informal, (hopefully) enjoyable activity designed to give you first-hand experience collecting research data
• Your Task: Go and research something of interest to you!
• Report on it informally in writing
• Give 5 minute oral report
• 10%, Due: April 9
March 26 Quantitative Research (con’t)
April 2 Quantitative Research
April 9 Preparing a Research Report
SDC Reports (in class)
April 16 Finish SDC Reports (if needed)
Research Project Presentations?
April 23 Research Project Presentations
Remaining Course Calendar
Notes About the Next RDA
Notes About the Next RDA
Final Project Rubric
Look for Email with this.
Dr. Lloyd RieberThe University of Georgia
Department of Educational Psychology& Instructional Technology
Athens, Georgia USA
EDIT 6900 Research in Instructional Technology
Part IV. Quantitative Research Methodologies
Chapters 9-11
Running an Olympic Marathon:
No Significant Difference?
• 26 miles, 385 yards
• Times of top 2 runners at 2004 Olympics in Athens, Greece:– 1. Stefano Baldini ITA 2:10:55– 2. Meb Keflezighi USA 2:11:29
• Is a difference of 34 seconds statistically significant?
Total votes cast forBush or Gore in 2000:
No Significant Difference?
Experimental Designs
Experimental design is used to identify cause-and-effect relationships.
The researcher considers many possible factors that might cause or influence a particular condition/phenomenon.
The researcher controls for all influential factors except those having possible effects.
Independent and Dependent Variables
Variable: any quality or characteristic in a research investigation that has two or more possible values.
Independent variable: a possible cause of something else (one that is manipulated)
Dependent variable: a variable that is potentially influenced by the independent variable.
The Importance of Control
Control the confounding variables
Keep some things constant.
Include a control group.
Randomly assign people to groups.
Assess equivalence before the treatment with one ore more pretests.
Expose participants to both or all experimental conditions.
Statistically control for confounding variables.
Types of Experimental Designs (1)
Pre-experimental designs
True experimental designs
Quasi-experimental designs
Overview of Experimental Designs (2)
Group Time
Group1
Group2Tx: indicates that a treatment (reflecting independent variable) is presented.
Obs: Indicates that an observation (reflecting the dependent variable) is made.
: Indicates that nothing occurs during a particular time period.
Exp: Indicates a previous experience ( an independent variable) that some participants have had and others have not; the experience has not been one that the researcher could control.
Pre-Experimental Designs
Design 1: One-shot experimental case study
Group Time
Group1 Tx Obs
Design 2: One-group pretest-posttest design
Group Time
Group1 Obs Tx Obs
Group Time
Random assignment
Group1 Obs Tx Obs
Group2 Obs Obs
True Experimental Designs (1)
Design 4: Pretest-posttest control group design
Design 5: Solomon focus-group designGroup Time
Random assignment
Group1 Obs Tx Obs
Group2 Obs Obs
Group3 Tx Obs
Group4 Obs
Group Time
Random assignment
Group1 Tx Obs
Group2 Obs
True Experimental Designs (2)
Design 6: Posttest-only control group design
Quasi-Experimental Designs
Group Time
Group1 Obs Tx Obs
Group2 Obs Obs
Design 8: Nonrandomized control group pretest-posttest design
Factorial Designs
Design 15: Randomized two-factor design
Group Time
Treatment related to the two variables may occur
simultaneously or sequentially
Treatment related to Variable 2
Treatment related to Variable 2
Group1 Tx1 Tx2 Obs
Group2 Tx1 Obs
Group3 Tx2 Obs
Group4 Obs
Inferential Statistics (1)
Estimating population parameters(1)
Inferential statistics can show how closely the sample statistics approximate parameters of the overall population. The sample is randomly chosen and representative of the total population. The means we might obtain from an infinite number of samples form a normal distribution. The mean of the distribution of the sample means is equal to the mean of the population from which the sample shave been drawn.The standard deviation of the distribution of sample means is directly related to the standard deviation of the characteristic in question for the overall population.
Inferential Statistics (2)
Testing Hypotheses (1)
Research hypothesis vs. statistical hypothesis Statistical hypothesis testing: comparing the distribution of data collected by a researcher with an ideal, or hypothetical distribution
- significance level/alpha (α): e.g., .05, .01 - statistically significant - reject the null hypothesis
Inferential Statistics (3)
Testing Hypotheses (2)
Making errors in hypothesis testing - Type 1 error: alpha error - Type 2 error: beta error
Inferential Statistics (4)
Testing Hypotheses (3)
Making errors in hypothesis testing -Increase the power of a statistical test 1) Use as large a sample size as is reasonably possible 2) Maximize the validity and reliability of your measures. 3) Use parametric rather than non parametric statistics whenever possible.
- Whenever we test more than one statistical hypothesis, we increase the probability of making at least one Type 1 error.
Inferential Statistics (5)
Examples of inferential statistical procedures
Parametric statistics Nonparametric statistics
Students’ t test Sign test
Analysis of variance (ANOVA)
Mann-Whitney U
Regression Kruskal-Wallis U
Factor analysis Wilcoxon matched-pair signed rank test
Structural equation modeling (SEM)
Chi-square goodness-of-fit test
Odds ratio
Fisher’s exact test
Inferential Statistics (6)
Example of reporting a test of a statistical hypothesis:
Percentage means and standard deviations are contained in Table 1. A significant main effect was found on the test of learning outcomes, F(1, 97) = 9.88, p <.05, MSerror = 190.51. Participants given the educational game scored significantly higher (mean =91.5%) than participants who were not given the game (mean=71.2%).
Your Task(This has already been emailed to you.)
1. Finish watching my pre-recorded presentation introducing quantitative research methods first.
2. Launch your Excel from last week. “Save as” with a new title.
3. Compute a t statistic from the data set emailed to you. Follow my video tutorial.
4. Email your spreadsheet to me as an attachment. (You do not have to finish this evening, but I expect most will.)
This is meant as a class activity. It is not a graded activity.
If you get stuck and become totally frustrated, stop and send
me what you have.
To do list• Follow the Course Learning Plan!