copyright © allyn & bacon (2007) final preparations before data collection graziano and raulin...

Copyright © Allyn & Bacon (2007)Copyright © Allyn & Bacon (2007)

Final PreparationsFinal PreparationsBefore Data Before Data CollectionCollectionGraziano and RaulinGraziano and RaulinResearch Methods: Chapter 14Research Methods: Chapter 14This multimedia product and its contents are protected under copyright law. The following are This multimedia product and its contents are protected under copyright law. The following are prohibited by law: (1) Any public performance or display, including transmission of any image prohibited by law: (1) Any public performance or display, including transmission of any image over a network; (2) Preparation of any derivative work, including the extraction, in whole or in over a network; (2) Preparation of any derivative work, including the extraction, in whole or in part, of any images; (3) Any rental, lease, or lending of the program.part, of any images; (3) Any rental, lease, or lending of the program.


Selecting Statistical Selecting Statistical ProceduresProcedures Selecting statistical analysis Selecting statistical analysis

procedures is part of the design procedures is part of the design processprocess

Based on the characteristics of the Based on the characteristics of the studystudy

Use the information about the study to Use the information about the study to follow the flowcharts to arrive at follow the flowcharts to arrive at decisions about statistics to usedecisions about statistics to use

Need not memorize the flowchartsNeed not memorize the flowcharts


Describing Your Describing Your Research (1/2)Research (1/2)

1.1. What is its level of constraint?What is its level of constraint?

2.2. What are the independent variables?What are the independent variables?

3.3. What are the levels of the What are the levels of the independent variables?independent variables?

4.4. What type of design is the research?What type of design is the research?

5.5. What are the dependent variables?What are the dependent variables?


Describing Your Describing Your Research (2/2)Research (2/2)

6.6. What is (are) the dependent What is (are) the dependent measure(s)?measure(s)?

7.7. What is the level of measurement of What is the level of measurement of each dependent measure?each dependent measure?

8.8. What type of data is generated for What type of data is generated for each dependent measure?each dependent measure?

9.9. List the research hypothesis List the research hypothesis (hypotheses)?(hypotheses)?

10.10. What kind of test is needed?What kind of test is needed?


Structure of the Structure of the FlowchartsFlowcharts Initial Flowchart (14.1)Initial Flowchart (14.1)

– Prompts for the details of the studyPrompts for the details of the study– Sends to other flowcharts based on data Sends to other flowcharts based on data

typetype Secondary FlowchartsSecondary Flowcharts

– Score Data Flowchart (14.2)Score Data Flowchart (14.2)– ANOVA Flowchart (14.3)ANOVA Flowchart (14.3)– Ordered Data Flowchart (14.4)Ordered Data Flowchart (14.4)– Nominal Data Flowchart (14.5)Nominal Data Flowchart (14.5)


Initial Initial FlowchartFlowchart


Score DataScore DataFlowchartFlowchart


ANOVAANOVAFlowchartFlowchart


OrderedOrderedDataDataFlowchartFlowchart


NominalNominalDataDataFlowchartFlowchart


Functional FlowchartFunctional Flowchart(Student Resource (Student Resource Website)Website) Functional Functional

flowchart walks you flowchart walks you through the through the process of selecting process of selecting a statistical testa statistical test

Links to the manual Links to the manual and computerized and computerized computation computation tutorialstutorials


Secondary AnalysesSecondary Analyses

Defined as analyses that do not directly Defined as analyses that do not directly test the hypothesis of the studytest the hypothesis of the study

Three types of secondary analysesThree types of secondary analyses– Post hoc analysesPost hoc analyses: Such as specific : Such as specific

means comparisons in ANOVAmeans comparisons in ANOVA– Secondary analyses to explain resultsSecondary analyses to explain results– Data snoopingData snooping: Exploring the data for : Exploring the data for

clues that might form hypotheses for later clues that might form hypotheses for later researchresearch


Caveats and Caveats and DisclaimersDisclaimers Flowcharts are a teaching device onlyFlowcharts are a teaching device only NOT every statistical procedure is NOT every statistical procedure is

includedincluded– Only the most common (e.g., testing mean Only the most common (e.g., testing mean

differences)differences) Procedures suggested by flowcharts, Procedures suggested by flowcharts,

although the correct procedures to although the correct procedures to use, are NOT the only correct use, are NOT the only correct procedures for that situationprocedures for that situation


Robust Statistical Robust Statistical ProceduresProcedures A statistical procedure is robust to an A statistical procedure is robust to an

assumption IF the assumption can be assumption IF the assumption can be violated without distorting resultsviolated without distorting results

Evaluated with Monte Carlo proceduresEvaluated with Monte Carlo procedures– Computer simulations of the effects of various Computer simulations of the effects of various

situations on the effectiveness of a statisticsituations on the effectiveness of a statistic Evidence suggests that assumption Evidence suggests that assumption

violations have little effect violations have little effect IFIF your sample your sample sizes about equal in each groupsizes about equal in each group


Pre-Data CheckPre-Data Check

A final check before beginning the A final check before beginning the researchresearch

Also serves as an excellent check Also serves as an excellent check during the planning for the studyduring the planning for the study

CoversCovers– Study definition and designStudy definition and design– Selection of all proceduresSelection of all procedures– Participant selection and assignmentParticipant selection and assignment– Preparation for running the studyPreparation for running the study


Checklist (1/8)Checklist (1/8)

Initial Problem DefinitionInitial Problem Definition– Literature reviewed?Literature reviewed?– Problem statement developed?Problem statement developed?– Variables operationally defined?Variables operationally defined?

Research HypothesesResearch Hypotheses– Clear hypotheses stating the Clear hypotheses stating the

direction and type of relationship direction and type of relationship expected?expected?



Statistical AnalysesStatistical Analyses– Descriptive statistics planned?Descriptive statistics planned?– Inferential statistics planned?Inferential statistics planned?– Post hoc and secondary analyses planned?Post hoc and secondary analyses planned?

Theory and operational definitionsTheory and operational definitions– Theoretical base for study clear?Theoretical base for study clear?– Do hypotheses and procedures address Do hypotheses and procedures address

the issues?the issues?



Independent Variable ManipulationIndependent Variable Manipulation– Manipulation carefully planned?Manipulation carefully planned?– Manipulation pretested?Manipulation pretested?– Manipulation check planned?Manipulation check planned?

Dependent MeasuresDependent Measures– Dependent measures operationally defined?Dependent measures operationally defined?– Measures pretested or piloted?Measures pretested or piloted?– Measures reliable and valid?Measures reliable and valid?– Procedures to measure reliability available?Procedures to measure reliability available?



Controls in placeControls in place– For threats to internal validity For threats to internal validity – For threats to external validityFor threats to external validity

ParticipantsParticipants– Adequately sampling target population?Adequately sampling target population?– Descriptive demographics collected?Descriptive demographics collected?– Sample large enough?Sample large enough?– Participants properly assigned to Participants properly assigned to

conditions?conditions?



Participants (continued)Participants (continued)– Groups carefully defined (differential Groups carefully defined (differential

research)?research)?– Matching information preserved (matched-Matching information preserved (matched-

subjects designs)?subjects designs)?– Participants available?Participants available?– Scheduling procedure ready?Scheduling procedure ready?– Participant incentive procedures in place Participant incentive procedures in place

(e.g., fees, credit)?(e.g., fees, credit)?



Research ethicsResearch ethics– IRB approval obtained?IRB approval obtained?– Informed consent forms ready (for Informed consent forms ready (for

human research)?human research)?– Debriefing and/or feedback Debriefing and/or feedback

procedures in place?procedures in place?– Ethical guidelines checked (for Ethical guidelines checked (for

animal research)?animal research)?



Preparation of setting (continued)Preparation of setting (continued)– Adequate space available?Adequate space available?– Space free of distractions?Space free of distractions?– Equipment checked out?Equipment checked out?– Enough research help available?Enough research help available?– Assistants adequately trained for Assistants adequately trained for

emergencies?emergencies?– Assistants trained on research procedures?Assistants trained on research procedures?– Blind procedures in place (if necessary)?Blind procedures in place (if necessary)?



Participant Preparation, Participant Preparation, Instruction, and ProceduresInstruction, and Procedures– Instructions to participants clear?Instructions to participants clear?– Instructions and procedures pilot Instructions and procedures pilot

tested?tested?


Checklist Complete?Checklist Complete?

ThenThen

Go for it!Go for it!


SummarySummary

Select statistical procedure as part of Select statistical procedure as part of design processdesign process– Statistical procedures selected before data Statistical procedures selected before data

collectioncollection– Selection based on study characteristicsSelection based on study characteristics

Pre-data checklistPre-data checklist– A final check before beginning researchA final check before beginning research– Covers design, ethical, and practical issuesCovers design, ethical, and practical issues

copyright © allyn & bacon (2007) final preparations before data collection graziano and raulin...

Documents

copyright allyn bacon

score data flowchart

ordered data flowchart

nominal data flowchart

anova flowchart slide

initial flowchart slide

flowcharts initial flowchart

copyright law