methodology & irb/urr

Download Methodology & IRB/URR

Post on 21-Jun-2015




0 download

Embed Size (px)


  • 1. See yourself graduate in 2014!

2. Methodology and IRB/URR Dr. James Lani Take Away Message Research design is a blueprint with several components: Research method: Quant/Qual/Mixed Operational constructs: How constructs are measured Sampling strategy and procedure What is Research Research follows the scientific method. Whats the scientific method? Formulation of testable questions or hypotheses Its organized knowledge: logical (theory) and evidence based (observable) Precise constructs Can be disproven (falsifiable) Parsimonious (simplest explanation) 3. Methodology: The Cookbook Metaphor Cooking Researching Making a stew Examining research questions Ingredients: 2 lbs beef, I clove garlic Preparation: Cube beef, mince garlic Cooking Instructions: Bake at 350 for one hour Celebrate: Youve made a replicable) stew! Ingredients: 20 participants, 15 item questionnaire, semi-structured interview. Preparation: Administer questionnaire before and after lecture; semi-structure interview with participants for 20 minutes using a tape recorder. Data analysis plan: Conduct dependent sample t-test; transcribe interviews then thematize participants responses. Celebrate: Youve conducted a (replicable) research study! Where the recipe can be replicated 4. Methodology Essential Ingredients Restate research question and hypotheses Overview Research Design Participants Materials/ Instruments Data Collection Procedures Data Analysis Plan Sample Size Limitations 5. Methods Quantitative Qualitative Mixed You can count it Not quantitative Both Goal Tend to answer What questions (What is relationship or differences) or When questions (when is theory supportedafter intervention Tend to answer Why questions (understand why people feel that way) or How questions (explore how they see things) Both Research methods/ Strategies of data collection Experimental (random assignment) and Non- experimental methods (no control group), Observations (time participant) Semi-structured interviews, Archive data, Observations (write down positive and negative feeling words) Both Operationalize variables Define constructs/ instruments Define constructs/ instruments Both Sampling strategy Discuss sampling process Discuss sampling process Both 6. Theoretical (or Conceptual) Framework Theory is a systematic explanation of behavior of phenomena. Theory guides analyses List existing theories and how your research questions relate to those theories E.g., Theory of mind: the ability to understand that others have their own beliefs, desires, intentions. Empathy. Tested by Faux Pas Task (ability to recognize a faux pas). Research question: Does alcohol abuse impact empathy? 7. Population The population is the group you want to generalize to. Describe characteristics of population Why is population relevant to problem (look at other peer reviewed studys justification) Distinguish the population from the sample 8. Sampling Frame and Sample Universe: theoretical population to generalize to Population: largest target population from universe you have access to or Sampling Frame Original Sample Final Sample Attrition 9. Survey Item Old (ones own SE) Change in item New (perceptions of others SE) 1 I feel that I have a number of good qualities. I have to she has I feel that she has a number of good qualities. 2 I feel I do not have much to be proud of. I do not to she does not I feel she does not have much to be proud of. 3 On the whole, I am satisfied with myself. I am to she is and myself to herself On the whole, she is satisfied with herself. If you need to amend instrument, use a change matrix; do not create your own instrument! 10. Materials: Informed Consent State purpose of project State procedure and how long it will take State voluntary nature of participation State risks (if any) Have them sign or state that by filing our survey they are agreeing to participate. 11. Constructs vs. Variables TheoreticalEmpirical Construct A: Social Environment Construct B: Personality Variable A: Birth-Order Variable B: Introversion/ extroversion 12. Constructs vs. Variables Constructs are the invisible abstract things were measuring (e.g., personality), while variables are the way were assessing (measure/operationalize) those invisible things. Constructs and variables need to be precise (is personality measured by introversion scale or by conscientiousness scale?) e.g., Intelligence is a construct, while the number of words remembered is a way of assessing intelligence. e.g., Personality is a construct, while the scores on an introversion/extroversion test is a way to assess an aspect of personality. 13. Variables Operationalize Constructs, Make Distinction between IVs/DVs, and Describe Level of Measurement Example: Does Empathy differ by group (alcohol abuse vs. no alcohol abuse)? Empathy is my dependent variable and Group is my independent variable. Empathy (my construct) is measured by scores on the Faux Pas task. Alcohol abuse (my construct) is measured by 5 or more drinks in one day. Empathy is a ratio-level variable measured with scores ranging from 0- 25, while my Group variable is a nominal-level (categorical-level) variable because participants re in one of two groups (alcohol abuse group or not in alcohol abuse group). 14. Reliability and Validity Valid but not Reliable Reliable but not Valid Reliable and Valid 15. Validity Internal Validity: IV causes a change in the DV (not time or other covariates, etc. External validity: can be generalized to the population Construct validity: does the scale measure the theoretical construct Translational validity: Face validity (items are reasonable) Content validity (items match the domains of interest) Criterion related validity: measures behave as theory predicts Convergent validity: how close the variable aligns with the construct (use EFA) Concurrent validity: construct relates to established instruments Predictive validity: measure can predict an outcome (GPAIncome) 16. Reliability Internal consistency: Cronbach alpha. Average inter-item correlation Inter-rater: if interval, correlate; if dichotomous, kappa Test-retest: administer same test at two times Split half: divide instrument into 2 parts and calculated totals, then correlate totals. 17. Brief Review: Units of Analyses Quantitative (e.g., Age) Nominal-level (Latin for name). Gender (M/F), Grouped (Old = 65+, middle age = 36-64, young = 35 or younger). Assign any number of groups (old = 1, middle = 2, young = 3). Ordinal-level is ranked (Latin for showing order). GPA (A-F), or age (group 1 = age 115, group 2 = age 16-25, group 3 = age 25-54, group 4 = age 65+) Interval/Ratio-level (also named scale or continuous; Latin for [equal] space [between numbers]). What is your age today in years? ____ (a number from 1-105) 18. Types of Methodology Models Theories explain phenomena, Models represent Phenomena. SEM and Path models Regression models (linear, logistic, ordinal) ANOVA models (repeated-measures) Time-series Heirarchical Linear Models (HLM) Correlational Models 19. Relationship Among Variables Employee Tenure Leadership Style Employee Satisfaction Customer Satisfaction Mediator Moderator 20. Data Collection Method -- Overview Describe the procedures used to administer the materials to the participants Remember to be as detailed as necessary so someone can literally replicate your study 21. Data Collection Qualitative Procedure for accessing participants Selection of data collected Number and duration of interviews How and when data is collected How data is recorded (e.g., hand notes, audiotaped) Role of researcher-relationship to researcher Quantitative Procedure to administer measures 22. Pilot Test (only if you develop instrument) Detect potential issues in the instrument Allows you to get feedback and to finalize your survey/interview items Makes sure participants understand survey items Assess typical responses to survey items or interview questions (were participants comfortable, long-winded, defensive, etc. Can you access data? 23. Data Analysis Plan Quantitative: Describe the analysis plan used to test each hypothesis, the assumptions of the statistical analyses, and a justification of the appropriateness of the analysis for each research question. Qualitative: Describe how the data will be analyzed (or thematized) Phenomenological Case Study Grounded theory 24. Sample Size Quantitative Varies by type of statistical analysis 1. Research questions in statistical language 2. Level of measurement of variables 3. Determine statistical analysis 4. Conduct power analysis Qualitative Varies by which qualitative approach is taken Phenomenological Case Study Grounded theory 25. Assumptions, Limitation, Delimitations Assumptions: Discuss things out of your control about the population and design, then justify assumptions (e.g., participants will answer honestly.) Limitations: Are out of your control and describe weaknesses in design, threats to validity (e.g., generalizability). Delimitations: Are in your control and relate to choices you will make to narrow the scope of the study (e.g., variables, research questions). 26. Ethical Considerations Describe informed consent procedures State whether your study will be anonymous or confidential with respect to the participants Describe considerations for children or vulnerable participants 27. Questions & Answers 28. See yourself graduate in 2014! 1-1 Personalized Dissertation Consulting 877-437-8600 29. Thank you for your participation and attention! Join us for our next webinar on Wednesday, April 23rd at 8:30pm ET. Results and Discussion Chapters, Editing, and Defense