class review week # 3

Research Methods for Counselors

COUN 597

Saint Joseph College

Class # 3

Copyright © 2007 by R. Halstead. All rights reserved.

Class Objectives

Basic Overview of MeasurementTrochim Chapter 3

Overview of Survey Research Trochim Chapter 4

Review of Correlation CoefficientsSalkind Chapter 5

Some Practice with Concepts

Topics Appropriate for Survey Research

There are a variety of uses for surveysIndividuals are the unit of analysis

Descriptive - (U.S. Census)Explanatory - (Attitudes individuals hold)Exploratory - (Discover some new aspect or

characteristic dimension such as in a needs assessment)

Self-Administered Questionnaires

Mail Distribution and ReturnElectronic SurveysMonitoring ReturnsFollow-up MailingsResponse Rates

Interview Surveys

The Role of the Survey InterviewerGeneral Rules for Survey Interviewing

Appearance and DemeanorFamiliarity with QuestionnaireFollowing Question Wording ExactlyAccurate Recording of ResponsesProbing for Responses

Coordination and Control

Telephone Surveys

Sampling ProblemsPhone ownership used to be a problem now it is cell

phone and access to the numbers which are unlistedOver use by telemarketing and political parties

AdvantagesRandom Digit DialingCost Effective - Computer Controlled Multi-

Number Dialing

Strengths and Weaknesses of Survey Research

StrengthsUseful in describing the characteristics in large

populationsAllows for obtaining very large sample sizesStandardization of the questionnaire allows for greater

control over factors that may serve to bias informant responses

Strengths and Weaknesses of Survey Research

WeaknessesStandardization of the questionnaire may limit the

uniqueness of informant responsesSurvey research rarely allows for an understanding of

the informants’ lives in social contextThere is an assumptive leap that how a person

answers the survey has some bearing on how that person actually operates a situation

Steps in the Survey Research Process

Questionnaire ConstructionSample SelectionSurvey Administration (Data Collection)Data AnalysisDrawing Conclusions

Secondary Analysis

One of the less expensive ways to engage in survey research is to conduct an analysis on research data that has been collected for another purpose.

Data archives allows researchers to do thisSee web link in syllabus to the Murray Center

Also see the ACA Code of Ethics that addresses release of data to researchers

Conceptualization & Measurement

Conceptualization – The birthing of an idea

Operational definitions – Clearly defining terms

Measurement – Adopting a method or method(s) for specifying and collecting data that can be later used for analysis.

Conceptualization

Identify personal conceptionsIdentify public constructsDevelop nominal definitionDevelop operational definition

(Operationalization) Terms: Concepts, Constructs, Indicators,

Dimensions

Conceptualization

Where Do Research Topics Come From?Practical problems in the fieldLiterature in the fieldYour own thinking

Is the Study Feasible?Tradeoff between rigor and practicalityHow long it will takeEthical constraintsNeeded cooperationCosts

Conceptualization

Conducting the Literature ReviewReview the scientific literature – What does it say?Are there inconsistencies that warrant further study?

Do the review early in the process

The literature review can help youSee if your idea has been triedInclude all relevant constructsSelect instrumentsAnticipate common problems

Operational Definitions

Points the way to how a variable will be measured

Specify observation procedures

Specify coding rules

Measurement

Levels of measurement

Measurement Error/Precision

Reliability

Validity

Levels of Measurement

Nominal Scale - Names and CategoriesExample: Gender, Martial Status, Race

Ordinal - Rank ordering Example: 1st, 2nd, 3th

Interval - Equal intervals between levels of an attributeExample: Age expressed in whole years.

Ratio - Continuous data can assume any value between two point along a continuum. Example: Time (2.35 seconds)

Levels of Measurement - So What?

Must be able to chose the level of measurement that will allow you to answer your question of interest. Here is an example.

MaleFemaleAsianInfant+

MaleFemaleAsianInfant? Divided by 4 = ?

Nominal Data

Measurement Error

Systematic Error - reflects a false picture because of some flaw in the system. AcquiescenceSocial desirabilityCulture bias

Random Error - Inconsistencies inherent to any form of measurement

Avoiding Systematic Measurement Error

Select appropriate instruments for your project

Select instruments that have demonstrated reasonable validity and reliability statistics

Look for elements that might suggest biasConduct a small pilot studyBe certain co-researchers are up to the taskAttend environmental factors

Reliability

Defined as the degree to which assessment measures are consistent, dependable, and repeatable.Inter-Rater ReliabilityTest-Retest ReliabilityParallel Forms or Alternate Forms

ReliabilitySplit-Half Reliability

Validity

A multidimensional concept use as a means of expressing the degree to which a certain inference drawn from a test is appropriate and meaningful.

An instrument measures what it purports to measure.

Types of Validity

Content Validity - The content of the items make sense - said to have Face Validity.

Criterion-related Validity - The test score is related to one or more outcome criteria of interest. (Concurrent and Predictive)

Construct Validity - Establishes that the instrument expresses an accurate measure of some construct.

The ProblemThe Problem

Concepts are not mutually exclusive.They exist in a web of overlapping

meaning.To enhance construct validity, you must

show where the construct is in its broader network of meaning.

Take a look at the next slide.

What Is the Goal?What Is the Goal?

The The constructconstruct

Other Other construct: Aconstruct: A

Other Other construct: Cconstruct: C

Other Other construct: Bconstruct: B

Other Other construct: Dconstruct: D

Measure Measure allall of the construct and of the construct and nothing else.nothing else.

Trochim, 2001

Example: You Want to MeasureSelf-EsteemExample: You Want to MeasureSelf-Esteem

Self -Self -esteemesteem

Trochim, 2001

Example: How Would You Distinguish Self-esteem From...Example: How Would You Distinguish Self-esteem From...

Self- Self- esteemesteem

Self-worthSelf-worth

ConfidenceConfidence

Positive Self- Positive Self- disclosuredisclosure

OpennessOpenness

Trochim, 2001

To Establish Construct ValidityTo Establish Construct Validity

You have to set the construct within a semantic (meaning) net.

You have to provide evidence that your data support the theoretical structure. (Constructs that should be more related, are more related and constructs that should be less related, are less related.)

Supply evidence that you control the operationalization of the construct (that your theory has some correspondence with reality).

Trochim, 2001

Summary

Reliability - speaks to the is consistency of measurement. Good reliability suggests that as the measure is used repeatedly, all factors being equal, the results will show little change.

Validity - speaks to the accuracy of the measure in labeling that which it is supposed to be measuring.

Correlation – A Review

Correlation is a statistical measurement that indicates the strength of relationship between two sets of data (two variables).

In essence, a correlation coefficient tells us how two sets of data “co-relate.”

Another way to think about this is how two variables change relative to one another.

Correlation - Continued

Samplesr xy is the correlation between variable x

and variable yr height-weight is the correlation between height

and weightr SAT-GPA is the correlation between SAT

and GPA


9 *8 *7 *6 *5 *4 *3 *2 *1 *0 1 2 3 4 5 6 7 8 9


StrengthPerfect Correlation + 1.00 Perfect Correlation - 1.00None 0.00

DirectionDirection - Positive (+)

and Negative (-)

Absolute Value and Strength

Which is the Stronger of the two correlations below?

+.50 - .70or

Answer: - .70 because the absolute value | .70 | is greater than the absolute value | .50 |.

Understanding What Correlation Means - A Rough Guideline

General Rule

Size of the Correlation General Interpretation.8 to 1.0 ---------------------- Very Strong.6 to .8 ---------------------- Strong.4 to .6 ---------------------- Moderate.2 to .4 ---------------------- Weak.0 to .2 ---------------------- Very Weak or at 0 No Relationship

Good method for a quick assessment

Understanding What Correlation Means - Coefficient of Determination

The Coefficient of DeterminationDefined: The percentage of variance in one

variable that is accounted for by the variance in the other variable.

Remember Variability? (Salkind Chapter 3, page 39)Variability (spread or dispersion) is a measure of how

different scores are from one another.We learned that usually variability is thought of as

measure of how much each score in a distribution differs from the mean.


Variables that share something in common tend to correlate with one another.Final grade in Appraisal and in Research have a

moderately high correlation for many reasons.Similar ConceptsHours of Study Put Into the Endeavor Consistent Trait Committed to Regular PracticeAptitude for Subject Matter

These factors and others account for the differences in students’ grades - variability


The more two variables share in common the more they will be related - correlate.

The two variables are said to share variability (the reasons why the final grades in Appraisal and Research tend to be similar).Similar ConceptsHours of StudyRegular PracticeAptitude for Subject Matter


To determine exactly how much of the variance (the dispersion or spread) one variable can be accounted for by the variance (the dispersion or spread) in another variable you just simply square the correlation coefficient.

Let’s look at an example of this.


Previously it was stated that we could make rough estimates regarding the strength of a correlation.

Size of the Correlation General Interpretation.8 to 1.0 ---------------------- Very Strong.6 to .8 ---------------------- Strong.4 to .6 ---------------------- Moderate.2 to .4 ---------------------- Weak.0 to .2 ---------------------- Very Weak or No Relationship


By computing the shared variance (the Coefficient of Determination) we can see why a r = .8 is strong as opposed to a r = .2

r = .8 squared = .64 or 64% of the variance for each variable is shared

r = .2 squared = .04 or 4% of the variance of eachvariable is shared

Understanding What Correlation Means

I know you have been holding back on asking the one question that is burning in the forefront of your mind - so let’s tackle that now.

What about that portion of the variance that is not shared? That portion that can not be explained by a coefficient of determination?

Understanding What Correlation Means - Coefficient of Alienation

The Coefficient of Alienation is the amount of the variance in one variable not explained by the variance in the other variable.

Logic would suggest, then, that the portion of unexplained variance must be due to factors (variables) that have not, as of yet, been taken into account.

Applying Correlation in Practice

Recall our December clients’ intakes.

5 7 14 16 18 20 20 20 24 26 32 33 37

3 5 9 8 7 20 19 24 15 17 10 12 11

Lets look at their levels of depression last August and arrange to data as matched pairs


December clients’ intakes.

Mean = 20.23 Md = 20 Mo = 20 Range = 33 Sd = 10.04

5 7 14 16 18 20 20 20 24 26 32 33 37

3 5 9 8 7 20 19 24 15 17 10 12 11

December clients in August.

Mean = 12.31 Md = 11 Mo = X Range = 22 Sd = 6.2


December clients intakes - Var X.

5 7 14 16 18 20 20 20 24 26 32 33 37

3 5 9 8 7 20 19 24 15 17 10 12 11

December clients in August - Var Y.


December clients intakes - Var X.

December clients in August - Var Y

5 7 14 16 18 20 20 20 24 26 32 33 37

3 5 9 8 7 20 19 24 15 17 10 12 11

r = .38

2

r = .14

CorrelationCoefficient of Determination


Correlation

Coefficient of Determination

r = .38

2

r = .14

What is this results of our data analysis saying?

What research questions might these results generate?

class review week # 3

Documents

correlation

similar concepts

survey research

applying correlation

correlation

validity

correlation

trochim